NEW YORK — Last week, astronomers, astrophysicists, data scientists and programmers came together at New York University to try to solve some of astronomy's toughest problems — in just five days.
The event, called Astro Hack Week, has only one rule: Everybody has to produce something. It might be a build of an astronomy data search algorithm, a series of programming tutorials or a bot that generates fake (and surprisingly plausible) tweets from one of the event creators. It might be planned from the get-go or something dreamed up based on a morning teaching session. But whatever it is, it must be built (or "hacked") entirely at Astro Hack Week and rely on the cooperation of programmers, scientists and engineers at all levels.
To deal with the huge complexity of data and simulation in astronomy today, many researchers are turning to data scientists and programmers for inspiration — or becoming programmers themselves. Space.com took a field trip last Thursday (Oct. 1) to see how Astro Hack Week is crafting that transition. [To Find Alien Earths, Scientists Comb Kepler Data (Video)]
"Many of the projects [that] people are working on didn't exist before, let's say, a week ago," said Phil Marshall, a staff scientist at the SLAC National Accelerator Laboratory at Stanford University in California and one of Astro Hack Week's organizers. "And some of them are emerging as we sit here," he told Space.com.
Astro Hack Week's mornings were filled with tutorials on different data techniques. During Thursday's afternoon hacking time, participants from around the world gathered in a large room in the NYU student center at several round tables, and the steady stream of discussion among pairs and trios adjusted volume as presenters came and went for pop-up lectures at the front.
The problems most groups were solving stemmed from dealing with massive amounts of data, often pulling in techniques tailored to the higher computing power available now that couldn't previously be applied to astronomy. For instance, one of the projects focused on speeding up a dark-matter simulation that required modeling hundreds of universes in a row, and another adapted a technique used in speech recognition to an analysis of star data from the Kepler space telescope (to figure out stars' internal rotation patterns). Many of the methods had only rarely been used in astronomical circles.
"Every field has its own jargon, but as soon as you overcome that, you realize that as long as you work on data problems, you kind of speak the same language, just with slightly different words sometimes," said Daniela Huppenkothen, a data scientist at NYU and a Hack Week organizer.
Huppenkothen's own hack, which she worked on throughout the event, concerned putting together a program for choosing attendants at big hack events with the right combinations of experience and expertise to spark new developments.
This is Astro Hack Week's second year, and its first in New York City. (The first was at the University of Washington last year.) Whereas astronomer and physicist David Hogg, also of NYU, spent the first year in an organizational role, he had a chance to work on a hack of his own this time around. He and a new collaborator worked on ways to reconstruct the three-dimensional structure of galaxies from two-dimensional images.
"For me, it's like time out of time," Hogg said. "There's nothing about the project that I'm doing that I couldn't have done on a longer timescale. On the other hand, hack week is like time out of time, where we can just focus 100 percent on what we're doing." For others, the projects they end up working on are completely unexpected or dreamed up right at the event: even previously-proposed projects "go through a mix-master" during the collaboration process.
Hogg noted that his results could go toward a new academic paper — the potential outcome for many of the new projects. [Researchers Create Massive Simulation Of The Universe (Video)]
At last year's event, Hogg said, there was a session on a machine learning method for classifying objects, called Random Forest, that had rarely been used in astronomy. With a machine learning program, a computer takes in many samples that have already been classified by humans to learn how to make the distinctions on its own. Many participants went on to implement that method to solve their own data classification problems during the event, and as those researchers publish their results, machine learning is becoming a solution to more and more astronomy problems. Hogg anticipates similar breakout techniques to come out of this event.
Hack Week is run and primarily sponsored by the Moore-Sloan Data Science Environment at NYU, whose mission is to change the way science is done by making data science methods a key, available part of the process. (Hogg is executive director of the program.) It's also sponsored by the Large Synoptic Survey Telescope in Chile, a massive project that will be able to map the whole sky in just a few nights after its construction in 2018 or 2020, and that will search for changes and new phenomena over the course of 10 years. The telescope's operators will need extraordinary data tools, developed in partnership with astronomers, to find the gems among the changing vistas.
Although the Hack Week concept isn't a new one to coders, it's one of the first to lend a "pop-up" approach to cracking tough problems astronomy researchers encounter.
"Hackers and coders have had hacking events for decades," Hogg said, "but scientists didn't think of themselves as hackers until recently."
Email Sarah Lewin at slewin@space.com or follow her @SarahExplains. Follow us @Spacedotcom, Facebook and Google+. Original article on Space.com.
Join our Space Forums to keep talking space on the latest missions, night sky and more! And if you have a news tip, correction or comment, let us know at: community@space.com.
