Monthly Highlights Archive

Hacking the cryosphere

The calving front of Crane Glacier in Antarctica Earth's cryospheric features, like Crane Glacier in Antarctica, can be vast, remote, and often shrouded in polar darkness. Technology is increasing important to assembling a picture of how the cryosphere is changing. Credit: T. Scambos/NSIDC

In 2012, NSIDC faced a challenge: how to unleash the creative potential of software development in the service of science. Scientists and informatics researchers needed better tools, while NSIDC software developers wanted to stretch their coding skills and work with data in innovative ways. “We had this brilliant staff of researchers, an enormous amount of data, and tech-savvy software developers,” Erik Jasiak, head of NSIDC software development, said, “but the three weren’t coming together on new ideas.” Nobody was speaking the same language. “It was a war of abbreviations,” Jasiak said. “The researchers didn’t understand new technologies, and the developers didn’t feel they had permission to push the envelope. They all had the data, though,” he said. As well, the tight confines of scientific funding may afford little space for exploration of new technologies. How could developers find the chance to show researchers what those technologies could do?

Space for creativity

NSIDC software developers NSIDC software developers work through a programming problem in their team room. Credit: A. Veale/NSIDC

Jasiak gained agreement from NSIDC stakeholders to try a practice common with software industry leaders: software “hack” sessions, aka hackathons. Developers periodically get a small amount of free time to code and demonstrate a new technology. The concept is simple: give the developers a high-level goal and a little room to explore, and trust in their instincts. NSIDC hackathon time is relatively small by industry standards, just one to two days each quarter per developer. Expectations are straightforward: participation is voluntary, the work must relate to NSIDC objectives, and a developer must always report back their results. They must either demonstrate a new technology relevant to science or informatics at NSIDC, or, just as valuable, discuss their learning experience on what technologies were problematic. Early hackathons revealed some of the gaps that needed to be bridged. In one of the first hackathons, developer Hannah Wilcox demonstrated overlaying NSIDC sea ice data sets in new mapping technologies. Scientists and analysts were impressed with the display, and what could be done with the format, but immediately began questioning the pixel colorations.  “I kept having to tell people, ‘This is not a done deal,’” Wilcox said. “’It’s not even correct yet; it just shows up.’” But the conversation about new technologies and research was beginning.

Building collaboration

A hackathon session at NSIDC showed scientists that this interactive Arctic sea ice chart was worth developing. A hackathon session at NSIDC showed scientists that this interactive Arctic sea ice chart was worth developing. It is now featured in NSIDC Arctic Sea ice News & Analysis as Charctic .

One year in, NSIDC hackathons are a growing success. One of the first projects teamed a relative newcomer to NSIDC, Kevin Beam, with an expert developer, Matt Savoie. Together, they produced an early version of an interactive Arctic sea ice extent chart (now live on the NSIDC Web site as Charctic ) in two days. “Matt knew the sea ice data, and had this idea for a Google finance chart for sea ice, and I knew the Ruby on Rails and JavaScript-fu [libraries] to make that work,” said Beam.  “It was kind of perfect because we had the data, and Matt knew what he wanted it to look like.  We just combined the knowledge and tools.” Charctic was an immediate hit with the NSIDC Arctic Sea Ice News and Analysis (ASINA) product team. "My first impression was 'Wow, this is so useful.’” said Walt Meier, a former NSIDC scientist (now with the NASA Goddard Cryospheric Sciences Lab). “We had always gotten comments from users asking to see specific years or other parts of the year than what we were showing at the time,” Meier said. Now instead of scientists having to manually make the relevant plots, users can make their own plots. "It's addicting to play around with, even for someone as familiar with the data as I am," Meier said. "The flexibility is amazing. It's great be able to hone in on a specific time of year, look at the specific data values, even pull up images of a given day's sea ice." The hackathon program continues to generate a storehouse of new ideas, tools, and libraries to help scientists with their data. Just as in scientific research, even failed experiments can produce useful knowledge. For example, a few ideas turned out be too cumbersome to set up during a hackathon session, and are now noted as having potential ramp-up issues—invaluable knowledge as researchers put together proposals. The conversations between researchers and developers about what is possible, and what is useful, improve every time.

A new way to find Arctic data

 
Icy water in the fjord of the Kangerdlugssuaq Glacier in eastern Greenland, as seen from NASA's P-3B aircraft during an Operation IceBridge flight. Diverse data on the Arctic, including IceBridge data, can be discovered through the Arctic Data Explorer. Image Credit: NASA/Jefferson Beck. Icy water in the fjord of the Kangerdlugssuaq Glacier in eastern Greenland, as seen from NASA's P-3B aircraft during an Operation IceBridge flight. Diverse data on the Arctic, including IceBridge data, can be discovered through the Arctic Data Explorer. Credit: NASA/Jefferson Beck.

When David Bailey, a climate modeler from the National Center for Atmospheric Research (NCAR), needs scientific data, he does one of two things: search for them using Google or write to the researcher who might have them. “I can’t even imagine the days before Google and e-mail,” he said. “You would hope to find some reference to the data in published literature, and that the authors point to where the data are available.” But this is seldom the case. Researchers also try their luck at major scientific conferences where they just might meet other researchers who have the data they need. Although usually effective, these data search methods can be time consuming. Researchers may go through hundreds of Web sites to find the data they need or run into dead ends when they write to other scientists for data. Lynn Yarmey, a data curator at the National Snow and Ice Data Center (NSIDC), said this is especially true when scientists look for data in another discipline. “They might be talking about the same data but will have different names for it,” she said. For example, what plant ecologists call photosynthesis is gross primary production to climate modelers. “If one community calls it something different than what others call it, then how would they know where to look?” Yarmey said.

Diverse data

A seal hunter prepares to jump back on his the sled in Qanaaq, Greenland. ADACIS data includes local traditional knowledge shared by Arctic residents and indigenous people. Cedit: Andy Mahoney, NSIDC A seal hunter prepares to jump back on his the sled in Qanaaq, Greenland. ADACIS data includes local traditional knowledge shared by Arctic residents and indigenous people. Credit: Andy Mahoney, NSIDC

Through the Advanced Cooperative Arctic Data and Information Service (ACADIS) project, NSIDC, in collaboration with NCAR, has put together a data search tool that will make data search easier for scientists who study the Arctic. The Arctic Data Explorer crosses disciplines and offers Arctic data on sea ice, biology, permafrost, meteorology, chemistry, demography, marine ecology, anthropology, oceanography, hydrography, biodiversity, and terrestrial ecology. A search for the term “polar bear” for example, gets results that include biological data like blood, muscle and breath analyses, behavioral data like location and movement, and local traditional knowledge of the animals in relation to changing sea ice conditions. A search for the term “fish” brings up data collected by groups as diverse as Arctic communities, research expeditions, and the oil industry. “There are so many pieces that all need to come together to answer these really big questions about climate change and the Arctic. We have to take a much broader view of science,” Yarmey said.

Minimizing clicks

The Arctic Data Explorer brings diverse data together by storing metadata from top Arctic data centers, including NSIDC, the ACADIS Gateway, the Earth Observing Laboratory/Computing, Data and Software Facility, the National Oceanographic Data Center, and the Norwegian Meteorological institute. “The Arctic Data Explorer searches through the metadata and connects the researcher with the data they are interested in, no matter which organization has that data set,” Yarmey said. “It really reduces the time and clicks that it takes for someone to get to the data.” NSIDC produced the Arctic Data Explorer as part of its work with ACADIS, which is reaching out to Arctic researchers to add their data to its growing collection. Bailey, who serves on the ACADIS Data Advisory Committee, said “We need to make it easy for observational scientists to put their data in the collection, and to make a search engine that will make end users want to keep coming back. It’s a huge challenge to make it useful at both ends.” Yarmey dreams that some day the process of contributing and finding data will be efficient and painless. “We can take some of the work out of sharing and finding data by building solid systems, automating metadata collection, and interconnecting data management systems. Then researchers can get the data they need quickly, and get back to doing research,” she said. For more information, see the Arctic Data Explorer at https://nsidc.org/acadis/search or contact support@aoncadis.org.