NASA DAAC Data Services Now Accessible as an API Endpoint

Featured

Are you looking to customize MODIS, GLAS, AMSR-E, or SMAP data? While customization options such as subsetting and reformatting are currently available through the NASA Earthdata Search web application, these services can now be accessed programmatically as a synchronous REST interface.

How does the API work?

Programmatic access to these data services is provided via an HTTPS URL. The programmatic access endpoint contains key-value pairs that utilize the Common Metadata Repository (CMR) to find the requested data, as well as the subsetting, reformatting, and reprojecting services to customize those data. When utilizing this API in a command line environment, the customized output is returned as either a single file or multi-file zip downloaded directly to your machine’s current working directory.

Programmatic access architecture

Architecture of the EOSDIS Service Interface. CMR and data processing services are called via programmatic API access. Source: https://developer.earthdata.nasa.gov/sdps/programmatic-access-docs

Why access data services programmatically?

While the NASA Earthdata Search application provides a comprehensive web-based search and order interface, you may be looking for a more direct access point for customized data. Using the new API feature, these data can now be easily accessed using a variety of approaches including URL transfer protocols (e.g. curl: https://curl.haxx.se/) and MATLAB readers (e.g. https://www.mathworks.com/help/matlab/ref/geotiffread.html). In addition, work is currently ongoing to provide OGC Web Coverage Service (WCS) compatibility, enabling data access through WCS-supported software such as ArcGIS, QGIS, and GeoServer.

Where can I learn more?

An overview guide to this API can be found at http://nsidc.org/support/how/how-do-i-programmatically-request-data-services. This guide walks you through the basic steps needed to construct the API, as well as resources to help you determine the customization options available for your data set(s) of interest. Comprehensive documentation provided by the API developers is also available on the Earthdata Developer Portal.

Comments? Concerns? Feedback?

Let us know! Email us at nsidc@nsidc.org to let us know how we can improve this API.

What is the difference between smoothed and unsmoothed data in the AE_L2A data set, and how should I use them together?

QandAEach frequency is looking at a different size footprint. Each frequency has its own feedhorn, and is thus susceptible to independent pointing errors. The most important benefit of resampling (or smoothing) is to create a suite of frequencies that are all looking at the same scene. So, when the higher-resolution channels are resampled to match the footprints of the lower-resolution channels, the data are smoothed.

In the not-resampled (unsmoothed) channels (denoted by ‘o‘ in Table 5):

  • The unsmoothed 23.8 GHz channel does not spatially correlate with any other channels, and should generally not be used in conjunction with any other frequencies
  • The 18.7, 36.5, and 89.0 GHz channels are not resampled to their own footprints, because it is intended that they be used in their native (not-resampled) form
  • The 89A and 89B GHz channels are not aligned with the lower-resolution channels, by design

In the resampled channels (denoted by ‘’ in Table 5):

  • Pointing errors in the 6.9 and 10.7 GHz channels led to resampling of these channels to their own footprints in order to line their boresights up (or spatially correlate) with the higher-resolution channels
  • The 23.8 GHz channel is resampled to spatially correlate with the 7.9, 10.7, and 18.7 GHz channels (it was determined that the footprint size of the 23.8 GHz channel was close enough to that of 18.7 that it did not warrant a suite of channels spatially correlated to the native 23.8 GHz footprint)
  • For 18.7 and 36.5, each higher frequency is resampled to correlate with those native footprints

The unsmoothed and smoothed channels (see Table 5) are useful for constructing sets of “spatially correlated” channels, such as:

Resolution 1: 6.9, 10.7, 18.7, 23.8, 36.5, 89.0
Resolution 2: 10.7, 18.7, 23.8, 36.5, 89.0
Resolution 3: (18.7), 23.8, 36.5, 89.0
Resolution 4: (36.5), 89.0
Resolution 5: (89.0)
where, () = unsmoothed

Over homogeneous areas of ocean, the different size footprint observed by each frequency is not a big problem. But if there is heterogeneous meteorology, each channel may be looking at different amounts of cloud, rain, land, etc.

Start with the lowest frequency (largest footprint) that is important to any particular application.

For example, 10.7 GHz wind speed retrieval. Then ask, what would 18.7 GHz say if it were looking at the exact same patch of earth as this 10.7 GHz observation? And how bright would this exact same patch of earth (“scene”) be at 36.5 GHz?

The unsmoothed 18.7 and 36.5 GHz channels do not answer these questions directly, but there are more than enough observations to construct “virtual” observations which do “measure” the same scene. These are the smoothed channels.

You can find this information in this Online Support article.

Fun with flags and other data quality information: AMSR-E’s new one-stop shop

The quality of a data set is very important to its user. Not knowing about data quality can lead to misleading interpretations and erroneous results. However, readily accessible data quality information enables the user to assess the limitations of a data set and to interpret the data accordingly.

Several of NSIDC’s data sets include quality information, which can range from flags, masks, or fields in the data or metadata to separate data quality summary files. Additional quality information is also often included in data set documentation and linked peer-reviewed literature. Thus, it can be a laborious process for data users to locate and interpret all of the quality information for a particular data set.

Fortunately, users of AMSR-E data sets can access all of this information in one convenient location. The new AMSR-E Data Quality Web pages provide information on the quality flags and files as well as data uncertainty reports provided by AMSR-E Principal Investigators. The data uncertainty reports are separated by parameter, such as sea ice and soil moisture, and provide the following:

  • A synopsis on sources of uncertainty
  • Best estimates of data uncertainty under optimal conditions for each measurement, including a confidence interval where possible
  • A description of how to interpret quality flags to understand the conditions under which uncertainty may be greater

If you are an AMSR-E data user, check out this one-stop shop for data quality information.

Flags at South Pole

International flags at the South Pole, pictured here, are symbolic of the Antarctic Treaty, just as data quality flags are symbolic of the quality of a data set. Credit: Ted Scambos, NSIDC

Tips for working with AMSR-E Daily Soil Moisture data

Soil moisture is a key variable in understanding land surface hydrology and in modeling ecosystems, weather, and climate. Among NSIDC’s most popular data sets is the “AMSR-E/Aqua Daily L3 Surface Soil Moisture, Interpretive Parameters, & QC EASE-Grids” (AE_Land3) data set. This data set is distributed in HDF-EOS format and one of the biggest hurdles encountered by many users is simply learning how to display the data. Here are some tips on how to get started.

The HDF Group provides sample code for access and visualization of HDF data into IDL, MATLAB, and NCL. Access to the sample code for AMSR-E HDF data is provided on the HDF Group’s HDF-EOS Comprehensive Examples Web page.

If you are more familiar with GeoTIFF format, you may choose to utilize the HDF-EOS to GeoTIFF (HEG) Tool to convert AMSR-E Daily Soil Moisture HDF data into GeoTIFF format. This tool also allows you to subset the data with spatial or parameter constraints, as well as change the output projection. These HEG Tool services are also available as an option when ordering these data through the Reverb search and order interface. Instructions on how to use these data services in Reverb can be found in this Online Support article.

If you are interested in importing the data into ArcGIS, you can either use the GeoTIFF files generated by the HEG Tool or downloaded from Reverb, or you can perform a few steps to import the native HDF-EOS files into ArcMap. Using the ArcToolbox, you can easily extract a data field from an HDF file and save it in a different raster format that you are more familiar with. Instructions detailing how to do this can be found in this Online Support article.

AMSR-E L3 Daily Soil Moisture plot, 07/01/2002

AMSR-E L3 Daily Soil Moisture plot, 07/01/2002.

Looking for sample code for reading and visualizing NSIDC’s HDF data?

Hierarchical Data Format (HDF) supports a variety of data types and allows for the transfer and manipulation of scientific data across diverse operating systems and computer platforms. It was developed by the HDF Group at the University of Illinois and is the standard data format for all NASA Earth Observing System (EOS) data.

A variety of HDF data are distributed at NSIDC, including data from the AMSR-E, MODIS, and GLAS sensors as well as the NISE data set.

Despite the versatility of HDF, some data users have had difficulty in reading and displaying HDF data and often ask if we have sample code for reading HDF data in programs such as IDL and MATLAB.

Fortunately, the HDF Group provides sample code for access and visualization of HDF data into IDL, MATLAB, and NCL. Access to the sample code for NSIDC HDF data is provided on the HDF Group’s HDF-EOS Comprehensive Examples Web page.

Also note that this information can be found in our Online Support under the AMSR-E, MODIS, GLAS, and NISE forums.

Sample Matlab plot of AMSR-E 12km Sea Ice

Sample plot generated in MATLAB from AMSR-E 12km Sea Ice data (from http://hdfeos.org/zoo/).