How do I access data using OPeNDAP?

OPeNDAP, the Open-source Project for a Network Data Access Protocol, is a NASA community standard DAP that provides a simple way for researchers to access and work with data over the internet. OPeNDAP's client/server software allows users to subset and reformat data using an internet browser, command line interface, or custom user interface such as a C NetCDF or Java NetCDF-compliant data analysis program. With a URL to an OPeNDAP server, users can browse data, perform subset operations, and open data directly in NetCDF-compliant software and tools like Matlab, R, IDL, and Panoply. The DAP server developed by the OPeNDAP group and used by the NSIDC Distributed Active Archive Center (NSIDC DAAC), called Hyrax, supports access to georeferenced gridded scientific data represented as arrays of cells arranged in rows and columns, and in NetCDF, GeoTIFF, JPEG2000, JSON, ASCII, and DAP2 (binary) formats. Data can be reformatted as NetCDF-3, NetCDF-4, ASCII, and DAP2 (binary) formats.

Using OPeNDAP for NSIDC DAAC data

Note that while OPeNDAP is enabled for most NSIDC DAAC data sets, it is currently only supported for select data sets (see Table 1). For these data sets, OPeNDAP outputs were tested for data integrity using Hyrax 1.13.1. Combinations of subsetting and reformatting were tested to ensure outputs were correct and values were consistent compared to the input files.

Table 1. NSIDC DAAC Data Sets Supported using OPeNDAP
Data Set Title and IDGet Data via OPeNDAP
MODIS/Terra Sea Ice Extent Daily L3 Global 1km EASE-Grid Day (MOD29P1D)OPeNDAP
MODIS/Terra Snow Cover Daily L3 Global 500m Grid (MOD10A1)OPeNDAP
MODIS/Aqua Sea Ice Extent Daily L3 Global 1km EASE-Grid Day (MYD29P1D)OPeNDAP
MODIS/Aqua Snow Cover Daily L3 Global 500m Grid (MYD10A1)OPeNDAP
Near-Real-Time SSM/I-SSMIS EASE-Grid Daily Global Ice Concentration and Snow Extent (NISE)OPeNDAP
SMAP L3 Radiometer Global Daily 36 km EASE-Grid Soil Moisture (SPL3SMP)OPeNDAP
SMAP L3 Radar Global Daily 3 km EASE-Grid Soil Moisture (SPL3SMA)OPeNDAP
SMAP L3 Radar/Radiometer Global Daily 9 km EASE-Grid Soil Moisture (SPL3SMAP)OPeNDAP
SMAP L3 Radar Northern Hemisphere Daily 3 km EASE-Grid Freeze/Thaw State (SPL3FTA)OPeNDAP
SMAP L3 Radiometer Global and Northern Hemisphere Daily 36 km EASE-Grid Freeze/Thaw State (SPL3FTP)OPeNDAP
SMAP Enhanced L3 Radiometer Global and Northern Hemisphere Daily 9 km EASE-Grid Freeze/Thaw State (SPL3FTP_E)OPeNDAP
SMAP Enhanced L3 Radiometer Global Daily 9 km EASE-Grid Soil Moisture (SPL3SMP_E)OPeNDAP
SMAP L4 Global Daily 9 km Carbon Net Ecosystem Exchange (SPL4CMDL)OPeNDAP
SMAP L4 Global 3-hourly 9 km EASE-Grid Surface and Root Zone Soil Moisture Geophysical Data (SPL4SMGP)OPeNDAP
SMAP L4 Global 3-hourly 9 km EASE-Grid Surface and Root Zone Soil Moisture Analysis Update (SPL4SMAU)OPeNDAP
SMAP L4 Global 9 km EASE-Grid Surface and Root Zone Soil Moisture Land Model Constants (SPL4SMLM)OPeNDAP

Using a command line interface

To access data from the command line, there are a few options depending on the tools that you have available, such as WGET and cURL

WGET Instructions - for command line in Mac and Unix/Linux



Example WGET command:

>wget --http-user=[USERNAME] --http-password=[PASSWORD] --load-cookies ~/.urs_cookies --save-cookies ~/.urs_cookies --keep-session-cookies --no-check-certificate --auth-no-challenge=on -r --reject "index.html*" -np -e robots=off < insert complete HTTPS OPENDAP URL >

The username and password should be your Earthdata Login username and password. When entering your username and password, remove the brackets [ ]. Also do not include the HTTPS URL brackets <>.

You can alternatively store your username and password in a .netrc file that can be used to log in using the following steps:

  1. Configure your username and password for authentication using a .netrc file

: ​

>echo "machine urs.earthdata.nasa.gov login <uid> password <password>" >> ~/.netrc

>chmod 0600 ~/.netrc




where <uid> is your Earthdata Login username and <password> is your Earthdata Login password. Do not include the brackets <>.

  1. Use the following WGET command:

>wget --load-cookies ~/.urs_cookies --save-cookies ~/.urs_cookies --keep-session-cookies --no-check-certificate --auth-no-challenge=on -r --reject "index.html*" -np -e robots=off < insert complete HTTPS OPENDAP URL >


WGET Instructions - for Windows


  1. Create a text file to store the website cookies returned from the HTTPS server, called "mycookies.txt". Store this in the wget installation directory.
  2. Use a WGET command to download your data, for example:

>wget --http-user=[USERNAME] --http-password=[PASSWORD] --load-cookies mycookies.txt --save-cookies mycookies.txt --keep-session-cookies --no-check-certificate --auth-no-challenge -r --reject "index.html*" -np -e robots=off <insert complete HTTPS OPENDAP URL>


WGET Tips

The WGET examples provided here will download files from the specified directory to a directory on your machine. The directory on your machine will have the title of the HTTPS host. For NSIDC OPeNDAP, this will be "n5eil01u.ecs.nsidc.org".

You can modify how the files are saved using the using the following WGET flags:
 -nd (or --no-directories)
-nH (or --no-host-directories)
--cut-dirs=number where number is the number of directories to cut, but does not include the host directory name (include -nH to remove that).

The GNU WGET 1.18 Manual provides more details on WGET options.

CURL Instructions - for command line in Mac and Unix/Linux


Example cURL command:

> curl -u "username:password" -b ~/.urs_cookies -c ~/.urs_cookies --location-trusted -O <insert complete HTTPS OPENDAP URL>

For Mac and Unix/Linux users, you can alternatively store your username and password in a .netrc file that can be used to log in using the following steps:

  1. Configure your username and password for authentication using a .netrc file: 



>echo "machine urs.earthdata.nasa.gov login <uid> password <password>" >> ~/.netrc

>chmod 0600 ~/.netrc




where <uid> is your Earthdata Login username and <password> is your Earthdata Login password. Do not include the brackets <>.

      2. Use the following cURL command:

>curl -b ~/.urs_cookies -c ~/.urs_cookies -L -n -O <insert complete HTTPS OPENDAP URL>

          The -O option on the cURL command downloads the file to the current working directory on your computer. 


Known issues & support options

The testing described above uncovered the issues listed in Table 2. All known issues occurred when reformatting the data, regardless of whether or not subsetting was performed. These issues were uncovered during testing for the SMAP Enhanced L3 Radiometer Global Daily 9 km EASE-Grid Soil Moisture (SPL3SMP_E) data set, and are some that you might encounter with any data set using OPeNDAP.

Table 2. Using OPeNDAP with NSIDC DAAC Data: Known Issues
Output FormatDescription of Issue
NetCDF-3,
NetCDF-4
  • If the data type is a string, blank values are returned.
  • If the data type is a string, grid dimensions are not consistent with the input/native data file (e.g. the UTC time field is three dimensional in the output rather than two dimensional as in the input/native file).
ASCII
  • If the data type is a string, blank values will be returned.
  • Real numbers are rounded (e.g. a input/native file is 0.5019599 whereas ASCII output is 0.50196).


    Still have questions? To learn how to use OPeNDAP, visit the OPeNDAP AsciiDoc site, which provides user guides and other helpful information about OPeNDAP and Hyrax. Our NSIDC User Services group is also available to provide basic support for your OPeNDAP questions. For assistance, email NSIDC User Services.

    Last updated: 10 January 2018