NASA Earthdata Cloud Data Access Guide
The NASA Earthdata Cloud is NASA’s cloud-based archive of Earth observations. It is hosted by Amazon Web Services (AWS). The NSIDC DAAC is currently transferring data products to the Earthdata Cloud. The list of NSIDC DAAC data products available in Earthdata Cloud is provided in NSIDC Data Search.
This guide provides an overview of how to access NSIDC DAAC data in Earthdata Cloud, covering two main approaches:
- Direct Cloud Access: Work with data directly in the cloud
- Local Download: Download cloud-hosted data to your computer
This guide also provides information on customization options for NSIDC DAAC data in the Earthdata Cloud.
Key things to know
- Downloading data from the Earthdata Cloud to your local computer or storage system is and will continue to be free for users. You don't need an AWS account to download data. For more general information on the Earthdata Cloud please see FAQs about NSIDC DAACs Earthdata Cloud migration.
- The NASA Earthdata Cloud Cookbook provides tutorials for finding data in Earthdata Search, Python, and R.
- This guide will be updated with more information as more tools for accessing data in the cloud become available.
Direct Cloud Access (S3)
Direct Cloud Access or simply Direct Access allows users to work directly with data in S3 buckets, streamlining cloud-based workflow. Users can open and load cloud-hosted data into memory without downloading them first.
What is S3?
Amazon S3 (Simple Storage Service) is the object storage system used by NASA Earthdata Cloud. Data are stored in S3 buckets, which function like digital containers. These buckets hold the file objects that include both the science data and its associated metadata.
Prerequisites for Direct Access
To access NSIDC DAAC data stored in Earthdata Cloud S3 buckets, you’ll need:
- An EC2 instance set up in the us-west-2 region. This article outlines how to create one.
- A free Earthdata Login account to access the data. If you don't have one, you can register here.
earthaccess
NSIDC DAAC provides a Python notebook demonstrating how to access ICESat-2 data in the NASA Earthdata Cloud via "direct S3 access" using the earthaccess library. This notebook is available at https://github.com/nsidc/NSIDC-Data-Tutorials/tree/main/notebooks/ICESat-2_Cloud_Access.
The earthaccess Python library simplifies searching, downloading, or streaming NASA Earth science data, requiring only a few lines of code.
For an additional example of using Python to directly access cloud-hosted data, refer to the NASA Earthdata Cloud cookbook at https://nasa-openscapes.github.io/earthdata-cloud-cookbook/how-tos/access-data/access-cloud-python.html. This example showcases how to use the earthaccess library to retrieve file URLs and subsequently access them using the xarray library.
Local Download
Web browser
Users can download data archived by the NSIDC DAAC from the NASA Earthdata Cloud using the HTTPS file system provided by CMR Virtual Directories. This file system is accessible through a web browser and supports single-click downloads. For users who need a large number of files, several programmatic options are also available.
From a data set landing page, you can access the HTTPS file system under Data Access & Tools.
Clicking the HTTPS File System link on a data set landing page will take you to the specific data set in the CMR Virtual Directories.
earthaccess
Users who want to download data to their local computer can also use the earthaccess Python library. For information on using earthaccess to download data to a local computer, refer to the Earthdata Cloud Cookbook at: https://nasa-openscapes.github.io/earthdata-cloud-cookbook/how-tos/access-data/access-local-python.html
Wget and cURL
Users familiar with WGET or cURL can continue using these command line tools to download data from Earthdata Cloud. Instructions and examples are provided in the Earthdata Cloud Cookbook at https://nasa-openscapes.github.io/earthdata-cloud-cookbook/how-tos/access-data/access-command-line.html
Customize Data in the Cloud with Earthdata Harmony
Select NSIDC DAAC datasets in the cloud can be customized using Harmony. This article provides step-by-step instructions for customizing cloud-based data using Harmony services available through NASA Earthdata Search: https://nsidc.org/data/user-resources/help-center/search-order-and-customize-nsidc-daac-data-nasa-earthdata-search.
The pages below offer guidance on using the Harmony API for programmatic data access to NASA NSIDC DAAC datasets in the Earthdata Cloud:
- The NASA Openscapes tutorial (https://nasa-openscapes.github.io/earthdata-cloud-cookbook/tutorials/IS2_Harmony.html) provides a step-by-step guide on using Harmony to subset ICESat-2 data using the harmony-py Python library.
- The NSIDC Harmony API Quickstart Guide (https://nsidc.org/data/user-resources/help-center/harmony-api-quickstart-guide-customizing-nasa-nsidc-daac-data-earthdata-cloud) explains how to use the API to request customized data with specific temporal and spatial filters. It covers topics such as formatting API endpoints, submitting jobs to Harmony, monitoring job status, and downloading processed data.
Resources
The NASA Earthdata Cloud Cookbook is a comprehensive resource that provides tutorials, guides, and examples for accessing and working with NASA data in the cloud: https://nasa-openscapes.github.io/earthdata-cloud-cookbook/
General Information about the Earthdata Cloud: Earthdata Cloud Cookbook - Cheatsheets, Guides and Slides
Tutorials for working with NASA DAAC data in the cloud, including tutorials on using the CMR API and direct S3 access: Earthdata Cloud Cookbook - Tutorials