NSIDC DAAC data pick up speed
November 2009
This year, access to data at NSIDC improved significantly, as a result of a substantial archive-technology shift and data migration by the NSIDC Distributed Active Archive Center (DAAC).
A solid archive
The NSIDC DAAC operates NASA's Earth Observing System Data and Information System (EOSDIS) Core System (ECS), which archives and distributes data from NASA's Earth Observing System (EOS) satellite programs. Previously, the NSIDC DAAC employed a StorageTek Powderhorn robotic tape library for storage and retrieval of all Earth science data products within the ECS. The tape library consisted of 9940A tapes, each with a capacity of approximately 200 gigabytes. When users requested data, the data were pulled from tape and temporarily placed on an FTP site. The most popular products were also added to an online disk cache, called the Data Pool, to provide faster access.
Faster access
With the availability of low-cost, high-capacity disk, the time was right to transition the ECS at NSIDC to an online disk archive with a tape backup system. First, the online Storage Area Network (SAN) disk array, currently used for the Data Pool, was expanded. Then NSIDC began a massive migration of data from the hundreds of 9940A tapes to the expanded Data Pool. The migration was performed with zero data loss. The Data Pool online archive now contains approximately 60 terabytes of EOS data, consisting of 8.5 million data files as well as associated metadata and browse files. A backup copy of the data was also made on a Quantum Scaler i500 tape library consisting of Linear Tape-Open version 4 (LTO-4) tapes, which have a capacity of 800 gigabytes. In contrast to the old, room-filling tape archive (pictured at right), the new backup system is the size of a small filing cabinet.
Speeding towards the future
The online, disk-based archive significantly reduces the latency of data orders and dramatically improves data accessibility. Having the data directly accessible online will also facilitate data discovery and analysis Web services. Ultimately, the new architecture will lead to more efficient science data support and improved user satisfaction.
