Close

Service Interruption

The NSIDC Web site and data services are currently having intermittent problems and may be unavailable. We are working to restore these services as soon as possible and apologize for any inconvenience this may cause. Please contact NSIDC User Services for assistance.

Libre

Advertise, Share, and Discover Data

Libre is supported by NASA Libre is supported by NSF Libre is supported by The Polar Information Commons

Support for Libre is provided by NASA, NSF, and the Polar Information Commons.

Creative Commons License
Rights to all Libre content, web applications, and APIs, are freely available under the Creative Commons By Attribution License.

Collection Caster Application Programming Interface

Overview

Note: The terms data set and data collection are often used interchangibly. However, for the purpose of this document, we will use the term data collection.

The Collection Caster Application Programming Interface (API) is a collection of Web services designed to help you advertise data collections so people can find them, and other applications and services can consume them through an Atom response feed. There are two main reasons to use the Collection Caster API:

The NSIDC Collection Caster API is compliant with the Federation of Earth Science Information Partners (ESIP) Discovery cluster's specification and uses the common Atom response format, Atom Syndication Format (RFC 4287), with extensions specific for Earth science data usage such as GeoRSS, OpenSearch and Time namespaces. Atom is an XML-based document format that describes lists of related information known as feeds. Feeds are composed of a number of items, known as entries, each with an extensible set of attached metadata. For example, each entry has an ID URL, a title, a summary, a start and end time, and at least one author. An Atom Feed can have zero, one, or many entries.

Since the API is ESIP compliant, than any aggregation service that is able to aggregate ESIP feeds will be able to discover and incorporate relevant entries from your feeds into their service; thus, greatly expanding the potential audience for your data.

You can use this API to create a collection cast feed, an XML document that you place on your Web site. In order to do this, you will need to create a header/footer file and a series of entry files, each of which describes a single data collection. The following documentation will explain these input files, the various outputs, and what to do with them. However, it is up to you to decide how you want to create the final collection cast file. It is recommended that you create the entry files in an automated manner using your metadata database or catalog of metadata records since essentially that is what the entry files are: files containing metadata about a particular data collection.

 


Creating the Header File

There are three parts to the Atom Feed XML document, the header file, the entry files, and the footer file. To generate the header file with this service, you will need to create either an XML input file or a JSON input file containing pertinent information. This information must also be constructed using the Federation of Earth Science Information Partners (ESIP) specifications to help ensure that Earth Science aggregators automatically retrieve and interpret your feed. To help you do that, we have created a template for you to use, or you can generate your own file without using our template paying close attention to the required fields and ESIP specifications.

  1. Choose either the XML or JSON format for creating the header file. Refer to Figures 1 and 2.
  2. Generate the header input file. Refer to Table 1 for the valid values for the variables listed in the header template.
  3. Save the file to the computer where you will be executing the commands to create the Atom Feed XML source content document.

XML Header File Template
JSON Header File Template

<?xml version="1.0" encoding="utf-8"?>

<feed>
<id>http://nsidc.org/data/api/example_collection_cast</id>
<title>National Snow and Ice Data Center</title>
<authors>
<email>jane.miller@nsidc.org</email>
<name>Jane Miller</name>
<simpleUrl>http://nsidc.org</simpleUrl>
</authors>
<updated>2011-09-26T12:09:54Z</updated>
</feed>

{"feed":{"id":"http://nsidc.org/data/api/example_collection_cast ","title":"National Snow and Ice Data Center ","authors":{"email":"jane.miller@nsidc.org ","name":"Jane Miller ","simpleUrl":"http://nsidc.org"},"updated":"2011-09-26T12:09:54Z "}}

Figure 1. Example of an XML Header Template File

Figure 2. Example of a JSON Header Template File



Table 1. Header Feed Input Variables
<feed>
<id></id>
<title></title>
<authors>
<email></email>
<name></name>
<simpleUrl></simpleUrl>
</authors>
<updated></updated>
</feed>
Variable
Description
<feed> Opening tag for the feed.
<id></id>1 Must be a IRI, which uniquely identifies your feed, for example, http://my.organization/feeds/my_data.
<title></title>1 This title tag is the headline for your feed. It should be carefully worded to attract the users you would like to reach.
<authors>1 Opening tag for author field. At least one author is required.
<email></email> E-mail address for the feed author.
<name></name>1 Organization or individual that administers this feed.
<simpleUrl></simpleUrl> URL associated with the organization or individual that administers this feed.
</authors> Closing tag for author field.
<updated></updated>1 Date and time the content of this feed was last refreshed. Must be an ISO time stamp.
</feed> Closing tag for the feed.
1 This is a required field.

Creating the Entry File

The entry file is an extensible set of metadata describing a particular data collection. For example, each entry file has an ID URL, a title, a summary, a start and end time, and at least one author. An Atom Feed can have zero, one, or many entries depending on how many data collections you want to advertise on your Atom Feed. Each entry is for one data collection. Thus, if you want to advertise four data collections on your Atom feed, you will need to create four separate entry files, one for each data set.

Before the XML entry source content file can be generated, you will need to create either an XML or JSON file containing pertinent entry information. This information must also be constructed using the ESIP specifications to help ensure that Earth Science aggregators automatically retrieve and interpret your feed. To help you do that, we have created a template for you to use, or you can generate your own entry file without using our template paying close attention to the required fields and ESIP specifications. It is recommended that you create the entry files using your metadata database or catalog since essentially that is what the entry file is, a file containing metadata about a particular data set.

  1. Choose either the XML or JSON format for creating the entry file. Refer to Figures 3 and 4.
  2. Generate the entry input file. Refer to Table 2 for the valid values for the variables listed in the entry template.
  3. Save the file to the computer where you will be executing the commands to create the Atom Feed XML source content document.
  4. Repeat Step 1 through Step 3 to create as many entry files as needed.

XML Entry File Template JSON Entry File Template

<?xml version="1.0" encoding="utf-8"?>

<entry>
<id>http://nsidc.org/data/g02171.html</id>
<title>Canadian Ice Service Arctic Regional Sea Ice Charts in SIGRID-3
Format</title>
<updated>2011-05-25T14:01:54Z</updated>
<summary>The Canadian Ice Service (CIS) produces digital Arctic regional sea ice charts for marine navigation, climate research, and input to the Global Digital Sea Ice Data Bank (GDSIDB). The ice charts are created through the manual analysis of in situ, satellite, and aerial reconnaissance data. The ice charts have information on ice concentration, stage of development, and ice form, following World Meteorological Organization terminology. These sea ice charts begin in 2006 and cover the following regions of the Canadian Arctic: Northern Canadian waters (Western Arctic, Eastern Arctic, and Hudson Bay) and Southern Canadian waters (Great Lakes and
East Coast). Each regional shapefile (.shp) (encoded in SIGRID-3 format) and associated metadata file (.xml) are combined into a tar archive file (.tar) for distribution. All data are available via FTP.</summary>
<links>
<href>href="http://nsidc.org/data/g02171.html"</href>
<rel>http://esipfed.org/ns/discovery/1.1/documentation# </rel>
<type>text/html</type>
</links>
<links>
<href>http://nsidc.org/cgi-bin/get_metadata.pl?id=g02171&amp;amp;format=FGDC</href>
<rel>http://esipfed.org/ns/discovery/1.1/metadata#</rel>
<type>application/xml</type>
</links>
<geoRSSsouth>40</geoRSSsouth>
<geoRSSnorth>42</geoRSSnorth>
<geoRSSwest>-116</geoRSSwest>
<geoRSSeast>-115</geoRSSeast>
<authors>
<email>jane.miller@nsidc.org</email>
<name>Jane Miller </name>
<simpleUrl>http://colorado.edu/AuthorsWebPage</simpleUrl>
</authors>
<startTime>2006-05-01</startTime>
<endTime>2011-09-01</endTime>
</entry>

{"entry":{"id":"http://nsidc.org/data/g02171.html","title":"Canadian Ice Service Arctic Regional Sea Ice Charts in SIGRID-3 Format ","updated":"2011-05-25T14:01:54Z ","summary":"The Canadian Ice Service (CIS) produces digital Arctic regional sea ice charts for marine navigation, climate research, and input to the Global Digital Sea Ice Data Bank (GDSIDB). The ice charts are created through the manual analysis of in situ, satellite, and aerial reconnaissance data. The ice charts have information on ice concentration, stage of development, and ice form, following World Meteorological Organization terminology. These sea ice charts begin in 2006 and cover the following regions of the Canadian Arctic: Northern Canadian waters (Western Arctic, Eastern Arctic, and Hudson Bay) and Southern Canadian waters (Great Lakes and East Coast). Each regional shapefile (.shp) (encoded in SIGRID-3 format) and associated metadata file (.xml) are combined into a tar archive file (.tar) for distribution. All data are available via FTP.","links":[{"href":"http://nsidc.org/data/g02171.html ","rel":"http://esipfed.org/ns/discovery/1.1/documentation#","type":"text/html"},{"href":"http://nsidc.org/cgi-bin/get_metadata.pl?id=g02171&format=FGDC","rel":"http://esipfed.org/ns/discovery/1.1/metadata#","type":"application/xml"}],"geoRSSsouth":40,"geoRSSnorth":42,"geoRSSwest":-116,"geoRSSeast":-115,"authors":{"email":"jane.miller@nsidc.org ","name":"Jane Miller ","simpleUrl":"http://colorado.edu/AuthorsWebPage"},"startTime":"2011-06-25T14:01:54Z","endTime":"2011-07-25T11:01:54Z"}}

Figure 3. Example of an XML Entry File Template

Figure 4. Example of a JSON Entry File Template



Table 2. Entry Input Variables
<entry>
<id></id>
<title></title>
<updated></updated>
<summary></summary>
<links>
<href></href>
<rel></rel>
<type></type>
</links>
<links>
<href></href>
<rel></rel>
<type></type>
</links>
<geoRSSsouth></geoRSSsouth>
<geoRSSnorth></geoRSSnorth>
<geoRSSwest></geoRSSwest>
<geoRSSeast></geoRSSeast>
<authors>
<email></email>
<name></name>
<simpleUrl></simpleUrl>
</authors>
<startTime></startTime>
<endTime></endTime>
</entry>
Variable
Description
<entry> Opening tag for the entry.
<id></id>1 The unique identifier for this data set or collection. Usually the URL for the data set.
<title></title>1 Name of data set or collection.
<updated></updated>1 Date/time that this entry was last updated, for example, last time the metadata changed. Must be an ISO time stamp.

Note: If this field is not entered, then the current time is used.
<summary></summary>1 A brief description of the data set that provides users with enough information to determine the usefulness of the data set. It should include brief statements of the following important information:
  • Parameters
  • Product type such as in situ instruments or sensor data record, and sensor/source information including if data were obtained from another institution, such as NOAA, NIC, etc.
  • Temporal coverage and any existing gaps in data set coverage.
  • Spatial coverage.
  • Units and resolution.
  • Data processing level/sampling (gridded, binned, swath) and any other parameters.
  • Ordering information.
  • Data format, data set size, and product media.
  • Brief discussion of ancillary data sets needed for processing.
  • Read software and analytical tools if available, no need for great detail in the summary, just mention existence.
<links> Opening tag for links.
<href></href> Must be a valid URL.

Optional URL to a Web page about this data collection, for example, href="http://nsidc.org/data/g02171.html"

Optional URL to a metadata record for this data collection, for example,
http://nsidc.org/cgi-bin/get_metadata.pl?id=g02171&amp;format=FGDC

Optional URL to access the data in this data collection, for example, href="ftp://sidads.colorado.edu/DATASETS/G02171/
<rel></rel>1 Must have at least one link with an ESIP rel attribute and one link with an alternate rel attribute

Optional URL to a Web page about this data collection, for example,
http://esipfed.org/ns/discovery/1.1/documentation#" type="text/html

Optional URL to a metadata record for this data collection, for example,
http://esipfed.org/ns/discovery/1.1/metadata#" type="application/xml

Optional URL to access the data in this data collection, for example,
http://esipfed.org/ns/discovery/1.1/data#" type="application/x-tar
<type></type> Must be valid mime type, for example, application/xml.
</links> Closing tag for links.
<geoRSSsouth></geoRSSsouth> Data set geospatial extent bounding box in geoRSS-simple format:

geoRSSsouth must be between -90 and 90
geoRSSnorth must be between -90 and 90
geoRSSwest must be between -180 and 180
geoRSSeast must be between -180 and 180.

Note 1: If this field is not entered, then it will default to -180, -90, 180, 90.
Note 2: Entering a south latitude that is greater than a north latitude will work, but will produce an invalid geoRSS bounding box.
<geoRSSnorth></geoRSSnorth>
<geoRSSwest></geoRSSwest>
<geoRSSeast></geoRSSeast>
<authors>1 Opening tag for author field. At least one author is required.
<email></email> E-mail address for user support.
<name></name>1 Organization or individual that administers this feed.
<simpleUrl></simpleUrl> URL associated with the organization or individual that administers this feed.
</authors> Closing tag for author field.
<startTime></startTime>1 Data set temporal coverage. Start date/time in ISO 8601 string format.
<endTime></endTime>1 Data set temporal coverage. Stop date/time in ISO 8601 string format.
</entry> Closing tag for the feed.
1 This is a required field.

Generating Content for the Atom Feed XML File

Now that the header file and entry input files have been created and saved, it is time to generate the content for the Atom Feed XML file. While a feed can be created interactively from the command line using a scripting language tool such as cURL, Python, Java, or Perl, we expect that most users will develop a script to automatically create/maintain their feed. For the purpose of this user manual, we used the cURL command in the example steps below.

There are three separate commands that you will use to create of the source content for the Atom Feed XML file. However, there is also a fourth command line that can be used to create an empty Atom Feed with no entries. For your convenience, Table 3 lists all the resource commands used to generate the Atom Feed XML source content document.

Generating the Header Content

  1. Depending on if you used XML or JSON format to create the header file, choose one of the following steps:

    1. If you used XML format, enter the following command:

      curl  -X POST -H "Content-Type: application/xml" -d @xxxx.xml   http://nsidc.org/libre/services/cast/collection/1.0/feed/header

    2. If you used JSON format, enter the following command:

      curl  -X POST -H "Content-Type: application/json" -d @xxxx.json   http://nsidc.org/libre/services/cast/collection/1.0/feed/header

      Where: xxxx = the directory path for the header/footer file.

  2. The following XML example output is generated from the command entered in Step 1.

    <?xml version="1.0" ?>
    <feed xmlns="http://www.w3.org/2005/Atom"><id>http://nsidc.org/data/api/example_collection_cast</id><title type="text">Test</title><updated>2011-09-26T12:09:54.000Z</updated><author><name>Jane Miller </name><email>jane.miller@nsidc.org</email><uri>http://nsidc.org</uri></author>jmiller@newice:~>

Generating the Footer Content

When generating the footer content, you are creating the closing tag for the Atom Feed XML file (</feed>). That is why you do not have to create a footer file, since there is no information needed to create a closing tag.

  1. Enter the following command:

    curl http://nsidc.org/libre/services/cast/collection/1.0/feed/footer

  2. The following XML example output is generated from the command entered in Step 1.

    </feed>jmiller@newice:~>

Generating the Entry Content

  1. Depending on if you used XML or JSON format to create the entry file, choose one of the following steps:

    1. If you used XML format, enter the following command:

      curl  -X POST -H "Content-Type: application/xml" -d @xxxx.xml   http://nsidc.org/libre/services/cast/collection/1.0/entry

    2. If you used JSON format, enter the following command:

      curl  -X POST -H "Content-Type: application/json" -d @xxxx.json   http://nsidc.org/libre/services/cast/collection/1.0/entry

      Where: xxxx = the directory path for the entry file.


  2. The following JSON example output is generated from the command entered in Step 1.

    <entry xmlns="http://www.w3.org/2005/Atom" xmlns:georss="http://www.georss.org/georss" xmlns:time="http://a9.com/-/opensearch/extensions/time/1.0/"><id>http://nsidc.org</id><title type="text">Title of data set</title><updated>2011-05-25T14:01:54.000Z</updated><summary type="text">summary of data set</summary><link href="http://nsidc.org/data/g02171.html" rel="alternate" type="text/html" /><link href="http://nsidc.org/cgi-bin/get_metadata.pl?id=g02171&amp;amp;format=FGDC" rel="http://esipfed.org/ns/discovery/1.1/metadata#" type="application/xml" /><author><name>Jane Miller </name><email>jane.miller@nsidc.org</email><uri>http://colorado.edu/AuthorsWebPage</uri></author><georss:box>40.0 -116.0 42.0 -115.0</georss:box><time:start>2011-06-25T14:01:54Z</time:start><time:end>2011-07-25T11:01:54Z
    jmiller@newice:~> curl -X POST -H "Content-Type: application/json" -d @/DEV/INTERNET/entry.json http://testsnowtest.nsidc.org/libre/services/cast/collection/1.0/entry

  3. You may have to repeat Steps 1 and 2 above depending on how many entry files you created and want to add to the Atom Feed. Or you may use a script or some other technology to generated all your entry content.

Generating an Empty Atom Feed Document

  1. Depending on if you used XML or JSON format to create the header/footer file, choose one of the following steps:

    1. If you used XML format, enter the following command:

      curl  -X POST -H "Content-Type: application/xml" -d @xxxx.xml   http://nsidc.org/libre/services/cast/collection/1.0/feed

    2. If you used JSON format, enter the following command:

      curl  -X POST -H "Content-Type: application/json" -d @xxxx.json   http://nsidc.org/libre/services/cast/collection/1.0/feed

      Where: xxxx = the directory path for the header/footer file.

  2. The following output is generated from the command entered in Step 1. This is an example of an XML output. The JSON output should be similar.

    <feed xmlns="http://www.w3.org/2005/Atom"><id>http://nsidc.org/data/api/example_collection_cast</id><title type="text">Test</title><updated>2011-09-26T12:09:54.000Z</updated><author><name>Jane Miller </name><email>jane.miller@nsidc.org</email><uri>http://nsidc.org</uri></author></feed>


Table 3. Commands Used to Execute the Atom Feed Document Content

All the commands used to generate the Atom Feed document content are listed in this table. Depending on if you created an XML file or a JSON file, use one of the command lines listed below, replacing the variables xxxx, yyyy, and zzzz with the values listed in Table 3.

XML Command Line

curl  -X xxxx -H "Content-Type: application/xml" -d @yyyy.xml   http://nsidc.org/libre/services/cast/collection/1.0/zzzz

JSON Command Line

curl  -X xxxx -H "Content-Type: application/json" -d @yyyy.json   http://nsidc.org/libre/services/cast/collection/1.0/zzzz

Where:

xxxx = the method used either GET or Post

yyyy = the directory path for the file being obtained, such as the entry file or the header/footer file.

zzzz = the resource used, such as entry or feed/header

Resource
Method
Produces
Consumes
Description
/help GET text/html N/A This document.
/example GET application/atom+xml N/A An example of an Atom Feed XML document.
/feed/footer GET text/plain N/A Command used to generate the content for the footer part of the Atom Feed XML document.
/entry POST application/atom+xml
text/xml
application/atom+xml
type=entry
application/xml
text/xml
application/json
Command used to generate the content for the entry/entries part of the Atom Feed XML document.
/feed/header POST text/plain application/xml
text/xml
application/json
Command used to generate the content for the header part of the Atom Feed XML document.
/feed POST application/atom+xml
text/xml
application/atom+xml
type=feed
application/xml
text/xml
application/json
Command used to generate an empty Atom Feed XML document.

Creating the Atom Feed XML Document

Now that you have the files needed to create the Atom Feed XML document, you are almost done. Using a text editor or programming language of choice, concatenate the output header file, the entry content files, and the footer file into one single Atom Feed XML document. However you choose to create the Atom Feed XML document is up to you, but you now have the pieces to create and publish your Atom Feed.