Landsat on AWS


The Multimedia Commons is a collection of audio and visual features computed for the nearly 100 million Creative Commons-licensed Flickr images and videos in the YFCC100M dataset from Yahoo! Labs, along with ground-truth annotations for selected subsets. The International Computer Science Institute (ICSI) and Lawrence Livermore National Laboratory are producing and distributing a core set of derived feature sets and annotations as part of an effort to enable large-scale video search capabilities. They have released this feature corpus into the public domain, under Creative Commons License 0, so it is free for anyone to use for any purpose.

This data set has known applied use in emergency management, commercial market research, and other fields. More broadly, this dataset could be useful for the next generation of computer vision, human mobility, machine learning, and social computing research.

AWS has made the images, videos, feature corpus and annotation sets for the Multimedia Commons freely available on Amazon S3. Now anyone can use the data on-demand in the cloud without worrying about storage costs and download time.

For more information, please visit http://www.multimediacommons.org. If you have any questions, please email multimedia-commons@icsi.berkeley.edu.

Landsat 8 data is available for anyone to use via Amazon S3. All Landsat 8 scenes from 2015 are available along with a selection of cloud-free scenes from 2013 and 2014. All new Landsat 8 scenes are made available each day, often within hours of production.

The Landsat program is a joint effort of the U.S. Geological Survey and NASA. First launched in 1972, the Landsat series of satellites has produced the longest, continuous record of Earth’s land surface as seen from space. NASA is in charge of developing remote-sensing instruments and spacecraft, launching the satellites, and validating their performance. USGS develops the associated ground systems, then takes ownership and operates the satellites, as well as managing data reception, archiving, and distribution. Since late 2008, Landsat data have been made available to all users free of charge. Carefully calibrated Landsat imagery provides the U.S. and the world with a long-term, consistent inventory of vitally important global resources.

AWS has made Landsat 8 data freely available on Amazon S3 so that anyone can use our on-demand computing resources to perform analysis and create new products without needing to worry about the cost of storing Landsat data or the time required to download it.

Learn more about how Landsat data is used on NASA's Landsat Science site.

Accessing Landsat on AWS

All Landsat 8 scenes from 2015 are available along with a selection of cloud-free scenes from 2013 and 2014. All new Landsat 8 scenes are made available each day, often within hours of production.

Landsat on AWS makes each band of each Landsat scene available as a stand-alone GeoTIFF and the scene’s metadata is hosted as a text file.

The data are organized using a directory structure based on each scene’s path and row. For instance, the files for Landsat scene LC81390452014295LGN00 are available in the following location: s3://landsat-pds/L8/139/045/LC81390452014295LGN00/

The “L8” directory refers to Landsat 8, “139” refers to the scene’s path, “045” refers to the scene’s row, and the final directory matches the scene’s identifier, which uses the following naming convention: LXSPPPRRRYYYYDDDGSIVV, in which:

  • L = Landsat
  • X = Sensor
  • S = Satellite
  • PPP = WRS path
  • RRR = WRS row
  • YYYY = Year
  • DDD = Julian day of year
  • GSI = Ground station identifier
  • VV = Archive version number

 

In this case, the scene corresponds to WRS path 139, WRS row 045, and was taken on the 295th day of 2014.

Each scene’s directory includes:

  • a .TIF GeoTIFF for each of the scene’s up to 12 bands (note that the GeoTIFFs include 512x512 internal tiling)
  • .TIF.ovr overview file for each .TIF (useful in GDAL based applications)
  • a _MTL.txt metadata file
  • a small rgb preview jpeg, 3 percent of the original size
  • a larger rgb preview jpeg, 15 percent of the original size
  • an index.html file that can be viewed in a browser to see the RGB preview and links to the GeoTIFFs and metadata files

 

For instance, the files associated with scene LC81390452014295LGN00 are available at:

s3://landsat-pds/L8/139/045/LC81390452014295LGN00/

or

http://landsat-pds.s3.amazonaws.com/L8/139/045/LC81390452014295LGN00/index.html

 

A gzipped csv describing all available scenes is available at

s3://landsat-pds/scene_list.gz

or

http://landsat-pds.s3.amazonaws.com/scene_list.gz

 

If you use the AWS Command Line Interface, you can access the bucket with this simple command:

aws s3 ls landsat-pds

Receive Data Notifications with Amazon SNS

The following ARN is for an Amazon SNS topic that provides notifications whenever a new Landsat scene has been added to Landsat on AWS:

arn:aws:sns:us-west-2:274514004127:NewSceneHTML

This topic publishes an Amazon S3 event message whenever a scene-level index.html file has been created, which is the last step in the process to make scene data available on Amazon S3. It will only accept subscriptions via Amazon SQS or AWS Lambda.

Learn more about subscribing to SNS topics.

Project Updates

To receive updates about Landsat on AWS, please fill out the form below.

About the Data

Source
U.S. Geological Survey
Category GIS, Sensor Data, Satellite Imagery, Natural Resource
Format GeoTIFF, txt, jpg
License There are no restrictions on the use of data received from the U.S. Geological Survey's Earth Resources Observation and Science (EROS) Center or NASA's Land Processes Distributed Active Archive Center (LP DAAC), unless expressly identified prior to or at the time of receipt. More information on licensing and Landsat data citation is available from USGS.
Storage Service Amazon S3
Location s3://landsat-pds in US West (Oregon) Region
Update Frequency New Landsat 8 scenes are added regularly as soon as they are available

Development Seed manages landsat-util, a popular command line utility that makes it easy to search, download, and process Landsat imagery. landsat-util has been updated to access Landsat on AWS.

Learn how Landsat on AWS helps Development Seed create Landsat power tools.

Esri has created a demonstration of how ArcGIS Online can quickly visualize Landsat on AWS data for visualization and analysis within the browser.

Visit Esri’s site to see how powerful and beautiful Landsat data can be.

Mapbox uses Landsat on AWS to power Landsat-live, a browser-based map that is constantly refreshed with the latest imagery from the Landsat 8 satellite.

Visit Mapbox to explore Landsat-live.

MathWorks has created a freely-downloadable tool for accessing, processing, and visualizing Landsat on AWS data in MATLAB. With this tool, you can create a map display of scene locations with markers that show each scene’s metadata.

Learn more about the tool and watch a demo video of it on the MathWorks blog.

EOS Data Analytics has created a web application for quick search for Landsat scenes on AWS with support of various filters and viewing any scene with any scale (zoom factor) using any of the band combinations, as well as calculation of spectral indexes on the fly.

Explore the Earth with EOS DA Landsat Viewer.

Planet Labs uses Landsat on AWS for image rectification, as a reference point for its own Earth observing satellites, and to create accurate and timely mosaics.

Learn how Planet Labs uses Landsat on AWS to quickly create better products for its customers.

If you would like to show us what you can do with Landsat on AWS or would like to receive updates on Landsat on AWS, please fill out the form below.

Educators, researchers and students can also apply for free credits to take advantage of the utility computing platform offered by AWS, along with Public Datasets such as the Landsat on AWS. If you have a research project that could take advantage of Landsat on AWS, you can apply for an AWS Grant.

Learn more about open data on AWS