Complete Sentinel-2 Archives Freely Available to Users

In a previous post, we shared a Q&A with Sinergise’s Grega Milcinski on Sentinel-2, an ongoing collection of satellite imagery of all land on Earth. We also discussed how to process and serve large amounts of data using serverless technologies, such as AWS Lambda.

To date, Sentinel-2 has already produced more than 1 PB of data. However, around 90 percent of this data goes unused. We continued the conversation with Grega on how to help more people make use of this data.

Can you tell us about the European Space Agency’s Sentinel program? What is its purpose and how much data is it producing?

Sentinels are part of Copernicus, an EU programme operated by the European Space Agency (ESA) focused on land observation. They provide insight into our planet on a weekly basis, up to 10-meter resolution globally. Datasets are available free of charge to anyone. Copernicus is one of the world’s largest Earth observation programmes. Sentinel-1 radar satellite and Sentinel-2 multi-spectral imaging satellite – which are the most interesting to the general public – generate more than 5 TB per day. This is expected to grow as the processing is stabilized.

Why is it important for this data to be openly accessible? How has AWS helped make this possible?

Now anyone has a view into what’s happening on Earth. This is important because we all need to be aware of environmental changes if we want to prevent further damage. The European Commission has invested more than 6 billion Euros into Sentinels based on the belief that data will generate new business opportunities.

Through AWS public datasets, Sentinel-2’s complete datasets are regularly archived and made freely available to users (both registered and non-registered). Open data on AWS makes it possible to build large-scale apps efficiently. We don’t need to invest in our own storage (over 1 PB). It affords a small company like Sinergise an opportunity to build and run a Sentinel Playground, a Google Maps-like application providing full and free insight into each and every image ever taken by Sentinel satellite. Sinergise is not alone in using the data; more than one billion file requests are now served by the AWS Sentinel-2 archive every month.

Can you share the challenges of effectively using the data, even while it’s freely available?

With so much data at our disposal, it becomes difficult to find the best possible scene at a specific location. The standard approach was to publish a list of images once per day. However, this required users to build their own databases and there was often lag after data was published. With data coming in every few minutes, this became tough to manage. Therefore, an advanced discovery service based on OpenSearch was established that will make it possible for anyone to query the data as soon as its available, pulling the results in the context of what they are working on.

How are you using this OpenSearch service in your own work? How do you expect others to use it in their work?

For Sinergise, a primary objective was to conduct research on OpenSearch standards and create something that would benefit the broader community. After establishing the Sentinel-2 repository on AWS, this service was a natural step. We are also exploring the option to replace our proprietary catalogue with an OpenSearch version.

For others, I’d hope they would integrate the service into their own processes based on a “just-in-time” strategy. This makes it possible to query the data and expect immediate results. With a few exceptions, there’s no need to store results or even the data itself. A user can also build full-fledged web applications without any back-end data processing. Web apps use OpenSearch to find available data, then Sentinel Hub OGC services to retrieve the relevant data using field-specific visualization. This makes the application cost effective and scalable.

“ESA is committed to open standards and actively collaborates with international organisations in their definition. ESA also actively promotes the adoption of these standards so that users can reap the benefit of the work. This is the case of the ongoing OGC Testbed 13 activity, where ESA is working with the OGC and a number of participants to test out new methods for data exploitation in cloud environments. For these tests, having access to a reliable service that offers discovery of Sentinel-2 data on AWS is an important enabler for the work being performed. In the future, the work of Testbed 13 will allow more commercial entities to offer similar services in public clouds directly to consumers,” said Cristiano Lopes, ESA.

What future applications do you envision will use this data and metadata?

We are amazed, observing what users are doing with Sentinel data. The options are limitless. Now, with almost two years of Sentinel-2 data available, and with consistent quality and availability, people are starting to realize the potential. And with the infrastructure and dedicated services to efficiently exploit it, I am confident we will see hundreds of applications helping farmers, disaster relief, environmental monitoring, and many others.

Monte Sarmiento in Tierra del Fuego, Sentinel-2, processed by Miha Kadunc

OpenSearch and WMS standards make it possible to integrate Sentinel-2 data in various open source applications, such as Cesium 3D viewer; Monte Sarmiento in Tierra del Fuego, processed by Miha Kadunc

Access Sentinel-2 on AWS and learn more about the close collaboration between ESA, industry, service providers, and data users.