Category: Open Source

Terabytes of Earth Observation (EO) data are collected each day, quickly leading to petabyte-scale datasets. By bringing these datasets to the cloud, users can use the compute and analytics resources of the cloud to reliably scale with growing needs. In this post, we show you how to set up a Pangeo solution with Kubernetes, Dask, and Jupyter notebooks step-by-step on Amazon Web Services (AWS), to automatically scale cloud compute resources and parallelize workloads across multiple Dask worker nodes.

The Allen Institute seeks to understand how our brains, cells, and immune systems work when we are healthy and, ultimately, how they go wrong in disease. Allen researchers have generated and shared atlases that map the brain, gene-edited stem cell lines, and many more resources that have been used by millions of scientists around the world to accelerate their research. In collaboration with AWS and the Registry of Open Data on AWS, they make many of their datasets publicly available. In celebration of Open Science Week, check out some of these open datasets from the Allen Institute, and their impact on the research community.

Weather radars see more than just the weather: they see smoke from fires, meteors, birds, mayflies, and almost anything else in the atmosphere. This makes weather radars an invaluable tool for scientists seeking to further the understanding of atmospheric processes and anything else that happens to be flying through the radar’s field of view. The Amazon Sustainability Data Initiative (ASDI) seeks to accelerate sustainability-related innovation and research by helping to minimizing the cost and time required to store, acquire, and analyze large weather and climate datasets.

Governments have similar problems and are not in competition, so using technology that others have developed and made freely available via open source to transform essential services makes sense. To make open source solutions easier to find for governments, AWS recently created an online repository: Open Government Solutions.

This is a guest post by Maria Teleńczuk, research engineer at the Paris-Saclay Center for Data Science (CDS), and Alexandre Gramfort, senior research scientist at INRIA, the French National Institute for Research in Digital Science and Technology. Maria and Alexandre explain how they adapted their open source data challenge platform RAMP to train the models submitted by student challenge participants using Amazon Elastic Compute Cloud (Amazon EC2) Spot instances, and how they leveraged AWS to support three student challenges.

Governments around the world want to accelerate their digital transformation to offer simpler access to citizen services online and earn trust with effective solutions. This includes things like being able to send notifications to users, providing a single log-in for government services, or publishing public health information in the wake of a pandemic. The AWS Government Transformation Team is here to help. We highlight available solutions and build new open source solutions that governments can leverage. Before developing new software, establishing a foundation with a sound design system is an important first step so our solutions are consistent, accessible, and simple to use. Read on to learn more about why starting with an open source design system is important and how we selected one for the software we’ve built and will build in the future.

If you are hosting an ArcGIS web app today, then you are probably hosting it on a Windows or Linux server using traditional web server software like IIS or Apache. With the web hosting capability of Amazon S3 you can remove the need to run these servers and the maintenance, management, and monitoring overhead that comes with it. Serverless services like Amazon S3 can scale automatically and can be as simple as copying over your website assets to get up and running in minutes. This blog focuses on web app implementations using ArcGIS API for JavaScript (as other ArcGIS web apps have additional considerations).

Today more than 200,000 volunteers around the world are helping accelerate research toward COVID-19 therapies—by walking away from their computers. That’s because of a concept called distributed computing, which allows anyone with a home computer, laptop, or virtual machine to contribute computing power to a common cause. This month, nonprofit Folding@home has started sharing one of the world’s largest public protein simulation databases as an AWS Open Data Set so that researchers around the world can easily access this data to speed up the search for therapies for COVID-19.

Governments, like the state government of California, are in the midst of a transition to a new way of delivering vital information, services and programs using technology and the cloud. Government organizations are adopting approaches pioneered in the technology industry, including user-centered design, agile development, data science, modular contracting, and the use of modern technology platforms. Many of these governments, like the state of California, are using Amazon Web Services (AWS) to respond quickly and scale to battle unprecedented challenges, like COVID-19, and help them quickly make decisions about how to protect their constituents.

This month, the National Oceanic and Atmospheric Administration (NOAA) celebrates 50 years of “science, service, and stewardship.” Over the past five decades, NOAA has demonstrated its ability to push the boundaries of technological innovation to collect and understand data, as well as share that knowledge and information with others. AWS supports NOAA’s mission, in particular by providing public access to the agency’s environmental datasets since 2015 through the Registry of Open Data on AWS.