AWS Storage Blog

Category: Amazon Simple Queue Service (SQS)

Orchestrating multi-agent AI architectures with Amazon S3 Files

​​​​Organizations are moving beyond single-model AI toward multi-agent architectures. In these systems, agents offload intermediate results to files rather than carrying everything in the prompt, because a large prompt inflates cost and degrades quality. A model’s context window is finite, so files become working memory that persists after a session ends. In multi-agent systems, a […]

s3-annotations-header-image

Track healthcare data lineage in real time with Amazon S3 annotations

Organizations that store sensitive data need to answer a simple question: where did this data come from, and what happened to it? In healthcare, this isn’t optional. Regulations like HIPAA, GDPR, and SOX require organizations to produce this information on demand. When an auditor asks, for example, “Show me every transformation that touched patient record […]

Scalable cross-cloud data migration to Amazon S3 with distributed rclone

Migrating petabytes of data across cloud providers is one of the most operationally demanding tasks an organization can take on. At this scale, simple transfer approaches break down. Teams lose track of what has been copied and what has failed. Transfers stall and require constant manual intervention to restart. In some cases, teams need to […]

Amazon S3 Tables

How Pendulum achieves 6x faster processing and 40% cost reduction with Amazon S3 Tables

Customer Story | Amazon S3 Tables – Pendulum migrated from self-managed Apache Iceberg tables to fully managed Amazon S3 Tables and saw faster processing, fewer weekly maintenance hours, and reduced costs for their analytics platform. Pendulum is an AI-powered analytics platform that aggregates and analyzes real-time data from social media, news, and podcasts. Designed to […]

AWS Transfer Family Featured Image

How FICO modernizes file transfers with ETL automation using AWS Transfer Family

FICO powers decisions that help people and businesses around the world prosper. Using FICO solutions, businesses in more than 80 countries do everything from protecting four billion payment cards from fraud, to improving financial inclusion, and increasing supply chain resiliency. As a global leader in credit scoring and analytics, FICO processes massive volumes of sensitive […]

Amazon S3 Express One Zone thumbnail

Prime Video improved stream analytics performance with Amazon S3 Express One Zone

Amazon Prime Video provides a selection of original content and licensed movies and TV shows that you can stream or download as part of the Amazon Prime subscription. Prime Video’s telemetry platform serves as the backbone for monitoring playback performance, saving data snapshots for failure recovery, providing business analytics, and generating real-time insights across its […]

Amazon S3 Express One Zone thumbnail

Lyrebird improves performance and reduces costs for generative AI workloads using Amazon S3 Express One Zone

Through accessible mobile apps, Lyrebird Studio aims to transform photography into a creative tool for everyone. Founded in 2011, the company is a leading global developer and software publisher for users who enjoy expressing themselves and creating social content. To give millions of users a responsive experience with minimal downtime, Lyrebird Studio needs its data […]

Amazon S3 featured image 2023

Secure data in a multi-tenant environment by automatically enforcing prefix-level encryption keys in Amazon S3

Many organizations need to store and process data that belong to multiple entities, commonly referred to as multi-tenancy. In these situations, it is important to secure every tenant’s data and ensure that a consumer can only access the data that they require for their responsibilities and nothing more. In particular, if a user or application […]

AWS Transfer Family Featured Image

Architecting secure and compliant managed file transfers with AWS Transfer Family SFTP connectors and PGP encryption

Users in industries such as financial services and healthcare regularly exchange files with their external business partners containing sensitive and regulated datasets, such as Personal Identifiable Information (PII) and financial records. These file transfers often happen over the Secure File Transfer Protocol (SFTP) and encrypting files using Pretty Good Privacy (PGP) before transfer is often […]

Figure 1: Multi-region FSx for NetApp ONTAP with SnapMirror replication for SQL Server DR

Automating retrievals from the Amazon S3 Glacier storage classes

Faced with increasing amounts of data and a tightening economic climate, enterprises are looking to save money on their storage costs by moving rarely needed data to archival storage options. The least costly options require your internal systems to support receiving data back in hours or days, often called asynchronous retrievals. With this time delay, […]