AWS Storage Blog

As your Spark datasets grow, storage latency often becomes the constraint, impeding application performance. Query runtimes stretch, and the bottleneck shifts from compute to how fast each node can read from Amazon Simple Storage Service (Amazon S3). We benchmarked this directly on Amazon EMR with TPC-DS at 3 TB scale. On an 8-node Graviton4 cluster […]

Organizations that replicate data across storage locations for disaster recovery, compliance, or analytics need a way to detect when replication stalls, because without early warning, unreplicated objects accumulate silently and drive unexpected storage costs. Most teams rely on automated retention policies to clean up the source copy after a defined period, but when replication breaks […]

AWS Backup centralizes data protection in your organization across hundreds of accounts and AWS Regions for more than 20 AWS services. At that scale, some jobs fail, and triaging each failure manually can take days. Most failures trace back to something small, such as a role missing permission, an encryption key policy blocking the backup […]

Nearmap captures high-resolution aerial imagery across populated areas of the United States, Canada, Australia, and New Zealand several times a year, at resolutions as fine as 1.5 inches. More than 10,000 customers globally use these images to assess insurance portfolios, size solar arrays, and track construction sites. Since 2007, Nearmap has completed more than 35,000 […]

With the introduction of AWS European Sovereign Cloud, organizations across Europe are adopting this dedicated partition to meet stringent data sovereignty requirements. However, organizations face significant operational challenges because they must maintain their existing AWS commercial partition deployments for global customers while simultaneously establishing AWS European Sovereign Cloud infrastructure. This dual-partition reality can cause data […]

​​​​Organizations are moving beyond single-model AI toward multi-agent architectures. In these systems, agents offload intermediate results to files rather than carrying everything in the prompt, because a large prompt inflates cost and degrades quality. A model’s context window is finite, so files become working memory that persists after a session ends. In multi-agent systems, a […]

WeatherBug is the third largest weather intelligence company in the US, delivering real-time forecasts, radar, lightning alerts, and interactive maps to over 10 million users. As their data footprint has grown across hundreds of Amazon Simple Storage Service (Amazon S3) buckets in a multi-account AWS environment, their storage costs rose steadily with no clear visibility […]

Enterprises face an evolving spectrum of threats to their backup data. From ransomware attacks that encrypt production systems and target recovery points, to accidental deletions by well-meaning administrators, to full account compromises that put every resource at risk, organizations can no longer assume that traditional backup approaches will keep their data recoverable. When backup infrastructure […]

Machine learning (ML) models used for inference on Kubernetes are often several gigabytes in size. When these models are embedded in container images, images become oversized and pod scheduling slows. More critically, inference pods are inherently stateful. Model weights, tokenizer files, compiled GPU kernels, and runtime caches must persist across pod restarts, node failures, and […]

A data warehouse is the analytical backbone of a modern enterprise, consolidating data from disparate sources into a single, authoritative view that enables complex queries, trend analysis, and confident decision-making. In financial services, this means sharper regulatory reporting, faster fraud detection, and deeper customer understanding. The operational reality is demanding. Enterprises juggle multiple source databases […]