AWS Storage Blog

Category: Compute

Amazon S3 Batch Operations featured image

Streamline data management at scale by automating the creation of Amazon S3 Batch Operations jobs

Over time, Enterprises may need to undertake operations or make modifications to their data as part of general data management, to address changing business needs, or to comply with evolving data-management regulations and best practices. As datasets being generated, stored, and analyzed continue to grow exponentially, the need for simplified, scalable, and reproduceable data management […]

Marqeta supercharges MySQL workloads using Amazon EBS io2 Block Express

Given the ubiquity of digital payments, cutting-edge fintech solutions hinge on seamless and highly available real-time transaction processing. Invariably, this needs the support of a performant, reliable, and secure datastore. And after considering technical requirements, fintech companies know that regulatory and compliance auditing never takes a back seat. Enterprise AWS customer Marqeta needed all of […]

Automating application-consistent Amazon EBS Snapshots for SAP HANA databases

Customers host their SAP HANA environments on AWS to run their business-critical processes, such as financial planning, data analytics, and supply chain management. Reliably backing up data in SAP HANA is paramount for users to restore the database in the event of disruptions and urgent business needs. To reliably back up SAP HANA, SAP documentation […]

Amazon S3 featured image 2023

How Visual Layer builds high quality datasets on Amazon S3

Companies from different industries use data to help their Artificial Intelligence (AI) and Machine Learning (ML) systems make intelligent decisions. For ML systems to work well, it is crucial to make sure that the massive datasets used for training ML models are of the highest quality, minimizing noise that can contribute to less-than-optimal performance. Processing […]

AWS DataSync Featured Image 2020

Automate data synchronization between AWS Outposts racks and Amazon S3 with AWS DataSync

Many organizations generate large quantities of data locally, including digital imagery, sensor data, and more. Customers require local compute and storage to ingest and enable real-time predications based on their data, and often preprocess this data locally before transferring to the cloud to unlock additional business value such analysis, reporting, and archiving. Automating transfers to […]

Amazon S3 Batch Operations featured image

Automate object processing in Amazon S3 directory buckets with S3 Batch Operations and AWS Lambda

Data, the lifeblood of any modern organization, is rarely static. For high-performance applications and workloads, enterprises need the ability to run operations on massive amounts of data, including modifying the data as is necessary for each use case, to further accelerate processing. This could include modifying uploaded images with a watermark, changing the bitrate of […]

Amazon S3 featured image 2023

Managing duplicate objects in Amazon S3

When managing a large volume of data in a storage system, it is common for data duplication to happen. Data duplication in data management refers to the presence of multiple copies of the same data within your system, leading to additional storage usage as well as extra overhead when handling multiple copies of the same […]

AWS Backup 2021 blog image

Automate the delivery of AWS Backup Audit Manager reports via email

Business continuity and disaster recovery plans include having a backup strategy for application workloads, whether on-premises or in Cloud. Furthermore, organizations need efficient methods to actively monitor their data protection posture and detect any failure for remediation while meeting the required recovery time objective (RTO) and recovery point objective (RPO) for the business. One efficient […]

Figure 1: Multi-region FSx for NetApp ONTAP with SnapMirror replication for SQL Server DR

Automating retrievals from the Amazon S3 Glacier storage classes

Faced with increasing amounts of data and a tightening economic climate, enterprises are looking to save money on their storage costs by moving rarely needed data to archival storage options. The least costly options require your internal systems to support receiving data back in hours or days, often called asynchronous retrievals. With this time delay, […]

Amazon S3 Object Lambda

Automatically modify data you are querying with Amazon Athena using Amazon S3 Object Lambda

Enterprises may want to customize their data sets for different requesting applications. For example, if you run an e-commerce website, you may want to mask Personally Identifiable Information (PII) when querying your data for analytics. Although you can create and store multiple customized copies of your data, that can increase your storage cost. You can […]