AWS Storage Blog

Category: AWS Lambda

AWS DataSync Featured Image 2020

How London Stock Exchange Group migrated 30 PB of market data using AWS DataSync

London Stock Exchange Group (LSEG) has 30 PB of Tick History-PCAP data, which is ultra-high-quality global market data that is based on raw exchange data, timestamped to the nanosecond. An additional 60 TB is generated every day. LSEG sought to migrate their data from Wasabi cloud storage, LSEG was looking for a new solution to […]

Monitoring AWS Storage Gateway health and performance using Amazon CloudWatch

When managing a hybrid-cloud infrastructure, monitoring system health is essential for maintaining business continuity. Setting up comprehensive monitoring provides visibility into performance and availability of infrastructure components. By establishing alert thresholds and promptly responding to alarms, administrators can identify degraded performance or outages early. Quickly diagnosing and fixing the issues maximizes uptime. AWS Storage Gateway, […]

Amazon S3 featured image 2023

Siemens builds Datalake2Go on AWS to analyze disparate data globally

Siemens is a technology company focused on industry, infrastructure, transport, and healthcare. From resource-efficient factories, resilient supply chains, and smart buildings and grids, to cleaner and more comfortable transportation and advanced healthcare, the company creates technology with purpose, adding real value for its customers. Siemens technology is everywhere, supporting the critical infrastructure and vital industries […]

Amazon S3 featured image 2023

Optimizing storage costs and query performance by compacting small objects

Applications produce log files that should be reliably stored for ad-hoc reporting, compliance, or auditing purposes. Over time, these collections of relatively small log files grow in volume and cost-effective storage and data management becomes crucial. Accessing the data in these files and querying them can also be useful for getting insight from the data. […]

A Pilot Light disaster recovery strategy for WordPress

In today’s digital ecosystem, maintaining an uninterrupted online presence and resilience is essential for businesses. WordPress platforms, whether e-commerce sites or news portals, must not only meet but exceed stringent Service Level Agreements (SLA’s) to maintain user trust, ensure continuity, and protect revenue. These SLAs, defining Recovery Time Objectives (RTO) and Recovery Point Objectives (RPO), […]

AWS Transfer Family Featured Image

Architecting secure and compliant managed file transfers with AWS Transfer Family SFTP connectors and PGP encryption

Users in industries such as financial services and healthcare regularly exchange files with their external business partners containing sensitive and regulated datasets, such as Personal Identifiable Information (PII) and financial records. These file transfers often happen over the Secure File Transfer Protocol (SFTP) and encrypting files using Pretty Good Privacy (PGP) before transfer is often […]

AWS Transfer Family Featured Image

Using Amazon Cognito as an identity provider with AWS Transfer Family and Amazon S3

In highly regulated industries, securely exchanging files business-to-business is a crucial business practice. When building out a Managed File Transfer (MFT) environment, it is common to consider using a third-party identity solution for authenticating users. This approach offers simplicity for businesses that already use an identity service, allowing them to maintain identities for a variety […]

Amazon S3 Batch Operations featured image

Streamline data management at scale by automating the creation of Amazon S3 Batch Operations jobs

Over time, Enterprises may need to undertake operations or make modifications to their data as part of general data management, to address changing business needs, or to comply with evolving data-management regulations and best practices. As datasets being generated, stored, and analyzed continue to grow exponentially, the need for simplified, scalable, and reproduceable data management […]

Amazon S3 Batch Operations featured image

Automate object processing in Amazon S3 directory buckets with S3 Batch Operations and AWS Lambda

Data, the lifeblood of any modern organization, is rarely static. For high-performance applications and workloads, enterprises need the ability to run operations on massive amounts of data, including modifying the data as is necessary for each use case, to further accelerate processing. This could include modifying uploaded images with a watermark, changing the bitrate of […]

Amazon S3 featured image 2023

Managing duplicate objects in Amazon S3

When managing a large volume of data in a storage system, it is common for data duplication to happen. Data duplication in data management refers to the presence of multiple copies of the same data within your system, leading to additional storage usage as well as extra overhead when handling multiple copies of the same […]