Category: Migration & Transfer Services

Genomics data is expanding at a rate exceeding Moore’s law according to the National Human Genome Research Institute. As more sequencing data is produced and researchers move from genotyping to whole genome sequencing, the amount of data produced is outpacing on-premises capacity. Organizations need cloud solutions that help manage data movement, storage, and analysis. The […]

Data is one of the most valuable assets that a business owns. Protecting that data from disasters, such as system or infrastructure failures, is an important part of business continuity planning. Companies have diverse IT infrastructures in which data can be presented from multiple sources. This includes on-premises network-attached storage devices via different file-sharing protocols, […]

You want to leverage cloud scalability, increase cost efficiency by paying only for utilized storage, decouple big data storage from processing, and increase capabilities for data analytics and machine learning using AWS. But how do you move your Hadoop cluster? To accelerate this transition, AWS DataSync recently launched support for moving data between Hadoop Distributed […]

UPDATE (1/19/2023): Some readers who followed the steps in this blog post to deploy an AWS DataSync agent on the KVM platform ran into issues, either because the hypervisor host does not support virtualization or it is not enabled on the platform. Therefore, I have added the steps to verify whether the hypervisor host supports […]

What do we do when our applications fail, and we must provide instance-level log data to external entities for troubleshooting purposes? It’s best to limit direct human interaction with our production resources, so we often see temporary access provided for a fixed period. For highly regulated industries, the approval process for production access can be […]

UPDATE (2/10/2022): Amazon S3 Batch Replication, which is not covered in this blog post, launched on 2/8/2022, allowing you to replicate existing S3 objects and synchronize your S3 buckets. See the S3 User Guide for additional details. UPDATE (5/1/2023): Updated the comparison table to reflect the latest capabilities of the mechanisms covered in the table. […]

It has become customary for us to share with our customers the new AWS Storage Gateway enhancements every year. As we did in 2020 and 2019, through this blog I’m reviewing all the new Storage Gateway launches from 2021. I’ll cover how the new enhancements make it easier for customers to access AWS Storage from […]

Customers who use the AWS Transfer Family service are typically exchanging files with their business partners who provide them with SSH public keys. In a large-scale deployment of the AWS Transfer Family service, public key management eventually becomes a time-consuming task to refresh expired keys and rotate keys for security. When using custom identity providers (custom IdP), […]

CloudEndure Disaster Recovery is a block level replication tool that uses Amazon EC2, Amazon EBS, and EBS Snapshot resources to create a safe and secure disaster recovery solution for customers. The CloudEndure Disaster Recovery console is purpose built to focus on disaster recovery, but does not provide visibility into underlying infrastructure costs. This means that […]

Update (March 4, 2022): Updated Amazon CloudWatch events section to utilize Amazon EventBridge rules instead, allowing you to further customize your serverless event architecture. AWS is also deprecating the requests module in botocore to improve flexibility and performance, and added the ability to use Layers within AWS Lambda to continue using this module. Code has […]