AWS Storage Gateway increases cache 4x and enhances bandwidth throttling

Customers’ hybrid-cloud architectures struggle to enable low latency access to hot data for growing datasets without sufficient local caching capabilities. Customers without sufficient local caching are also more vulnerable to network issues between their data centers and the cloud. Moreover, in a hybrid architecture, the decentralization of data across locations, gateways, and instances can also negatively affect operational latency. In addition, customers can struggle with strained network usage between their data centers and the cloud if they are unable to execute efficient bandwidth management.

Customers use AWS Storage Gateway’s Tape and Volume Gateway as hybrid cloud storage mechanisms within their overall data infrastructure. Using these services enables them to achieve greater scalability and agility, all while lowering costs and simplifying management as their business needs grow. You can use AWS Storage Gateway to back up and archive data to AWS, provide on-premises file shares backed by cloud storage, and provide on-premises applications low latency access to in-cloud data.

Today, AWS Storage Gateway released two new enhancements. The first is 64-TB local cache support. The second is schedule-based network bandwidth throttling for Tape Gateway and Volume Gateway.

In this blog, we cover these new capabilities and how they can further assist you and your business to overcome some of the potential roadblocks mentioned as hindrances to hybrid cloud architectures.

AWS Storage Gateway increases local cache support 64-TB cache size (4x increase) for Tape and Volume Gateway

The local cache of AWS Storage Gateway already helps you enable low latency access to your hot data, where all your data is stored in the AWS Cloud. This cache size increase enables you to manage larger datasets in the local cache, closer to the on-premises applications to enable additional benefits such as:

• Easily scale the gateway’s local cache to manage more application data closer to on-premises applications and users.
• Low latency access to more hot data stored and served from the local cache.
• Helps buffer data for scenarios where you have limited network bandwidth or when there are challenges with your network links.
• Simplify hybrid cloud storage and decrease costs to centralize data assets and consolidate storage on a lesser number of gateways, with each supporting up to 64 TB of local cache.

You can easily allocate larger underlying local cache disks to AWS Storage Gateway at setup time.

We are using 64 TB for simplicity. The actual value you must specify is in tebibytes (TiB).

Note, this increased cache size capability is available only for gateways running on virtual machines (VM).

Schedule-based network bandwidth throttling for Tape and Volume Gateway

AWS Storage Gateway optimizes data transfer to AWS by sending only changed data and compressing data; this helps customers optimize their network utilization to AWS. This new capability of schedule-based network bandwidth throttling provides you with a new bandwidth management feature, which helps to further optimize your network usage between your data center and AWS.

This feature allows you to minimize network congestion during peak network traffic times. By using all the available bandwidth during off-peak hours, you can efficiently use the network connection between your on-premises gateway and AWS. Before this enhancement, Tape and Volume Gateways supported network bandwidth throttling, but lacked the time dimension. As you deploy these gateways in your data centers, remote sites, and branch offices, where each location can have varying types of network connectivity and available bandwidth, you need flexibility in how you can use your network bandwidth during peak business hours.

You can easily configure the schedule-based network bandwidth throttling based on your desired day, start time, end time, download rate, and upload rate values.

Benefits for Tape Gateway customers

The increased local cache support to 64 TB allows you to:

• Cache more operational recovery data, such as your daily backup data via the gateway’s local cache, enabling faster access and restores of that hot data.
• Reduce costs, where a less number of Tape Gateways are required to manage 64 TB of local cache.

The schedule-based network bandwidth throttling allows you to:

• Throttle the bandwidth to a lower value during peak times (that is, Monday to Friday during business hours) to minimize any network congestion with other workloads that use a network link to AWS.
• During off-peak times, use all the available bandwidth to allow faster backup data uploads to AWS (that is, weekly or monthly full backups that could occur over the weekend).

Benefits for Volume Gateway customers

Volume Gateway presents on-premises storage in either stored mode or cached mode. Previously, you could leverage a maximum of 16 TB for cached mode. If you wanted to store more data locally on premises in cache for low latency frequently accessed data, you were required to deploy multiple Volume Gateways. With the announcement of increased cache to 64 TB, you can store more data on premises. This can help you reduce the number of Volume Gateways you create, and you can also store more on-premises cached data for low latency access. An added benefit is reduced load on the network as the increased cache can act as a buffer for slower or reduced bandwidth networks. This helps reduce how often data may need to be requested from the cloud due to not being stored in the cache.

Conclusion

In this blog post, we highlighted two key AWS Storage Gateway enhancements that provide you with even more functionality.

The increase to 64-TB local cache support for Volume and Tape Gateway brings it inline with the existing capability provided with File Gateway. This feature enables you to seamlessly cache more of your data locally, simplify hybrid cloud storage, and decrease costs, all while enabling low latency access to more of your frequently accessed data. The schedule-based network bandwidth throttling capability provides you with greater control over the behavior of data synchronization activities to allow optimized network usage between your data center and AWS, during peak and off-peak periods.

Thanks for reading this blog post! We encourage you to start leveraging these new capabilities today. Also, remember to join us on November 10, 2020, for the AWS Storage Day virtual event, to learn what is new across the AWS Storage portfolio. If you have any comments or questions, please don’t hesitate to leave them in the comments section.