How CineSend manages their media content using S3 Intelligent-Tiering

Is your organization managing terabytes (or even petabytes) of data stored as objects across hundreds if not thousands of buckets on Amazon S3? What are the chances that the access patterns and application requirements for all of these objects are the same? For most companies out there – slim to none.

We operate in a fast-changing business landscape where data producers and consumers are constantly evolving, as is the data itself. In fact, we hear from our customers that the speed and agility the cloud provides is a competitive advantage. At the same time, customers want to ensure they control their storage costs to support future growth.

In this post, we explore how CineSend, a leading provider of secure video solutions, leveraged S3 Intelligent-Tiering to drive operational efficiency and achieve storage cost optimizations even as their data growth accelerated at an unprecedented pace.

About CineSend

CineSend is a leading provider of secure video storage, encoding, and delivery solutions. CineSend serves a diverse range of customers including flagship airlines, Hollywood studios, global financial institutions, top colleges, and other trail-blazing enterprises. To deliver their solutions, CineSend uses various AWS services, including Amazon S3 as its primary storage provider, AWS Elemental on AWS for media processing, and Amazon CloudFront for delivering content in a fast and reliable manner.

CineSend’s customers have access to cutting-edge (proprietary) video security technologies in their media workflows including, Digital Rights Management (DRM), visible and forensic watermarking, geoblocking (down to a specific ZIP code, if needed), viewing limits, and anti-link-sharing protections.

Cinesend’s storage requirements

Today, CineSend stores media assets in Amazon S3 and later processes the media to add security features and prepare the videos for streaming playback. This media preparation process demands fast access to uploaded content for a short period of time, after which the uploaded files may not be accessed for some time, depending on how the customer interacts with the content in their account. This workflow means CineSend is often storing content in Amazon S3 that needs to be readily accessible even if it becomes infrequently accessed thereafter, depending on customer-specific access patterns that are impossible to predict.

There are thousands of customers that all interact with their media differently, and CineSend was looking for a way to quickly optimize their storage costs without the operational overhead of having to build case-by-case S3 Lifecycle policies or having to run detailed storage analyses.

CineSend’s biggest data-driving customers are typically in the media and entertainment space. When film and TV content is freshly out of production and moves into its distribution phase – when it’s being delivered to film festivals, movie theaters, streaming platforms, or broadcasters, customers expect their media to be readily available. However, as the content ages, access patterns change and data retrieval becomes less frequent.

S3 Intelligent-Tiering

S3 Intelligent-Tiering turned out to be the perfect product for this use case. Immediately after upload, files are placed into the Frequent Access tier. After an initial period of sharing and distribution that typically follows a file upload, it automatically moves to the Infrequent Access tier, which happens automatically 30 days after any individual file is last-accessed. Now, with the newly launched Archive Instant Access tier, S3 Intelligent-Tiering delivers additional storage cost savings of 68% for any content that is not accessed for 90 consecutive days (see Table 1).

Table 1: S3 Intelligent-Tiering storage by access tier

It’s important to note that S3 Intelligent-Tiering has a per object Monitoring & Automation charge but the larger the objects are, the higher the savings. CineSend stores assets larger than 10 GB on average and the per object Monitoring & Automation charge accounts for less than 1% of the overall storage costs. Using S3 Intelligent-Tiering, CineSend’s storage cost savings averages 33% per month, and with the new Archive Instant Access tier, expect to realize storage cost savings in excess of 40% per month (see Table 2).

Table 2: S3 Intelligent-Tiering Storage Cost Savings compared to S3 Standard *Includes per Object Monitoring & Automation charge

If for any reason the file needs to be shared or downloaded again after it has already been moved into Infrequent or Archive Instant tiers, it will move back into the Frequent Access tier automatically and with no noticeable access latency to the end user. Given the degree of variability involved with customer business-cases, it would be challenging for CineSend to architect an efficient mechanism to move files between access tiers automatically. Fortunately, they can simply toggle on S3 Intelligent-Tiering and move these assets between access tiers automatically.

Optimizing very-rarely accessed data

Looking ahead, CineSend is adding policies that move files into the Deep Archive Access tier after 180 days of inactivity to achieve the highest storage cost savings possible for content that is not being used. Since the Deep Access tier standard retrievals are within twelve hours, it’s necessary for CineSend to be notified when access tiers change. This is now easily accomplished with the newly launched EventBridge notifications. These notifications allow CineSend to indicate to users that files have been moved into the Deep Archive Access tier so that they may plan retrieval times accordingly (see Image 1).

Image 1: S3 Event archive notifications for objects moved to Deep Archive Access

In the CineSend ecosystem, uploaded main video files are typically encoded and encrypted into streaming formats using AWS Elemental MediaConvert for secure delivery to end users. CineSend also stores these streaming files in buckets using S3 Intelligent-Tiering. This means that the larger main files and smaller streaming files can be “decoupled” and individually moved into the optimal storage class, without reducing quality of service for either downloading or streaming.

Key takeaways

CineSend found in Amazon S3 Intelligent-Tiering the answer to one of their largest cost optimization challenges. Using S3 Intelligent-Tiering, CineSend was able to easily store tens of thousands of objects across multiple buckets, without having to worry about writing and managing a plethora of lifecycle policies themselves. They could rely on S3 Intelligent-Tiering to keep track of each object and its individual access patterns, and accordingly move these objects across the requisite storage tiers in a completely automated fashion. As a result, CineSend’s engineering team has been able to focus their efforts on further developing their core product, serving their users and cementing their position as industry leaders.

“Using S3 Intelligent-Tiering allowed us to use a “set-it-and-forget-it” model for stored media content. Confident that frequently and infrequently accessed files are in their correct storage class and that costs are being kept to an efficient minimum, my team is able to focus on our mandate: deliver secure video content across the globe with cutting-edge technology.”

– D’Arcy Rail-Ip, VP Technology, CineSend

For most companies today, the speed and agility at which they can move and pivot their business will play a defining role in their success. This rings even more true for customers in their early growth stages. It’s crucial for young companies to find technology solutions that can scale with their growth, in the most efficient way possible. Amazon S3 Intelligent-Tiering allows AWS customers to do exactly that – grow rapidly in a cost-effective manner. Moreover, if you happen to store lots of data with rapidly changing access patterns – which is most companies today – by using S3 Intelligent-Tiering you will be able to put time back into your hands to focus on the core competencies of your business.

Thanks for reading this blogpost outlining Cinesend’s storage solution using Amazon S3 Intelligent-Tiering. If you have any comments or questions, feel free to leave them in the comments section below.

Some things to keep in mind

• Durability and availability: S3 Intelligent-Tiering is designed for 99.9% availability and 99.999999999% durability.
• Pricing: You pay for monthly storage requests, and data transfers. When using S3 Intelligent-Tiering, you pay for a small monthly per-object fee for monitoring and automation. There is no retrieval fee in S3 Intelligent-Tiering and no fee for moving data between tiers.
  • Objects in the Frequent Access tier are billed at the same rate as S3 Standard.
  • Objects stored in the Infrequent Access tier are billed at the same rate as S3 Standard-Infrequent Access
  • Objects stored in the Archive Access tier are billed at the same rate as S3 Glacier.
  • Objects stored in the Deep Archive access tier are billed at the same rate as S3 Glacier Deep Archive.
• API and CLI access: You can use S3 Intelligent-Tiering through the Amazon S3 CLI and S3 API operations with the INTELLIGENT_TIERING storage class. You can also configure the S3 Intelligent-Tiering archive using PUT, GET, and Delete configuration APIs for a specific bucket.
• Feature support: S3 Intelligent-Tiering supports features like S3 Inventory to report on the access tier of objects, and S3 Replication to replicate data to any AWS Region.is no retrieval fee in S3 Intelligent-Tiering nor is there a fee for moving data between tiers.

Objects in the Frequent Access tier are billed at the same rate as S3 Standard, objects stored in the Infrequent Access tier are billed at the same rate as S3 Standard-Infrequent Access, objects stored in the Archive Instant Access tier are billed at the same rate as S3 Glacier Instant Retrieval, objects stored in the Archive Access tier are billed at the same rate as S3 Glacier and objects stored in the Deep Archive access tier are billed at the same rate as S3 Glacier Deep Archive.