CoMix Wave Films Leverages AWS Rendering Management Tools and Storage, Creates Hit Film Suzume with a 300-node Cloud Render Farm

CoMix Wave Films (hereinafter, "CWF") covers everything from writer management, production of animated films, theatrical distribution, package sales, and overseas selling. For the studio’s latest work, Suzume, the Systems Administration department reflected on the previous film, Weathering with You, aiming to support creativity by developing, operating, and maintaining systems while controlling the IT environment. Its two main challenges were rendering and file storage.

For Weathering with You, the studio used different rendering management tools for computer graphics (CG) and compositing. This complicated troubleshooting as errors needed to be checked for each tool. The multiple tools also swallowed more rendering resources, with the number of rendering servers increasing from 14 to around 40 over the project. “Adding hardware created more problems, such as kitting; sorting and grouping to specifications; power requirements and heat generation; and securing storage,” says Masahide Miyakogawa in Systems Administration at CWS. “We struggled to cope.”

CWF was also keen to avoid the storage problems it encountered during production of Weathering with You, where it started with a single on-premises shared storage device, but as production progressed, various problems arose and more storage devices were added each time.

“Storage sprawl made data management decentralized and fragmented,” says Miyakogawa. “We also had a lot of other issues, such as more user connections than expected and problems with transfer speeds.”

For Suzume, CWF solved its rendering issues by unifying rendering tools with AWS Thinkbox Deadline and building a cloud rendering farm. AWS Thinkbox Deadline leverages the cloud and various other resources for large-scale rendering processing while offering compatibility with popular content production software.

“We wanted to use AWS Thinkbox Deadline from the beginning, so there was no point in examining other clouds,” explains Miyakogawa. “The attentive support of the AWS media and entertainment team across storage, networks, and several other aspects was very reassuring when developing the cloud rendering farm.”

CWF built its cloud rendering farm with Amazon Elastic Compute Cloud (Amazon EC2) Spot Instances for better scaling and cost optimizations. A major goal was to enable users to submit work to on-premises and cloud servers without a noticeable difference. After unifying rendering tools across the company with AWS Thinkbox Deadline, CWF started with an on-premises rendering server in September 2021 before spinning up a cloud server for the CG department a year later. AWS Direct Connect linked the on-premises and cloud servers, and users employed on-premises or cloud rendering servers via AWS Thinkbox Deadline.

CWF also unified its main storage. Despite the studio having solved numerous other challenges, a decline in speed due to IOPS (I/O per second) impeded production, spurring CWF to adopt Amazon FSx for NetApp ONTAP in July 2022.

“Our initial infrastructure capped out at 200 IOPS, which wasn’t enough for a production with innumerable small files,” says Miyakogawa. “Switching to Amazon FSx for NetApp ONTAP accelerated IOPS to 21,600, and we were able to clear the IOPS hurdle within budget.”

Although CWF’s local infrastructure communicated with AWS over a 10 Gbps connection with AWS Direct Connect, geographical distance caused a delay when sending and receiving a large number of small files. However, the company overcame the latency challenge by adding FlexCache caching to the local environment.