Posted On: Mar 22, 2023
AWS Batch now enables you to configure ephemeral storage up to 200 GiB in size on AWS Fargate type jobs. This means that you can expand the available ephemeral storage beyond the default value of 20 GiB for your jobs when using Fargate as the compute platform on AWS Batch. With this launch, you no longer need to limit the size of the Docker images to run machine learning inferences or the size of the data sets in the container for data processing to be under 20GiB. You can configure the ephemeral storage to be as large as 200GiB to run larger sized workloads.
You can define the size of ephemeral volume needed by an AWS Batch job using the ephemeralStorage parameter that gives the option to set the ephemeral storage ranging from 21GiB to up to 200GiB in job definitions. This parameter is available only when using Fargate as the compute platform and can be done using the RegisterJobDefinition API operation.
Now you can configure the size of the ephemeral storage needed for a job using Fargate as the compute platform on AWS Batch. You no longer have the restriction of using the default limit of 20 GiB. This configuration is now available in all AWS Regions where AWS Batch is currently available. To learn more about AWS Batch, see the AWS Batch User Guide. To learn more about the AWS Batch API, see the AWS Batch API Reference.