AWS Deep Learning AMI GPU PyTorch 2.6 (Ubuntu 22.04)

For help getting started, please see the AWS Deep Learning AMI Developer Guide.
 

AMI Name format:

• Deep Learning OSS Nvidia Driver AMI GPU PyTorch 2.6.${PATCH_VERSION} (Ubuntu 22.04) ${YYYY-MM-DD}

Supported EC2 Instances:

• Please refer to Important changes to DLAMI 
• Deep Learning with OSS Nvidia Driver supports G4dn, G5, G6, Gr6, P4, P4de, P5, P5e, P5en.

The AMI includes the following:

• Supported AWS Service: EC2
• Operating System: Ubuntu 22.04
• Compute Architecture: x86
• Python: /opt/pytorch/bin/python
• NVIDIA Driver: 
  • OSS Nvidia driver: 570.86.15
• NVIDIA CUDA12.1 stack:
  • CUDA, NCCL and cuDDN installation path: /usr/local/cuda-12.6/
  • Default CUDA: 12.6
    • PATH /usr/local/cuda points to /usr/local/cuda-12.6/
    • Updated below env vars:
      • LD_LIBRARY_PATH to have /usr/local/cuda/lib:/usr/local/cuda/lib64:/usr/local/cuda:/usr/local/cuda/targets/x86_64-linux/lib
      • PATH to have /usr/local/cuda/bin/:/usr/local/cuda/include/
  • Compiled system NCCL Version present at /usr/local/cuda/: 2.24.3
  • PyTorch Compiled NCCL Version from PyTorch conda environment: 2.21.5
• NCCL Tests Location: 
  • all_reduce, all_gather and reduce_scatter: /usr/local/cuda-xx.x/efa/test-cuda-xx.x/
  • To run NCCL tests, LD_LIBRARY_PATH is already with updated with needed paths.
    • Common PATHs are already added to LD_LIBRARY_PATH: 
      • /opt/amazon/efa/lib:/opt/amazon/openmpi/lib:/opt/aws-ofi-nccl/lib:/usr/local/lib:/usr/lib
    • LD_LIBRARY_PATH is updated with CUDA version paths
      • /usr/local/cuda/lib:/usr/local/cuda/lib64:/usr/local/cuda:/usr/local/cud/targets/x86_64-linux/lib
• EFA Installer: 1.38.0
• Nvidia GDRCopy: 2.4.1
• Nvidia Transformer Engine: v1.11.0
• AWS OFI NCCL: 1.13.2-aws
  • Installation path: /opt/aws-ofi-nccl/ . Path /opt/aws-ofi-nccl/lib is added to LD_LIBRARY_PATH.
  • Note: PyTorch package comes with dynamically linked AWS OFI NCCL plugin as a conda package aws-ofi-nccl-dlc package as well and PyTorch will use that package instead of system AWS OFI NCCL.
• AWS CLI v2 as aws2 and AWS CLI v1 as aws
• EBS volume type: gp3
• Python version: 3.11
• Query AMI-ID with SSM Parameter (example region is us-east-1):
  • OSS Nvidia Driver:
    • aws ssm get-parameter --name /aws/service/deeplearning/ami/x86_64/oss-nvidia-driver-gpu-pytorch-2.6-ubuntu-22.04/latest/ami-id --region us-east-1 --query "Parameter.Value" --output text
• Query AMI-ID with AWSCLI (example region is us-east-1):
  • OSS Nvidia Driver:
    • aws ec2 describe-images --region us-east-1 --owners amazon --filters 'Name=name,Values=Deep Learning OSS Nvidia Driver AMI GPU PyTorch 2.6.? (Ubuntu 22.04) ????????' 'Name=state,Values=available' --query 'reverse(sort_by(Images, &CreationDate))[:1].ImageId' --output text

Notices

PyTorch Deprecation of Anaconda Channel

Starting with PyTorch2.6, Pytorch has deprecated support for Conda (see official announcement ). As a result, Pytorch 2.6 and above will move to using Python Virtual Environments. Tto activate the pytorch venv, please use source /opt/pytorch/bin/activate

P5/P5e Instances:

• DeviceIndex is unique to each NetworkCard, and must be a non-negative integer less than the limit of ENIs per NetworkCard. On P5, the number of ENIs per NetworkCard is 2, meaning that the only valid values for DeviceIndex is 0 or 1. Below is the example of EC2 P5 instance launch command using awscli showing NetworkCardIndex from number 0-31 and DeviceIndex as 0 for first interface and DeviceIndex as 1 for rest 31 interrfaces.

aws ec2 run-instances --region $REGION \
--instance-type $INSTANCETYPE \
--image-id $AMI --key-name $KEYNAME \
--iam-instance-profile "Name=dlami-builder" \
--tag-specifications "ResourceType=instance,Tags=[{Key=Name,Value=$TAG}]" \
--network-interfaces "NetworkCardIndex=0,DeviceIndex=0,Groups=$SG,SubnetId=$SUBNET,InterfaceType=efa" \
 "NetworkCardIndex=1,DeviceIndex=1,Groups=$SG,SubnetId=$SUBNET,InterfaceType=efa" \
 "NetworkCardIndex=2,DeviceIndex=1,Groups=$SG,SubnetId=$SUBNET,InterfaceType=efa" \
 "NetworkCardIndex=3,DeviceIndex=1,Groups=$SG,SubnetId=$SUBNET,InterfaceType=efa" \
 "NetworkCardIndex=4,DeviceIndex=1,Groups=$SG,SubnetId=$SUBNET,InterfaceType=efa" \
 ....
 ....
 ....
 "NetworkCardIndex=31,DeviceIndex=1,Groups=$SG,SubnetId=$SUBNET,InterfaceType=efa"

Kernel:

• Kernel version is pinned using command: 
  • • echo linux-aws hold | sudo dpkg —set-selections
      echo linux-headers-aws hold | sudo dpkg —set-selections
      echo linux-image-aws hold | sudo dpkg —set-selections
• We recommend users to avoid updating their kernel version (unless due to a security patch) to ensure compatibility with installed drivers and package versions. If users still wish to update they can run the following commands to unpin their kernel versions: 
  • • echo linux-aws install | sudo dpkg —set-selections
      echo linux-headers-aws install | sudo dpkg —set-selections
      echo linux-image-aws install | sudo dpkg —set-selections
    • apt-get upgrade -y
• For each new version of DLAMI, latest available compatible kernel is used.

Release Date: 2025-02-21

AMI Names:

• Deep Learning OSS Nvidia Driver AMI GPU PyTorch 2.6.0 (Ubuntu 22.04) 20250220

Added

• Initial release of Deep Learning AMI GPU PyTorch 2.6 (Ubuntu 22.04) series. Including a Python virtual environment pytorch (source /opt/pytorch/bin/activate), complimented with NVIDIA Driver R570, CUDA=12.6, cuDNN=9.7, PyTorch NCCL=2.21.5, and EFA=1.38.0.
  • Starting with PyTorch2.6, Pytorch has deprecated support for Conda (see official announcement ). As a result, Pytorch 2.6 and above will move to using Python Virtual Environments. Tto activate the pytorch venv, please activate using source /opt/pytorch/bin/activate