Use SMB storage with Windows containers on AWS Fargate

Introduction

Customers can run their Windows container-based applications on Amazon Elastic Container Service (Amazon ECS) using AWS Fargate to avoid the undifferentiated heavy lifting of managing servers. A common use case for Windows Containers on AWS is to run web applications hosted using Internet Information Services (IIS). When using common files in a web farm, virtual directories are used to map external storage to the web site, so that files aren’t stored on individual web server nodes. In this post, we discuss how customers can use external Server Message Block (SMB) storage, such as Amazon FSx for Windows File Server, in their Windows applications on AWS Fargate.

Distributed Windows applications frequently have a need to read or write to a centralized Windows file share. A common solution is to mount a file-share to the Windows container host and then expose that volume to running containers using SMB Global Mappings. The volume is then visible to the container as a mount point within the container. This solution requires control over the underlying container host instances, including joining the instances to a Microsoft Active Directory (AD) domain and configuring the instances to include the appropriate permissions to access the remote file system. If a customer doesn’t want to manage the underlying container instance, or if they want to use serverless Windows containers with AWS Fargate, then this method won’t work.

This post addresses this gap and unlocks SMB storage on AWS Fargate for Windows Containers. We demonstrate a technique for accessing Windows file shares directly from Windows Containers running on AWS Fargate, without any dependence on the underlying container host. This removes the need for the customer to manage the underlying container host, while maintaining application performance, flexibility, and security.

Solution overview

Figure 1: Highly available Windows containers running on AWS Fargate with persistent storage

This solution introduces a method to map an Amazon FSx for Windows file server (FSxW) share directly as a drive on each provisioned container. When the container starts, a PowerShell script using the cmdlet New-SMBGlobalMapping is used to map the Amazon FSxW file server share so that it is available to applications hosted in the container. We move the drive mapping into the container session versus utilizing the container host to proxy the SMB traffic.

Given that each environment is different, you need to consider how this works within your organizations network, Active Ditectory (AD), and Domain Name System (DNS) structure.

Options to deploy the solution

There are two ways this post can be useful for you. The first is by deploying the demonstration site that I use to show how the solution works. This is a simple HTML site that uses a Virtual Directory in IIS to display an image hosted on our SMB storage. To provide a comprehensive learning experience, we use the Windows Containers on AWS workshop to walk through this solution step-by-step.

The second way this post is helpful, is by using this information to retrofit the solution into your existing environment. This approach is meant for teams already using Windows containers and are familiar with the foundations of operating Amazon Elastic Container Service (Amazon ECS), AD, security groups, and file shares. Given that every organization is different, this approach will require you to take the core components of the solution and map them to your container build process (see Figure 2). If you are deploying this into your own account, then the minimum environment prerequisites are (see Figure 1):

A foundational environment deployed. At a minimum, this includes a VPC with private and public subnets, an internet gateway, and NAT gateways.
Provision AWS Managed AD domain controllers into the private subnets or provide network access to existing AD Domain Controllers.
Provision your file share—we use Amazon FSx for Windows File server in this post—into your private subnets utilizing a custom security group to allow access to AD and clients to connect to Amazon FSx.
Create an Amazon ECS cluster to run your AWS Fargate tasks in.

Figure 2: Windows containers running on AWS Fargate with persistent storage workflow

Create a Service Account in AD to be used to access the share.
Create entries in AWS Systems Manager parameter store (AWS Secrets Manager is an alternative option) for the username, password, and file share you want your containers to connect to. The file share will be the FQDN and top-level directory in the folder in the share you want to access.
Copy your web content to your file share.
Create an Amazon ECS task execution AWS Identity and Access Management (AWS IAM) role that allows the container to access AWS Systems Manager parameter store and has the AmazonECSTaskExecutionRolePolicy and AmazonSSMReadOnlyAccess.
Configure your VPC DNS to integrate with your AD DNS.
Configure NT File System (NTFS) permissions for your service account on your SMB share.
Create the PowerShell script to be included inside your Windows container image build (e.g., inside the same folder as your application binaries).
Create a Dockerfile that includes your container startup script that pulls in the secrets for your AD user account, map the drive with the user, and remove the environment variables for the user account.
Build your Docker Container image that uses your startup script.
Create an Amazon ECR repository and push your image to Amazon ECR for deployment.
Create a new Amazon ECS task definition that includes the AWS Fargate launch type, which injects the AWS Systems Manager parameters, such as environment variables, and your latest container image.
Create an Application Load Balancer for redundancy and allows external traffic to your private AWS Fargate tasks.
Deploy your container image to Amazon ECS with the appropriate network configuration that allows access to your workload traffic (i.e., external load balancer [ELB]), AD domain controllers, and your file share. This would include the appropriate Security groups and subnets with configured network connectivity to those resources.
Test that your application has access to the network share you specified in your startup script.

To allow the container access to the Amazon FSx file share, credentials are injected into the container on startup as environment variables. A PowerShell script invoked during startup uses these credentials to authenticate to the domain and map the file share. Parameters like file share, username, and password are stored in AWS Systems Manager Parameter Store.

The container has the ability to interact with your network and resources just as an Amazon Elastic Compute Cloud (Amazon EC2) instance does. This is made possible because AWS Fargate uses the network mode of awsvpc, which places an Elastic Network Interface (ENI) within the VPC and subnet you specify in Amazon ECS task definition. When the container starts, it receives a DHCP address from the subnet you specified allowing it to operate as a standalone resource. The container receives the DNS options you provided for that subnet. A common practice is to use an Amazon Route53 resolvers to help forward DNS queries for your AD domain to the closest domain controllers. This means that the AD infrastructure, as well as the Amazon FSx file share, don’t need to be hosted in the same subnet and VPC as your AWS Fargate tasks.

For simplicity, in our walkthrough we have our AD domain controllers, Amazon FSx for Windows File shares, and AWS Fargate containers running in the same private subnets. As long as the AWS Fargate container has IP connectivity to the Amazon FSx file share and your AD domain and your security groups are configured to allow traffic, the container maps the drive—even if the share is across accounts or routed through an AWS Transit Gateway. This design allows you to bring any SMB storage (e.g., Amazon FSx for Netapp ONTAP or Storage gateway) to your Windows Containers hosted on AWS Fargate.

When the container startup is complete, and the drive has been mapped, the credentials and environment variables are removed from the environment variables.

Walkthrough

In this section, we distill down the basic components that are a startup script and a DockerFile. Step-by-step instructions are provided in a workshop format.

1. PowerShell startup script for our container

A script called mapdrive.ps1 gets copied into the container build folder and run as a startup script.

[LANGUAGE: POWERSHELL]
# specify the log file path (change to your desired path)
$logFilePath = "C:\Users\Administrator\Desktop\map.txt"
New-Item -ItemType File -Path $logFilePath

# start transcript logging
Start-Transcript -Path $logFilePath -Append

# log the current date and time
Write-Output "Script started: $(Get-Date)"

# set error action preference
$ErrorActionPreference = "Stop"

# set variables
Try {
$username = $env:username
$fsxpassword = $env:password
$fsx = $env:fileshare
} Catch [System.Exception] {
    Write-Output "Failed to set variables $_"
    Exit 1
}

# set credential
Try {
$password = $fsxpassword | ConvertTo-SecureString -asPlainText -Force
$credential = New-Object System.Management.Automation.PSCredential($username, $password)
} Catch [System.Exception] {
    Write-Output "Failed to set credentials $_"
    Exit 1
}

# map drive
Try {
New-SmbGlobalMapping -RemotePath $fsx -Credential $credential -LocalPath N: -Persistent $true -RequirePrivacy $true
} Catch [System.Exception] {
    Write-Output "Failed to set map SMB storage $_"
    Exit 1
}

# add virtual directory to iis
Try {
c:\windows\system32\inetsrv\appcmd.exe add vdir /app.name:'Default Web Site/' /path:/images/ /physicalPath:N:\
} Catch [System.Exception] {
    Write-Output "Failed to configure IIS virtual directory $_"
    Exit 1
}

# configure demo site
Try {
Invoke-WebRequest -UseBasicParsing -Uri 'https://dotnetbinaries.blob.core.windows.net/servicemonitor/2.0.1.6/ServiceMonitor.exe' -OutFile 'C:\\ServiceMonitor.exe'; echo '<div class="SMB for Fargate"><h1>Logo hosted on persistent storage, IIS site running in a Windows container on Fargate</h1><img src="/images/logo.png" alt="logo" /></div>' > C:\\inetpub\\wwwroot\\iisstart.htm; C:\\ServiceMonitor.exe 'w3svc'
} Catch [System.Exception] {
    Write-Output "Failed to configure IIS site $_"
    Exit 1
}

# reset env variables
Try {
Remove-Item -Path "Env:$username"
Remove-Item -Path "Env:$password"
Remove-Item -Path "Env:$fileshare"
[System.Environment]::SetEnvironmentVariable('USERNAME', $null, [System.EnvironmentVariableTarget]::User)
[System.Environment]::SetEnvironmentVariable('password', $null, [System.EnvironmentVariableTarget]::User)
[System.Environment]::SetEnvironmentVariable('fileshare', $null, [System.EnvironmentVariableTarget]::User)
[System.Environment]::SetEnvironmentVariable('USERNAME', $null, [System.EnvironmentVariableTarget]::Machine)
[System.Environment]::SetEnvironmentVariable('password', $null, [System.EnvironmentVariableTarget]::Machine)
[System.Environment]::SetEnvironmentVariable('fileshare', $null, [System.EnvironmentVariableTarget]::Machine)
} Catch [System.Exception] {
    Write-Output "Failed to reset environment variables $_"
    Exit 1
}

# log the script completion date and time
Write-Output "Script completed: $(Get-Date)"

# stop transcript logging
Stop-Transcript

# run service monitor
C:\ServiceMonitor.exe w3svc

All necessary startup configuration steps are moved into the PowerShell script that’s invoked at container startup.

2. Build a container image that uses the startup script

For our demonstration application we use the Windows Server 2019 server core IIS base image, and this solution can also be used in Windows Server 2022. The demonstration creates a basic HTML site that displays a logo that’s hosted on our SMB drive. We utilize IIS Virtual Directories, which allow you to map a directory name to a physical local or remote path connected to the container.

FROM mcr.microsoft.com/windows/servercore/iis:windowsservercore-ltsc2019

SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop'; $ProgressPreference = 'Continue'; $verbosePreference='Continue';"]

# map FSxW drive startup script
WORKDIR /
COPY mapdrive.ps1 /mapdrive.ps1

ENTRYPOINT ["C:\\windows\\system32\\WindowsPowerShell\\v1.0\\powershell.exe", "C:\\mapdrive.ps1"]

Now, build your container image that contains your startup script.

Docker build -t fargate-smb .

3. Step-by-step using the Windows Containers on AWS workshop

The Windows containers on AWS workshop is an educational resource for learning Windows Containers on AWS. It allows teams to learn many core operational tasks required when running Windows Containers on Amazon ECS and Amazon Elastic Kubernetes Service (Amazon EKS).

Lab 8 contains all of the steps necessary to provision this solution step-by-step. This allows you to map the solution to an existing environment, automate the steps using your tools of choice, and understand the mechanics of how each component works together.

Troubleshooting

There are multiple components to this solution that could cause the share to not to map to the container. To help with any troubleshooting, you can enable Amazon ECS Exec for AWS Fargate. The instructions to run tasks that can be interactively administered can be found in this post.

Common problems are:

Network path between the container ENI, the SMB share, and the AD domain controllers is blocked. A great tool to use is the AWS VPC Reachability Analyzer.
Misconfigured Systems Manager Parameter Store parameters: Incorrect password, wrong username, etc.
Permissions on the SMB share: You align the AD service account that you’re using to map the drive to have the permissions you want the container based application to have within the container session.
Reset your AD user password.
Run your container with the Amazon EC2 Launch type so you can establish a PowerShell session to the fouled container.
Utilize Amazon ECS Exec for AWS Fargate to establish a PowerShell session with your Windows Containers.
If you are using these step-by-step instructions, be aware of having multiple task definitions, which causes the ecs-service creation to fail.

Conclusion

In this post, we showed you a solution that can be used to provide high performance persistent SMB storage to your Windows Containers running in AWS Fargate on Amazon ECS. This solution allows any SMB storage to be utilized depending on your workload.

Containers