Containers

Sharing Amazon ECR repositories with multiple accounts using AWS Organizations

Customers are adopting multi-account deployments in AWS given the improved security and separation of duties it provides. Some AWS services, like Amazon Elastic Container Registry (ECR), support scalability when a single instance is shared between accounts to reduce management overhead and increase visibility. AWS accounts have become an elastic resource, where the account number and other account-specific metadata is generated so frequently, a hands-off, automated approach must be adopted. By making a few updates, you can now leverage AWS Organizations for fine-grained access control to your ECR repository across a multi-account environment.

AWS Organizations has enabled customers to create logically separated yet related groupings of their AWS accounts. Each organizational unit (OU) within a hierarchy could contain hundreds or even thousands of accounts. Because of how much customization power Organizations provides, customers sharing resources between accounts becomes more precise and easier to scale-out as their needs increase. ECR images are an example of an artifact, which can be hosted in a centralized architecture for easy manageability, but shared between many accounts to create standards and improve efficiency. When you combine AWS Organizations OU’s and the ECR repository policy, a solution to meet these needs can be realized.

Although an organizational unit can’t be used as the “principal” of an IAM policy, it can be used as a condition using AWS Global Condition Context Keys. Because this feature of IAM has many different capabilities, we can focus on a piece of metadata that cross-account requests include when the caller’s origin is an account within an OU. The “aws:PrincipalOrgPaths” condition can be included in access policies where the resource the request is destined for is in another account, likely within a different OU altogether. Taking a step back, imagine a shared services account that provides key enterprise-wide components of a CI/CD pipeline, that is separate from the development and production accounts and thus easier to secure. Because version control systems like AWS CodeCommit have adopted robust ways to integrate with build systems like AWS CodeBuild, a developer only needs to make a commit to the repository or create a pull request between branches to trigger a pipeline, which will build and deploy their app’s artifact using these resources that are orchestrated cross-account. AWS commitment to solving customer problems has made architectures that benefit multi-account solutions just as easy to implement as services in a single AWS account. Let’s walk through how these features have converged to enable this.

Solution overview

In this blog, we walk through an example of performing a blue/green deployment from a multi-account ECR to an Amazon Elastic Container Service (ECS) cluster. These features can also benefit customers using Elastic Kubernetes Service (EKS). We will also be using AWS CodeDeploy and AWS CloudFormation to orchestrate the infrastructure necessary for supporting an application, in this case a simple “hello world” app. The following architecture diagram shows the AWS resources across the example shared services and development organizational units, which we’ll cover further in this blog.

In this tutorial, you create two container images, one for the blue version of the “hello world” application, and one for the green version.  These images are pushed to a repository in Amazon ECR, which exists in your shared services account belonging to your shared services organizational unit. The ECR repository policy is configured using the condition key “aws:PrincipalOrgPaths” to provide the read access to all the development accounts belonging to the development organizational unit. When a new development account is created and added into the development OU, access to the ECR repository is automatically granted. When the new account tries to access the ECR repository, AWS verifies that the account’s OU of the originating request matches the one specified in the policy. In the development account, you use a CloudFormation template to deploy the blue version of the “hello world” application to an ECS Service running on AWS Fargate. Then you update the CloudFormation stack to deploy the green version of the same application.

Solution tutorial

Prerequisites

To follow along with this solution, you need to complete the following prerequisites:

  • An AWS Organization environment with an OU for shared services and development. Newly created accounts will be assigned as necessary, in this example 1 in each OU.
  • A local or cloud-based development environment with Docker and AWS CLI installed. For more information, see getting started using the AWS CLI.

 

Step 1: Download the source code to your local environment

The source code to implement this solution can be downloaded to your local development environment from amzon-ecr-orgpath-sample GitHub repo or cloned using the command below.

git clone https://github.com/aws-samples/amazon-ecr-orgpath-sample.git

The following files are included in the source code.

  • cft-service-ecr-repo.yaml – CloudFormation template to deploy all the required resources in the service account
  • cft-dev-codeploybluegreen-create.yaml – CloudFormation template to deploy the first version (blue) of “hello world” on Amazon ECS in the development account
  • cft-dev-codeploybluegreen-update.yaml – CloudFormation template to deploy the second version (green) of “hello world” on Amazon ECS in the development account
  • Dockerfile.blue – A sample Dockerfile to build a simple hello world web app, version 1 (blue version)
  • Dockerfile.green – A sample Dockerfile to build a simple hello world web app, version 2 (green version)

 

Step 2: Create an AWS CloudFormation stack in the service account to provision an Amazon ECR repository

At the AWS CloudFormation page in the AWS Management Console of your service account, choose “Create stack.” In the Select Template section, under Specify Template, select “Upload a template file.” Select “Choose File” to upload the cft-service-ecr-repo.yaml file, and then choose “Next.” For more information, see Step 3: Create the Stack.

On the Specify Details screen, provide values for the following fields:

  • Stack name: Enter a name for the stack.
  • DevOrgUnitID: Enter your AWS development Organizational Unit ID.
  • OrgID: Enter your AWS Organizational Unit ID.

On the Create page, acknowledge the creation of IAM resources if asked and then choose “Create.” To monitor the stack creation progress, check the Events tab of the CloudFormation stack. After the CloudFormation stack is complete, you can verify that the resources have been created by browsing the Resources tab of that stack and navigating to them with the management console.

The stack provisions an Amazon ECR repository demoecshwapp-ecr-repo in your service account. The output of the stack is the ECR repository URI, which is used in the subsequent steps for container image push and pull operations. The URI format is:

{aws-account-id}.dkr.{aws-region}.amazonaws.com/demoecshwapp-ecr-repo

The stack also defines a read-only ECR repository policy as shown below. The organization ID and the development organizational unit ID are passed to the condition “aws:PrincipalOrgPaths”, which allows all the accounts in the development OU to pull the images from the ECR repo in the service account.

 

{

  “Version”: “2012-10-17”,

  “Statement”: [

    {

      “Sid”: “AllowPull”,

      “Effect”: “Allow”,

      “Principal”: “*”,

      “Action”: [

        “ecr:BatchCheckLayerAvailability”,

        “ecr:BatchGetImage”,

        “ecr:DescribeImages”,

        “ecr:DescribeRepositories”,

        “ecr:GetDownloadUrlForLayer”

      ],

      “Condition”: {

        “ForAnyValue:StringLike”: {

          “aws:PrincipalOrgPaths”: “o-xxxxxxxxxx/*/ou-xxxx-xxxxxxxx/*”

        }

      }

    }

  ]

}

Step 3: Create hello world Docker images and push images to the ECR repo in the service account

In your local development environment, go to the directory where Dockerfile.blue and Docketfile.green are located. Set the following environment variables in your shell. Replace {aws-account-id} and {aws-region} with your numeric AWS Account ID and the AWS Region where your registry endpoint is located. Or you can just copy the output value from the stack created in step 2 and paste to the value of variable REPOURI.

REPOURI={aws-account-id}.dkr.{aws-region}.amazonaws.com/demoecshwapp-ecr-repo
AWS_REGION={aws-region}

Log your Docker client into ECR.

aws ecr get-login-password --region ${AWS_REGION} | docker login --username AWS --password-stdin ${REPOURI}

Now build two Docker images for blue and green version of hello world app and tag them with blue and green respectively for the demonstrating purpose. Push both images to Amazon ECR.

docker build -t ${REPOURI}:blue -f Dockerfile.blue .
docker push ${REPOURI}:blue
docker build -t ${REPOURI}:green -f Dockerfile.green .
docker push ${REPOURI}:green

The images pushed to Amazon ECR can be verified using command “aws ecr”.

aws ecr describe-images --repository-name demoecshwapp-ecr-repo --region ${AWS_REGION}

Step 4: Create an AWS CloudFormation stack to deploy hello world blue version on AWS Fargate in the development account

Repeat the stack creation process described in step 2 for the cft-dev-codeploybluegreen-create.yaml file. On the Specify Details page, provide the following values:

  • ImageUri: enter a “hello world” blue version image URI (sample input: {aws-account-id}.dkr.{aws-region}.amazonaws.com/demoecshwapp-ecr-repo:blue).
  • Subnet1: choose a public subnet in your development account.
  • Subnet2: choose a public subnet in your development account.
  • VPC: choose a VPC in your development account.

This stack provisions the resources as shown below. They include the common resource for both blue and green application environments and blue specific ECS task related resources.

  • An Application Load Balancer
  • An Application Load Balancer listener and a listener rule
  • Two Application Load Balancer target groups, one for blue and one for green environment
  • An Amazon ECS cluster
  • An Amazon ECS task definition for blue image
  • An Amazon ECS task set for blue resource
  • An Amazon ECS primary task set for blue resource
  • Two security groups, one for the Application Load Balancer and one for the ECS service
  • An IAM role for ECS task execution

The stack template includes a Transform section that invokes the AWS:CodeDeployBlueGreen transform, and a Hook section that invokes the AWS:CodeDeployBlueGreen hook, which defines the traffic routing settings. For more information, see perform ECS blue/green deployments through CodeDeploy using AWS CloudFormation.

After the completion of the stack creation, click Application Load Balancer DNS name on the outputs tab of the stack. You will see the message “Hello World! (Blue)” displayed on the web page. The application is deployed on AWS Fargate successfully.

 

Step 5: Update the AWS CloudFormation Stack to deploy hello world green version on AWS Fargate in the development account

On the AWS CloudFormation page in the AWS Management Console of your development account, choose the stack created in step 4. Click “Update”, choose “Replace” current template. Select “Choose File” to upload the cft-dev-codeploybluegreen-update.yaml file, and then choose “Next.” On the Specify Details page, keep all the existing values unchanged except for parameter ImageUri. For parameter ImageUri, replace blue with green in the input value so the docker image tagged with green is deployed. The sample input is {aws-account-id}.dkr.{aws-region}.amazonaws.com/demoecshwapp-ecr-repo:green. Then click “Next” and acknowledge the creation of IAM resources on the Update page and then choose “Update Stack.”

The only difference in the CloudFormation template cft-dev-codeploybluegreen-update.yaml, compared with the original creation template cft-dev-codeploybluegreen-create.yaml, is the increase of the memory from 512 to 1024 in the ECS task definition resource. The stack update triggered by this change will enable blue/green deployment. A green environment is created using the container image tagged as green. Based on the green target group’s health check defined in the CloudFormation template, ECS determines whether the new deployment is working successfully. If an error occurs at any stage prior to traffic being routed to the green stack, CloudFormation will roll-back to its state prior to the deployment being initiated. The TrafficRoutingConfig “Type” property, in the hook section of the deployment’s CloudFormation template, allows you to define how traffic is routed to the green stack with the option “AllAtOnce” being used in this example. The blue environment is removed after the time interval specified in the “TerminationWaitTimeInMinutes” property is reached.

After the completion of the stack update, click Application Load Balancer DNS name on the outputs tab of the stack or refresh the web page accessed in step 4. You will see the message “Hello World! (Green)” displayed on the web page. The application is updated with the green version successfully.

 

Cleanup

To clean up the resources created by this blog, delete the stack created in your service account and development account.

 

Conclusion

In this blog, we walked through an example where the AWS Organizations OU and the ECR repository policy are combined using IAM policy condition “aws:PrincipalOrgPaths”.  The blue-green deployment of a simple ECS hello world app in a development account pulls Docker images from the service account in the shared services OU. With the access configured at OU level, all the development accounts belonging to the same OU can pull images in a different account without extra configuration steps. This feature can be included in your CI/CD pipelines to make the cross-account access much simpler, manageable and more secure.