Building a CI/CD pipeline to update an AWS CloudFormation StackSets

AWS CloudFormation StackSets can extend the functionality of CloudFormation Stacks by enabling you to create, update, or delete one or more stack across multiple accounts. As a developer working in a large enterprise or for a group that supports multiple AWS accounts, you may often find yourself challenged with updating AWS CloudFormation StackSets. If you’re building a CI/CD pipeline to automate the process of updating CloudFormation stacks, you can do so natively. AWS CodePipeline can initiate a workflow that builds and tests a stack, and then pushes it to production. The workflow can either create or manipulate an existing stack; however, working with AWS CloudFormation StackSets is currently not a supported action at the time of this writing.

You can update an existing CloudFormation stack using one of two methods:

• Directly updating the stack – AWS immediately deploys the changes that you submit. You can use this method when you want to quickly deploy your updates.
• Running change sets – You can preview the changes AWS CloudFormation will make to the stack, and decide whether to proceed with the changes.

You have several options when building a CI/CD pipeline to automate creating or updating a stack. You can create or update a stack, delete a stack, create or replace a change set, or run a change set. Creating or updating a CloudFormation StackSet, however, is not a supported action.

The following screenshot shows the existing actions supported by CodePipeline against AWS CloudFormation on the CodePipeline console.

This post explains how to use CodePipeline to update an existing CloudFormation StackSet. For this post, we update the StackSet’s parameters. Parameters enable you to input custom values to your template each time you create or update a stack.

Overview of solution

To implement this solution, we walk you through the following high-level steps:

1. Update a parameter for a StackSet by passing a parameter key and its associated value via an AWS CodeCommit
2. Create an AWS CodeBuild
3. Build a CI/CD pipeline.
4. Run your pipeline and monitor its status.

After completing all the steps in this post, you will have a fully functional CI/CD that updates the CloudFormation StackSet parameters. The pipeline starts automatically after you apply the intended changes into the CodeCommit repository.

The following diagram illustrates the solution architecture.

The solution workflow is as follows:

1. Developers integrate changes into a main branch hosted within a CodeCommit repository.
2. CodePipeline polls the source code repository and triggers the pipeline to run when a new version is detected.
3. CodePipeline runs a build of the new revision in CodeBuild.
4. CodeBuild runs the changes in the yml file, which includes the changes against the StackSets. (To update all the stack instances associated with this StackSet, do not specify DeploymentTargets or Regions in the buildspec.yml file.)
5. Verify that the changes were applied successfully.

Prerequisites

To complete this tutorial, you should have the following prerequisites:

• An AWS account.
• Access to either AWS Cloud9 or the AWS Command Line Interface (AWS CLI).
• Basic knowledge of AWS CloudFormation.
• A CloudFormation StackSet. You can create a StackSet via the AWS Management Console or the AWS CLI. For instructions, see Create a StackSet. You can also use one of the sample templates provided by AWS CloudFormation device to create a StackSet. For more information, please visit the Create a StackSet
• An AWS Identity and Access Management (IAM) service role to allow CodeBuild to build this project.

Retrieving your StackSet parameters

Your first step is to verify that you have a StackSet in the AWS account you intend to use. If not, create one before proceeding. For this post, we use an existing StackSet called StackSet-Test.

1. Sign in to your AWS account.
2. On the CloudFormation console, choose StackSets.
3. Choose your StackSet.

For this post, we modify the value of the parameter with the key KMSId.

4. On the Parameters tab, note the value of the key KMSId.

Creating a CodeCommit repository

To create your repository, complete the following steps:

1. On the CodeCommit console, choose Repositories.
2. Choose Create repository.

3. For Repository name, enter a name (for example, Demo-Repo).
4. Choose Create.

5. Choose Create file to populate the repository with the following artifacts.

A buildspec.yml file informs CodeBuild of all the actions that should be taken during a build run for our application. We divide the build run into separate predefined phases for logical organization, and list the commands that run on the provisioned build server performing a build job.

6. Enter the following code in the code editor:

YAML

phases:

  pre_build:

    commands:

      - aws cloudformation update-stack-set --stack-set-name StackSet-Test --use-previous-template --parameters ParameterKey=KMSId,ParameterValue=newCustomValue

The preceding AWS CloudFormation command updates a StackSet with the name StackSet-Test. The command results in updating the parameter value of the parameter key KMSId to newCustomValue.

7. Name the file yml.
8. Provide an author name and email address.
9. Choose Commit changes.

Creating a CodeBuild project

To create your CodeBuild project, complete the following steps:

1. On the CodeBuild console, choose Build projects.
2. Choose Create build project.

3. For Project name, enter your project name (for example, Demo-Build).
4. For Description, enter an optional description.

5. For Source provider, choose AWS CodeCommit.
6. For Repository, choose the CodeCommit repository you created in the previous step.
7. For Reference type, keep default selection Branch.
8. For Branch, choose master.

To set up the CodeBuild environment, we use a managed image based on Amazon Linux 2.

9. For Environment Image, select Managed image.
10. For Operating system, choose Amazon Linux 2.
11. For Runtime(s), choose Standard.
12. For Image, choose amazonlinux2-aarch64-standard:1.0.
13. For Image version, choose Always use the latest for this runtime version.

14. For Service role¸ select New service role.
15. For Role name, enter your service role name.

16. Chose Create build project.

Creating a CodePipeline pipeline

To create your pipeline, complete the following steps:

1. On the CodePipeline console, choose Pipelines.
2. Choose Create pipeline

3. For Pipeline name, enter a name for the pipeline (for example, DemoPipeline).
4. For Service role, select New service role.
5. For Role name, enter your service role name.

6. Choose Next.
7. For Source provider, choose AWS CodeCommit.
8. For Repository name, choose the repository you created.
9. For Branch name, choose master.

10. Choose Next.
11. For Build provider, choose AWS CodeBuild.
12. For Region, choose your Region.
13. For Project name, choose the build project you created.

14. Choose Next.
15. Choose Skip deploy stage.
16. Choose Skip
17. Choose Create pipeline.

The pipeline is now created successfully.

Running and monitoring your pipeline

We use the pipeline to release changes. By default, a pipeline starts automatically when it’s created and any time a change is made in a source repository. You can also manually run the most recent revision through your pipeline, as in the following steps:

1. On the CodePipeline console, choose the pipeline you created.
2. On the pipeline details page, choose Release change.

The following screenshot shows the status of the run from the pipeline.

3. Under Build, choose Details to view build logs, phase details, reports, environment variables, and build details.

4. Choose the Build logs tab to view the logs generated as a result of the build in more detail.

The following screenshot shows that we ran the AWS CloudFormation command that was provided in the buildspec.yml file. It also shows that all phases of the build process are successfully complete.

The StackSet parameter KMSId has been updated successfully with the new value newCustomValue as a result of running the pipeline.  Please note that we used the parameter KMSId as an example for demonstration purposes. Any other parameter that is part of your StackSet could have been used instead.

Cleaning up

You may delete the resources that you created during this post:

• AWS CloudFormation StackSet.
• AWS CodeCommit repository.
• AWS CodeBuild project.
• AWS CodePipeline.

Conclusion

In this post, we explored how to use CodePipeline, CodeBuild, and CodeCommit to update an existing CloudFormation StackSet. Happy coding!

About the author

Karim Afifi is a Solutions Architect Leader with Amazon Web Services. He is part of the Global Life Sciences Solution Architecture team. He is based out of New York, and enjoys helping customers throughout their journey to innovation.