Automate safe AWS CloudFormation deployments from GitHub

AWS CloudFormation, an Infrastructure as Code (IaC) service that lets you model, provision, and manage AWS and third-party resources, now supports using Git sync to automatically trigger a deployment whenever a tracked Git repository is updated. This enables developers to significantly speed up the development cycle for CloudFormation by integrating into their Git workflow and reducing time lost to context switching. The new integration works directly with GitHub, GitHub Enterprise, GitLab, and Bitbucket.

In this post, you’ll explore what a modern development experience looks like using both GitHub’s native tooling as well as the native CloudFormation Git sync integration. You’ll be creating a cloud development environment using GitHub CodeSpaces, integrating direct feedback into Pull Requests using GitHub Actions and the CloudFormation Linter, and automating safe deployments.

Requirements

• An AWS account. You will be deploying Amazon Virtual Private Cloud (Amazon VPC) resources, which are provided at no extra cost. AWS Accounts may only have five (5) VPCs without a limit increase.
• A GitHub account with access to Codespaces and Actions. These have a free tier but may incur charges if exceeded.
• A CodeStar Connection to GitHub

Creating an empty repository

For this, you’ll start with a new GitHub repository. GitHub allows you to create new Git repositories for free. For this example, you’ll create a repository called git-sync.

Setting up a Codespace

Once you create the repository, you’ll have the option to create a Codespace. Codespaces are remote development environments managed by GitHub that allows you to develop from anywhere on standardized virtual machines.

Codespaces uses Visual Studio Code as its code editor of choice. Visual Studio Code is an open-source code editor that has excellent flexibility and extensibility due to the ability to install extensions.

Once it finishes creating, you can set up the environment much like you would your local development environment. You’re going to be adding the CloudFormation Linter extension to provide fast in-editor feedback when developing your template. This lets you avoid having to send CloudFormation your templates for validation and instead have good confidence that your templates are valid before you submit them for provisioning. You’ll install it both using the command line and an extension to Visual Studio Code itself. In the terminal, run:

pip3 install cfn-lint

Once that installs, you can install the linter in the extensions panel:

Next, you’ll create your template in the base directory called vpc.yaml. As you start typing, the linter extension will offer recommendations and auto-complete for you.

Copy the following template into our newly created vpc.yaml file:

AWSTemplateFormatVersion: "2010-09-09"
Resources:
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.0.0.0/16

This template creates a VPC with a CIDR block of 10.0.0.0/16.

You can verify the template gives no errors by running cfn-lint in the terminal and verifying it returns no errors.

cfn-lint -t vpc.yaml

Adding the deployment file

In order to support many different types of deployments, Git sync supports a deployment file to provide flexibility for managing CloudFormation stacks from within a Git repository. This config file manages the location of the template file, and any parameters or tags you may be interested in using. I strongly encourage you to use a config file for managing your parameters and tags, as it enables easy auditability and deterministic deployments.

You’ll be creating a new file called deployment-file.yaml in your repository. Since this stack doesn’t have parameters or tags, it’ll be relatively simple:

template-file-path: ./vpc.yaml

You also have the ability to add this file in the console later.

Adding Pull Request actions

Now that you’ve configured your development environment just the way you want it, you want to ensure that anyone who submits a pull-request will receive the same high-quality feedback that you’re getting locally. You can do this using GitHub Actions. Actions are a customizable workflow tool that you can leverage to enable pull-request feedback and CI builds.

To do that, you’ll have to create the following folder structure: .github/workflows/pull-request.yaml. The contents of this file are as follows:

name: Pull Request workflow

on:
  - pull_request

jobs:
  cloudformation-linter:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout
        uses: actions/checkout@v3
      - name: Linter install
        uses: scottbrenner/cfn-lint-action@v2
        with:
          command: cfn-lint -t ./vpc.yaml

With this configured, you’ll now get feedback on a pull request with the linter’s findings. Push your work to the remote branch.

git add -A
git commit -m "add pull request linting workflow, add base vpc template"
git push origin main

Now you’ll add a subnet to your VPC and intentionally make a mistake by adding an invalid property called VpcName, instead of VpcId.

AWSTemplateFormatVersion: "2010-09-09"
Resources:
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.0.0.0/16

  Subnet:
    Type: AWS::EC2::Subnet
    Properties:
      VpcName: !Ref VPC
      CidrBlock: 10.0.0.1/16

The linter will immediately inform you this is invalid:

You can ignore these for now. To create your pull request, you have to create a new branch and commit your local changes. You can do that using:

git switch -c add-subnet
git add -A
git commit -m "add subnet"
git push origin add-subnet

Once you push these commits, GitHub will allow you to create a pull request against your main branch. However, once you create it, you’ll notice that your checks fail when your GitHub Actions finish running.

You can see what went wrong by checking the “Files changed” tab. Your linter action will provide feedback directly on the pull request and block your merge action if you’ve set up your branch protection. This repository requires at least one reviewer and all checks to pass, so you’ll have to resolve both these failures.

Now that you have the high-quality feedback as well as the offending line numbers, you can go back to your template and make the necessary fix of changing VpcName to VpcId.

AWSTemplateFormatVersion: "2010-09-09"
Resources:
  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: 10.0.0.0/16

  Subnet:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      CidrBlock: 10.0.0.1/16

The local linter is happy, and after you commit again you’ll see that your remote linter is equally happy. After getting another approver, you can merge your commit into your main branch.

Enabling Git sync

You now have a high-quality cloud development environment and your pull request process ensures your templates are linted before merging. You can be sure that a CloudFormation template that makes it to the main branch is ready to be deployed. Next, you’ll be leveraging the newly released Git sync feature of CloudFormation to automatically sync your deployed stack with this new template.

First, create the role that will deploy our CloudFormation template. Be sure to note the name you select for this as you’ll be using it to manage your stack later. This example uses vpc-example-cloudformation-deployment-role.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "ec2:CreateVpc",
        "ec2:CreateSubnet",
        "ec2:DescribeVpcs",
        "ec2:DescribeSubnets",
        "ec2:DeleteVpc",
        "ec2:DeleteSubnet",
        "ec2:ModifySubnetAttribute",
        "ec2:ModifyVpcAttribute"
      ],
      "Resource": "*",
      "Condition": {
        "ForAnyValue:StringEquals": {
          "aws:CalledVia": ["cloudformation.amazonaws.com"]
        }
      }
    }
  ]
}

Once the role has been created, you’ll have to create a new stack:

Here, you can see the new option to select Sync from Git template source, which you can configure on the next screen. Since you already created your stack deployment file, you can select I am providing my own file in my repository.

Next, you can configure your Git integration to choose your repository. Since it’s your first time, you’ll need to use the CodeStar Connection you created beforehand and select your repository.

Select GitHub, your connection, the repository, and branch, the deployment file location.

Finally, you will select New IAM Role to create a service managed role. This role will enable Git sync to connect to your repository. You’ll only need to do this once; in the future you’ll be able to use the existing role you’ll create here.

On the next page, you’ll select the IAM Role you created to manage this stack. This role controls the resources that CloudFormation will deploy. Stacks managed by Git sync must have a role already created.

Finally, you can see the status of your sync in the new “Git sync” tab, including the configuration you provided earlier as well as the status of your sync, your previous deployments, and the option to retry or disconnect the sync if needed.

Conclusion

At this point, you’ve configured a remote development environment to get high-quality feedback when creating and updating your CloudFormation templates. You also have the same high-quality feedback when creating a pull request. Finally, when a template does get merged to the main branch, it will be automatically deployed to your stack. This represents a robust and extensible CI/CD system to manage your infrastructure as code. I’m excited to hear your feedback about this feature!