Deploying AWS Lambda functions using AWS CloudFormation (the portable way)

Using AWS CloudFormation to deploy AWS Lambda functions provides a reliable, reproducible and version-able deployment mechanism. But while simple deployments are easily achieved, it can be challenging to produce templates that seamlessly deploy to any AWS Region supported by Lambda. One common challenge is providing Amazon Simple Storage Service (Amazon S3) storage for Lambda zip files. Lambda requires the bucket to reside in the same AWS Region as the function, but creating a single template that references a single bucket confines your template to a single Region. In this post, we will discuss three patterns to address this challenge, each with its own benefits.

Custom resource

When an AWS CloudFormation stack is created, an Amazon S3 bucket is created in the same Region as the stack and populated with the Lambda zips by an AWS CloudFormation custom resource.

Using this pattern, only one bucket needs to contain the Lambda zip file in advance, and no additional work is required to have newly launched Regions supported. Because of its flexibility, this is the preferred approach for most use cases. It does, however, require the most additional resources in the template: an inline Lambda function with the code to copy the zips, an associated Identity and Access Management (IAM) role, an Amazon S3 bucket, and the custom resource.

The following snippet creates a new Amazon S3 bucket in the AWS CloudFormation stack Region (LambdaZipsBucket) and then copies zip files (CopyZips.Properties.Objects) from a source bucket (CopyZips.Properties.SourceBucket) into the new bucket.

  Resources:
    LambdaZipsBucket:
        Type: AWS::S3::Bucket
    CopyZips:
        Type: Custom::CopyZips
        Properties:
          ServiceToken: !GetAtt 'CopyZipsFunction.Arn'
          DestBucket: !Ref 'LambdaZipsBucket'
          SourceBucket: !Ref 'QSS3BucketName'
          Prefix: !Ref 'QSS3KeyPrefix'
          Objects:
            - functions/packages/MyFunction/lambda.zip

Notice that Objects takes a list of paths, so that if you have multiple Lambda functions in your template, all the zips can be added to a single CopyZips resource.

Now to declare our Lambda function and point it to the newly created bucket.

When creating the function, we point MyFunction.Properties.Code.S3Bucket to LambdaZipsBucket.

  MyFunction:
    DependsOn: CopyZips
    Type: AWS::Lambda::Function
    Properties:
      Code:
        S3Bucket: !Ref 'LambdaZipsBucket'
        S3Key: !Sub '${QSS3KeyPrefix}functions/packages/MyFunction/lambda.zip'
      ...

Notice the DependsOn key pointing to our CopyZips custom resource. By default, AWS CloudFormation will attempt to launch resources that are not dependent on each other in parallel, so this is required to be sure that the zips have already been copied before our Lambda function is created.

To implement this in your own template you will need the CopyZips Lambda code and the related IAM role deployed as well. A full example implementation is available in the AWS Quick Start examples repository.

Regional buckets

If you require support for only a few AWS Regions that don’t change often, then simply copying zips up to regional buckets in advance can be a viable approach. When Lambda function code is published or updated, a CD process can copy the .zip file to a set of buckets. To make templates portable, the bucket names should consist of a common prefix followed by the Region name. Your template code can then use the Fn::Sub intrinsic function to add the Region to the end of the common prefix.

  MyFunction:
    Type: AWS::Lambda::Function
    Properties:
      Code:
        S3Bucket: !Sub 'lambda-zips-${AWS::Region}'
      ...

Inline function code

Simple functions that are written in Node.js or Python can be declared inline in an AWS CloudFormation template, so no zips are necessary. AWS CloudFormation supports a maximum of 4096 characters, so this is viable only for small functions.

Resources:
  MyFunction:
    Type: AWS::Lambda::Function
    Properties:
      Code:
        ZipFile: |
          import json
          def handler(event, context):
              print("Event: %s" % json.dumps(event))
      ...

Because the code is embedded in YAML or JSON, external libraries cannot be used, certain characters must be escaped, and in JSON templates, each line must be separately declared and joined using the AWS CloudFormation Fn::Join intrinsic function. These requirements make it harder to use an IDE’s syntax checking and code completion functionality, so the template requires additional testing to ensure that code is rendered as expected when extracted from the template. For these reasons, this approach is best suited for function code that is simple and does not change frequently.

Summing up

In this post we’ve discussed approaches to authoring AWS CloudFormation templates containing AWS Lambda functions that are easily portable across different AWS regions. We’d love to hear from you on how these work—and if there’s something you are doing that works really well, let us know by commenting below or creating an issue on GitHub.