Automatically Deploy from Amazon S3 using AWS CodeDeploy

AWS CodeDeploy does software deployments to any instance, including Amazon EC2 instances and instances running on-premises. It helps avoid downtime during deployments and also provides centralized control over your applications, instances, deployments and deployment configurations. You can learn more about CodeDeploy here. This post explains how to automatically start a CodeDeploy deployment when you upload your applications to Amazon S3. We will use AWS Lambda to notify CodeDeploy as soon as your new application revision is uploaded.

AWS Lambda is a compute service that runs your code in response to events. An event is generated by supported AWS services and Lambda executes your code as soon as these events fire. You can learn more about Lambda here.

Prerequisites

Lambda Execution Role

We need to set up an execution role to allow Lambda to execute our function that creates a deployment. To create an execution role copy the following policy

{
 "Version": "2012-10-17",
 "Statement": [
  {
    "Effect": "Allow",
    "Action": [
      "logs:*"
    ],
    "Resource": "arn:aws:logs:*:*:*"
  },
  {
    "Effect": "Allow",
    "Action": [
      "s3:GetObject"
    ],
    "Resource": [
      "arn:aws:s3:::BUCKET_NAME/*"
    ]
  },
  {
    "Effect": "Allow",
    "Action": "codedeploy:GetDeploymentConfig",
    "Resource": "arn:aws:codedeploy:us-east-1:123ACCOUNTID:deploymentconfig:*"
  },
  {
    "Effect": "Allow",
    "Action": "codedeploy:RegisterApplicationRevision",
    "Resource": "arn:aws:codedeploy:us-east-1:123ACCOUNTID:application:*"
  },
  {
    "Effect": "Allow",
    "Action": "codedeploy:GetApplicationRevision",
    "Resource": "arn:aws:codedeploy:us-east-1:123ACCOUNTID:application:*"
  },
  {
    "Effect": "Allow",
    "Action": "codedeploy:CreateDeployment",
    "Resource": "arn:aws:codedeploy:us-east-1:123ACCOUNTID:deploymentgroup:*"
   }
 ]
}

Make sure you replace BUCKET_NAME with the S3 bucket to which you’ll upload your CodeDeploy application revision. You will also need to replace “us-east-1” if you are using a different region and replace “123ACCOUNTID” with your AWS account ID that is found on your Account Settings page.

Go to IAM Policies page.
Click on Create Policy.
Select Create Your Own Policy.
Set the Policy Name to CodeDeployDeploymentPolicy, paste the above CodeDeploy deployment policy in the Policy Document section.
Go to IAM Roles page.
Click on Create New Role.
Input the name LambdaExecutionRole and click Next Step.
Under Select Role Type choose AWS Service Roles and choose AWS Lambda and click Next Step.
In the Attach Policy page select CodeDeployDeploymentPolicy and click Next Step.
Click on Create Role to create the role.

Once you’ve created the Lambda execution role, the Trust Relationships policy will look like

{
 "Version": "2012-10-17",
 "Statement": [
   {
     "Sid": "",
     "Effect": "Allow",
     "Principal": {
       "Service": "lambda.amazonaws.com"
     },
     "Action": "sts:AssumeRole"
   }
 ]

This role provides access to write to Amazon CloudWatch Logs, perform GetObject operations on the specified S3 bucket, and perform deployment using CodeDeploy. The CloudWatch Logs permission is optional. It lets us log exceptions if something goes wrong. This will be handy during debugging. The trust policy grants Lambda permission to perform the above allowed actions on the user’s behalf.

Lambda Function

The Lambda function gets a notification from Amazon S3. The event contains the source bucket and key. It tries to detect the object’s type by looking at the extension. By default, it assumes tar. It then calls getObject on the s3 bucket and key to read the object’s metadata. It expects two parameters in object metadata.

These values represent the CodeDeploy application and deployment group to which you want to auto deploy this S3 object. We will talk about how these values are passed to the Lambda function soon.

var aws = require('aws-sdk');
var s3 = new aws.S3({apiVersion: '2006-03-01'});
var codedeploy = new aws.CodeDeploy();
 
exports.handler = function(event, context) {
    var artifact_type;
    var bucket;
    var key;
 
    /* runtime functions */
    function getS3ObjectAndCreateDeployment() {
    // Get the s3 object to fetch application-name and deploymentgroup-name metadata.
	    s3.headObject({
		    Bucket: bucket,
		    Key: key
	    }, function(err, data) {
            if (err) {
                context.done('Error', 'Error getting s3 object: ' + err);
            } else {
                console.log('Creating deployment');
                createDeployment(data);
            }
        });
    }
 
 
    function createDeployment(data) {
        if (!data.Metadata['application-name'] || !data.Metadata['deploymentgroup-name']) {
            console.error('application-name and deploymentgroup-name object metadata must be set.');
            context.done();
        }
        var params = {
            applicationName: data.Metadata['application-name'],
            deploymentGroupName: data.Metadata['deploymentgroup-name'],
            description: 'Lambda invoked codedeploy deployment',
            ignoreApplicationStopFailures: false,
            revision: {
                revisionType: 'S3',
                s3Location: {
                    bucket: bucket,
                    bundleType: artifact_type,
                    key: key
                }
            }
        };
        codedeploy.createDeployment(params, 
            function (err, data) {
                if (err) {
                    context.done('Error','Error creating deployment: ' + err);
                }
                else {
                    console.log(data);           // successful response
                    console.log('Finished executing lambda function');
                    context.done();
                }
        });
    }
 
    console.log('Received event:');
    console.log(JSON.stringify(event, null, '  '));
 
    // Get the object from the event
    bucket = event.Records[0].s3.bucket.name;
    key = event.Records[0].s3.object.key;
 
    tokens = key.split('.');
    artifact_type = tokens[tokens.length - 1];
    if (artifact_type == 'gz') {
        artifact_type = 'tgz';
    } else if (['zip', 'tar', 'tgz'].indexOf(artifact_type) < 0) {
        artifact_type = 'tar';
    }
 
    getS3ObjectAndCreateDeployment();
};

Registering the Lambda Function

Registering a Lambda function is simple. Just follow these steps:

Go to the AWS Lambda console.
Click Create A Lambda Function.
Provide a Name and Description for the function.
Paste the above code snippet in the Lambda Function Code section, replacing the default contents.
Leave the handler name as ‘handler’.
For Role name, select the Lambda execution role that you created.
You can increase the Memory and Timeout under Advanced Settings if you are going to upload a large application revision of size greater than 5 Mb for example. Otherwise use the default values.
Click Create Lambda Function.
Select the Lambda function, click Actions and select Add event source.
Select the Event source type to S3.
Choose your S3 bucket.
Select the Event type to Object Created.
Click Submit.

Upload to Amazon S3

Once all the above steps are done, you should be able to upload new revisions of your application to the configured S3 bucket. Make sure you specify the following custom metadata while uploading.

application-name
deploymentgroup-name

This way Lambda understands which application and deployment group to create the deployment on. If you are uploading via the Amazon S3 console, do the following:

Go to S3 Console and click on your bucket to which you are going to upload your application revision.
Click Upload.
Add your CodeDeploy bundle by clicking AddFiles.
Click SetDetails>.
Click SetPermissions>.
Click SetMetadata>.
Click Add more metadata.
After adding the required metadata, click Start Upload.

Note: Custom object metadata should be prefixed with x-amz-meta-. For example, x-amz-meta-application-name or x-amz-meta-deploymentgroup-name. Amazon S3 uses this prefix to distinguish the user metadata from other headers.

If you forget to specify the above object metadata during S3 upload, the Lambda function would log an error similar to the following in AWS CloudWatch.

Other Integrations

This Lambda function is just a simple example that showcases how to link AWS CodeDeploy with other AWS services. You can create similar functions to perform other CodeDeploy actions in response to other events. We would love to hear about your ideas or questions in the comments here or over in our forum.

AWS DevOps & Developer Productivity Blog