AWS Security Blog

How to Automatically Update Your Security Groups for Amazon CloudFront and AWS WAF by Using AWS Lambda

Note from April 1, 2021: Before implementing the steps in this blog post, please request an EC2 limit increase for “rules per security group.” Ask for 220 rules per security group in the AWS Region where your security groups will be.

Note from December 3, 2019: The features and services described in this post have changed since the post was published and the procedures described might be out of date and no longer accurate. If we update this post or create a replacement, we’ll add a notification about it here.

Update on August 23, 2018: We revised the “Configure your Lambda function’s trigger” procedure.

Update on June 14, 2018: We removed an out-of-date code sample.

Amazon CloudFront can help you increase the performance of your web applications and significantly lower the latency of delivering content to your customers. Recently announced, AWS WAF (a web application firewall) gives you control over which traffic to allow or block by defining customizable web security rules. In conjunction with AWS WAF, CloudFront now can also help you secure your web applications. This blog post will show you how to create an AWS Lambda function to automatically update VPC security groups with AWS internal service IP ranges to ensure that AWS WAF and CloudFront cannot be bypassed.

When using AWS WAF to secure your web applications, it’s important to ensure that only CloudFront can access your origin; otherwise, someone could bypass AWS WAF itself. If your origin is an Elastic Load Balancing load balancer or an Amazon EC2 instance, you can use VPC security groups to allow only CloudFront to access your applications. You can accomplish this by creating a security group that only allows the specific IP ranges of CloudFront. AWS publishes these IP ranges in JSON format so that you can create networking configurations that use them. These ranges are separated by service and region, which means you’ll only need to allow IP ranges that correspond to CloudFront.

In the past, you would use these IP ranges to manually create a security group rule in the AWS Management Console and supply only the prefixes marked for CloudFront. But what would you have done if the IP ranges changed? One solution was to poll the IP ranges’ endpoint periodically with a simple cron job to make sure they were current. This meant you needed infrastructure to support the task. However, you ended up with another host to manage, complete with the typical patching, deployment, and monitoring. As you can see, a small task could quickly become more complicated than the problem it aimed to solve.

An Amazon Simple Notification Service (SNS) topic is generated whenever the AWS IP ranges change. Therefore, you can build an event-driven, zero-infrastructure solution using a Lambda function that is triggered in response to the SNS notification. Let’s get started!

Create a security group

The first thing you need to do is create a security group. This security group will allow only traffic from CloudFront and AWS WAF into your Elastic Load Balancing load balancers or EC2 instances.

Then, add both security groups to your Amazon EC2 instance or Elastic Load Balancing load balancer and configure the AWS Lambda script.

In the EC2 console:

  1. Click Security Groups > Create Security Group.
  2. Give your security group a meaningful name and description.
  3. Next, view the security group you just created, and add three tags that our Lambda function will use to identify security groups it needs to update: set Name to cloudfront_g, AutoUpdate to true, and Protocol to either http or https.
  4. Repeat this process above with the Protocol and AutoUpdate tags with a new name of cloudfront_r.

Note: If you use both http and https to your origin, create new security groups and set the other protocol you did not use in the above steps.

Create an IAM policy and execution role for the Lambda function

When creating a Lambda function, it’s important to understand and properly define the security context to which the Lambda function is subject. Using IAM, you will create the Lambda execution role that determines the AWS service calls that the function is authorized to complete. (Learn more about the Lambda permissions model.)

  1. Before you can create the IAM role, you need to create an IAM policy that you will attach to it. In the IAM console, click Policies > Create Policy > Select (next to Create Your Own Policy).
  2. Supply a name for your policy, and then copy and paste the following policy document into the Policy Document box. 
    {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Resource": "arn:aws:logs:*:*:*"
    },
    {
      "Effect": "Allow",
      "Action": [
        "ec2:DescribeSecurityGroups",
        "ec2:AuthorizeSecurityGroupIngress",
        "ec2:RevokeSecurityGroupIngress"
      ],
      "Resource": "*"
    }
  ]
}

To explain what this policy allows, let’s look closely at both statements in the policy. The first statement allows the Lambda function to write to Amazon CloudWatch logs, which is vital for debugging and monitoring our function. The second statement allows the function to get information about existing security groups and to authorize and revoke ingress permissions. It’s an important best practice that your IAM policies be as granular as possible, to observe the principal of least privilege.

Now that you have created your policy, you can create your Lambda execution role using that policy:

  1. In the IAM console, click Roles > Create New Role, and then name your role.
  2. To select a role type, select AWS Service Roles > AWS Lambda.
  3. Attach the policy you just created.
  4. After confirming your selections, click Create Role.

Create your Lambda function

Now that you have created your Lambda execution role, you are ready to create your Lambda function:

  1. Go to the Lambda console and choose Create function. On the next page, choose Author from scratch. (Because I’ll be providing the code for your Lambda function, you can skip the blueprint step, but for other functions, blueprints can be a great way to get started.)
  2. On the Configure triggers page, choose Next.
  3. Give your Lambda function a name and description, and select Python 3.7 from the Runtime menu.
  4. Paste the Lambda function code from the aws-cloudfront-samples GitHub repository. Important note: By default, Lambda configures the SDK in its own region. If the security groups are in a different region than the Lambda function, you must update the SDK client with the correct region (client = boto3.client(‘ec2’,region_name=‘yourregion’)).
  1. Below the code window for Lambda function handler and role, select the execution role you created earlier.
  2. Under Advanced settings, increase the Timeout to 5 seconds.  If you are updating several security groups with this function, you might have to increase the timeout by even more time. Finally, click Next.
  3. After confirming your settings are correct, click Create function.

Test your Lambda function

Now that you have created your function, it’s time to test it and initialize your security group:

  1. In the Lambda console on the Functions page, choose your function, choose the Actions drop-down menu, and then Configure test event.
  2. Enter the following as your sample event, which will represent an SNS notification. 
    {
  "Records": [
    {
      "EventVersion": "1.0",
      "EventSubscriptionArn": "arn:aws:sns:EXAMPLE",
      "EventSource": "aws:sns",
      "Sns": {
        "SignatureVersion": "1",
        "Timestamp": "1970-01-01T00:00:00.000Z",
        "Signature": "EXAMPLE",
        "SigningCertUrl": "EXAMPLE",
        "MessageId": "95df01b4-ee98-5cb9-9903-4c221d41eb5e",
        "Message": "{\"create-time\": \"yyyy-mm-ddThh:mm:ss+00:00\", \"synctoken\": \"0123456789\", \"md5\": \"7fd59f5c7f5cf643036cbd4443ad3e4b\", \"url\": \"https://ip-ranges.amazonaws.com/ip-ranges.json\"}",
        "Type": "Notification",
        "UnsubscribeUrl": "EXAMPLE",
        "TopicArn": "arn:aws:sns:EXAMPLE",
        "Subject": "TestInvoke"
      }
    }
  ]
}
  1. After you’ve added the test event, click Save and test. Your Lambda function will be invoked, and you should see log output at the bottom of the console similar to the following. 
    Updating from https://ip-ranges.amazonaws.com/ip-ranges.json
MD5 Mismatch: got 2e967e943cf98ae998efeec05d4f351c expected 7fd59f5c7f5cf643036cbd4443ad3e4b: Exception
Traceback (most recent call last):
  File "/var/task/lambda_function.py", line 29, in lambda_handler
    ip_ranges = json.loads(get_ip_groups_json(message['url'], message['md5']))
  File "/var/task/lambda_function.py", line 50, in get_ip_groups_json
    raise Exception('MD5 Missmatch: got ' + hash + ' expected ' + expected_hash)
Exception: MD5 Mismatch: got 2e967e943cf98ae998efeec05d4f351c expected 7fd59f5c7f5cf643036cbd4443ad3e4b

You will see a message indicating there was a hash mismatch. Normally, a real SNS notification from the IP Ranges SNS topic will include the right hash, but because our sample event is a test case representing the event, you will need to update the sample event manually to have the expected hash.

  1. Edit the sample event again, and this time change the md5 hash highlighted in red to be the first hash provided in the log output. In this example, we would update the sample event with the hash “2e967e943cf98ae998efeec05d4f351c”.
  2. Click Save and test, and your Lambda function will be invoked.

This time, you should see output indicating your security group was properly updated. If you go back to the EC2 console and view the security group you created, you will now see all the CloudFront IP ranges added as allowed points of ingress. If your log output is different, it should help you identify the issue.

Configure your Lambda function’s trigger

After you have validated that your function is executing properly, it’s time to connect it to the SNS topic for IP changes. To do this, use the AWS Command Line Interface (CLI). Enter the following command, making sure to replace <Lambda ARN> with the Amazon Resource Name (ARN) of your Lambda function. You will find this ARN at the top right when viewing the configuration of your Lambda function.


aws sns subscribe --topic-arn arn:aws:sns:us-east-1:806199016981:AmazonIpSpaceChanged --protocol lambda --notification-endpoint <Lambda ARN>

You should receive an ARN of your Lambda function’s SNS subscription.

Now add a permission that allows the Lambda function to be invoked by the SNS topic. The following command adds the Lambda trigger, as well:


aws lambda add-permission --function-name <Lambda ARN> --statement-id lambda-sns-trigger --action lambda:InvokeFunction --principal sns.amazonaws.com --source-arn arn:aws:sns:us-east-1:806199016981:AmazonIpSpaceChanged

Summary

As you followed this blog post, you created a security group and a Lambda function to update the security group’s rules dynamically whenever AWS publishes new internal service IP ranges. This solution has several advantages:

  • The solution is not designed as a periodic poll, so it only executes when it needs to.
  • It is automatic, so you don’t need to update security groups manually.
  • It is simple because you have no extra infrastructure to maintain.
  • It is cost effective. Because the Lambda function fires only when necessary and only runs for a few seconds, this solution only costs pennies to operate.

And this is just the tip of the iceberg for AWS WAF. In the coming year, we hope to provide you additional blog posts about how to use AWS WAF.

If you have any questions or comments, please add them in the comments section below or on the Lambda forum. If you have any other use cases for using Lambda functions to dynamically update security groups or even other networking configurations such as VPC route tables or ACLs, we’d love to hear about them as well!

Want more AWS Security news? Follow us on Twitter.