How to Automatically Tag Amazon EC2 Resources in Response to API Events

Note: As of March 28, 2017,  Amazon EC2 supports tagging on creation, enforced tag usage, AWS Identity and Access Management (IAM) resource-level permissions, and enforced volume encryption. See New – Tag EC2 Instances & EBS Volumes on Creation on the AWS Blog for more information.

Access to manage Amazon EC2 instances can be controlled using tags. You can do this by writing an Identity and Access Management (IAM) policy that grants users permissions to manage EC2 instances that have a specific tag. However, if you also give users permissions to create or delete tags, users can manipulate the values of the tags to gain access and manage additional instances.

In this blog post, I will explore a method to automatically tag an EC2 instance and its associated resources without granting ec2:createTags permission to users. I will use a combination of an Amazon CloudWatch Events rule and AWS Lambda to tag newly created instances. With this solution, your users do not need to have permissions to create tags because the Lambda function will have the permissions to tag the instances. The solution can be automatically deployed in the region of your choice with AWS CloudFormation. I explain the provided solution and the CloudFormation template in the following sections.

Solution overview

This solution has two parts. I first explore the AWS Identity and Access Management (IAM) policy associated with my IAM user, Bob. Then, I explore how you can tag EC2 resources automatically in response to specific API events in my AWS account.

The IAM configuration

My IAM user, Bob, belongs to an IAM group with a customer managed policy called [Your CloudFormation StackName]–TagBasedEC2RestrictionsPolicy–[Unique ID], as shown in the following screenshot. For convenience, I will refer to this policy as TagBasedEC2RestrictionsPolicy in the remainder of this post. (Throughout this post, you should replace placeholder values with your own AWS information.)

The content of the customer managed policy, TagBasedEC2RestrictionsPolicy, follows.

{
               "Version" : "2012-10-17",
               "Statement" : [
                   {
                       "Sid" : "LaunchEC2Instances",
                      "Effect" : "Allow",
                       "Action" : [
                           "ec2:Describe*",
                           "ec2:RunInstances"
                       ],
                       "Resource" : [
                           "*"
                       ]
                   },
                   {
                       "Sid" : "AllowActionsIfYouAreTheOwner",
                       "Effect" : "Allow",
                       "Action" : [
                           "ec2:StopInstances",
                           "ec2:StartInstances",
                           "ec2:RebootInstances",
                           "ec2:TerminateInstances"
                       ],
                       "Condition" : {
                           "StringEquals" : {
                               "ec2:ResourceTag/PrincipalId" : "${aws:userid}"
                           }
                       },
                       "Resource"  : [
                           "*"
                       ]
                   }
               ]
}

For example, an IAM user cannot see the unique identifier, UserId, from the IAM console, but you can retrieve it with the AWS CLI by using the following command.

aws iam get-user --user-name Bob

The following output comes from that command.

{
    "User": {
        "UserName": "Bob",
        "PasswordLastUsed": "2016-04-29T20:52:37Z",
        "CreateDate": "2016-04-26T19:26:43Z",
        "UserId": "AIDAJ7EQQEKFAVPO3NTQG",
        "Path": "/",
        "Arn": "arn:aws:iam::111122223333:user/Bob"
    }
}

In other cases, such as when assuming an IAM role to access an AWS account, the UserId is a combination of the assumed IAM role ID and the role session name that you specified at the time of the AssumeRole API call.

role-id:role-session-name

For a full list of values that you can substitute for policy variables, see Request Information That You Can Use for Policy Variables.

Because you cannot possibly memorize all these IDs, the automation defined later in this post not only tags resources with the UserId, but also it retrieves the actual userName (or RoleSessionName), and uses this value to define a more human-readable tag, Owner, that can be used to filter or report resources in an easier way.

Tag automation

The IAM user has EC2 rights to launch an EC2 instance. Regardless of how the user creates the EC2 instance (with the AWS Management Console or AWS CLI), he performs a RunInstances API call (#1 in the following diagram). CloudWatch Events records this activity (#2).

A CloudWatch Events rule targets a Lambda function called AutoTag and it invokes the function with the event details (#3). The event details contain the information about the user that completed the action (this information is retrieved automatically from AWS CloudTrail, which must be on for CloudWatch Events to work).

The Lambda function AutoTag scans the event details, and extracts all the possible resource IDs as well as the user’s identity (#4). The function applies two tags to the created resources (#5):

• Owner, with the current userName.
• PrincipalId, with the current user’s aws:userid value.

When Amazon Elastic Block Store (EBS) volumes, EBS snapshots and Amazon Machine Images (AMIs) are individually created, they invoke the same Lambda function. This way, you can similarly Allow or Deny actions based on tags for those resources, or identify which resources a user created.

CloudFormation automation

This CloudFormation template creates a Lambda function, and CloudWatch Events trigger that function in the region you choose. Lambda permissions to describe and tag EC2 resources are obtained from an IAM role the template creates along with the function. The template also creates an IAM group into which you can place your user to enforce the behavior described in this blog post. The template also creates a customer managed policy so that you can easily apply it to other IAM entities, such as IAM roles or other existing IAM groups.

Note: Currently, CloudWatch Events is available in six regions, and Lambda is available in five regions. Keep in mind that you can only use this post’s solution in regions where both CloudWatch Events and Lambda are available. As these services grow, you will be able to launch the same template in other regions as well.

The template first defines a Lambda function with an alias (PROD) and two versions (Version $LATEST and Version 1), as shown in the following diagram. This is a best practice for continuous deployment: the definition of an alias decouples the trigger from the actual code version you want to execute, while versioning allows you to work on multiple versions of the same code and test changes while a stable version of your code is still available for current executions. For more information about aliases and versions, see AWS Lambda Function Versioning and Aliases.

The Lambda function

You will find the full Lambda function code in the CloudFormation template. I will explain key parts of the code in this section.

The code starts with the definition of an array called ids. This array will contain all the identifiers of EC2 resources found in a given event. Resource IDs could be instances, EBS volumes, EBS snapshots, ENIs, and AMI IDs.

Array initialization

ids = []

The Lambda function extracts the region, the event name (that is, the name of the invoked API, such as RunInstances and CreateVolume), and the user’s principalID and userName. Depending on the user type, the code extracts the userName from the event detail, or it defines it as the second part of the principalID.

Information extraction

        region = event['region']
        detail = event['detail']
        eventname = detail['eventName']
        arn = event['detail']['userIdentity']['arn']
        principal = event['detail']['userIdentity']['principalId']
        userType = event['detail']['userIdentity']['type']

        if userType == 'IAMUser':
            user = detail['userIdentity']['userName']

        else:
            user = principal.split(':')[1]

Then, the code continues with EC2 client initialization.

EC2 client initialization

        ec2 = boto3.resource('ec2')

After a few input validations, the code looks for the resource IDs in the event detail. For each API call, the resource IDs can be found in different parts of the response, so the code contains a sequence of if/else statements to map these differences.

In the case of an EC2 instance, the function describes the identified instance to find the attached EBS volumes and ENIs, and adds their IDs to the array’s IDs.

Resource IDs extraction for each API call

        if eventname == 'CreateVolume':
            ids.append(detail['responseElements']['volumeId'])


        elif eventname == 'RunInstances':
            items = detail['responseElements']['instancesSet']['items']
            for item in items:
                ids.append(item['instanceId'])

            base = ec2.instances.filter(InstanceIds=ids)

            #loop through the instances
            for instance in base:
                for vol in instance.volumes.all():
                    ids.append(vol.id)
                for eni in instance.network_interfaces:
                    ids.append(eni.id)

        elif eventname == 'CreateImage':
            ids.append(detail['responseElements']['imageId'])

        elif eventname == 'CreateSnapshot':
            ids.append(detail['responseElements']['snapshotId'])

        else:
            logger.warning('Not supported action')

• Owner, with the value of the variable user.
• PrincipalId, with the value of the variable principal.

Resource tagging

       if ids:
            ec2.create_tags(Resources=ids, Tags=[{'Key': 'Owner', 'Value': user}, {'Key': 'PrincipalId', 'Value': principal}])

The CloudWatch Events rule

In order to trigger the Lambda function, you create a CloudWatch Events rule. You define a rule to match events and route them to one or more target Lambda functions. In our case, the events we want to match are the following AWS EC2 API calls:

• RunInstances
• CreateVolume
• CreateSnapshot
• CreateImage

The target of this rule is our Lambda function. We specifically point to the PROD alias of the function to decouple the trigger from a specific version of the Lambda function code.

The following screenshot shows what the CloudFormation template creates.

IAM group

The CloudFormation template creates an IAM group called [Your CloudFormation StackName]-RestrictedIAMGroup-[Unique ID]. The customer managed policy TagBasedEC2RestrictionsPolicy is associated with the group.

The solution in action

First, deploy the CloudFormation template in the region of your choosing. Specify the following Amazon S3 template URL: https://s3.amazonaws.com/aws-security-blog-content/public/sample/autotagec2resources/AutoTag.template, as shown in the following screenshot.

The template creates the stack only if you confirm you have enabled CloudTrail, as shown in the following screenshot.

The deployment should be completed within 5 minutes. The following screenshot shows the status of the stack as CREATE_COMPLETE.

Now that you have deployed the required automation infrastructure, you can assign IAM users to the created IAM group (ManageEC2InstancesGroup in the following screenshot).

Similarly, if you are using federation and your users access your AWS account through an IAM role, you can attach the customer managed policy, TagBasedEC2RestrictionsPolicy, to the IAM role itself.

In my case, I have added my IAM user, Bob, to the IAM group created by CloudFormation. Note that you have to add the IAM users to this group manually.

When Bob signs in to my AWS account, he can create EC2 instances with no restrictions. However, if he tries to stop an EC2 instance that he did not create, he will get the following error message.

In the meantime, the instance Bob created has been automatically tagged with his user name and user ID.

Bob can now terminate the instance. The following screenshot shows the instance in the process of shutting down.

What’s next?

Now that you know you can tag resources with a Lambda function in response to events, you can apply the same logic to other resources such as Amazon Relational Database Service (RDS) databases or S3 buckets. With resource groups, each user can focus on just the resources he created, and the IAM policy provided in this post assures that no Stop/Start/Reboot/Terminate action is possible on someone else’s instance.

Additionally, tags are useful in custom billing reports to project costs and determine how much money each individual owner is spending. You can activate the Owner tag in the billing console from the Cost Allocation Tags of your billing console to include it in your detailed billing reports. For more information, see Applying Tags.

If you have any comments, submit them in the “Comments” section below. If you have questions about the solution in this blog post, please start a new thread on the EC2 forum.

– Alessandro

Want more AWS Security how-to content, news, and feature announcements? Follow us on Twitter.