AWS CloudTrail – Capture AWS API Activity

Do you have the need to track the API calls for one or more AWS accounts? If so, the new AWS CloudTrail service is for you.

Once enabled, AWS CloudTrail records the calls made to the AWS APIs using the AWS Management Console, the AWS Command Line Interface (CLI), your own applications, and third-party software and publishes the resulting log files to the Amazon S3 bucket of your choice. CloudTrail can also issue a notification to an Amazon SNS topic of your choice each time a file is published. Each call is logged in JSON format for easy parsing and processing.

The API call history logged by CloudTrail is designed to support a wide variety of use cases. Here are some ideas to get you started:

• Compliance Aid – You have access to information needed to demonstrate that AWS resources were managed according to rules and regulatory standards.
• Resource Life Cycle Tracking -You can track an AWS resource from creation through deletion.
• Operational Troubleshooting – You can identify the most recent changes made to resources in your environment.
• Security Analytics – You can see which user activities failed due to inadequate permissions.

The data produced by CloudTrail can help you to answer questions such as:

• What actions did a given user take over a specific time period?
• For a given resource, which AWS user has taken action on it over a given time period?
• What is the source IP address of a particular activity?

CloudTrail logs API calls on the following AWS services:

• Elastic Compute Cloud (EC2)
• Elastic Block Store (EBS)
• Virtual Private Cloud (VPC)
• Relational Database Service (RDS)
• Identity and Access Management (IAM)
• Security Token Service (STS)
• Redshift
• CloudTrail

We plan to add support for additional services over time; please click here to let us know which services are most important to you.

There’s no charge for CloudTrail; you will pay the usual S3 and SNS charges to store the data and to receive the notifications.

CloudTrail Architecture
Here is a picture to help you understand how CloudTrail works:

Enabling and Configuring CloudTrail
You can enable CloudTrail from the AWS Management Console. You can route the log files to the S3 bucket of your choice. You can consolidate logs from multiple AWS accounts in a single bucket.

Here is how you enable CloudTrail using the AWS Management Console:

Click the Get Started button, then enter the configuration information. You can create a new S3 bucket or you can use an existing one. If you choose the default option and create a new bucket, CloudTrail will apply a suitable access policy to it. If you use an existing bucket, you will need to configure the policy yourself.

Using CloudTrail
Once you have enabled CloudTrail, log delivery will commence. Events will be delivered to the bucket, typically within 15 minutes after they take place. The log files are encrypted using S3’s Server Side Encryption and will remain in the bucket until you choose to delete or archive them (see my post on Archiving Amazon S3 Data to Glacier for more information). If you are using CloudTrail to pursue your compliance and auditing goals, you can add an additional layer of security by enabling MFA Delete on the bucket.

The log files will be organized by AWS Account Id, Region, Service Name, Date, and Time:

Each file contains events for all supported AWS Services. Each event is represented as a single JSON object. Note that the data includes AWS access keys. The following data represents a single Stop Instance request from the AWS Management Console:

{    "awsRegion":"us-east-1",    "eventName":"StopInstances",    "eventSource":"ec2.amazonaws.com",    "eventTime":"2013-08-30T06:11:17Z",    "requestParameters":{      "force":false,      "instancesSet":{        "items":[          {            "instanceId":"i-14678937"          }        ]      }    },    "responseElements":{      "instancesSet":{        "items":[          {            "currentState":{              "code":64,              "name":"stopping"            },            "instanceId":"i-14678937",            "previousState":{              "code":16,              "name":"running"            }          }        ]      }    },    "sourceIPAddress":"1.2.3.4",    "userAgent":"AWS Console",    "userIdentity":{      "accessKeyId":"ASxxxxxxxxxxxxxxxxxx",      "accountId":"123456789012",      "arn":"arn:aws:iam::123456789012:user/jbarr",      "principalId":"AIxxxxxxxxxxxxxxxxxxx",      "sessionContext":{        "attributes":{          "creationDate":"Thu Aug 29 14:36:54 PDT 2013",          "mfaAuthenticated":"false"        }      },      "type":"IAMUser",      "userName":"jdoe"    }  }  

Learn more in the CloudTrail User Guide.

CloudTrail Partners
We have been working with partners to make sure that you have access to a wide variety of analytics tools for your CloudTrail logs. Here are some screen shots from some of these tools (visit the CloudTrail Partner Page for more information):

Blaze a Trail
CloudTrail is available now in the US East (Northern Virginia), and US West (Oregon) Regions. We’ll be adding support for additional Regions as quickly as possible.

— Jeff;