Retrieving previous versions of S3 bucket policies

When dealing with applications or storage, it is often helpful to log previous policy configurations and have the ability to retrieve them. This can help you managed and troubleshoot configuration changes and comply with security regulations.

Amazon Simple Storage Service (Amazon S3) bucket policies are resource-based policies that you can use to grant access permissions to your S3 bucket and the objects in it. In order to retrieve previous version of a policy you can look at configuration change events.

Creating or updating cloud resources generates API events that contain configuration parameters of the cloud resource. AWS provides services that allow logging and auditing of API activities, such as AWS CloudTrail and Amazon EventBridge. AWS Config gathers an inventory of AWS resources and enables you to dive into the history of a specific asset configuration.

In this blog, we guide you through three options to retrieve a previous version of an S3 bucket policy configuration. First, we’ll cover how to review CloudTrail’s Event history to quickly identify resource changes and troubleshoot any associated issues. This feature is available to all accounts and comes at no additional cost, providing access to 90 days of events. Next, we’ll demonstrate how to utilize AWS Config history to obtain a detailed view of your AWS resource configurations, enabling policy retrieval. Finally, we’ll showcase a sample serverless solution that leverage Amazon EventBridge and the power of AWS Lambda and Amazon DynamoDB to let you easily backup and restore bucket policies.

Option 1: CloudTrail

AWS CloudTrail allows tracking of user and API activities across your AWS infrastructure to help you meet your operational and risk auditing, governance, and compliance requirements.

To retrieve a previous version of a bucket policy, we’re interested in the S3 PutBucketPolicy API call, which contains the bucket policy document applied to a bucket. This API call is recorded as a CloudTrail event.

As of today, there are three ways to query events recorded by CloudTrail:

• CloudTrail Event history, which stores events for 90 days
• CloudTrail Lake an AWS-managed CloudTrail query engine
• Amazon Athena, an interactive query service that makes it easy to retrieve CloudTrail events stored in Amazon S3 using standard SQL

The following comparison table shows the three solutions to query CloudTrail events.

By default, you can access bucket policies created in the last 90 days in the Event History Dashboard.

If you must query events older than 90 days, then these must have been persisted prior to your query. There are two options to store CloudTrail events: either you create a CloudTrail Trail, which stores events into an S3 bucket, or you create an event data store from the CloudTrail Lake managed service.

Next, let’s see an example of retrieving a previous bucket policy version using CloudTrail Event history feature.

Prerequisites

You only need to have access to the CloudTrail Event history to retrieve a bucket policy from the past 90 days.

Make sure that the AWS Identity and Access Management (IAM) user or IAM role performing the action has permissions as described in the following managed policy:

arn:aws:iam::aws:policy/AWSCloudTrail_ReadOnlyAccess

Solution

1. Go to the CloudTrail console and choose the AWS Region where your S3 bucket is.

2. Go to Event history.

3. Choose the Event name.

4. Enter the PutBucketPolicy as the lookup value and choose the time range where the previous bucket policy was set. Pick the time and the resources for which you’re looking.

5. Go to the Event record section and find the bucket policy under “requestParameters”.

Solution cost

There is no charge. More information on CloudTrail pricing can be found here.

Option 2: AWS Config

AWS Config is a fully managed service that provides you with an AWS resource inventory, configuration history, and configuration change notifications. It provides a detailed view of your AWS resource configurations.

This option is interesting for you if your bucket policy is more than 90 days old, and you can’t retrieve it with the option 1. AWS Config lets you define compliance rules and obtain a detailed report for each resource type. For example, you could verify if the encryption is enabled on your S3 buckets, a security group has an unrestricted IP/port configuration, or the S3 object versioning feature is enabled.

Prerequisites

To retrieve your bucket policies with AWS Config you must have AWS Config enabled for S3 bucket resources (see enabling AWS Config) prior to the update of the bucket policy.

Solution

1. Go to the AWS Config console, and select Resources.

2. Fill Resource identifier with the bucket name, and select the result.

3. Select Resource Timeline.

4. Choose Configuration events in the Event type dropdown list, and choose the appropriated Start date.

5. Finally, select the Configuration change event that you want and find the previous S3 bucket policy.

Solution cost

More information on AWS Config pricing can be found here.

Option 3: Sample S3 bucket policies backup and restore solution

The following serverless architecture offers you the possibility to have backup and restore capabilities for S3 bucket policies. An Amazon EventBridge rule captures the CloudTrail event PutBucketPolicy and triggers a Lambda function to persist the bucket policy into a DynamoDB table. Then, another Lambda function can easily be called to restore a previous policy.

Here’s the solution architecture diagram:

Amazon EventBridge is a serverless event bus that makes it easier to build event-driven applications at scale using events generated from your applications and AWS services. Within EventBridge you can define rules. A rule matches incoming events and sends them to targets for processing, for example a Lambda function. Rules are based either on an event pattern or a schedule.
The following event pattern is used in the EventBridge rule that triggers the Register BucketPolicy Lambda function.

{
  "detail-type": ["AWS API Call via CloudTrail"],
  "source": ["AWS.s3"],
  "detail": {
    "eventSource": ["s3.amazonAWS.com"],
    "eventName": ["PutBucketPolicy"]
  }
}

Once the event has been detected, the EventBridge rule triggers a Lambda function to persist the new bucket policy update data into a DynamoDB table.

The primary key of the DynamoDB table is the hash code (md5) of the bucket policy document. This will always be unique and is calculated by the Lambda function.

Unicity of the hash code: if a statement in the bucket policy presents values in the resource field different than the name of the bucket, then the Amazon S3 API throws an exception. Therefore, creating a hash code out of the bucket policy document always results in a unique value, and we can use it as the primary key of the DynamoDB table.

If the same bucket policy document is configured several times on the same bucket, then the RegisterBucketPolicy Lambda function updates the timestamp attribute of the existing DynamoDB table record, as well as the user identity attribute, which refers to the IAM entity that executed the API call.

The registering function persists the bucket policy in the DynamoDB table with the following attributes:

Note that bucket policies are limited to 20KB in size, which is less than the maximum record size of a DynamoDB table, 400KB.

Prerequisites

The following prerequisites are required for this section:

• You need to have deployed the solution to your AWS account prior to the update of the bucket policy. See this Github Repository and follow the instructions.
• You need the hash of the BucketPolicy that you wish to restore. You can find it in the DynamoDB table.
• You have the right IAM permissions to invoke a Lambda function.

Solution

1. To restore a previously configured bucket policy, you invoke the RestoreBucketPolicy Lambda function and pass the digest code of the bucket policy as the parameter. You can invoke the Lambda function, either through the AWS Management Console or the AWS Command Line Interface (AWS CLI).

{
"BucketPolicyHash": "48a4346945d1349981f96a9323d718f2"
}

2. If you want to invoke a Lambda function through the AWS CLI, then you can execute the following command:

aws lambda invoke --function-name RestoreBucketPolicyFunction --payload '{ "BucketPolicyHash": " 48a4346945d1349981f96a9323 d718f2" }' response.json

Solution cost

AWS resources are billed only for the usage duration. For example, Lambda functions are billed for the execution duration and the allocated memory. Check out the Lambda, DynamoDB, and EventBridge pricing pages.

Conclusion

In this blog, we provided a comprehensive guide on how to retrieve previous versions of S3 bucket policies. First, we explored the ability to seek and retrieve past API calls via AWS CloudTrail. Then we used the configuration history provided by AWS Config to retrieve past versions of S3 bucket policies. Finally, we showed a sample serverless solution to version S3 bucket policies.

CloudTrail is the go-to option for reviewing bucket policy configuration change for the last 90 days, at no cost. For changes older than 90 days, take advantage of CloudTrail’s long-term storage solutions or leverage AWS Config’s detailed view of your AWS resources configuration history. Use the sample serverless solution as an efficient way to backup and restore bucket policies using AWS Lambda and Amazon DynamoDB.

With the solutions provided, you can identify issues with new configurations, comply with regulations, and review previous configurations. Additionally, the sample serverless solution provides an easy way to backup and restore bucket policies, which can save time and effort.

To learn more about the services and feature presented in this blog post, please check out the following resources.

Additional resources

• CloudTrail Lake announcement blog
• Create a CloudTrail Trail
• Getting start with AWS Config