Automatically scan for public Amazon S3 buckets and block public access

Data is a valuable asset for an organization and users are always looking for simple tools to protect their data from unauthorized access. While some use cases do require data to be publicly accessible at times, most enterprise use cases and data privacy depend on strictly managed permissions and no public access.

Enterprises use Amazon S3 object storage for a wide variety of use cases, such as cloud applications, dynamic websites, content distribution, mobile, and gaming applications. As Amazon S3 becomes an organization’s multi-faceted and centralized datastore, it’s crucial to keep data private unless public access is absolutely necessary. By default, new S3 buckets have S3 Block Public Access enabled at the bucket level, access control lists (ACLs) disabled, and all new objects encrypted. To make sure that public access to your S3 buckets and objects is blocked, you can turn on S3 Block Public Access at the account level. Although these features create a strong security posture, inadvertent or mistaken actions, users with overly permissive access, or even malicious activity may result in an S3 bucket created with S3 Block Public Access disabled or an existing bucket having S3 Block Public Access disabled when that is not desired.

In this post, we walk you through a proactive, event-driven solution that detects public S3 buckets using AWS Security Hub then automatically enables S3 Block Public Access at the bucket level. In certain cases, public access to an S3 bucket may be necessary, and in those scenarios we recommend you tag those S3 buckets (for example, tag_key = “bucket.status” & tag_value = “public.bucket”). This solution is built on an event-driven architecture that is cost effective and easy to deploy, and it can improve the overall security posture and operational reliability of your organization by ensuring buckets that are meant to be private stay private.

Solution overview

The following diagram illustrates the different components of this event-driven architecture.

At a high level, the solution works as follows:

1. AWS Security Hub checks the S3 bucket compliance and security.

2. Findings are sent to Amazon EventBridge.

3. EventBridge invokes two targets:

a. An AWS Lambda function that includes logic to block public access to S3 buckets based on specific resource tags.

b. An Amazon CloudWatch log group triggers a CloudWatch alarm, which sends a notification to an Amazon SNS topic.

4. The SNS topic then generates an email notification that a public S3 bucket was detected.

Prerequisites

For this walkthrough, you need the following to get started:

• AWS account
• S3 bucket
• AWS Lambda execution IAM role
• An S3 bucket policy that provides the Lambda function access to update S3 Block Public Access settings

Walkthrough

The following sections walk you through the solution:

1. Configuring Security Hub

2. Set up AWS Lambda function

3. Configuring EventBridge rule with targets

4. Setting up CloudWatch metrics and alarms

1) Configuring Security Hub

In a previous post, “Find public Amazon S3 buckets in your AWS account,” we walked you through the different AWS services that you can use to detect public S3 buckets across different AWS Regions in your organization. One of those was Security Hub, a cloud security posture management (CSPM) service that performs security best practice checks, aggregates alerts, and enables automated remediation.

You can use Security Hub to monitor public S3 buckets by using the controls of the AWS Foundational Security Best Practices (FSBP) standard. Security Hub uses service-linked AWS Config rules to perform security checks for most controls. To get started, follow these steps:

1. Enable AWS Config on all accounts in each AWS Region.

2. Enable Security Hub by integrating with AWS Organizations or manually.

3. Enable the AWS FSBP standard in Security Hub.

For this post, we are using the FSBP security control [S3.8] S3 general purpose buckets should block public access. This control checks whether an S3 general purpose bucket blocks public access at the bucket level. The following image is a Security Hub finding of a public S3 bucket.

Security Hub automatically sends new findings and updates to existing findings to EventBridge as Security Hub Findings – Imported events. This means that every time Security Hub detects a public S3 bucket during continuous monitoring, an event is sent to EventBridge.

2) Set up AWS Lambda function

Lambda is a serverless compute service for running code without provisioning a server. In this solution, an EventBridge event invokes a Lambda function (“BlockPublicS3Bucket”) to block public access of the S3 bucket. This function does the following:

1. Identifies the S3 bucket name using the JSONPath expression.

2. Iterates through the S3 bucket tags and checks if a specific tag key-value pair is present. The following is the sample tag key-value pair used for the purpose of this post:

bucket_tag_key = "bucket.status"
bucket_tag_value = "public.bucket"

3. If the tag is not found, then the function proceeds to block the public access configuration for the bucket using the put_public_access_block API call.

4. If the tag is found, then the function skips the public access block and logs a message.

This way, the function only blocks public access if the specified tag is not present on the S3 bucket. The following are the prerequisites for the Lambda function:

• A Lambda execution IAM role that grants permission to access AWS services and resources.
• An S3 bucket policy that provides the Lambda function access to update the S3 Block Public Access settings. The following is a sample policy.

{
    "Version": "2012-10-17",
    "Id": "BlockPublicAccess",
    "Statement": [
        {
            "Sid": "LambdaAccess",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::123456789012:role/service-role/BlockPublicS3Bucket-role-123"
            },
            "Action": [
                "s3:GetBucketTagging", 
                "s3:PutBucketPublicAccessBlock"
            ],
            "Resource": "arn:aws:s3:::my-bucket"
        }
    ]
}

You can find the Lambda function in this AWS sample.

3) Configuring EventBridge rule with targets

EventBridge is a serverless service for building scalable event-driven applications. Using EventBridge, you can monitor and audit your AWS environments, and respond to operational changes in your applications in near real-time to prevent infrastructure vulnerabilities. An EventBridge rule watches for specific types of events. When a matching event occurs, the event is routed to the targets associated with the rule. Rules use event patterns to select events and send them to targets, and you can associate a rule with one or more targets. In this section, I’ll walk through creating an EventBridge rule and its two associated targets.

1. Create an EventBridge rule to monitor the public S3 bucket event generated by Security Hub. Refer to the documentation on creating rules that react to events. You can find a sample Security Hub event here.

2. Select Rule with an event pattern. Use the following event pattern to match the public S3 bucket event generated by Security Hub.

{
  "source": ["aws.securityhub"],
  "detail-type": ["Security Hub Findings - Imported"],
  "detail": {
    "findings": {
      "Compliance": {
        "Status": ["FAILED"],
        "SecurityControlId": ["S3.8"]
      },
      "RecordState": ["ACTIVE"],
      "Workflow": {
        "Status": ["NEW"]
      }
    }
  }
}

3. Add the Lambda function for this solution and a CloudWatch log group as targets (as shown in screenshot following the code block):

a. Add the Lambda function “BlockPublicS3Bucket” from the preceding section.

b. Add a CloudWatch log group as a target. CloudWatch enables you to monitor your complete stack and use logs, metrics, and alarms to take automated actions and reduce mean time to resolution. To deliver the event data to the target log group, EventBridge needs permission to access the target log group. The following example policy document demonstrates the permissions you must define in the resource-based policy of your log group. For further instructions, refer to CloudWatch logs permissions.

{
  "Version": "2012-10-17",
  "Id": "CloudWatchLogforEventBridge 
  "Statement": [
    {
      "Action": [
        "logs:CreateLogStream",
        "logs:PutLogEvents"
      ],
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "events.amazonaws.com",
          "delivery.logs.amazonaws.com"
        ]
      },
      "Resource": "arn:aws:logs:<region>:<123456789012>:log-group:/aws/events/*:*",
      "Sid": "TrustEventsToStoreLogEvent"
    }
  ]
}

4. Configure tags, review the rule, and choose Create rule.

4. Setting up CloudWatch metrics and alarms

Once you have an event sent to your CloudWatch log group, configure CloudWatch metrics and alarms.

1. Create a CloudWatch metric filter to capture the event, [S3.8] S3 general purpose buckets should block public access, and turn it into a numerical CloudWatch metric. You can use metric filters to transform log data into actionable metrics. Filter patterns only return the log events that contain the terms you define. It’s a cost-effective way to create an alarm or route log events to other AWS services. To create a metric filter, refer to the documentation on creating a metric filter for a log group.

2. Create a CloudWatch alarm using the custom metric. For instructions, refer to the documentation with examples on creating CloudWatch alarms for CloudTrail events. On the Configure actions page, choose Notification, and then choose In alarm, which indicates that an action to send a notification to an SNS topic is taken when a threshold is breached. Use Amazon SNS to send an email notification to subscribing endpoints or clients when the CloudWatch alarm you created changes its state. To create an SNS topic, follow the steps in the documentation on creating an Amazon SNS topic.

Cleaning up

To avoid ongoing charges in your AWS account, be sure to clean up any resources you created for the blog post. Delete the Lambda function, S3 bucket, CloudWatch alarm, SNS topic, and remove any other AWS services you provisioned so you don’t incur unwanted charges.

Conclusion

In this post, we walked you through a proactive, event-driven approach to detect public S3 buckets using AWS Security Hub then automatically enable S3 Block Public Access at the bucket level. Security Hub checks the S3 bucket compliance and security and sends findings to Amazon EventBridge. EventBridge then invokes an AWS Lambda function that blocks public access to S3 buckets based on specific resource tags and an Amazon CloudWatch log group that triggers an alarm notifying that a public S3 bucket was detected.

You should design and implement automatic remediation workflows with caution, and thorough testing is recommended to avoid unintended consequences or disruptions to your application. Unless you intend to have your S3 buckets publicly accessible, you should enable bucket- and account-level Amazon S3 Block Public Access. Securing data from unintended access is crucial, and the solution featured in this post can help you make sure you aren’t unintentionally exposing or sharing data publicly. Make sure to check out the respective pricing pages for each service option mentioned in this post before getting started.

For more information, visit the following resources;

• AWS Well-Architected Framework
• Architecture Best Practices for Storage
• Security best practices for Amazon S3