AWS Cloud Operations Blog

Detect and respond to security threats in near real-time using Amazon Managed Grafana

Security is “job zero” at AWS. It’s crucial to gain deeper insights into your AWS infrastructure’s security posture to respond quickly to threats. The ability to centrally monitor and visualize the security findings make it easier for you to identify any security threats or gaps and also keep the principle of least privilege in focus. You can visualize the insights into those security findings across multiple accounts using Amazon Managed Grafana that retrieves and refreshes the data periodically.

AWS Security Hub offers a comprehensive overview of your AWS security status, allowing you to compare it with industry standards and best practices. Organizations may have additional requirements to centralize AWS Security Hub findings and integrate them with other operational data. Some of the benefits include consolidating AWS Security Hub findings across regions into a single dashboard view, and centralizing and correlating various security & compliance data along with operational data into one dashboard.

In this post we will show you how you can centralize your AWS Security Hub findings, that provide a single-pane-of-glass on workloads running on AWS cloud using Amazon Managed Grafana.

We will demonstrate similar integration with Amazon GuardDuty and Amazon Inspector through another post.

Architecture Overview:

Figure 1: Architecture Overview

Please refer to the details on components of the solution depicted in the above architecture:

  1. AWS Security Hub – AWS Security Hub is a cloud security posture management service that performs security best practices checks, aggregates alerts, and enables automated remediation.
  2. Amazon EventBridge Rule – Amazon EventBridge rule will be triggered on a new finding or an update to existing finding in AWS Security Hub.
  3. Custom AWS Lambda to process the Event – An AWS Lambda function to extract and transform the complex nested AWS Security Hub finding to a simpler JSON format that is more suitable to perform further analytics.
  4. Amazon Simple Storage Service (Amazon S3) bucket to receive the processed event – The AWS Lambda function delivers each finding to this Amazon S3 bucket.
  5. AWS Glue crawler and AWS Glue Data Catalog – Configure an AWS Glue crawler to run every hour to parse the JSON data stored in Amazon S3 and update the AWS Glue Data Catalog containing the JSON table schema, metadata, and database partitions. This metadata is required to perform queries against the data in Amazon S3.
  6. Amazon Athena Workgroup – Amazon Athena is used to look at the schema from the AWS Glue Data Catalog and query the data in S3. The Amazon Managed Grafana Dashboard must have permissions to execute queries against the Amazon S3 data using Amazon Athena Workgroup and AWS Glue Data Catalog.
  7. Amazon Managed Grafana – Deploy a dashboard using Amazon Managed Grafana to visualize data.

Using the AWS CloudFormation template provided in this blog, deploy the required resources in the same AWS Region and AWS account containing the Security Hub service where findings of all regions and accounts are aggregated into.

Prerequisites:

  1. Enable AWS Security Hub in member accounts. Enable required security standards, AWS native service integration, and partner integrations in all the member accounts across your AWS Regions.
  2. Set up your AWS Security Hub administrator account. Designate one of the AWS accounts within your AWS Organizations to be a delegated administrator for AWS Security Hub. This account can manage and receive findings across member accounts.
  3. Enable Cross Region Aggregation.
  4. Set up Amazon Athena workgroups.
  5. Set up Amazon Managed Grafana workspace. For information, and steps for creating the Amazon Managed Grafana workspace, see Creating a Workspace.

Once you have all the prerequisites in place, follow the instructions below for visualizing AWS Security Hub findings on Amazon Managed Grafana.

Deployment Steps:

Step 1: Launch the AWS CloudFormation template

Download and launch this AWS CloudFormation template to deploy AWS Lambda, Amazon S3 Bucket, AWS Glue Crawler, AWS Glue Database and its related components.

Note: Some of the resources that this stack deploys incur costs when in use.

Follow these steps to generate your resources utilizing an AWS CloudFormation template:

  1. Sign in to the AWS Management Console.
  2. Navigate to the AWS CloudFormation console > Create Stack > “With new resources”.
  3. Upload the yaml template file and choose Next.
  4. Specify a “Stack name” and choose Next.
  5. Leave the “Configure stack options” at default values and choose Next.
  6. Review the details on the final screen and under “Capabilities” check the box for “I acknowledge that AWS CloudFormation might create IAM resources with custom names”.
  7. Choose Submit.


Figure 2: Acknowledgement

Note: You can review the progress of your new stack under AWS CloudFormation > Stacks > [StackName] > Events tab

Once the Stack is created successfully, you will see the following resources deployed: Amazon EventBridge scheduler, AWS Lambda Function, Amazon S3 Bucket, AWS Glue Crawler, AWS Glue Database and the corresponding AWS IAM Roles and Policies are created successfully.

Step 2: Create View in Amazon Athena using the below queries created as part of the AWS CloudFormation stack

  1. Go to Amazon Athena > Query editor > Saved queries tab and choose the query named “AWS-securityhub”.
    Note: Workgroup created is named “Primary”

Figure 3: Amazon Athena Saved Queries

  1. On the Query editor, verify the Data source, Database and Table names while running the query. Upon successful execution, the query creates a View named “security_hub_findings”.

Figure 4: Amazon Athena Query Editor

Step 3: Configure Amazon Athena Data Source in Amazon Managed Grafana

  1. Access the Amazon Managed Grafana console through the provided Amazon Managed Grafana workspace URL and log in using the user credentials you’ve set up.
  2. Navigate to Administration > Data sources and select Amazon Athena from the options.
  3. Adjust the Amazon Athena settings by specifying the Default Region (us-east-1), Data source (AWSDataCatalog), Database (aws-security-hub-db), Workgroup (primary), and set the Output Location for your Amazon Athena query.
  4. Choose “Save & test” to confirm that the data source is functioning properly. You can now begin querying and visualizing metrics from the AWS environment.
    Note: If you encounter a permission denied error, ensure that the Amazon Managed Grafana service role permissions, as discussed in the previous step, are correctly configured.

Figure 5: Amazon Athena as Data source

Step 4: Create an Amazon Managed Grafana Dashboard

Amazon Managed Grafana is a fully managed service designed to simplify the process of creating, configuring, and sharing interactive dashboards and charts for monitoring your data. It offers the ability to establish alerts and notifications based on specific conditions or thresholds, enabling swift identification and response to issues.

In this next step, we will utilize Amazon Managed Grafana to generate a new AWS Security Hub findings detection dashboard.

  1. Retrieve the AWS Security Hub findings dashboard JSON file from this link.
  2. Import the dashboard by navigating to Dashboards > New and selecting Import in the Amazon Managed Grafana console. For additional information on exporting and importing dashboards, refer to the documentation.

Figure 6: Amazon Managed Grafana Dashboard

Finally, AWS Security Hub findings are integrated into Amazon Managed Grafana. This dashboard updates every 5 minutes, querying the materialized views established in Amazon Athena.

Furthermore, Amazon Managed Grafana’s alerting system delivers strong and actionable alerts, enabling us to swiftly identify system issues as soon as they arise. For further insights into Amazon Managed Grafana alerting, please visit the “Alerts in Amazon Managed Grafana” section.

Clean up

To avoid incurring future charges, delete all resources used in this post.

In this blog post, we showed how you can visualize and analyze your AWS Security Hub with Amazon Managed Grafana. By identifying potential threats quickly, sensitive data can be safeguarded more effectively. Near real-time dashboards allow for proactive measures, ensuring that critical workload remains secure.

To learn more and get hands-on experience on AWS observability services, check the One Observability Workshop.

About the authors:

Sameeksha Garg

Sameeksha is a Technical Account Manager at AWS committed to accelerate the cloud journey for AWS Global Enterprise customers. She has 7+ years of industry experience across cloud security, cloud operations, cloud infrastructure management and customer advocacy. She is passionate about cloud security technologies and strives to help customers secure their workloads in the cloud.

Anjali Sharma

Anjali Sharma is a Technical Account Manager (TAM) at AWS with more than 7 years of IT experience. Her diverse career includes roles such as Cloud Consultant and Operations Engineer at AWS Managed Services. In her current position, she collaborates with global customers to develop sustainable software solutions. She has a passion for troubleshooting and enhancing operational excellence for her customers.

Yash Bindlish

Yash is an Enterprise Support Manager at Amazon Web Services. He has more than 17 years of industry experience including roles in cloud architecture, systems engineering, and infrastructure. He works with Global Enterprise customers and help them build, scalable, modern and cost effective solutions on their growth journey with AWS. He loves solving complex problems with his solution-oriented approach.

Scott Kellish

Scott Kellish is a Sr Partner Solutions Architect / WW Tech Lead for Cloud Operations., with 8+ years at AWS, working with AWS Partners and customers on all things related to cloud operations. Outside of work, he enjoys working outside, exploring new places by bicycle with his wife and sitting on a beach in Nantucket being thankful for life.