AWS Cloud Operations Blog

Using AWS CloudTrail data events to audit your Amazon SNS and Amazon SQS workloads

Customers in highly regulated industries, such as Financial Services or Healthcare and Life Sciences, often need to audit every action made in environments with sensitive data. Regulations like HIPAA or FFIEC, and industry frameworks like the PCI DSS, require granular log entries that record user and administrative actions within an environment containing sensitive data, and especially interactions with that data. AWS CloudTrail helps by providing you a record of actions taken by a user, role, or an AWS service in your account. With CloudTrail you can extend that functionality with the capability to also record data events, which are events that provide additional information about the resource operations performed on or in a resource.

CloudTrail helps by providing added observability and supports data events for a wide variety of services. These data events can be used to help you meet your critical compliance, risk, and security objectives. Some examples of these type of events include object-level API activities such as delete, update, and put items. Examples of the enhanced visibility provided by CloudTrail data events include API activity on an agent alias or knowledge base in Amazon Bedrock, activity on an application or data source in Amazon Q Business, or Sagemaker API activity on a feature store. These deliver key risk management benefits such as:

  • Monitoring access to personal data and sensitive information
  • Visibility into personal data and sensitive data modifications
  • Auditing activities in applications that handle personal data and sensitive information
  • Detecting potential data breaches and privacy incidents
  • Facilitating privacy audits and compliance reporting

Regulations and compliance frameworks worldwide call for strict controls and auditing capabilities for access to personal data and sensitive data. CloudTrail data events can help support these needs. Some examples include:

  • Privacy regulations such as the General Data Protection Regulation (GDPR) and California Consumer Privacy Act (CCPA) require organizations to implement strict controls and auditing capabilities for access to and handling of personal data and sensitive information.
  • The Federal Information Security Management Act (FISMA) requires federal agencies, departments, and contractors to adequately safeguard information systems and assets. The underlying requirements for systems that handle government data come from NIST Special Publication (SP) 800-53. Data event granular activity logging is needed for requirements that include, but are not limited to, AU-2, AU-3, AU-6, IR-5, and IR-6.
  • Using these data events to capture item-level activity supports the HIPAA technical safeguard §164.312(b), Audit Controls, by capturing detailed information about data access, modification, and deletion activities of electronic protected health information (ePHI).
  • For workloads containing credit card information and in scope for the Payment Card Industry Data Security Standard (PCI DSS), CloudTrail data events are needed to satisfy Requirement 10.2.1.1 and capture object-level activity of interactions with an Amazon S3 object that contains PCI data.

In this blog we will highlight some of the business and regulatory requirements these data events support, example events for Amazon Simple Notification Service (Amazon SNS) and Amazon Simple Queue Service (SQS), and how you can use them to audit your workloads.

According to the Amazon SNS security best practices and Amazon SQS security best practices it recommends to consider using VPC endpoints to access Amazon SNS and Amazon SQS. For example, if you have Amazon SNS topics or Amazon SQS queues that you must be able to interact with, but which must absolutely not be exposed to the internet, using VPC endpoints to control access to only the hosts within a particular VPC to publish or send messages to an Amazon SNS topic or Amazon SQS queue.

In the following solution, we will show how you can leverage CloudTrail data events to provide evidence that API activity related to publishing or sending messages to an Amazon SNS topic or Amazon SQS queue are using a VPC endpoint.

Prerequisites

The following sample queries used in this blog for CloudTrail data events requires that you have an existing CloudTrail Lake event data store and Amazon SQS and Amazon SNS resources.

Using CloudTrail Data Events for Amazon SNS

Amazon SNS can support messaging use cases in regulated market segments. Enabling data events for Amazon SNS in CloudTrail you will be able to audit Publish and PublishBatch API actions for all your Amazon SNS topics.

In this sample scenario we will be showing how to use data events for Amazon SNS to audit VPC Endpoints being used when publishing messages to Amazon SNS topics from instances within your VPC, without traversing the internet. For additional information in setting this up, look at the documentation for publishing an Amazon SNS message from Amazon VPC.

  1. Navigate to the CloudTrail console.
  2. In the left-hand navigation menu, choose Lake.
  3. Choose the Event Data Stores.
  4. Choose your Event Data Store for CloudTrail events.
  5. Under Data events, choose Edit.
  6. Choose Add data event type under Data events.
  7. Select SNS topic for Data event type.
  8. Choose Custom under Log selector template.
  9. Choose resources.ARN for the ‘Field’ under the Advanced event selectors section.
  10. Select starts with for the Operator.
  11. In the Value field, enter the ARN of your Amazon SNS topic.
Data event screen displaying the advanced event selectors.

Figure 1: AWS CloudTrail data events

  1. Choose Save changes.

CloudTrail Lake will start to record all data events for the Amazon SNS topic we added to the event selector.

Using CloudTrail Lake to query Data Events for Amazon SNS topics

Next, we will use CloudTrail Lake, to query data events for Amazon SNS topics API actions within a specific time period (in this case, 1 day).

  1. Navigate to the CloudTrail console.
  2. In the left-hand navigation menu, choose Lake.
  3. Choose Query.
  4. Copy the below query and paste it into your editor window. (Note: you must replace $EDS_ID with the id of your event data store. Also, you can adjust the eventtime for the time range you would like to query)

SQL Query:

SELECT eventTime,
	substr(userIdentity.arn, strpos(userIdentity.arn, '/') +1) as IAM, 
	recipientAccountId, 
	awsRegion, 
	eventName,sourceIPAddress, 
	substr(element_At(requestParameters, 'topicArn'), 
	strpos(element_At(requestParameters, 'topicArn'), '.com/') +18) as Topic, 
	vpcEndpointId
FROM $EDS_ID
	WHERE eventSource = 'sns.amazonaws.com'
	AND eventName = 'Publish'
	AND eventtime >= '2024-06-24 00:00:00'
	AND eventtime <= '2024-06-24 23:59:59'
  1. Click Run and then your results will show under Query Results.
Query results screen displaying the results of the query.

Figure 2: AWS CloudTrail query results

The query results show API actions for the Publish events from Amazon SNS, indicating messages where being published to an Amazon SNS topic. The results will also show which IAM entity took this action and the specific VPC endpoint ID the message was sent. However, if there is no VPC endpoint ID then the Publish API call was made without using a VPC Endpoint. This provides evidence for your compliance requirement that your Amazon SNS traffic use your VPC endpoint.

Using CloudTrail Data Events for Amazon SQS

Amazon SQS can also support the same regulated market use case as Amazon SNS. By enabling data events for Amazon SQS in CloudTrail you will be able to audit all API related activity for data plane level events for Amazon SQS, similar to Amazon SNS.

In this sample scenario we will be showing how to use data events for Amazon SQS to ensure that the SendMessage API action will audit that VPC Endpoints are being used when sending messages to Amazon SQS queues from instances within your VPC, without traversing the internet. For additional information in setting this up, take a look at the documentation for Sending a message to an Amazon SQS queue from Amazon Virtual Private Cloud.

  1. Navigate to the CloudTrail console.
  2. In the left-hand navigation menu, choose Lake.
  3. Choose the Event Data Stores.
  4. Choose your Event Data Store for CloudTrail events.
  5. Under Data events, choose Edit.
  6. Choose Add data event type under Data events.
  7. Select SQS for Data event type.
  8. Choose Custom under Log selector template.
  9. Choose resources.ARN for the Field under the Advanced event selectors section.
  10. Select starts with for the Operator.
  11. In the Value field, enter the ARN of your Amazon SQS queue.
Data event screen displaying the advanced event selectors.

Figure 3: AWS CloudTrail data event

  1. Choose Save changes.

CloudTrail Lake will then start to record all data events for the Amazon SQS queue we added to the event selector.

Using CloudTrail Lake to query Data Events for Amazon SNS topics

Next, we will use CloudTrail Lake, to query data events for Amazon SQS API actions within a specific time period (in this case, 1 day).

  1. Navigate to the CloudTrail console.
  2. In the left-hand navigation menu, choose Lake.
  3. Choose Query.
  4. Copy the below query and paste it into your editor window. (Note: you must replace $EDS_ID with the id of your event data store. Also, you can adjust the eventtime for the time range you would like to query)

SQL Query:

SELECT eventTime, 
	substr(userIdentity.arn, strpos(userIdentity.arn, '/') +1) as IAM, 
	recipientAccountId, 
	awsRegion, 
	eventName,sourceIPAddress, 
	substr(element_At(requestParameters, 'queueUrl'), 
	strpos(element_At(requestParameters, 'queueUrl'), '.com/') +18) as Queue, 
	vpcEndpointId
FROM $EDS_ID
WHERE eventSource = 'sqs.amazonaws.com'
	AND eventName = 'SendMessage'
	AND eventtime >= '2024-06-24 00:00:00'
	AND eventtime <= '2024-06-24 23:59:59'
  1. Click Run and then your results will show under Query Results.
Query results screen displaying the results of the query.

Figure 4: AWS CloudTrail query results

Similarly as what was shown for the Amazon SNS query, the results will show API actions for the SendMessage events from Amazon SQS, indicating messages were being sent to a specific Amazon SQS queue. These results will also show which IAM entity took the action and the specific VPC endpoint ID to which the message was sent. This provides evidence for your compliance requirement that your Amazon SQS traffic use your VPC endpoint.

Cleanup

To prevent incurring additional charges, remove the Amazon SNS and Amazon SQS data events that were created during this walkthrough.

Conclusion

This post demonstrates how to use CloudTrail data events for Amazon SNS and Amazon SQS workloads to help provide an audit trail for resources that have specific compliance requirements. Then, we demonstrated how to leverage CloudTrail Lake to run query for specific actions related to data events to provide evidence of your compliance requirements. To learn more about how you can use CloudTrail data events for other type of workloads, please look at some of our other blogs:

Auditing generative AI workloads with AWS CloudTrail

Using CloudTrail data events with Athena and CloudWatch to create an audit trail for DynamoDB tables events

About the authors

Isaiah Salinas

Isaiah Salinas is a Senior Specialist Solution Architect with the Cloud Operations Team. With over 10 years of experience working with AWS technology, Isaiah works with customers to design, implement, and support complex cloud infrastructures. He also enjoys talking with others about how to use AWS services to provide solutions to their problems.

Brad Gilomen

Brad is a Principal Product Specialist for AWS CloudOps, focused on AWS Config and CloudTrail. His experience includes leading Federal Financial Enterprise Support at AWS for three years before becoming a Product Specialist in 2020. He currently works with our customers to reduce risk while staying secure and compliant while operating in the cloud.

Ted Tanner

Ted is a Principal Assurance Consultant and GRC SME with AWS Security Assurance Services, and has more than 25 years of IT, security, and compliance experience. He leverages this to provide AWS customers with guidance on compliance and security in the cloud, and how to build and optimize their cloud compliance programs. He is author and co-author of several whitepapers and guides to manage compliance in the AWS cloud.