Deploy Datadog’s AWS Integration into new accounts using AWS Control Tower Account Factory Customization

Multi-account AWS environments are an important best practice for meeting regulatory and compliance needs and achieving resource isolation. AWS Control Tower establishes a well-architected, multi-account baseline and continues to offer customers the ability to offload manual processes and customize their accounts to suit their specific needs. See Getting Started with AWS Control Tower for more information about the service.

Datadog is a member of the AWS Partner Network (APN) and is available in AWS Marketplace. As a unified monitoring and security platform for cloud applications, Datadog brings together end-to-end traces, metrics, and logs for centralized observability and troubleshooting on dynamic architectures. It brings together telemetry data from across your technology stack and enables you to monitor the health and performance of your AWS-hosted infrastructure and applications at scale.

In this post, Ryan and I show you how to deploy Datadog’s AWS integration with AWS Control Tower using a predefined Datadog AWS Integration blueprint. With Account Factory Customization, you can quickly and easily configure your accounts to meet your business needs. In addition, this integration enables Datadog to start automatically monitoring the health and performance of the AWS resources in your newly provisioned AWS account without additional manual configuration.

Prerequisites

You must complete the following prerequisites before implementing the Datadog and AWS Control Tower Account Factory Customization integration solution:

1. Create a new Datadog account or sign in to your existing account.
2. Follow these steps to create a Datadog API and Application Key, which you’ll use in step B.4.
3. Set up AWS Control Tower and create or use an existing landing zone.

Solution overview

Datadog’s integration with AWS Control Tower Account Factory Customization enables you to automatically set up Datadog monitoring of your AWS accounts upon account provisioning. This solution performs the following steps:

1. Deploys the Datadog AWS integration CloudFormation template in a new account managed by AWS Control Tower.
2. Automatically creates the necessary IAM role and policies.
3. Automatically registers the account in Datadog and initiates ingestion of AWS CloudWatch metrics and events from the AWS resources in your account.
4. Optionally sets up a Lambda function for sending logs to Datadog.
5. Optionally configures Datadog Cloud Security Management to monitor resource misconfigurations in your AWS account.

Architecture diagram

The following architecture diagram illustrates the components of AWS Control Tower and Datadog integration.

1. The management account deploys the Datadog AWS Integration blueprint (CloudFormation stack) in the home Region of a newly managed account using Account Factory Customization. Refer to the following diagram.

2. A centralized account stores the Datadog credentials used for configuring the integration for the newly managed account in AWS Secrets Manager. Refer to the following diagram.

3. In the newly managed account, the Datadog AWS Integration CloudFormation stack creates the IAM role used by Datadog to fetch metrics and events. It also adds the new AWS account details to Datadog and optionally sets up a Lambda function that can be used for pushing logs from AWS services to Datadog. Refer to the following diagram.

Solution walkthrough: Deploy Datadog’s AWS integration to new accounts in AWS Control Tower

A. Setting up your Datadog AWS integration

1. Navigate to the AWS Service Catalog console and sign in.
2. In the AWS Service Catalog console left side bar, under Administration, choose the Getting Started Library. To show the relevant AWS managed blueprint templates, filter the current product list by entering Control Tower Blueprints in the Search Products search bar.
3. Select Datadog AWS Integration blueprint from the list. This opens a new tab with the blueprint’s product details and description.
4. In the top right corner, select Add to portfolio.
5. If you have an existing AWS Service Catalog portfolio you want to use, choose Select Existing Portfolio. Alternatively, you can enroll the template into a new portfolio by selecting Add to new portfolio. This is a one-time operation and does not need to be done for each new account you provision.

B. Adding new accounts to your AWS Control Tower environment

1. On the AWS Control Tower console left sidebar, choose Account factory. Choose Create a new account.
2. On the Create account form, enter the account details, access configuration, organizational unit, and other details.
3. Under the Account Factory Customization section, select the Datadog AWS Integration product.
4. Using the information from Prerequisite step 2, configure the required template parameters by entering your Datadog API Key, App Key, and Site. Alternatively, you can supply an AWS Resource Number (ARN) for the Datadog credentials stored in AWS Secrets Manager. Optionally adjust the parameters to include or exclude setting up metrics collection, the Lambda for sending logs to Datadog, and Datadog’s Cloud Security Management product.
5. Under the Deployment Regions section, select Home Region for deployment and then select Create account. Once the template deploys successfully, an account is created with the Account Factory baseline and Datadog AWS Integration blueprint. AWS Control Tower makes API calls to provision the product directly from the Account Factory console.
6. After the CloudFormation stack is successfully created, wait 10 minutes for data to be collected, and then you can sign in to Datadog to start monitoring any resources in your AWS account. Datadog will automatically start monitoring and collecting events and Amazon CloudWatch metrics for the AWS services you are using.

C. Monitoring your accounts with Datadog

To create an EC2 dashboard in Datadog, do the following:

1. Log in to Datadog. From the left sidebar, select
2. In the search box, enter EC2, and select the pre-built AWS EC2 Overview dashboard.

The following screenshot of the AWS EC2 Overview dashboard shows a sample overview Datadog dashboard for Amazon EC2, one of over 90 AWS integrations Datadog supports. The left pane shows EC2 events and a host map. The middle pane has an overview of active instances by type showing 92 t2.micro, 54 m5a.2xlarge and other instances. It also shows EC2 alerts and has seven alerts showing. The right pane has a network section, showing network in by instance type, network out by instance type, and total network in and out. The I/O section shows disk reads by instance type and bytes read. Refer to the following screenshot.

After setting up the Datadog AWS integration blueprint in AWS Control Tower Account Factory, this dashboard will be populated with CloudWatch metrics and maintenance or state-change events related to the EC2 instances in your account.

Cleanup

To remove resources created during this tutorial, do the following:

1. Navigate to the AWS CloudFormation console and choose the Stacks page. Select the CloudFormation stack for Datadog’s AWS Integration that you want to delete in the child AWS account.
2. In the Stack Details pane, choose Delete.
3. When prompted, select Delete stack.

Conclusion

In this blog post, we showed you how to deploy the Datadog AWS Integration blueprint in newly provisioned AWS accounts using AWS Control Tower Account Factory Customization. Instead of manually configuring Datadog’s AWS integration for each new account, you can now automatically deploy the integration during account creation with Account Factory. Once deployed, Datadog immediately starts monitoring your newly added AWS account, giving you comprehensive visibility into your AWS services.

The content and opinions in this post are those of the third-party authors, and AWS is not responsible for the content or accuracy of this post.

About the authors

Fionce Siow is a Product Marketing Manager at Datadog, owning the go-to-market strategy for AWS integrations and Datadog adoption among AWS customers. In her spare time, she enjoys catching the latest film releases and brewing her own kombucha.

.

.

.

.
Ryan Warrier is a Senior Product Manager at Datadog working on AWS Integrations. In his spare time, he enjoys trying the best new local restaurants, playing golf, and occasionally scuba diving.