Manage your AWS DMS endpoint credentials with AWS Secrets Manager

When configuring AWS Database Migration Service (AWS DMS) endpoints, you previously had to maintain the source and target credentials in plain text. In December 2020, we announced the integration of AWS DMS and AWS Secrets Manager, which allows you to take advantage of the built-in credential management capability in Secrets Manager to manage, retrieve, and rotate AWS DMS endpoint credentials.

In this post, we provide an end-to-end overview of managing AWS DMS endpoint credentials with Secrets Manager for an Oracle database running in conjunction with Oracle Automatic Storage Management (ASM) for file storage.

Prerequisites

For this walkthrough, you should have the following prerequisites:

An AWS account. If you don’t have an account, you can create and activate one.
An on-premises Oracle database that also utilizes Oracle ASM for file storage.
Access to AWS Identity and Access Management (IAM) to access or create roles.
Access to Secrets Manager to create or access secrets.

Solution overview

To authenticate to an Oracle Database and Oracle ASM utilizing Secrets Manager, you need to provide the following:

SecretsManagerSecretId – The full Amazon Resource Name (ARN), partial ARN, or friendly name of a secret that you created for Oracle Database access in Secrets Manager
SecretsManagerAccessRoleArn – The ARN of a secret access role that you created in IAM to provide AWS DMS access to the SecretsManagerSecretId secret on your behalf
SecretsManagerOracleAsmSecretId – The full ARN, partial ARN, or friendly name of a secret that you created for Oracle ASM access in Secrets Manager
SecretsManagerOracleAsmAccessRoleArn – The ARN of a secret access role that you created in IAM to provide AWS DMS access to the SecretsManagerOracleAsmSecretId secret on your behalf

The following diagram shows the process flow when creating an AWS DMS endpoint that utilizes Secrets Manager as the authentication mechanism.

When creating an AWS DMS endpoint that has been configured to use Secrets Manager as the authentication mechanism, AWS DMS connects to IAM and verifies whether the user making the create or modify endpoint API request has the IAM pass role permissions on the SecretsManagerAccessRoleARN. Next, AWS DMS connects to Secrets Manager to verify whether the SecretsManagerSecretId and SecretsManagerOracleASMSecretId provided is valid.

The following diagram shows the process flow when connecting to an AWS DMS endpoint with Secrets Manager as the authentication mechanism.

After the secret details are validated, it’s stored in the AWS DMS endpoint securely. During a migration when connecting to your database, AWS DMS assumes the IAM role, fetches the secret, and connects to the database.

To manage your AWS DMS Oracle Database and Oracle ASM endpoint credentials through Secrets Manager, we complete the following high-level steps:

Create Oracle Database and Oracle ASM related secrets in Secrets Manager.
Allow AWS DMS to access the Oracle Database and Oracle ASM secrets maintained in Secrets Manager.
Create an AWS DMS endpoint for Oracle, choose Secrets Manager to access the endpoint database, and provide the secret ID and IAM role details for both the Oracle and Oracle ASM instances.

Create Oracle Database and Oracle ASM secrets

In this step, we first take you through the steps to create the Secrets Manager secrets that store the credentials to access your Oracle database. Next, we outline the steps to create the Secrets Manager secrets for Oracle ASM instances.

To create a secret that AWS DMS can use to authenticate a database for source and target endpoint connections, complete the following steps:

On the Secrets Manager console, choose Store a new secret.
For Select secret type, select Other type of secrets.
On the Plaintext tab, enter the following JSON, replacing the appropriate values:

{"username": db_username,
"password": db_user_password,
"port": db_port_number,
"host": db_server_name
}

Provide db_username (the name of the user accessing the database), db_user_password (the password of the database user), db_port_number (the port number to access the database), and db_server_name (the database server name on the web) in the JSON code.

For Select the encryption key, choose an AWS Key Management Service (AWS KMS) key to encrypt your secret information.

You can accept the default encryption key created for your service by Secrets Manager or choose an AWS KMS key that you created.

For Secret name, enter a name.

This is the friendly name that you use as the value for SecretsManagerSecretId or SecretsManagerOracleAsmSecretId.

For Description, enter an optional description.
Choose Next.

Review and store your secret in Secrets Manager.

Next, you follow similar steps to create a secret for the Oracle ASM instance.

On the Secrets Manager console, choose Store a new secret.
For Select secret type, select Other type of secrets.
On the Plaintext tab, when creating a secret for the Oracle ASM instance, provide the asm_username (the name of the user accessing Oracle ASM), asm_user_password (the password of the Oracle ASM user), and asm_server_name (the Oracle ASM server name on the web) in the following JSON format:
```
{
  "asm_user": asm_username,
  "asm_password": asm_user_password,
  "asm_server": asm_server_name
}
```
Complete the remaining steps as outlined earlier to finish creating your secret.

When you’re done creating the secrets, you can look up each secret by its friendly name in Secrets Manager, then retrieve the secret ARN as the value for SecretsManagerSecretId or SecretsManagerOracleAsmSecretId as appropriate to authenticate access to your endpoint database connection and Oracle ASM.

Allow AWS DMS to access Oracle Database and Oracle ASM secrets

In this step, we create the access policies and roles to allow AWS DMS to access Oracle Database and Oracle ASM related secrets.

On the IAM console, choose Policies, then choose Create policy.
On the JSON tab, enter the following policy to enable access to and decryption of your secret:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "secretsmanager:GetSecretValue",
            "Resource": secret_arn,
        },
        {
             "Effect": "Allow",
             "Action": [
                        "kms:Decrypt",
                        "kms:DescribeKey"
                      ],
             "Resource": kms_key_arn,
        }
     ]
}

In the preceding code, secret_arn is the ARN of your secret, which you can get from either SecretsManagerSecretId or SecretsManagerOracleAsmSecretId as appropriate, and kms_key_arn is the ARN of the KMS key that you’re using to encrypt your secret. Note that you created secret_arn and kms_key_arn in the previous section.

The following screenshot is the example JSON for creating an IAM policy for your Oracle Database instance.

The following screenshot shows the example JSON for creating an IAM policy for your Oracle ASM instance.

Review and create the policy with a friendly name and optional description.
In the navigation pane, choose Roles, then choose Create role.
Choose AWS service as the type of trusted entity.
Choose DMS from the list of services as the trusted service.
Choose Permissions.
Look up and attach the policy you created.
Add any tags and review your role.
Edit the trust relationships for the role to use your AWS DMS Regional service principal as the trusted entity.

This principal has the following format: dms.region-name.amazonaws.com, in which region-name is the name of your Region. In the following example, the Region is us-east-1. Therefore, an AWS DMS Regional service principal for this Region is dms.us-east-1.amazonaws.com. To find your Region code, see Regions, Availability Zones, and Local Zones.

After you edit the trusted entity for the role, create the role with a friendly name and optional description.

You can now look up your new role by its friendly name in IAM, then retrieve the role ARN as the SecretsManagerAccessRoleArn or SecretsManagerOracleAsmAccessRoleArn value to authenticate your endpoint database connection.

Create an AWS DMS endpoint for Oracle ASM

Next, we create an AWS DMS endpoint for Oracle ASM, allow Secrets Manager to access the endpoint database, and provide the secret ID and IAM role details for both the Oracle and Oracle ASM instances.

On the AWS DMS console, choose Endpoints and then choose Create endpoint.
For Endpoint type, select Source endpoint.
For Endpoint identifier, enter a name.
For Description, enter an optional description.
For Source engine, choose Oracle.
For Access to endpoint database, select Choose AWS Secrets Manager.
Provide the details of the Oracle Database secret ID, the IAM role that provides access to the Oracle Database secret, Oracle ASM secret ID, and IAM role that provides access to the Oracle AMS secret.

After you create the endpoint connection, you should test the connection to verify.

On the AWS DMS console, choose Endpoints.
Choose the endpoint that you created in the previous step to see its details.
On the Connections tab, choose Test connections.
Choose a replication instance to validate the connection.

If you don’t have a replication instance, you can create one.

Choose Run test.

If the endpoint is correctly configured, the status shows as successful. If the connection is unsuccessful, recheck your configurations. In addition, contact your administrator to understand whether you have authorization to perform any action in AWS DMS and are authorized to perform the iam:PassRole action.

Conclusion

In this post, we demonstrated how you can centrally manage your AWS DMS endpoint credentials for your Oracle Database instance and Oracle ASM instances. You can take advantage of the built-in credential management capability provided by Secrets Manager, including auditing, updating, and rotating the database credentials.

For more information on AWS DMS endpoints, see Creating source and target endpoints. If you have any questions or comments about this post, share your thoughts in the comments section.

About the Authors

Darshan Thimmaiah is a Product Manager with the Database Migration Service (DMS) team at Amazon Web Services. He owns the DMS product roadmap and helps customers modernize their legacy databases and move to managed databases on AWS.

Mahesh Kansara is a Database Engineer with the Database Migration Service (DMS) team at Amazon Web Services. He works on DB migrations related challenges and works closely with customers to help them realize the true potential of the DMS service. He has helped migrate 100s of databases into the AWS cloud using DMS and SCT.

AWS Database Blog