Create and share encrypted backups across accounts and Regions using AWS Backup

Enterprises and organizations in more security-conscious industries often protect their data through encryption, restricting data access to those with the necessary permissions and improving their security posture. Creating backups of data resources is often another critical component of a secure and resilient architecture. Additionally, backing up encrypted data is also important, even across geographical regions or administrative accounts.

AWS Backup enables you to centralize and automate data protection across AWS services and accounts. With the release of the cross-account backup feature, you can copy your encrypted backups for Amazon EC2 instances, Amazon EBS volumes, Amazon RDS databases (including Amazon Aurora clusters), AWS Storage Gateway volumes, and Amazon EFS file systems between accounts. AWS Backup can also make your backups available to your organization across accounts and across AWS Regions. This helps you to meet your security, compliance, and business continuity requirements.

In this blog post, I walk through the process of creating a backup of encrypted Amazon EBS volumes. Then, I verify the AWS Key Management Service (AWS KMS) keys used to encrypt the backups. Once the backup is created in the source account’s Region, I perform a one-time copy of the backup to another Region and account. Lastly, I discuss how to address common errors you may experience when performing a cross-account copy.

Solution overview

When your resources like Amazon EC2, Amazon EBS, Amazon RDS (including Aurora clusters), and AWS Storage Gateway volumes are encrypted, cross-account copy can only be performed if they are encrypted by AWS KMS keys, with an exception for Amazon EFS backups. The default vault is encrypted using SMKs. Therefore, to perform cross-account backups, you must use KMS key encrypted vaults instead of using your default backup vault.

For Amazon EFS, you can perform cross-account backups using any Amazon EFS backup vault because AWS Backup independently manages the encryption for each Amazon EFS backup vault.

When your cross-account copy job is performed, two roles are used by AWS Backup to perform the copy operation:

1. In the source account, AWS Backup uses the role that is provided with the backup selection or the role that is explicitly provided while creating a copy job.
2. In the destination account, AWS Backup uses service-linked roles. The AWS Backup service-linked role is AWSServiceRoleForBackup.

AWS Backup uses the source role to share/unshare and list tags from the source recovery point and uses the destination role to copy backup and monitor copy operation. If your resources are encrypted with a customer managed key, you must share the customer managed key used to encrypt the resources in the source account, with the destination account.

The following diagram illustrates the solution discussed in this blog post, wherein I use an encrypted Amazon EBS volume.

Here is an overview of the solution:

1. In the source account, create a backup of a customer managed key encrypted EBS volume.
2. Give the target account access to the customer managed AWS Key Management Service (AWS KMS) encryption key used by the source account EBS volume.
3. Copy the encrypted snapshots to the target account, which would re-encrypt them using the target vault account’s AWS KMS encryption keys in the target Region⁺.
4. Verify the KMS key that the recovery point created from the backup job will use.

⁺ Before you proceed with step 3, make sure you have a backup vault encrypted using a customer managed key created in the destination account. Also, make sure this vault is shared with the different account using vault policy.

Prerequisites

For this walkthrough, you need two AWS accounts present in the same AWS Organization.

1. The source account where you have a KMS key encrypted EBS volume.
2. The target account in which you will copy the encrypted snapshots.
3. Make sure to opt in to cross-account backup.
4. An IAM role that can be used to perform cross-account backups. You can also make use of the AWSBackupDefaultServiceRole.
5. A backup vault encrypted with KMS key in the target account.

In a recent blog post, we covered steps 3 through 5 of prerequisites and how you can make use of backup plans to copy unencrypted backups from source account to destination account. In this blog, I am using one-time backup and copy jobs to demonstrate how you can protect and copy your encrypted resources using AWS Backup. You can also use the same approach when setting up a backup plan to perform a cross-account copy.

Note: You can also automate a sequence of cross-account and cross-region copies for your most supported resources, except for Amazon RDS and Amazon Aurora. For Amazon RDS and Aurora snapshots, AWS Backup only supports automating either cross-account or cross-region copies due to way the encryption keys are structured with those services.

In this blog, I use two account IDs, 66XXXX9480 and 08XXX0204, for the source account and the target account, respectively. Both the accounts are a part of the same organization.

If you are following along with this post, be sure to change these account IDs to match your own. In addition, you will create a KMS key in the source account-source Region in addition to a KMS key in the target account-target Region. When you create a KMS key for the target account, select the Region where you plan to copy the backup. It is important to note that KMS keys are never transmitted outside the AWS Regions in which they were created. For simplicity, I created two KMS keys, cmkSource under account 66XXXX9480 in us-east-1, and cmkTarget under account 08XXXX0204 in eu-west-1.

Note: You cannot share snapshots encrypted with your default KMS key with another account. Resources encrypted under a service default KMS key can only be shared within the same account. If you are starting with snapshots encrypted under the default Amazon EBS KMS key or default Amazon RDS KMS key (with the key alias, aws/ebs or aws/rds respectively), copy those snapshots and re-encrypt them under a custom KMS key you created in KMS. You will then be able to modify the key policy on the custom KMS key to be able to grant access to the key to any number of external accounts.

Deploying the solution

Here, I show you how to take an on-demand backup of an encrypted EBS volume, verifying the KMS key being used by the backup that was created. Once I have created a backup, I share the KMS key to the destination account and perform an on-demand copy.

Step 1: Create a backup of an encrypted EBS volume

In this walkthrough, I create an on-demand backup in the us-east-1 Region of an EBS volume encrypted with KMS key ‘cmksource,’ as shown in the following screenshot. For more information on creating an EBS volume, see the documentation on creating an EBS volume.

1.a Create an on-demand backup

1. Sign in to the AWS Management Console, and open the AWS Backup console.
2. From the AWS Backup console dashboard, choose Create on-demand backup.
3. On the Create on-demand backup page, choose the resource type that you want to back up. Here, I have selected EBS for Amazon EBS volumes.
4. Choose the name or ID of the resource that you want to protect.
5. Ensure that Create backup now is selected. This initiates a backup immediately and enables you to see your saved resource sooner on the Protected resources page.

6. Choose an existing backup vault or create a new one.

Here, I have selected Create new backup vault to create a new source vault, which is encrypted using a different KMS key than the EBS volume, as you can see in the following screenshot.

Choosing Create new backup vault opens a new page to create a vault and then returns you to the Create on-demand backup page when you are finished. So, once you are back on the on-demand page for IAM role, choose Default role.

1.b Setting backup lifecycle and creating the backup

Before we proceed with the creation of an on-demand backup job, I specify the Retention period after which the backup created is automatically deleted. Once the configuration is complete, I will create an on-demand backup job and will also show you how you can check the details on the job.

1. Specify the expire value for your recovery point.
2. If you want to assign tags to your on-demand backup, enter a key and optional value, and choose Add tag.
3. Choose Create on-demand backup. This takes you to the Jobs page, where you will see a list of jobs.

4. Choose the Backup job ID for the resource that you chose to back up, to see the details of that job. Click on the recovery point ARN (Amazon Resource Name), which will redirect you to the vault where the backup resides.

1.c Verify the KMS key used for the recovery point

Amazon EBS snapshots are automatically encrypted with the same encryption key that was used to encrypt the source EBS volume. In order to check the KMS key being used to encrypt the snapshot created in Step 1.a, complete the following steps.

1. Navigate to the EC2 console.
2. From the left navigation pane, select Snapshots.
3. Search for the snapshot id in the search box.
4. Under Description of the snapshot, look for KMS Key Aliases or KMS Key ARN. You will see that it is encrypted using the KMS key (cmkSource) of the EBS volume instead of the KMS key of the source backup vault as shown in the following view.

Step 2: Sharing the source KMS key with the destination account

As the Amazon EBS volume is encrypted with a KMS key, I share this KMS key with the destination account. This allows the destination account permission to decrypt and re-encrypt the snapshot. The following steps give the target account permission to access the EBS volume KMS key.

1. From the AWS Key Management Service console, click Customer managed keys in the left pane, and select the source account’s KMS key (cmkSource).
2. Look for the Other AWS accounts subsection, click Add other AWS accounts, and type the target account ID as you can see in the following screenshot.

Read the blog post on sharing custom encryption keys more securely between accounts using AWS Key Management Service for more information.

Step 3: Creating a copy of an encrypted snapshot

Here, I create an on-demand copy of the Amazon EBS snapshot created in step 1.

Before proceeding with the copy operation, make sure you have a destination vault created with a KMS key as seen in the following screenshot.

Also, make sure that you set the vault policy to allow AWS Backup permissions to access the external (source) account, as shown in the following screenshot.

3.a On-demand copy of the backup

1. Open the AWS Backup console in the source account.
2. Choose Backup vault where the source backup resides.
3. Choose the Recovery point ID of the backup you want to copy.
4. Choose Copy.

3.b Verifying recovery point details and configuring copy job details

Here, I will verify the details of the backup and then will provide parameters for configuring a copy job.

1. Expand Backup details to see information about the recovery point you are copying as in the following screenshot.

2. In the Copy configuration section, choose a destination Region.
3. Choose Copy to another account’s vault. The option turns blue when selected. Enter the Amazon Resource Name (ARN) of the destination account vault. In this example, we use “arn:aws:backup:us-east-1:08XXXX0204:backup-vault:CABTargetVault.” AWS Backup copies the backup to the destination account’s vault. The destination Region automatically updates to the Region in the external vault ARN.
4. For IAM role, specify the IAM role that AWS Backup will assume when creating and managing backups on your behalf. We recommend choosing the default role.
5. Choose Copy.

3.c Creating an on-demand copy

Once the configuration for a copy job is completed, I trigger the on-demand copy job and check the details of the job. Depending on the size of the resource you are copying, this process could take several hours to complete. When the copy job completes, you will see the copy in the Copy jobs tab in the Jobs menu.

Once the copy job is completed, you can check the Copy job details to get the Destination recovery point ARN.

Congratulations! You have successfully copied the snapshot to the destination account. Now, let’s verify the KMS key being used for the copied backup.

Step 4. Verify the KMS key that the recovery point created from the backup job will use

You can follow the steps to check the encryption key being used by the destination recovery point ARN.

1. Navigate to the EC2 console.
2. From the left navigation pane, select Snapshots.
3. Search for the Snapshot ID in the search box.
4. Under Description of the snapshot, look for KMS Key Aliases or KMS Key ARN. Notice that it is encrypted using the KMS key (cmkTarget) of the destination backup vault, as you can see in the following snapshot.

Troubleshooting common errors

In this section, I cover a few of the common errors that you might encounter when setting up cross-account backup jobs as a part of backup plan or as part of an on-demand job.

Error 1

Copy job failed. Both source and destination account must be a member of the same organization.

For this error, make sure that the source and destination accounts belong to same organization.

Error 2

AMI snapshot copy failed with error: Given key ID is not accessible. You must have DescribeKey permissions on the default CMK.

This error can surface if the destination IAM role does not have access to the source snapshot KMS key. In order to verify if the destination IAM role is not being authorized, you can also check the CloudTrail events in the destination account with the source as “kms.amazonaws.com” ; you will see the following error.

User: arn:aws:sts::destination-account:assumed-role/AWSServiceRoleForBackup/AWSBackup-AWSServiceRoleForBackup is not authorized to perform: kms:CreateGrant on resource: arn:aws:kms:us-east-1:source-account:key/09c84b51-xxxxx-4ee8-bb64-xxxxxxxxx

To resolve this, while deploying the solution, add the destination account in their source snapshot KMS key as explained in the section “Step 2: Sharing the Source KMS key with the destination account.”

Error 3

Access Denied trying to call AWS Backup service

This error generally occurs when the destination AWS Backup vault is not shared by the external account. AWS Backup allows you to share a backup vault with one or multiple accounts, or your entire organization in AWS Organizations. You can share a destination backup vault with a source AWS account, user, or IAM role.

Error 4

Snapshots encrypted with the AWS Managed CMK can’t be shared. Specify another snapshot. (Service: AmazonEC2; Status Code: 400; Error Code: InvalidParameter; Request ID: 35b614cc-e511-40da-93ad-ac6fbc829c13; Proxy: null)

This error can occur when your EBS snapshots in the source account are encrypted with an AWS managed key that is, aws/ebs. To copy them successfully, you should encrypt source snapshots with a customer managed key. Encryption keys used by an Amazon EBS volume can’t be changed. However, you can create a snapshot of the volume and then use the snapshot to create a new, encrypted copy of the volume. While creating the new volume, specify the KMS key encryption key. You can find complete steps in this article.

Error 5

Copy job failed because the destination Backup vault is encrypted with the default Backup service managed key. The contents of this vault cannot be copied. Only the contents of a Backup vault encrypted by a customer master key (CMK) may be copied.

The destination vault cannot be the default vault or a vault encrypted with AWS managed keys. This is because the default vault is encrypted with a key that cannot be shared with other accounts.

Error 6

Copy job from us-west-2 to us-east-1 cannot be initiated for RDS resources. Feature is not supported for provided resource type.

Currently, AWS Backup only supports automating either cross-account or cross-region copies due to how Amazon RDS/Aurora services create their encryption keys.

Cleaning up

To avoid incurring future charges, follow these steps to remove the example resources you may have created while following along.

1. Delete recovery points by following this guide.
2. Delete the backup vaults by following this guide.
3. Delete the KMS key created as a part of this setup.
   • Note: Delete a KMS key only when you are sure that you don’t need to use it anymore. After a KMS key is deleted, you can no longer decrypt the data that was encrypted under that KMS key, which means that data becomes unrecoverable.

Conclusion

In this blog post, I showed you how to back up an EBS encrypted resource and copy this snapshot to another account and Region. I also verified the encryption key used by the source and destination snapshots. In addition, I provided examples of common errors you may encounter while setting up a cross-account copy using AWS Backup.

Using AWS Backup’s cross-account backup and cross-region copy capabilities, you can quickly restore important data to the source or destination accounts in your preferred AWS Region. This enables AWS Backup users to minimize business impact in the event of an account compromise, unexpected disaster, or service interruption. You can create consistent EBS volumes globally based on your backups, in order to meet your security and compliance requirements, while improving your agility and availability by distributing your workload.

Thanks for reading this blog post. If you have any comments or questions, please do not hesitate to leave them in the comments section.