Best Practices from Innovative Solutions for Migrating Databases in AWS Amplify Deployments

By Michael Krauklis, Chief Architect – Innovative Solutions
By Venkat Gomatham, Partner Management Solutions Architect – AWS

Innovative Solutions

Managing database objects via deployment automation is a best practice that improves system quality, reduces operational overhead, and decreases overall time to market. However, AWS Amplify does not provide support for database migrations (schema updates) as part of its in-built deployment automation.

Furthermore, there is currently no way to access resources within virtual private clouds (VPCs) such as Amazon Relational Database Service (Amazon RDS) during Amplify deployments.

In this post, we will explain how to implement database software development lifecycle (SDLC) automation when leveraging the AWS Amplify framework to quickly create a three-tier web application following cloud engineering best practices.

This solution leverages the AWS Code suite coupled with the Amplify command line interface (CLI) to integrate database migrations against an Aurora Serverless V1 PostgreSQL cluster with deployment of an Amplify project (frontend and backend). Most importantly, the solution leverages the ability to execute an AWS CodeBuild container within a VPC.

Innovative Solutions is an AWS Premier Tier Services Partner and professional services company that gives companies confidence to grow in the cloud with well-architected reviews, cloud migrations, managed cloud services, application modernization, and cloud cost optimization.

Solution Overview

The purpose of this post is to demonstrate a pattern that can be used to create a deployment pipeline for AWS Amplify applications that require access to VPC resources.

In this example, the specific need that’s solved is automatically deploying database migrations to a database residing in a VPC alongside an Amplify frontend and backend deployment (via amplify publish).

AWS Cloud Development Kit (AWS CDK) is the infrastructure as code (IaC) framework used to provision all of the core infrastructure required for this reference solution. Alembic is used to execute the database migrations.

Deployment

The source for this project can be found in this GitHub repository, and the high-level steps to deploy the solution are as follows:

• Install prerequisites
• Clone GitHub Git repository
• Update VPC CIDR in cdk.json
• Install node modules
• Deploy baseline project using AWS CDK
• Update AWS Secrets Manager access keys
• Clone the AWS CodeCommit Git repository created during AWS CDK baseline deployment
• Initialize baseline React application
• Initialize the AWS Amplify application
• Push the Amplify backend
• Add Amplify hosting (leveraging Amazon S3 and Amazon CloudFront)
• Publish the Amplify application
• Install Alembic
• Push changes to AWS CodeCommit

These steps will create a functional React application deployed with AWS CodePipeline, including database migrations.

Prerequisites

This example was developed and intended for Amazon Linux 2 environments. Mileage may vary with other flavors of Linux or Windows.

Other requirements include the following:

• AWS CLI v1.19.112 or later
• AWS CDK v2.17 or later
• NodeJS v12.x or later
• NPM v5.x or later
• Git v2.14.1 or later
• AWS Amplify CLI @aws-amplify/cli v7.6.19 or later
• Python 3.8 (3.9 had issues)

Ensure your environment has programmatic access set up (configure a local environment, run on an Amazon EC2 instance, or leverage AWS Cloud9). This project was built using AWS Cloud9.

Clone the Git Repository

Clone the repository using Git:

git clone https://github.com/innovativeSol/solution-amplify-with-database-migrations.git

Optional: Update VPC CIDR

The VPC CIDR is defined in cdk.json and defaults to 10.1.100.0/24. This can be updated if necessary, but must have at least 256 addresses to fit the four 64-address subnets.

Figure 1 – VPC CIDR definition in cdk.json.

Install Node Modules

Install node module dependencies:

npm install

Deploy Baseline

Deploy the CDK application in your environment:

cdk deploy

This will create three outputs:

• AmplifyAppId
• CodeCommitHTTPCloneUrl
• CodeCommitRepositoryName

Figure 2 – CDK deploy output.

Note that the CDK deployment does not create the above outputs as environment variables. However, below they are referred to as such. Wherever you see such an environment variable (such as $CodeCommitHTTPCloneUrl) simply replace it with the value of the CDK deploy output.

Deploying the CDK stack will create the following:

• VPC and subnets
• Aurora Serverless PostgreSQL cluster and secret
• AWS CodeCommit repository
• Empty AWS Amplify application
• AWS CodePipeline:
  • AWS CodeBuild project for Amplify
  • AWS CodeBuild project for database deployment
  • Supporting security groups, IAM roles, AWS Systems Manager parameters, and more

You can see in the console this creates a CodePipeline with all of the required steps, and kicks off an execution of the pipeline. This initial execution will fail because we have not yet checked in any code and do not yet have a master branch.

Figure 3 – AWS CodePipeline console.

Update AWS Secrets Manager Access Keys

The cdk deploy will create two secrets to store the access key ID and secret. These must be updated with actual values from an access key tied to a user that has all of the required Amplify permissions. This can be done through the AWS console:

• /app/AwsAmplifyCodepipelineDbMigrationsMainStack/CodeBuild/dev/AMPLIFY_USER_SECRET_ACCESS_KEY
• /app/AwsAmplifyCodepipelineDbMigrationsMainStack/CodeBuild/dev/AMPLIFY_USER_ACCESS_KEY_ID

Clone CodeCommitHTTPCloneUrl

Clone the newly-created repository. A warning will be presented that an empty repository has been cloned. This can be ignored as it is expected.

git clone $CodeCommitHTTPCloneUrl

Initialize React Application

Use npx to initialize a React application in the directory that was just cloned.

npx create-react-app $CodeCommitRepositoryName

CD Into Application Directory

Change directories into the directory that was just cloned/initialized.

cd $CodeCommitRepositoryName

Initialize AWS Amplify App

Initialize the AWS Amplify environment using the $AmplifyAppId output of cdk deploy. Use the defaults up to authentication method. For the authentication method, use “AWS access keys” and specify the same values that were updated in AWS Secrets Manager.

amplify init --appId $AmplifyAppId

Figure 4 – AWS Amplify init.

This will initialize a backend environment named dev that can be seen in the console.

Figure 5 – AWS Amplify console for backend environments.

Push AWS Amplify Backend

Verify the environment by pushing the backend. You should see that no changes are detected.

amplify push

Figure 6 – AWS Amplify push output.

Optional: Install Amplify React UI Module

Install the Amplify React user interface (UI) module. While not explicitly required for this demo, now is a good time to do so.

npm install --save aws-amplify @aws-amplify/ui-react

Add AWS Amplify Hosting

Add non-Amplify hosting to the Amplify project. This creates the template for the hosting stack (specific resources depend on DEV/PROD configuration).

It’s important to choose “Amazon CloudFront and S3” as the hosting module to execute. For this demo, both DEV and PROD configurations will work. Choose defaults for the rest of the options.

amplify add hosting

Figure 7 – AWS Amplify add hosting command output.

Publish Amplify

The “amplify publish” command will deploy the backend and build/deploy the frontend based on the hosting configuration.

amplify publish

Once published, the React application will be available at the URL specified in the output of the amplify publish command.

Figure 8 – AWS Amplify publish command output.

Edit React Application

Edit the src/App.js file in such a way that we’ll be able to see changes once deployed.

Figure 9 – App.js file.

Push to AWS CodeCommit

Commit the changes and push to AWS CodeCommit. This triggers the CodePipeline to execute automatically.

git add .
git commit -m "Amplify project initialized."
git push

The CodePipeline will partially succeed. The CodeBuild project that deploys the Amplify application will succeed, and the CodeBuild project that runs the database migrations will fail since the alembic migrations have not been setup within the application.

Figure 10 – AWS CodePipeline console.

Since the Amplify deployment was successful the changes that were made to src/App.js are reflected in the published website.

Install Alembic

Install Alembic and initialize an Alembic project. This will create an Alembic directory in the root of the project.

yum install -y python3-devel postgresql-devel
python3 -m venv env
source env/bin/activate
pip install psycopg2-binary
pip install postgres
pip install SQLAlchemy
pip install alembic
pip install boto3
pip install pytest

alembic init --template generic alembic

Next, create the first migration script using the “alembic version” command.

git commit -m "Added alembic baseline."

This will create a migration file under the alembic/versions directory.

Push to AWS CodeCommit

Commit the changes and push to AWS CodeCommit. This triggers the AWS CodePipeline to execute automatically.

git add .
git commit -m "Added alembic baseline."
git push

The CodePipeline will now fully succeed.

Figure 11 – AWS CodePipeline console after updates.

In looking at the logs for the DatabaseDeploy step, we can see the database migrations were successfully executed.

Figure 12 – ‘DatabaseDeploy’ step output.

Cleanup

Clean up the reference example by calling the CDK destroy command.

cdk destroy

Conclusion

This reference solution lays the foundation for integrating AWS Amplify deployments with steps that require access to private VPC resources. It can do this without having to expose the VPC resources to the public internet or implement a complicated method of proxying communication into the VPC.

.

.

Innovative Solutions – AWS Partner Spotlight

Innovative Solutions is an AWS Premier Tier Services Partner and professional services company that gives companies confidence to grow in the cloud with well-architected reviews, cloud migrations, managed cloud services, application modernization, and cloud cost optimization.

Contact Innovative Solutions | Partner Overview | AWS Marketplace