Front-End Web & Mobile

Building Real-time Serverless APIs with PostgreSQL, CDK, TypeScript, and AWS AppSync

September 14, 2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. See details.

AWS AppSync is a managed serverless GraphQL API service that simplifies application development by letting you create a flexible interface to securely access, manipulate, and combine data from one or more data sources with a single network call and API endpoint. With AppSync, developers can build scalable applications on a range of data sources, including Amazon DynamoDB NoSQL tables, Amazon Aurora Serverless relational databases, Amazon OpenSearch Service (successor to Amazon Elasticsearch Service) clusters, HTTP APIs, and serverless functions powered by AWS Lambda.

In my last post I showed how to use AWS CDK to deploy AWS AppSync as a real-time API layer integrated with Amazon DynamoDB. One of the strengths of AppSync is that it is database agnostic – because the resolvers are just functions, the service integrates well with any database within AWS or elsewhere using either direct resolvers or Lambda resolvers (which we will be using in this tutorial).

You are also not limited to a single database per API or even per API request. A single API can be backed by a combination of Amazon DynamoDB, Amazon Aurora, Amazon OpenSearch Service, or any other database and a single request can return a response combining all of these datasources at once.

In this tutorial, you will learn how to build a real-time AppSync API using Amazon Serverless Aurora PostgreSQL as the database. The infrastructure as code tool will be CDK written in TypeScript. The API will implement common CRUD and List operations as well as real-time event-based functionality using GraphQL subscriptions for all create, update, and delete operations to build a blogging back end. At the end of the post, we’ll use the CDK CLI to create an outputs file that can be integrated into a client application using AWS Amplify.

We’ll also look at how to consume the API from a web application using the Amplify JavaScript library.

CDK Overview

The AWS Cloud Development Kit (AWS CDK) is an open source software development framework to model and provision your cloud application resources using familiar programming languages. CDK can be written using a variety of programming languages like Python, Java, TypeScript, JavaScript, and C#. In this tutorial we will be using TypeScript

To work with CDK, you will need to install the CDK CLI. Once the CLI is installed, you will be able to do things like create new CDK projects, deploy services to AWS, deploy updates to existing services, and view changes made to your infrastructure during the development process.

In this post we’ll be using the CDK CLI to provision and deploy API updates.

CDK with AppSync

An AppSync API is usually composed of various AWS services. For example, if we are building an AppSync API that interacts with a database and needs authorization, the API will depend on these resources being created.

With this in mind, we will not only be working with CDK modules for AppSync but also various other AWS services. Part of building an AppSync API is learning how to use all of these things and making them work well together, including managing IAM policies in certain circumstances in order to enable access between services or to configure certain types of data access patterns.

The AppSync CDK constructs and classes take this into consideration and enable the configuration of various resources within an AppSync API using AWS services also created as part of the CDK project.

Click here to view the CDK documentation. Click here to view the AWS AppSync documentation.

Getting Started

If you’d like to see the code for this project at any time, it is located here.

First, install the CDK CLI:

npm install -g aws-cdk

You must also provide your credentials and an AWS Region to use AWS CDK, if you have not already done so. The easiest way to satisfy this requirement is to install the AWS CLI and issue the following command:

aws configure

Next, create a directory called appsync-cdk-rds and change into the new directory:

mkdir appsync-cdk-rds
cd appsync-cdk-rds

Next, we’ll create a new CDK project using the CDK CLI:

cdk init --language=typescript

The CDK project should now be initialized and you should see a few files in the directory, including a lib folder which is where the boilerplate for the root stack has been created.

Now that we’ve created the CDK project, let’s install the necessary dependencies we’ll need. Since we’ll be working with several different CDK packages, we’ll need to go ahead and install them now:

npm install @aws-cdk/aws-appsync @aws-cdk/aws-lambda @aws-cdk/aws-ec2 @aws-cdk/aws-rds

Running a build

Because the project is written in TypeScript, but will ultimately need to be deployed in JavaScript, you will need to create a build to convert the TypeScript into JavaScript before deploying.

There are two ways to do this, and the project is already set up with a couple of scripts to help with this.

Watch mode

You can run npm run watch to enable watch mode. The project will automatically compile to JavaScript as soon as you make changes and save files and you will also be able to see any errors logged out to the terminal.

Manual build

You can also create a build at any time by running npm run build.

Creating the API

Now that the project is set up, we can start writing some code! Some boilerplate code for the stack has already been created for you. This code is located at appsync-cdk-rds/lib/appsync-cdk-rds-stack.ts. This is the root of the CDK app and where we will be writing the code for our app.

To get started, let’s first go ahead and import the CDK modules we’ll be needing:

import * as cdk from '@aws-cdk/core';
import * as ec2 from '@aws-cdk/aws-ec2';
import * as lambda from '@aws-cdk/aws-lambda';
import * as rds from '@aws-cdk/aws-rds';
import * as appsync from '@aws-cdk/aws-appsync';

Next we’ll use the appsync CDK module to create the AppSync API. Update the Stack with following code:

export class AppsyncCdkRdsStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    // Create the AppSync API
    const api = new appsync.GraphqlApi(this, 'Api', {
      name: 'cdk-blog-appsync-api',
      schema: appsync.Schema.fromAsset('graphql/schema.graphql'),
      authorizationConfig: {
        defaultAuthorization: {
          authorizationType: appsync.AuthorizationType.API_KEY,
          apiKeyConfig: {
            expires: cdk.Expiration.after(cdk.Duration.days(365))
          }
        },
      },
    });
  }
}

We’ve defined a basic AppSync API with the following configuration:

  • name: Defines the name of the AppSync API
  • schema: Specifies the location of the GraphQL schema
  • authorizationConfig: This allows you to define the default authorization mode its configuration, as well as (optional) additional authorization modes
  • apiKeyConfig: This will set the API key expiration to 1 year from the time of deployment
  • xrayEnabled: Enables AWS X-Ray debugging

Next, we need to define the GraphQL schema. Create a new folder in the root of the project named graphql and within it, create a new file named schema.graphql:

type Post {
  id: String!
  title: String!
  content: String!
}

input CreatePostInput {
  id: String
  title: String!
  content: String!
}

input UpdatePostInput {
  id: String!
  title: String
  content: String
}

type Query {
  listPosts: [Post]
  getPostById(postId: String!): Post
}

type Mutation {
  createPost(post: CreatePostInput!): Post
  deletePost(postId: String!): Post
  updatePost(post: UpdatePostInput!): Post
}

type Subscription {
  onCreatePost: Post
    @aws_subscribe(mutations: ["createPost"])
  onUpdatePost: Post
    @aws_subscribe(mutations: ["updatePost"])
  onDeletePost: Post
    @aws_subscribe(mutations: ["deletePost"])
}

This schema defines a data model for a Blogging application with two queries and three mutations for basic CRUD + List functionality. There are also subscription definitions for enabling real time updates when a post is created, updated, or deleted.

Now, or any time, we can run run a build and use the CDK diff command to see what changes will be deployed:

npm run build && cdk diff

The diff command compares the current version of a stack defined in your app with the already-deployed version and displays a list of differences.

From here, we can build and deploy the API to see it in the AppSync console. To do so, run the following command from your terminal:

npm run build && cdk deploy

When the build finishes, you should see JavaScript files compiled from the TypeScript files in your project.

Once the deployment is complete, you should be able to see the API (cdk-blog-appsync-api) in the AppSync console.

? Congratulations, you’ve successfully deployed the AppSync API using CDK!

Adding the Amazon Aurora Serverless data source

Now that we’ve created the API and defined the GraphQL schema, we need a way to connect the GraphQL operations in our API with a data source. We will be doing this by mapping these operations into a Lambda function that will be interacting with an Amazon Serverless Aurora database cluster.

To build out this functionality, we’ll now need to create the database. To set up a clustered database (like Aurora), we’ll need to define a ServerlessCluster or DatabaseCluster. You must always launch these types of databases in a VPC, so we’ll also need to create a VPC using EC2.

To create these resources, add the following code below the API definition in lib/appsync-cdk-rds-stack.ts.

// Create the VPC needed for the Aurora Serverless DB cluster
const vpc = new ec2.Vpc(this, 'BlogAppVPC');
// Create the Serverless Aurora DB cluster; set the engine to Postgres
const cluster = new rds.ServerlessCluster(this, 'AuroraBlogCluster', {
  engine: rds.DatabaseClusterEngine.AURORA_POSTGRESQL,
  parameterGroup: rds.ParameterGroup.fromParameterGroupName(this, 'ParameterGroup', 'default.aurora-postgresql10'),
  defaultDatabaseName: 'BlogDB',
  vpc,
  scaling: { autoPause: cdk.Duration.seconds(0) } // Optional. If not set, then instance will pause after 5 minutes 
});

The database is configured with the following settings:

  1. Defining a database name of BlogDB.
  2. Setting the engine to be PostgreSQL.
  3. Setting the PostgreSQL version to 10.
  4. Turning autoPause off in the scaling configuration.
    1. Leaving autoPause on will cause your cluster to automatically pause itself after a period of 5 minutes by default. It usually takes between 30-60 seconds for the cluster to unpause and resume upon a new query. Turning autoPause off will mean your cluster will always be running and immediately available. This will cost 0.06 per Aurora Capacity Unit (ACU) Hour. By default, your instance will be configured with 2 ACUs meaning that the running cluster will cost around $85 per month when unpaused. Note that the autoPause setting can be left on for development purposes or circumstances when this latency is not a factor.

Creating the Lambda function

Next, we need to create the Lambda function and add it as a data source to the AppSync API. To do so, add the following code below the cluster definition in lib/appsync-cdk-rds-stack.ts.

// Create the Lambda function that will map GraphQL operations into Postgres
const postFn = new lambda.Function(this, 'MyFunction', {
  runtime: lambda.Runtime.NODEJS_10_X,
  code: new lambda.AssetCode('lambda-fns'),
  handler: 'index.handler',
  memorySize: 1024,
  environment: {
    CLUSTER_ARN: cluster.clusterArn,
    SECRET_ARN: cluster.secret?.secretArn || '',
    DB_NAME: 'BlogDB',
    AWS_NODEJS_CONNECTION_REUSE_ENABLED: '1'
  },
});
// Grant access to the cluster from the Lambda function
cluster.grantDataApiAccess(postFn);
// Set the new Lambda function as a data source for the AppSync API
const lambdaDs = api.addLambdaDataSource('lambdaDatasource', postFn);

This code creates the Lambda function and configures the following:

  1. Sets the runtime as Node.js 10.x.
  2. Defines the code location and the handler function.
  3. Sets the amount of memory, in MB, that is allocated to the Lambda function.
  4. Creates environment variables that we’ll need in the function to talk to the database
  5. Grants IAM access from the Lambda to the cluster
  6. Sets the new Lambda function as a data source

Now that the Lambda function is created and configured, the last thing we need to do is define the resolvers that we’d like to enable in our API. To do so, add the following code below the Lambda function definition in lib/appsync-cdk-rds-stack.ts:

// Define resolvers to map GraphQL operations to the Lambda function
lambdaDs.createResolver({
  typeName: 'Query',
  fieldName: 'listPosts'
});
lambdaDs.createResolver({
  typeName: 'Query',
  fieldName: 'getPostById'
});
lambdaDs.createResolver({
  typeName: 'Mutation',
  fieldName: 'createPost'
});
lambdaDs.createResolver({
  typeName: 'Mutation',
  fieldName: 'updatePost'
});
lambdaDs.createResolver({
  typeName: 'Mutation',
  fieldName: 'deletePost'
});

Printing out resource values for client-side configuration

If we’d like to consume the API from a client application, we’ll need the values of the API key, GraphQL URL, and project region to configure our app. We could go inside the AWS console for each service and find these values, but CDK enables us to print these out to our terminal upon deployment as well as map these values to an output file that we can later import in our web or mobile application and use with AWS Amplify.

To create these output values, add the following code below the GraphQL resolver definitions in lib/appsync-cdk-rds-stack.ts.

// CFN Outputs
new cdk.CfnOutput(this, 'AppSyncAPIURL', {
  value: api.graphqlUrl
});
new cdk.CfnOutput(this, 'AppSyncAPIKey', {
  value: api.apiKey || ''
});
new cdk.CfnOutput(this, 'ProjectRegion', {
  value: this.region
});

Adding the Lambda function code

The last thing we need to do is write the code for the Lambda function. The Lambda function will map the GraphQL operations coming in via the event into a call to the Aurora database. We will have functions for all of the CRUD and List operations. The Lambda handler will read the GraphQL operation from the event object and call the appropriate function.

Create a folder named lambda-fns in the root directory. Next, change into this directory and initialize a new package.json file and install the uuid library as well as the data-api-client library, which we will be using to interact with the database:

cd lambda-fns
npm init --y
npm install data-api-client uuid

In the lambda-fns folder, create the following files:

  • index.ts
  • Post.ts
  • db.ts
  • createPost.ts
  • updatePost.ts
  • deletePost.ts
  • listPosts.ts
  • getPostById.ts

Post.ts

// lambda-fns/Post.ts
type Post = {
  id: string;
  title: string;
  content: string;
}

export default Post

The Post type should match the GraphQL Post type and will be used in a couple of our files.

db.ts

// lambda-fns/db.ts
const db = require('data-api-client')({
  secretArn: process.env.SECRET_ARN,
  resourceArn: process.env.CLUSTER_ARN,
  database: process.env.DB_NAME
});

export default db;

The database configuration is stored here and will be used for any call to Aurora. The configuration is reading the environment variables created via CDK.

index.ts

// lambda-fns/index.ts
import createPost from './createPost';
import listPosts from './listPosts';
import updatePost from './updatePost';
import deletePost from './deletePost';
import getPostById from './getPostById';
import Post from './Post';

type AppSyncEvent = {
  info: {
    fieldName: string
  },
  arguments: {
    post: Post,
    postId: string
  }
}

exports.handler = async (event:AppSyncEvent) => {
  switch (event.info.fieldName) {
    case 'createPost':
      return await createPost(event.arguments.post);
    case 'listPosts':
      return await listPosts();
    case 'updatePost':
      return await updatePost(event.arguments.post);
    case 'deletePost':
      return await deletePost(event.arguments.postId);
    case 'getPostById':
      return await getPostById(event.arguments.postId);
    default:
      return null;
  }
}

The handler function will use the GraphQL operation available in the event.info.fieldname to call the various functions that will interact with the Aurora database.

listPosts.ts

// lambda-fns/listPosts.ts
import db from './db';

async function listPosts() {
    try {
        const result = await db.query(`SELECT * FROM posts`);
        return result.records;
    } catch (err) {
        console.log('Postgres error: ', err);
        return null;
    }
}

export default listPosts;

getPostById.ts

// lambda-fns/getPostById.ts
import db from './db';

async function getPostById(postId: string) {
    try {
        const query  = `SELECT * FROM posts WHERE id = :postId`;
        const results = await db.query(query, { postId });
        return results.records[0];
    } catch (err) {
        console.log('Postgres error: ', err);
        return null;
    }
}

export default getPostById;

createPost.ts

// lambda-fns/createPost.ts
import Post from './Post';
import db from './db';
const { v4: uuid } = require('uuid');

async function createPost(post: Post) {
    if (!post.id) post.id = uuid();
    const { id, title, content } = post;
    try {
        const query = `INSERT INTO posts (id,title,content) VALUES(:id,:title,:content)`;
        await db.query(query, { id, title, content });
        return post;
    } catch (err) {
        console.log('Postgres error: ', err);
        return null;
    }
}

export default createPost;

deletePost.ts

// lambda-fns/deletePost.ts
import db from './db';

async function deletePost(postId: string) {
    try {
        const query = `DELETE FROM posts WHERE id = :postId`;
        const result = await db.query(query, { postId });
        if (result.numberOfRecordsUpdated === 1) return postId;
        return null;
    } catch (err) {
        console.log('Postgres error: ', err);
        return null;
    }
}

export default deletePost;

updatePost.ts

// lambda-fns/updatePost.ts
import Post from './Post';
import db from './db';

async function updatePost(post: Post) {
  if (Object.entries(post).length === 1) return;
  let query =  `UPDATE posts SET`;
  const updateVariables: { [key: string]: string } = { id: post.id };
  Object.entries(post).forEach(item => {
      if (item[0] === 'id') return;
      updateVariables[item[0]] = item[1];
      if (Object.keys(updateVariables).length > 2) {
        query = `${query},`;
      }
      query = `${query} ${item[0]} = :${item[0]} `;
  })
  query = query + 'where id = :id';
  try {
      await db.query(query, updateVariables)
      return post
    } catch (err) {
        console.log('Postgres error: ', err);
        return null;
    }
}

export default updatePost;

Deploying and testing

Now we are ready to deploy. To do so, run the following command from your terminal in the root directory of your CDK project:

npm run build && cdk deploy -O cdk-exports.json

Now that the updates have been deployed, you should see a new file called cdk-exports.json created in the root of your project.

Next, we will create the table and begin testing the API.

Creating the posts table

Visit the RDS dashboard and click on Query Editor. From the dropdown menu, choose the database (it should begin with appsynccdkrdsstack-aurorablogcluster).

For the Database username, choose Connect with a Secrets Manager ARN.

To sign in, you will need the ARN from the secret that was created by CDK. To get this secret, in a new window open AWS Secrets manager. Here, click on the secret that was created by CDK (it should start with AuroraBlogClusterSecret). Copy the Secret ARN to your clipboard and go back to the RDS Query Editor.

Next, use the Secret ARN as the Secrets Manager ARN and BlogDB as the name of the database. Next, press enter and click on Connect to Database.

Once signed in, create the posts table by executing the following query:

CREATE TABLE posts (
 id text UNIQUE,
 title text,
 content text
);

Testing the API

Next, visit the AppSync console and click on the API name to view the dashboard for your API.

Next click on Queries in the left hand menu to view the query editor. From here, we can test out the API by running the following queries and mutations:

mutation createPost {
  createPost(post: {
    id: "001"
    title: "My first post!"
    content: "Hello world!"
  }) {
    id
    title
    content
  }
}

query listPosts {
  listPosts {
    id
    title
    content
  }
}

query getPostById {
  getPostById(postId: "001") {
    id
    title
    content
  }
}

mutation updatePost {
  updatePost(post: {
    id: "001"
    title: "My updated post!"
  }) {
    id
    title
  }
}

mutation deletePost {
  deletePost(postId: "001")
}

Connecting a Client application

You can connect to an AppSync API using the Amplify libraries for iOS, Android, or JavaScript.

In this example, we’ll walk through how you can make API calls from a JavaScript application.

Client project setup

You first need to install the AWS Amplify libraries using either NPM or Yarn.

npm install aws-amplify

Next, configure the Amplify app at the root of your project using the Project Region, API Key, and GraphQL URL. This configuration is available in the cdk-exports.json file created by the CDK deployment. This configuration is usually done at the entry-point of your app:

  • Angular – main.ts
  • Vue – main.js 
  • React – index.js
  • Next.js – _app.js
// Your app entrypoint
import Amplify from 'aws-amplify';
import { AppsyncCdkRdsStack } from './cdk-exports.json';

Amplify.configure({
  aws_appsync_region: AppsyncCdkRdsStack.ProjectRegion,
  aws_appsync_graphqlEndpoint: AppsyncCdkRdsStack.AppSyncAPIURL,
  aws_appsync_apiKey: AppsyncCdkRdsStack.AppSyncAPIKey,
  aws_appsync_authenticationType: "API_KEY", //Primary AWS AppSync authentication type
});

Fetching data (queries)

To query data, you can use the API category, passing in the query that you’d like to use. In our case, we’ll use the same query from above:

import { API } from 'aws-amplify'

const query = `
  query listPosts {
    listPosts {
      id title content
    }
  }
`

async function fetchPosts(){
  const data = await API.graphql({ query })
  console.log('data from GraphQL:', data)
}

Creating and updating data (mutations)

To create, update, or delete data, you can use the API category, passing in the mutation that you’d like to use along with any variables. In this example, we’ll look at how to create a new post:

import { API } from 'aws-amplify'

const mutation = `
  mutation createPost($post: CreatePostInput!) {
    createPost(post: $post) {
      id title content
    }
  }
`

async function createPost() {
  await API.graphql({
    query: mutation,
    variables: {
      post: { id: '001', title: 'My first post', content: 'Hello World' }
    }
  })
  console.log("post successfully created!")
}

Real-time data (subscriptions)

To subscribe to real-time updates, we’ll use the API category and pass in the subscription we’d like to listen to. Any time a new mutation we are subscribed to happens, the data will be sent to the client application in real-time.

import { API } from 'aws-amplify'

const subscription = `
  subscription onCreatePost {
    onCreatePost {
      id title content
    }
  }
`

function subscribe() {
  API.graphql({
    query: subscription
  })
  .subscribe({
    next: postData => {
      console.log('postData: ', postData)
    }
  })
}

Conclusion

If you’d like to continue building this example out, you may look at implementing things like additional data sources, argument-based subscriptions, or creating and then querying against a multiple tables.

If you’d like to see the code for this project, it is located here.