Front-End Web & Mobile
Introducing Merged APIs on AWS AppSync
AWS AppSync is a serverless GraphQL service that makes it easy to create, manage, monitor and secure your GraphQL APIs. Within an AppSync API, developers can access data across multiple different data sources including Amazon DynamoDB, AWS Lambda, and HTTP APIs. As the service continues to grow in adoption, our customers have faced challenges related to team collaboration across multiple teams and AWS accounts within an organization. Each AppSync API has a single GraphQL schema and configured data sources, resolvers, and functions.
When exposing multiple microservices through a single GraphQL API endpoint, multiple teams owning a portion of the API have to work simultaneously in the same API. Multiple developer teams working on the same API without proper isolation and guardrails can lead to accidental breaking changes, with no effective way for such teams to avoid stepping on each other’s toes when, for example, a developer in team A pushes an API change that could break functionality team B previously implemented and vice-versa. Another challenge is support and maintenance; when you have a single GraphQL API, it is difficult to troubleshoot whether there is an issue in one part of the Graph. For Enterprises who have multiple business domains in the same Graph, no single human can understand the entire Graph and maintenance becomes a nightmare.
Today, we are pleased to announce the general availability of Merged APIs in AWS AppSync. Merged APIs enable teams to merge resources, including types, data sources, functions, and resolvers, from multiple source AppSync APIs into a single, unified AppSync endpoint. At launch, you can merge up to 10 source AppSync APIs into a Merged API. In the frontend, clients only need to interact with a single endpoint to retrieve data across multiple source APIs. In the backend, development teams can create, update, test, and deploy their independent source APIs as part of a CI/CD pipeline. Once they have approved their changes, they can merge their changes to the Merged API endpoint in order to make them available to clients without blocking on other changes from other source APIs. Execution of queries, mutations, and subscriptions on a Merged API is handled by the AppSync service, providing the same monitoring and performance experience as a source AppSync API today.
The diagram below shows an example how a Merged API is configured with multiple source APIs across different teams:
Figure 1: Example of Merged API configured across multiple Source APIs
Schema Directives
When there are conflicts across Source Schema definitions, new GraphQL directives can be used to provide the flexibility to resolve conflicts.
- @canonical: if two or more source APIs have the same GraphQL type or field, one of the APIs can annotate their type or field as canonical, which takes precedence when merging the schemas. Conflicting types without this directive in other source APIs are ignored when merged.
- @hidden: teams may want to remove or hide specific types or operations in the target API so only internal clients can access specific typed data. With this directive attached, types or fields are not merged to the Merged API target.
- @renamed: There are use cases where different APIs have the same type or field name. However they all need to be available in the merged schema. A simple way to have these types in the Merged API target is by renaming one of them for handling any naming conflicts.
Auto Merge
AppSync provides multiple ways of merging the changes from the source API(s) to the Merged API. By default, these merge operations are manual. However, you can enable the auto merge mode which will submit a merge operation to the Merged API whenever a source API has an update. The way this works is that the Merged API subscribes for changes to the source API which has enabled auto-merge. When there is a change, the AppSync service assumes the configured merged API execution role on the Merged API and submits a StartSchemaMerge
operation on behalf of the Merged API owner to merge in the changes. Whenever you change the types, data sources, resolvers, or functions of a source API, auto merge will trigger an update to the corresponding Merged API.
Comparison to other approaches
There are many solutions and patterns in the GraphQL community for combining GraphQL schemas and enabling team collaboration through a shared graph. AppSync Merged APIs adopts a “build time” approach to schema composition, where source APIs are combined into a separate, Merged API. An alternative approach is to layer a “run time” router across multiple source APIs or sub-graphs. In this approach the router receives a request, references a combined schema that it maintains as metadata, constructs a request plan, and then distributes request elements across its underlying sub-graphs/servers.
AppSync Merged APIs can only be associated with AppSync source APIs. If you need support for schema composition across AppSync and non-AppSync sub-graphs you can connect one or more AppSync GraphQL API and/or Merged APIs into a router-based solution.
Example Scenario : Creating a Merged API for Book Reviews and Recommendations
In the following example, we will create a Merged API that will power the backend of a book review and recommendations website, similar to Goodreads. In this example, we will have multiple source APIs which are owned and developed by separate teams including:
- Books Source API → This team is responsible for storing the data related to each book in the sites catalog. The team manages a source AppSync API which provides the ability to query the book metadata and add, update, and delete books from the catalog.
- Authors Source API → This team is responsible for storing the data related to authors for the books in the site catalog. The Authors source AppSync API will provide the ability to query information related to an Author including name and email as well as mutations to add, update, and delete authors in the catalog.
- Reviews Source API → This team is responsible for storing the data related to reviews for the books in the sites catalog. Users of the site can add reviews for a book including comments and ratings. The Reviews source AppSync API will provide the ability to query reviews by book or by author as well as add, update and delete reviews.
- Users Source API → This team is responsible for handling the data related to users in the site. Users of the site can review books as well as create reading lists. The Users source AppSync API will provide the ability to query information related to a given user as well as the reading list of a given user on the site. User entries can be added, updated, and deleted within this source API.
Creating the Source AppSync APIs
In this section, we will create the source AppSync APIs that will make up our Merged API for the book review site. This example uses Amazon DynamoDB as the data source. However you can use other data sources as well supported by AWS AppSync based on your use case.
Creating the Books Source API
- Navigate to the AppSync console and click on
Create API
. In Step 1 of the API creation wizard selectGraphQL APIs option
,Design from scratch
and clickNext.
- In step 2 of the wizard, we add the Books API metadata so that the Merged API owner can contact the books team when investigating any issues if necessary.
Note: You can also use private API features while creating the Source API if needed for your use case.
- In step 3 of the wizard, we generate a Book type backed by a DynamoDB table in order to store the Book metadata.
- Note that in our model, we include an
authorId
field which is required for each book. This field is the key which will be used to retrieve metadata associated with an Author from the Authors source API. When building the source schema, it is recommended to identify your types by unique keys such as an ID field to make it easier to join data across different source APIs. - In the DynamoDB table, we add indexes which will allow us to query the books by a given author or a given publisher.
- Note that in our model, we include an
- In Step 4 of the wizard, we review the API details and then click on
Create API
. It starts creation of the API and DynamoDB table.
Creating the Authors Source API
- Again, navigate to the AppSync console and click on
Create API
. In Step 1 of the API creation wizard selectGraphQL APIs option, Design from scratch
and clickNext.
- In step 2 of the wizard, we add the Authors API metadata so that the Merged API owner can contact the authors team when investigating any issues if necessary.
- In step 3 of the wizard, we generate an Author type backed by a DynamoDB table. The model for this table includes id, name, bio, contactEmail, and nationality of the author.
- In Step 4 of the wizard, we review the API details and then click on
Create API
. It starts creation of the API and DynamoDB table.
Creating the Reviews Source API
- Again, navigate to the AppSync console and click on
Create API
. In Step 1 of the API creation wizard selectGraphQL APIs option, Design from scratch
and clickNext.
- In step 2 of the wizard, we add the Reviews API metadata so that the Merged API owner can contact the reviews team when investigating any issues if necessary.
- In step 3 of the wizard, we generate the Review type backed by a DynamoDB table. The Review type will include a
reviewerId
which references the user that wrote the review,authorId
that references the author of the book being reviewed, and abookId
that references the book the review is about as well as anid
,comment
,rating
,createdAt
timestamp, andupdatedAt
timestamp. We will add indexes to the DynamoDB table to query reviews for a given book and reviews by a given reviewer.
Creating the Users Source API
- Again, navigate to the AppSync console and click on
Create API
. In Step 1 of the API creation wizard selectGraphQL APIs option, Design from scratch
and clickNext.
- In step 2 of the wizard, we add the Users API metadata so that the Merged API owner can contact the users team when investigating any issues if necessary.
- In step 3 of the wizard, add a User type backed by a generated DynamoDB table with id, name, and email fields.
- In Step 4 of the wizard, we review the API details and then click on
Create API
. It starts creation of the API and DynamoDB table.
Creating the Merged AppSync API
Now that we have created all four source AppSync APIs, it is time to create the Merged API.
- Navigate to the AppSync console and click on
Create API
. In Step 1 of the API creation wizard selectMerged APIs option
and clickNext.
- Similar to before we add the API name and contact info in step 2.
- For this example, we will create a new service role for the Merged API.
- The Merged API execution role is responsible for securely accessing the merged source API resources during queries and mutations using a new IAM action:
appsync:SourceGraphQL
. - AppSync requires this permission on the top level field ARNs within a request for each top level field that has a configured resolver merged from a source API. You can choose to allow or deny specific top-level fields similar to how the IAM authorization mode works in AppSync today for the
appsync:GraphQL
permission. For non-top level fields, AppSync requires permission on the source API ARN itself. - For example, here is the policy of the Merged API execution role.
- The Merged API execution role is responsible for securely accessing the merged source API resources during queries and mutations using a new IAM action:
-
- If you want to restrict access to just certain GraphQL operations, you can do this for the root `Query`, `Mutation`, and `Subscription` fields.
-
- For more on IAM authorization for Merged APIs, refer to the documentation.
- Next, we will add source API associations to the Merged API in step 3. AppSync supports adding source APIs from your account and source APIs from other accounts which have been shared via AWS Resource Access Manager (AWS RAM).
Note: You can also use private API features while creating the Merged API if needed for your use case.
- For now, we will add the source APIs from our account that we have just created. In a follow up blog, we will go through how to share a source API from another account in AWS RAM.
- Click the
Add Source APIs
button and check theUsersAPI, AuthorsAPI, ReviewsAPI, and BooksAPI
sources. Finally, clickAdd Source APIs
to confirm. Finally, clicknext
to finish step 3. - Note : Associating a source API requires permissions on both resources in the association. The operation requires the
appsync:AssociateSourceGraphqlApi
action on the Merged API and theappsync:AssociateMergedGraphqlApi
on the Source API. In contrast, disassociation operations only require permission on one of the resources in the association. For example, theDisassociateSourceGraphqlApi
requires permission on the Merged API to remove a source API from it.DisassociateMergedGraphqlApi
can be used to disassociate a source API from its Merged API and this action does authorization on the source API.
- In step 4 of the wizard, we will select the authorization mode. In this example, we will use the default API key primary authentication mode to match the primary authentication mode of all source APIs.
- In order for associations to be compatible, a Merged API must include the primary authentication mode of each source API either as a primary or additional authentication mode in its configuration.
- In step 5 of the wizard, review the configuration and click
Create API.
- The creation of the Merged API will associate all the source APIs that were configured and merge the source resources to the unified endpoint. The flash bar at the top of the page is used to indicate whether any of these associations failed due to a conflict or not.
- The status of each source API association is visible in the main page for your Merged API. Here you can find information regarding the current merge status, last successful merge, owner contact, and links to the source APIs that have been associated.
Note: You can hover and click over the merge status column to get a detailed message related to the success or failure of each individual source API merge:
- Now that your Merged API is created, we can view the schema to see that all types from all source APIs are available in the Merged API endpoint.
Joining Data Across Source APIs
Now that we have successfully merged the schemas from four different Source APIs, we can access the data from each source API independently. However, the true power of the Merged API architecture is the ability to join data across different teams. In this example, we have many opportunities to join the data in order to provide a more useful experience for clients. For example, when querying the Merged API endpoint we may want the ability to get the basic information about an author while also returning a paginated list of books in the catalog that author wrote. Going even a step further, we might want to also retrieve the list of reviews for each book in the list.
In order to join data across different source APIs, we must ensure that each type we are joining has a well known primary key field or fields that can be used to reference this relation in another source API. It is common to use an ID field to identify each type as we have done in this example. For each Book type in the Books source API, we store an authorId
field which can be used to join data about the Author from the Authors source API. The authorId acts as the “key
” for retrieving data about an author. Next, we will walk through how we can use these “key
” fields, to join data.
Joining Books Data with Authors Data Using a Field Resolver
- The first step in joining the data between the Books and Authors types is to define a Book type within the Authors source API. This Book type will act as a sub-set of the Book definition and only include the fields which are relevant in the Authors source API.
- Navigate to the Authors source API from the link in the Merged API table and add the following definition to the schema. Click
Save Schema
in the top right hand corner.
- Next, we add a resolver on the
Book.author
field which will return the author of a given book. In the Book service, it stores the id of the author as theauthorId
field. Our resolver will access this authorId field from the context.source object as it will be retrieved by the parent resolver owned by the Book source API. - Create a function called
GetAuthorByParentAuthorId
in the Authors source API by navigating to functions and clickingCreate function
in the top right hand corner. This function will use the DynamoDB AuthorsTable as a data source.
- The function will simply call a
GetItem
on the DynamoDB table using theauthorId
from thecontext.source
object. We write the function code in JavaScript like this:
export function request(ctx) {
return {
operation: 'GetItem',
key: {
id: {
'S': ctx.source.authorId
}
}
}
}
export function response(ctx) {
if (ctx.error) {
util.error(ctx.error.message, ctx.error.type, ctx.result)
}
return ctx.result
}
- Click
Create
in the top right hand corner to create the function. - Now that the function is created, we will use it in a Pipeline resolver attached to the
Book.author
field in the Authors source API. Navigate toBook.author
field in Resolvers section and clickAttach
.
- We use the default request and response code for the Pipeline resolver and add the function we just created. Click
Create
in the top right hand corner to create the resolver.
- Now that we have created this resolver, we can properly link the data between these source APIs. In order to make this resolver available in the Merged API, we need to merge this update. We can do this by navigating to the settings page of the Authors Source API and clicking the
Merge Now
button.
- Now that the resolver is successfully merged, verify that the Book type in the Merged API includes a field for author and that there is a pipeline resolver defined on that author field.
Joining Reviews Data with Authors Data Using a Field Resolver
- Next, we will add a resolver to join the reviews data with the authors data.
- Staying in the Authors source API, we will add another type to the Source API schema:
- This type will be used to add the ability to retrieve the author for a given Review in a single query.
- We now create a pipeline resolver on the
Review.author
field in the Authors source API. The pipeline resolver should use the default request and response resolver code. We will add the same function namedGetAuthorByParentAuthorId
as we used above to this resolver as well because the Review type also references authors using anauthorId
field. - Once the pipeline resolver is added, we again merge the update to the Merged API via the
Merge Now
button in the settings page.
Joining Books Data with Reviews Data Using a Field Resolver
- Next, we will add a resolver to join the reviews with the books from the sites catalog.
- Navigate to the Reviews source API.
- Since each Review type has a
bookId
indicating which Book the review is about, we can add a reference Book type in the Reviews source API which adds a paginated list of Reviews for a given book like so:
- Since each Review type has a
- We now create a function with the
ReviewsTable
as a data source that will query the reviews table by book id in order to return a paginated list of reviews for each book. The book id is accessible via the id field of the Book type. We access this id field through thectx.source
object. We will use the following code to JavaScript code for the function and name itGetReviewsByBook
.
export function request(ctx) {
return {
operation: 'Query',
query: {
expression: '#bookId = :bookId',
expressionNames: {
'#bookId': 'bookId'
},
expressionValues: {
':bookId': util.dynamodb.toDynamoDB(ctx.source.id)
}
},
index: 'reviews-by-book-index',
scanIndexForward: true,
select: 'ALL_ATTRIBUTES'
}
}
export function response(ctx) {
if (ctx.error) {
util.error(ctx.error.message, ctx.error.type, ctx.result)
}
return ctx.result
}
- After creating the new function in the Reviews source API, we repeat the same process as above, adding this function to a pipeline resolver on the
Book.reviews
field and merging this update to the Merged API.
Joining Authors Data with Reviews Data Using a Field Resolver
- Staying in the Reviews source API, we also want to join the data between authors and reviews in order to be able to retrieve reviews for a given author. We add the following author type in the Reviews source API to enhance the type and provide the ability to join this data.
- Again, we create a function with the Reviews table as a data source. It will query the reviews table by author id in order to return a paginated list of authors for each book. The id for the author type of interest is found in the
ctx.source
object. We will use the following code to JavaScript code for the function and name itGetReviewsByAuthor
.
export function request(ctx) {
return {
operation: 'Query',
query: {
expression: '#authorId = :authorId',
expressionNames: {
'#authorId': 'authorId'
},
expressionValues: {
':authorId': util.dynamodb.toDynamoDB(ctx.source.id)
}
},
index: 'reviews-by-author-index',
scanIndexForward: true,
select: 'ALL_ATTRIBUTES'
}
}
export function response(ctx) {
if (ctx.error) {
util.error(ctx.error.message, ctx.error.type, ctx.result)
}
return ctx.result
}
- Again, we add this function to a pipeline resolver with default request and response resolver code. This time we attach it to the
Author.reviews
field we added. We merge the update to the Merged API by clicking theMerge Now
button in the settings page for the Reviews source API.
Joining Authors Data With Books Data Using a Field Resolver
- Next, we will add the ability to get the books of a given author by joining the books data.
- First, we navigate to the Books source API and create a new reference Author type. The type will include the primary key id field as well as the new field that returns a paginated list of books written by a given author.
- The id for the Author will be accessible in the
ctx.source
object for this resolver. We will use this id to lookup all entries for books with anauthorId
that matches. - Create a new function in the Books source API called
GetBooksForAuthor
which uses the Books table as a data source. Add the following function code in JavaScript:
export function request(ctx) {
return {
operation: 'Query',
query: {
expression: '#authorId = :authorId',
expressionNames: {
'#authorId': 'authorId'
},
expressionValues: {
':authorId': util.dynamodb.toDynamoDB(ctx.source.id)
}
},
index: 'author-index',
scanIndexForward: true,
select: 'ALL_ATTRIBUTES'
}
}
export function response(ctx) {
if (ctx.error) {
util.error(ctx.error.message, ctx.error.type, ctx.result)
}
return ctx.result
}
- Add the function to a default pipeline resolver on the
Author.books
field and save the pipeline resolver. Merge the update to the Merged API using theMerge Now
button in the settings page.
Joining Reviews Data With Books Data Using a Field Resolver
- Finally, we will add a resolver to get the book information for a given review to join the book data with the review data. Each Review type in the Reviews source API has a
bookId
field which can be used in performing this join. - Staying in the Books source API, create a new reference type to add a book field to the Review type like so:
- Create a new function called
GetBookForReview
that will use thebookId
found in thectx.source
object and call aGetItem
on the Books table to retrieve the data related to this book. We will use the following code for this function:
export function request(ctx) {
return {
operation: 'GetItem',
key: {
id: {
'S': ctx.source.bookId
}
}
}
}
export function response(ctx) {
if (ctx.error) {
util.error(ctx.error.message, ctx.error.type, ctx.result)
}
return ctx.result
}
- Add the function to a default pipeline resolver on the
Review.book
field and create the pipeline resolver. Merge the update to the Merged API using theMerge Now
button.
Challenge: Add your own resolver in the Users source API to retrieve the User data for the User that reviewed a given book by adding a user field to the Review type. You can use the reviewerId
as the key for joining the User data. Merge the resolver to the Merged API to ensure that this data can be added.
Enabling Auto Merge
To Enable Auto Merge on a Source API, Navigate to Source API Settings, and click Edit . Then select Automatic merging
and click Save.
Testing the Merged API
First, lets add some test data to populate the data sources. Navigate to Merged API (Books Reviews Merged Api
) and click on Query Editor. Once you run a mutation mentioned below, note down the Ids to be used in subsequent queries/mutations.
Now we can test the response for queries that span multiple source APIs.
Here is an example of Listing Books along with Author and Review Information :
Other queries to try :
Testing the Source APIs Independently
In the above step, we used id field keys to join the data across different source APIs. These keys form the “interface” of the source API, enabling us to test the Source APIs independently without needing to setup a Merged API itself. Suppose we are members of the Books source API development team. We have created two resolvers to link the Books data to other source APIs on the Author.books
and Review.book
fields.
- In order to test the
Author.books
resolver in the Books source API, we can use the AppSync test endpoint to ensure that our JavaScript code for theGetBooksForAuthor
function is generating the correct request and response function evaluations. You can learn more about testing AppSync resolvers here . - Testing in the console or via the
EvaluateMappingTemplate
orEvaluateCode
operations does not actually call the data source that is configured for a given function or resolver. - In order to test the source API end to end, we can make use of the
@hidden
directive and create mock resolvers to mock the author id key. In order to do this, we can add a fieldgetAuthor
to the Query type in the Books source API that is hidden. Save the schema after adding this field.
- Navigate to the data sources menu in the Books source API and click
Create data source
. Create a newNone
data source calledMockAuthorDatasource
which will be used for this mock resolver. - Create a new function which uses the
None
data source created calledMockGetAuthor
. We will use the following code for the function:
export function request(ctx) {
return {
payload: {
id: ctx.args.id
}
};
}
export function response(ctx) {
return ctx.result;
}
- Add the
MockGetAuthor
function to a default pipeline resolver attached toQuery.getAuthor
. Save the pipeline resolver in order to allow the ability to mock the input to theAuthor.books
resolver. - Now, we will test the source API resolver itself. First we add some mock data for books using mutations:
- Next, we query the books of a given mock author id.
- Using the mocked
None
resolver, we have now successfully validated the source API resolver is working properly.
Cleanup
- Navigate to AppSync console, Select
Books Review Merged API,
click Delete , confirm the API Name and click Delete again. - Repeat step 1 for all the Source APIs as well.
- Navigate to DynamoDB console, Select tables associated with Source APIs (
BooksTable
,AuthorsTable
,ReviewsTable
andUsersTable
) , click Delete , select confirm and click Delete again.
Important things to know
- At Launch, A single Source API can be associated only with one Merged API.
- At Launch, A Merged API cannot be associated as a Source API to another Merged API.
- The number of source APIs per Merged API has a limit of 10 at launch.
- Merged API specific resolvers and functions are not supported at this time. To add custom logic specific to Merged APIs, you can add functions and resolvers to the Source APIs and merge the changes to your Merged APIs.
Get started today!
AWS AppSync Merged APIs are generally available in all AWS regions where AWS AppSync is available today. You can refer to AWS Regional Services List to find out the regions where AppSync is available. To learn more about Merged APIs, refer to the AppSync documentation or visit the AppSync product page for more general information on AWS AppSync. We can’t wait to see what you will build!
About the authors
Nicholas is a Senior Software Engineer who has been working on AWS AppSync for the past 3 years. He spends his work days focused on improving GraphQL query execution performance and weekends roaming around San Francisco with his dog Pippa. |
Venugopalan Vasudevan is a Senior Specialist Solutions Architect focusing on AWS Front-end Web & Mobile services. Venu helps customers build their front-end and mobile strategies on AWS, including maturing and enhancing their DevOps practices. |