AWS Public Sector Blog

How to build a multilingual document summarization application using Amazon Bedrock

AWS branded background design with text overlay that says "How to build a multilingual document summarization application using Amazon Bedrock"

We introduced Amazon Bedrock to the world in April 2023, delivering an entirely new way to build generative artificial intelligence (AI) applications. It’s exciting to experiment with and test the various capabilities of Amazon Bedrock in the AWS Management Console. However, integrating these new generative AI capabilities in your application requires understanding various aspects of AWS serverless services.

India has 28 states and eight union territories. Today, the Indian constitution recognizes 22 major languages of India, including Hindi, Marathi, Gujarati, Malayalam, Tamil, Telugu, Assamese, Bengali, and Punjabi. Most of the state government policies and documents are available primarily in a local language, English, or both. Extracting actionable intelligence and summarizing information in those large numbers of documents is a common challenge faced by government staff, and it currently involves a huge amount of manual effort. You can address this using the Retrieval-Augmented Generation (RAG) approach with Amazon Bedrock.

Imagine you have hundreds of documents in any of the major languages used in India describing state policies and other important governance-related content. Due to the limited size of prompts accepted by many large language models (LLMs), you may have to select relevant parts of these documents to be included as context into prompts. The solution is to transform all your documents into embeddings and store them in a vector database, such as OpenSearch.

When a user wants to query a corpus of documents, you can transform the user’s natural language query into a vector embedding and perform a similarity search on the vector database to find the most relevant content for answering the query. This helps the LLM generate more accurate and relevant answers.

In this post, we showcase a RAG application that can search and query across multiple Indian languages using the Cohere Embed – Multilingual model and Anthropic Claude 3 on Amazon Bedrock. This post focuses on Indian languages, but you can use the approach with other languages that are supported by the LLM.

In this post, we explore how to use serverless architecture to build a multilingual Q&A and document summarization application using Amazon Bedrock, Amazon Simple Storage Service (Amazon S3), Amazon API Gateway, and AWS Lambda.

The sample code in this post is only for testing purposes. You are advised to write your own code for Lambda functions in production environments.

What is Amazon Bedrock?

Amazon Bedrock is a fully-managed service that offers a choice of high-performing foundation models (FMs)—along with a broad set of capabilities—to build generative AI applications while simplifying development and maintaining privacy and security. With knowledge bases for Amazon Bedrock, you can connect FMs to your data sources for RAG in just a few clicks. Vector conversions, retrievals, and improved output generation are all handled automatically.

Solution overview

The following diagram illustrates the application architecture.

Figure 1. Architecture diagram of multilingual RAG application.

The workflow includes the following steps:

  1. The user sends a prompt for querying the documents to the REST API.
  2. Amazon API Gateway sends the user prompt as an event to a Lambda function.
  3. The Lambda function invokes Amazon Bedrock API and sends the prompt to the Anthropic Claude 3 Sonnet model.
  4. The LLM parses the prompt and does a similarity search with vector embeddings.
  5. Enhanced context from the knowledge base is used to generate a text response.
  6. The final text response is returned by Amazon Bedrock to the Lambda function.
  7. The user receives the final response through RESI API.

This is an event-driven architecture composed of individual AWS services that are loosely integrated with each other, with each service handling a specific function. It uses AWS serverless technologies, allowing you to build and run your application without having to manage your own servers. All server management is done by AWS, providing many benefits such as automatic scaling and built-in high availability, letting you take your idea to production quickly.

Prerequisites

In this section, we go through the prerequisite steps to complete before you can set up this solution.

  1. Pick an AWS Region from the Amazon Bedrock supported Regions. This post uses the us-east-1 Region
  2. Enable model access through Amazon Bedrock.

You can add access to a model from the Amazon Bedrock console. For this walkthrough, you need to request access to the Anthropic Claude 3 Sonnet and Cohere Embed – Multilingual model on Amazon Bedrock. For more information, see Manage access to Amazon Bedrock foundation models.

Figure 2. Model access in AWS console.

Walkthrough

To build the solution, create an S3 bucket, Amazon Bedrock knowledge base, Lambda function, an inline AWS Identity and Access Management (IAM) policy, and then test the results by integrating the Lambda function with API Gateway.

Create an S3 bucket

  1. Use the steps in Create your first S3 bucket to create a private S3 bucket.
  2. Upload a PDF document to the S3 bucket. In this post, we use a document in the Marathi language. This public-facing PDF document contains the electric vehicle policy for the government of Maharashtra, India. You may use documents in any of the supported Indian languages.

Follow the steps to create a knowledge base

  1. On the Choose data source page, select the S3 bucket created earlier.
  2. For Chunking and parsing configurations, select Default.
  3. On the Select embeddings model and configure vector store page, choose Embed Multilingual v3 by Cohere.
  4. In the Vector database section, choose Quick create a new vector store.
  5. Select Create knowledge base.
  6. Once knowledge base is successfully created, choose the data source and select Sync.

The time it takes to create the knowledge base depends on the amount of data you provided. When the knowledge base is finished being created, the Status of the knowledge base changes to Ready. Take note of the Knowledge base ID.

Follow the steps to create and configure a Lambda function.

1. Create a Python function by following the steps in Building Lambda functions with Python.

2. Go to the Functions page of the Lambda console and select the name of a function.

3. Select Configuration, then General configuration, and choose Edit. Change the timeout to 1 minute and choose Save.

4. Choose Configuration, and then choose Permissions. Choose the default role. This opens the Roles page in the IAM.

5. Under Add permissions, choose Create inline policy.

6. Select JSON and paste the following policy in Policy editor. This inline policy restricts access to only the specific knowledge base created earlier and Anthropic Claude 3 Sonnet. This configuration supports the principle of least privilege. Provide the appropriate AWS Account ID and Knowledge base ID noted in the earlier step.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "bedrock:InvokeModel",
                "bedrock:Retrieve"
            ],
            "Resource": [
                "arn:aws:bedrock:us-east-1:<AWS Account ID>:knowledge-base/<Knowledge base ID>",
                "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-sonnet-20240229-v1:0"
            ]
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": "bedrock:RetrieveAndGenerate",
            "Resource": "*"
        }
    ]
}

7. Choose Next. Enter a Policy name.

8. Choose Create Policy. The inline policy will be listed under Permissions policies of the Lambda execution role.

9. The following sample Python code illustrates how Amazon Bedrock API can be invoked to retrieve and generate a response from an Amazon Bedrock knowledge base. Paste the code in Lambda code source window and choose Deploy.

import json
#1. import boto3 library
import boto3
#2 Create a client object for the AWS Bedrock Agent Runtime service using the `boto3.client()` method.
client_bedrock_knowledgebase = boto3.client('bedrock-agent-runtime')
def lambda_handler(event, context):
    #3  Print and store the value of the 'prompt' key from the `event` dictionary. The 'prompt' key contains the user's input or query.
    print(event['prompt'])
    user_input=event['prompt']
    # 4.Invoke the `retrieve_and_generate` method of the `client_bedrock_knowledgebase` object
    client_knowledgebase = client_bedrock_knowledgebase.retrieve_and_generate(
    input={
        'text': user_input
    },
    retrieveAndGenerateConfiguration={
        'type': 'KNOWLEDGE_BASE',
        'knowledgeBaseConfiguration': {
            'knowledgeBaseId': '<knowledge_base_ID>',
            'modelArn': 'arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-sonnet-20240229-v1:0'
                }
            })

# 5. Print the response from the retrieve_and_generate method. This response contains the generated text based on the user's input or query.

    print(client_knowledgebase)
    final_response=client_knowledgebase['output']['text']
    return {
        'statusCode': 200,
        'body': final_response
    }

10. Use test event to invoke and test the Lambda function. Figure 3 shows a sample event, which provides a prompt. The event JSON is converted into an object and passed to your code’s handler method for processing.

Figure 3. Test event for Lambda function.

You will see a similar result under the Execution results tab:

Figure 4. Lambda execution result.

Follow the steps to create and configure an API Gateway REST API.

1. On the API Gateway console, choose Get started with API Gateway

2. Choose to build REST API. For API details, choose New API.

3. Enter a suitable API name, such as multilingualapi. Choose Create API.

4. Choose Create Resource. Enter an appropriate Resource name. Choose Create Resource.

5. Choose Create method. Select GET for Method type. For Integration type, choose the Lambda function created earlier. Choose Create Method.For Method request, choose Edit.

6. For URL query string parameters, choose Add query string. For Name, enter prompt and select Required.

7. For Request validator, select Validate query string parameters and headers. Choose Save. For Integration request, choose Edit.

8. For Mapping templates, choose Add mapping template. For Content type, enter application/json.

9. Enter the template body, as shown in the following example using the Velocity Template language (VTL).

{
"prompt":"$input.params('prompt')"
}

10 Choose Save. Choose Deploy API. For Stage, select New stage.

11. For Stage name, enter Dev, then choose Deploy.

Follow the steps to test the REST API

  1. Select Resources, select GET method, and choose Test.
  2. For Query strings, enter prompt= इलेक्ट्रिक व्हेईकल खरेदी करणाऱ्या ग्राहकाला कुठले फायदे होतील?
  3. Choose Test. You will see a response similar to Figure 5.

You can also test the API with third-party tools such as Postman.

Figure 5. Test result of REST API.

Integrate REST API with the UI

You can use open source frameworks such as Streamlit to efficiently create interactive web-based applications in pure Python. You can integrate the application with Amazon Cognito for user authentication and authorization. For more information, refer deploy-streamlit-app in GitHub.

You can also choose any other framework of your choice to build the UI front, following the requirements.

Figure 6. Sample chatbot output.

Cleanup

To avoid future charges in your account, delete the resources you created in this walkthrough.

  1. Delete the S3 bucket.
  2. Delete the knowledge base.
  3. Delete the REST API.
  4. Delete the Lambda function.

Conclusion

In this post, we discussed how to build a serverless application to query a Marathi language document using API Gateway, Lambda, Amazon Bedrock, and Cohere Embed – Multilingual model. This solution can be used for other Indian languages including Hindi, Gujarati, Malayalam, Tamil, Telugu, Assamese, Bengali, Punjabi as well as English. You can further enhance security by using a private API and data encryption. This solution can help state and local governments in India to quickly harness the power of Amazon Bedrock and transform their operations. With the tools discussed in this post, you can get started with building powerful and scalable generative AI applications with broad model choice, consistent security, and responsible AI practices.