Breaking Language Boundaries: Multilingual GenAI Solutions with Amazon Bedrock

By Kelvin Kok, Chief Architect & CEO – Axrail
By Cara Lee, Account Manager – Axrail
By Brendan Child, Sr. AI/ML Partner Development Specialist – AWS
By Vasileios Vonikakis, Sr. AI/ML Partner Solution Architect – AWS

Axrail

Introduction

Generative AI enhanced solutions, such as virtual assistants and enterprise search, have already demonstrated their potential in transforming businesses through increased productivity and greater customer experience. In regions such as in Southeast Asia, where English is rarely the primary or only language, one of the greatest areas of opportunity is the ability to allow businesses to engage with their customers seamlessly in their chosen language.

In this blogpost, we’ll delve into how the multilingual capabilities of Generative AI are helping remove language barriers when enhancing customer experience and optimizing business processes. Specifically, we will breakdown 2 real-world multilingual Generative AI solutions by Axrail.AI, an AWS Advanced Tier Consulting Partner: a virtual assistant chatbot, and Generative AI-powered intelligent document processing, both developed using Large Language Models (LLMs) in Amazon Bedrock,. We will also shed light on how their multilingual capabilities are adding business value for customers who have adopted them.

Multi-lingual Generative AI Chatbot Assistant

With the increasing engagement of consumers through digital channels, effective customer engagement is a key priority across the B2C landscape. With Generative AI, customer service chatbots can be vastly improved from the traditional rule-based workflows, instead allowing users to interact in their preferred language, using a natural tone. This enhances the customer experience both in terms of functionality and multilingual accessibility.

Unlike traditional AI chatbots, Generative AI chatbots allow for contextualized understanding of user queries. This means an engagement isn’t interrupted due to misspelled words or the use of slang, and generates responses that are customized with the context of each individual customer query.

Offering multilingual capabilities is of paramount importance, especially in regions like ASEAN, where English is often not the customer’s first language. Additionally, by letting the Generative AI models handle the multilingual aspect, the architecture is simplified and decreases the time-to-market and allowing development teams to focus on new features.

High-level architecture

The following figure depicts the high-level architecture of the Multi-lingual Generative AI Chatbot assistant.

Fig. 1. High-level architecture flow of Axrail.AI’s multilingual Generative AI chatbot.

The high-level architecture flow of Axrail.AI’s Multi-lingual Generative AI Chatbot assistant solution is as follows:

A collection of documents, such as PDFs containing Frequently Asked Questions (FAQs), are uploaded to an Amazon S3 bucket and ingested to an Amazon Kendra Index. These documents act as the knowledge base for the LLM. The documents can either be only in English, or in one of the supported languages of Amazon Kendra.
A user submits a question through the chatbot interface on WhatsApp.
An AWS Lambda function acts as an orchestrator and sends the incoming question to Amazon Comprehend in order to identify the dominant language of the question. Alternatively, this step could also be accomplished by prompting an LLM.
Optionally, the question can be translated to English using Amazon Translate (more details on the section about multilingualism).
At the same time, the AWS Lambda function sends a retrieve query to the Amazon Kendra index, based on the user’s question (translated to English or not). The Amazon Kendra index returns relevant search results, including excerpts from the uploaded documents.
The AWS Lambda function formulates a prompt for Amazon Bedrock, incorporating the user’s query, the retrieved data as context, as well as the detected language of the question, and prompts the selected LLM on Amazon Bedrock to “return the answer in the same language”.
The LLM generates a concise response to the user’s question, in the respective language, based on the provided context.
The chatbot app sends the generated response back to the user.
Application data, such as conversation logs, is stored in Amazon DynamoDB for future use or analysis.

Multilingualism

In order to handle multilingual queries, the solution utilizes a layered approach.

First, the dominant language of the question is detected using Amazon Comprehend. If the language of the question is found to be different than the languages of the indexed documents, then it is translated to one of them using Amazon Translate. This will allow Amazon Kendra to find relevant context from within the indexed documents.

Then, following a Retrieval Augmented Generation (RAG) approach, the retrieved context, along with the question (original or translated) is passed to Amazon Bedrock, where an LLM will generate an answer to the question, given the retrieved context. More importantly, the LLM is prompted to return the response in the dominant language of the original question (as detected by Amazon Comprehend). As such, the chatbot’s responses are not only contextually accurate but also linguistically coherent.

The Axrail team has found that Anthropic’s Claude LLMs, versions 2, 2.1 and especially 3, exhibit strong multilingual capabilities for the ASEAN languages (e.g. Bahasa Indonesia, Malay, Mandarin, Tagalog, Vietnamese, Tamil, etc.), being able to receive the question and context in one language and generate the correct answer in another.

This layered approach enables the Axrail.AI chatbot to interact effectively with users in multiple languages such as English, Chinese, Bahasa Melayu, and Indonesian to name a few, even if all the documents are indexed in a single language like English. In case the original question is posed in a language that is not supported by the LLM, the chatbot can inform the user that it is not able to answer the question, and it will suggest alternative ways of reaching out to the team (e.g. email, live human agent, etc).

An alternative approach, which can also result in multilingualism, is to use multilingual embeddings (e.g. Amazon Titan Text Embeddings). If you are interesting to know more about this approach, please take a look at this blogpost.

Customer references

In this section we will describe the business impact that Axrail.AI’s multilingual Generative AI chatbot assistant had on 2 customers who have adopted it.

PETRONAS Sepang International Circuit (PETRONAS SIC)
Sepang International Circuit (SIC) is a motorsport race track in Sepang, Selangor, Malaysia. Dubbed as the crown jewel of Malaysian motorsports, SIC is best known for its endless motorsports events and activities, such as the F1 Grand Prix (F1), MotoGP™ and others, as well as corporate, lifestyle, and exhibition events.

Previously, while SIC had utilized a WhatsApp channel to respond to customer inquiries, these queries were responded to manually by a customer service team. The key business challenges faced were the number of agents required to manage customer queries which increased linearly with the volume of inquiries. During peak hours, this resulted in longer waiting times for customers. Moreover, customer service responses were only available when the agent was present, and the language support was limited by their linguistic capabilities.

Axrail.AI adapted their Multi-lingual Generative AI Chatbot assistant to SIC’s needs, using WhatsApp as the interface, to redefine the way fans experience races at the iconic circuit. The choice of WhatsApp as the user interface allows for ease of adoption by users, who largely leverage this application on a daily basis. This solution provides 24/7 availability and makes sure that customer queries can be responded promptly on demand. Axrail.AI’s Generative AI Chatbot, is powered by Amazon Bedrock and offers the following features:

Frequently-Ask-Questions (FAQ): The chatbot can answer multiple questions about upcoming or ongoing racing events within WhatsApp similar to a human agent.
Merchandise Purchases: The chatbot allows audiences to efficiently browse through the available items, add items to their cart and make secure payments.
Multilingual Responses: The chatbot is able to handle multiple local languages, such as English, Mandarin (Chinese), Bahasa Melayu and Tamil, while automatically adjusting to the language of the user’s request.

Fig. 2 depicts various UI snapshots of the SIC Multi-lingual Generative AI Chatbot assistant. You can watch the promotional video, which describes the above features here.

Fig 2. UI snapshots of the SIC Multi-lingual Generative AI Chatbot assistant.

Axrail.AI’s Multi-lingual Generative AI Chatbot assistant was introduced 2 months before the Petronas MotoGP race, on 9^th-12 ^thNovember 2023, which garnered a global audience size of more than 160,000 spectators across 3 days. During that trial period, before and during the racing event, hundreds of users interacted with the chatbot, with an average response time of 9 seconds for all of the features mentioned above. The chatbot handled 60% of English, 20% Malay, and 20% Mandarin queries and automatically generated responses to suit the users’ needs.

During the trial period, SIC tested in parallel their existing chatbot system, based on conversational flow technology and human agents, with Axrail.AI’s Multilingual Generative AI Chatbot assistant. The timeliness and quality of responses of Axrail.AI’s solution was found to be superior compared to the existing approach and helped SIC to migrate from a traditional AI chatbot to a Generative AI based chatbot.

Azhan Shafriman Hanif, Chief Executive Officer of Sepang International Circuit has mentioned:
“The introduction of a MotoGP™ Malaysia Generative AI chatbot, developed by Axrail.AI leveraging Amazon Web Services (AWS) services, offers an unparalleled user journey and effortlessly supports multiple languages, provides pinpoint location services, and simplifies merchandise browsing to a mere touch. This helped us to reduce the need of human agents and traditional AI chatbots, and elevate fans’ experience during the race and better engage fans internationally with the multilingual feature”.

National Library Board of Singapore

The National Library Board (NLB) promotes reading, learning, information literacy and a greater appreciation of Singapore’s heritage and identity by offering a diverse range of programmes and services, resources, and digital innovations.

Axrail.AI worked with NLB in order to adapt their Multi-lingual Generative AI Chatbot assistant technology to allow users to ask questions about the content of specific books, and if interested, borrow them using their NLB account. The Generative Chatbot assistant is called ChatBook and supports Q&A functionality for the content of the book “Seven Hundred Years: A History of Singapore”. In this use case, the chatbot is configured to respond solely in English to all queries, regardless of the query’s language. This decision is specific to NLB’s use case – to uphold the academic standards required to deliver educational content based on the book and learning materials, and to preserve the integrity and authenticity of the author’s original work. Nevertheless, the solution remains highly proficient in understanding and processing queries in a variety of language, while maintaining quality and reliability of the bot’s content. The underlying technology remains flexible and robust, catering to the requirements of various projects.

Fig. 3 depicts ChatBook UI snapshots demonstrating its capability to provide accurate and contextually appropriate responses based on the user’s input, regardless of the language used to pose the question.

Fig 3. UI screenshots of the ChatBook agent, allowing users to ask questions about the content of specific books, across different languages, and potentially borrow them.

NLB plays a pivotal role in providing access to a vast repository of books and documents that are crucial for educational purposes. However, students and teachers often face challenges in efficiently accessing and retrieving information related to Singapore’s history content. These challenges include navigating through extensive collections to find relevant information such as historical events and figures. Moreover, there may be varied levels of digital literacy among readers, making it difficult for some to leverage the full potential of the book and library resources. To keep on top of trends and continue serving the public in a time when books seem passé, NLB partnered with Axrail to develop ChatBook to draw upon the materials that NLB has, and provide instant, on-demand access to detailed explanations, summaries and reliable responses. This has since garnered more than a thousand users who interacted with ChatBook.

Gene Tan, Chief Innovation Officer of National Library Board, has mentioned:
“In commitment to nurture Readers for Life, Learning Communities and a Knowledgeable Nation across Singapore, NLB has adopted Generative AI solutions to enhance how Singaporeans discover, learn and experience. In this journey, Axrail has partnered with us to test a versatile chatbot – both web-based and WhatsApp, powered by Amazon Bedrock, to
empower the public with quick access to information about books in a conversational manner. The current interactive chatbot prototype, ‘ChatBook’ allows users to input queries and receive comprehensive responses, based on the book, ‘Seven Hundred Years: A History of Singapore’ and NLB’s online resources. User feedback on ChatBook has been positive.”

Intelligent Document Processing powered by Generative AI

Aside from multilingual chatbots which can improve customer experience, Generative AI has the potential to also transform business operations. Currently, many organizations grapple with the burden of manually processing large volumes of various document types in multiple languages. In light of these challenges, many organizations are exploring the adoption of Optical Character Recognition (OCR) and Intelligent Document Processing (IDP) technologies.

Axrail.AI has introduced their SmartEye solution, an Intelligent Document Processor that leverages Amazon Textract as the key AWS service. In combination with Amazon Bedrock, it can leverage the power of LLMs to navigate the intricacies of multiple languages. As such, it can not only enhance the accuracy of data extraction, but also open up new possibilities for business operations at a global scale. Furthermore, it allows for handling of complex data formats such as paragraphs, sentences, keywords, and contextualization of natural language queries. It does not rely on predefined templates and excels at learning on the fly, unlocking barriers and limitations of Robotic Process Automation (RPA) solutions which require manual configuration to adapt to changes in processes or systems. Across Axrail.AI’s customers, SmartEye has helped to increase operational efficiencies – accelerating document processing duration from hours to minutes; a customer reduced the size of the manpower required for this process from 5 to 2 employees. Moreover, it enhances detection accuracy while reducing human errors. Subsequently, it allows business workflows to be streamlined when integrated with existing Enterprise Resource Platform (ERP) and Customer Relationship Management (CRM) systems.