Improving Patient Experience with Digital Front Door

In this blog, we demonstrate the capabilities of the digital front door using AWS services. The reader will understand the approach used to store a plan benefits document, create embeddings, process the user questions, and deliver an interactive chatbot. By demystifying policy details, healthcare payors can create a member experience that works better for everyone.

Healthcare payors are constantly seeking innovative ways to improve patient experience and engagement. One common issue that affects patient experience and engagement is understanding plan benefits documents. These are confusing, lengthy documents that patients must deal with to understand their plan benefits and coverage. Too often, patients are left in the dark about their healthcare coverage and what they owe, leading to financial hardship and avoidance of care.

One way to address this issue is by employing a cutting-edge “digital front door” chatbot, built using artificial intelligence (AI) services on Amazon Web Services (AWS). This allows patients to access plan benefits information 24/7, from their smartphones or computers, without having to dig through lengthy documents. Patients can get insights into covered services, out-of-pocket costs, and maximize their benefits.

This empowers patients to take charge of their healthcare, understand financial impacts, and manage their health effectively. When patients have a clear understanding of their plan benefits, they are more likely to schedule preventive care visits, adhere to prescription regimens, and effectively manage chronic conditions. This leads to better patient-provider relationships and delivers superior care management. Doctors can monitor patient health more effectively, improve patient engagement, leading to better outcomes.

Overview

Advances in natural language processing (NLP) and generative AI present new opportunities to automate certain customer service interactions. Chatbots and virtual assistants can act as a helpful digital front door, fielding common questions so agents can focus on high-priority issues. In this blog, we demonstrate the capabilities of a generative AI powered plan benefits chatbot powered by Amazon Bedrock. The solution uses embeddings and retrieval augmented generation (RAG).

Embeddings

Machine learning embeddings capture the semantic relationships between words, allowing algorithms to find words with similar meanings. Embeddings emphasize the notion of representing data in a numeric way giving a meaning, context, or description to data points. Neural embeddings are vector representations of text that capture semantic meaning. The solution described by this blog creates neural embeddings of a plan benefits document using Amazon Titan Embeddings from Amazon Bedrock.

These vector representations allow the LLM to understand the semantic meaning of the document. When a user asks a question, the chatbot can match the question to relevant passages in the document. This allows the chatbot to comprehend the plan details and respond accurately to user questions. Without neural embeddings, the chatbot would not have any context about the plan benefits and would not be able to meaningfully answer questions.

Neural embeddings are a critical component enabling NLP for this chatbot. By vectorizing the raw text of the benefits document, the chatbot matches user questions to relevant passages and generates accurate, contextual responses. The embeddings allow the large language model to comprehend the domain content like a human expert.

Retrieval Augmented Generation

The RAG technique in NLP combines the strengths of retrieval models with generative models. A retrieval model searches a large collection of documents to find relevant excerpts and passages to a question or conversation. The relevant excerpts are provided as additional inputs to a generative model, like one of the LLMs in Amazon Bedrock, to produce a response.

This technique allows the generative model to use external knowledge and produce more domain-specific, factual responses – critical for healthcare conversations. RAG models can provide detailed, nuanced answers by retrieving key medical contexts across plan documents, research papers, and standard operating procedures (SOP). For example, in a complex patient benefits conversation, RAG retrieves the relevant sections of the patient’s specific insurance policy, based on the incoming user query. The generative model then uses those relevant sections to answer the user’s question using the information contained in the document. This has the added benefit of enabling the model to cite where in the document it found the information used to formulate its response.

Compared to a standalone generative model, RAG reduces hallucination (an AI generated response that contains false or misleading information) and improves accuracy. And, compared to just a retriever, RAG allows for more fluent, conversational responses. By combining retrieval and generation, RAG takes the best of both worlds. The high-precision of retrieval complements the language mastery of LLMs. This makes RAG ideal for sensitive digital health tools like chatbots and virtual assistants.

Solution Approach

Adding a sample plan benefits document to an Amazon Simple Storage Service (S3) bucket triggers an AWS Lambda function. The lambda generates vector embeddings of the document content using Bedrock’s Titan Embeddings model. Then, Amazon OpenSearch Service indexes the embeddings.

Vector Engine for Amazon OpenSearch Serverless provides a simple, scalable, and high-performing vector storage and search capability. The chatbot API built with Amazon API Gateway and the Lambda function queries the indexed content using RAG, allowing the chatbot to read and understand the plan documents. Conversational history is maintained in Amazon DynamoDB so the chatbot can recall context from prior aspects of the conversation, just like a human would.

Testing queries across multiple foundation models from Bedrock demonstrates how this architecture could enable payors to offer 24/7 access to accurate, personalized plan explanations that drive better patient engagement.

Architecture

Consider a sample Summary of Benefits (SBC) coverage document from the Centers for Medicare & Medicaid Services as the plan benefits document. The document in Figure 1 contains detailed benefits coverage and the costs for the respective medical service. This document is filled with information that can be confusing for a member not familiar with healthcare terminology. The digital front door architecture simplifies this experience by using generative AI to summarize portions of the document relevant to the member’s query.

Figure 1: Summary of Benefits coverage document

The architecture has two main components: document ingestion and content retrieval.

Figure 2: Digital front door architecture

Document Ingestion

To ingest the plan benefits document, we create the embeddings and then persist those embeddings in Amazon OpenSearch Serverless. The ingestion steps are:

1. The plan document is uploaded to an Amazon S3 bucket. The document can be in human readable formats such as pdf, docx, json, csv.

2. An event notification trigger is set on the S3 bucket to invoke an AWS Lambda function when a new document is uploaded. The trigger invokes a Lambda function to ingest the document.

3. The Lambda function retrieves the document from the S3 bucket and then uses Amazon Bedrock’s Titan Embeddings to create embeddings for the document.

4. The Lambda function persists these embeddings by ingesting and indexing them into Amazon OpenSearch Serverless. OpenSearch uses these embeddings later to match user questions to relevant passages in the document when powering the chatbot.

Content Retrieval

The chatbot uses the embeddings to find relevant passages within the document based on the similarity of said passages to the incoming user question.

5. Amazon API Gateway invokes an AWS Lambda function with the relevant arguments such as user question, model name and search index.

6. The Lambda function accesses the LLM using Amazon Bedrock

7. The Lambda queries the embeddings in Amazon OpenSearch using a Python package called LangChain.

8. Amazon DynamoDB stores the conversation history so the model can understand context.

By leveraging multiple Bedrock models like Anthropic’s Claude, Meta’s LLama2 and AI21 Lab’s Jurassic-2, the solution can test responses from multiple models. Healthcare payors evaluate the responses from these models for the same question and choose the right model for the task at hand.

Healthcare Payors can embed the chatbot in their websites, member portals, customer service applications and mobile apps. The service can even be embedded into the Interactive Voice Response (IVR) system which will provide voice responses. Members use the chatbot to ask questions on their plan benefits and seek clarification.

Below are some example questions that the chat bot can answer:

• What is my deductible?
• What is the copay for a specialist visit?
• Are mental health services covered?

Foundation Model selection

Amazon Bedrock supports foundation models from many different providers as well as Amazon’s own large language and embedding models. Healthcare Payors can compare the responses from these different models to determine the foundation model most suitable for their needs. Inference parameters and prompt engineering can then be leveraged to further calibrate the responses generated by the model.

Operation

Security is job zero at AWS. Ensure you apply authentication and authorization to manage protected health information (PHI) and protected identify information (PII). Like every other regulated industry workload, follow the security, data protection, and compliance best practices.

Finally, user questions are passed as part of the prompt to the content retrieval API Gateway. The request invokes the content retrieval lambda which queries the indexed content in OpenSearch. The most relevant passages and the incoming user query are then passed to the LLM with the instructions to ‘answer the question based on the provided excerpts.”

In this example, the members ask a question “What is the copay for a doctor visit?” The solution sends this question to different models and gets the responses. Responses vary across the different models (see responses below), e.g. the AI21 model gives a concise response and Cohere’s is more detailed. Here are the responses:

• AI21
  • Based on the context provided, the copay for a primary care doctor is $35
• Claude V1
  • Based on the context provided, the copay for a primary care doctor visit is $35
• Claude V2
  • Based on the context provided, the copay for a primary care provider (PCP) office visit is $35. The copay for a specialist visit is $50
• Cohere
  • The copay for a doctor visit depends on the type of doctor visit and the patient’s insurance plan. The patient’s insurance card should have details about the copay for a primary care visit, specialist visit or, preventive care visit. In the context of the information provided, it appears that the patient has a $35 copay for a primary care visit to treat an injury or illness. However, if the patient is visiting a specialist, they may have to pay a $50 copay per visit, and the coinsurance may vary depending on the type of specialist visit. It is important to note that the patient’s insurance plan may have specific requirements, such as obtaining preauthorization for certain specialist visits, to ensure that the services are covered under their plan. It is recommended that the patient consult their insurance card or contact their insurance provider directly for accurate and up-to-date information about their copay and coinsurance responsibilities.
• Llama2
  • Based on the information provided, the copay for a Doctor Visit is $35.00

To improve the relevance, the Payor can add additional context and apply more prompt engineering. As the member logs into the member portal, the Payor recognizes the member as a Medicare member. The question from the member remains “What is the copay for a doctor visit?” The solution takes the context “Medicare” and uses prompt engineering to change the request to “What is the copay for a doctor visit for Medicare?”. Below, the responses are now specific to Medicare coverage:

• AI21
  • The copay for a doctor visit for Medicare is $35.00
• ClaudeV1
  • Based on the context provided: -There is a $35 copay for each covered specialist office visit. There is a $0 copay for each covered service you get at a retail health clinic (clinic inside a retail pharmacy). There is a $0 copay for each Medicare-covered dental visit for care that is not considered routine. There is a $35 copay for each Medicare-covered hearing exam to diagnose a hearing condition. There is a $0 copay for defined Medicare-covered telehealth services from your primary care provider (PCP) office visit. For the “Welcome to Medicare” preventive visit, there is no coinsurance, copayment, or deductible. So in summary, the copay for a doctor visit for Medicare would typically be $35, but some visits like those to a retail health clinic or a PCP visit via telehealth would have a $0 copay according to this context.
• ClaudeV2
  • Based on the context provided, the copay for a doctor’s office visit under this Medicare plan is: $35 copay for a primary care provider (PCP) office visit. $35 copay for a specialist office visit. The details indicate there is a $35 copay for a PCP office visit and a $35 copay for a specialist office visit. No other copay amounts are mentioned for general doctor’s office visits under Medicare in the given context.
• Cohere
  • The copay for a doctor visit depends on whether the doctor is an in-network or out-of-network provider. In-network doctor visits have no copay, while out-of-network visits have a $35 copay. Original Medicare covers the Welcome to Medicare preventive visit, which is a one-time visit that includes a health review, counseling, and referrals. This visit is covered at no cost to you, as long as you receive it within the first 12 months of your Medicare Part B coverage. Please note that there may be additional costs associated with doctor visits, such as the deductible and coinsurance, depending on your specific Medicare plan and the type of services you receive. It’s always a good idea to verify the cost of a doctor visit with your healthcare provider before receiving services
• Llama2
  • Based on the information provided, there is no copay for the “Welcome to Medicare” preventive visit. However, after the first 12 months of having Medicare Part B, the copay for doctor visits is $35.00 for each covered specialist office visit.

Payors can further programmatically control the response using inference parameters. The model responses are different based on the inference parameter changes. Compared to prompt engineering, there are noticeable changes to the responses (Figure 5), especially so for AI21 and Claude V1 models. AI 21 was giving a terse response in Figure 4 and it gives a more detailed and contextual response in Figure 5. Claude V1 was giving a verbose response not to the point in in the previous responses and it gives a wordier response here:

• AI21
  • The copay for a doctor visit for Medicare is $35.00. However, there is no copay for covered services at a retail health clinic, a primary care visit to treat an injury or illness, a Medicare-covered dental visit for care that is not considered routine, a Medicare-covered hearing exam to diagnose a hearing condition, or defined Medicare-covered telehealth services from your primary care provider’s office.
• Claude V1
  • Primary care doctor visit copay is $35. Specialist visit copay is $50. Retail health clinic, dental, and telehealth visits have $0 copay. The “Welcome to Medicare” preventive visit is also free
• ClaudeV2
  • The copay for a primary care provider office visit is $35. The copay for a specialist visit is $50. There is no copay for the “Welcome to Medicare” preventive visit, Medicare-covered dental visits for non-routine care, Medicare-covered hearing exams, Medicare-covered telehealth services from your primary care provider’s office/clinic, or services at a retail health clinic.
• Cohere
  • The copay for a doctor visit depends on the type of doctor. For a primary care physician (PCP), the copay is $35.00. For a specialist, the copay is $50.00. There is no copay for a visit to retail health clinic or for a Medicare-covered dental or hearing exam. There is also no copay for a “Welcome to Medicare” preventive visit, but this plan only covers this type of visit within the first 12 months of having Medicare Part B
• Llama2
  • The copay for doctor visits for Medicare is $35.00. There are no copays for preventive visits, telehealth services, or dental visits for non-routine care.

Payors can use these different techniques to contextualize the request based on user input and get desired outcomes. Throughout this example, the foundation models remain the same. Based on the response, Payors decide the model, inference parameters and prompt engineering for respective use cases.

Conclusion

Healthcare payors have an opportunity to transform the patient experience by providing easy access to personalized plan benefits explanations. A conversational AI “digital front door” chatbot built on AWS services can empower patients to understand their coverage, costs, and maximize benefits. This leads to better engagement, financial preparedness, and care management. Generative AI based chatbots enable Payors to create a frictionless member experience for clarifying complex policy details, improving efficiency for patients, providers, and the healthcare system overall. The result is superior access, better outcomes, and reduced costs. As next steps, Payors should consider prototyping solutions like this to demonstrate value and scale what works. Reach out to your AWS account team to learn more about the underlying technology involved in this blog and start your cloud journey.