Alnylam Transforms Product Complaint Management with Generative AI Using Amazon Bedrock

Founded in 2002 and based in Cambridge, Massachusetts, Alnylam operates in compliance with regulatory mandates, with an obligation to triage and manage product complaints related to clinical and commercial drug products to regulatory bodies such as the Food and Drug Administration. Previously, teams relied on manual processes to understand, analyze, categorize, and investigate hundreds of drug product quality complaint reports. “We were taking 3–4 days to complete the steps for customers across different time zones, languages, and reporting channels,” says Kristen Dvorscak, Alnylam’s head of global quality systems. “With the new process powered by generative AI, we are reinventing existing workflows, making them near real time.”

In June 2023, Alnylam and teams from Amazon Web Services (AWS) began work on a solution that uses generative AI to significantly cut down reporting and initial evaluation and response time. “When we started on this journey, we knew we needed a solid partner like AWS with strong technical expertise,” says Murtaza Cherawala, head of data management and AI at Alnylam. “The AWS team describes technical complexities in a way that is understood by developers as well as business leaders.” Together, Alnylam and AWS developed a product complaints intake and triage prototype in 3 months, in alignment with company requirements for legal compliance, data privacy, security, validation, and quality.

In parallel, Alnylam and AWS teams also sought to enhance employee productivity with generative AI. They used Amazon Bedrock to create a Slack-integrated chatbot called AskALNY that is used by more than 2,000 employees and an additional 1,000 contractors to find information easily, summarize insights from complex documents, and generate content for a range of use cases. “Using AskALNY, our teams accurately find information hidden in different policy documents along with the source link,” says Cherawala. For example, teams use AskALNY to create job descriptions, write meeting agendas, summarize medical literature and other documents into plain language, search compliance and legal policies, access guidance around finance and procurement, identify guardrails around travel or timecard submissions, and even find facilities information.

While respecting GxP compliance, the Alnylam and AWS teams defined a range of parameters for the use cases, including model selection. They ran iterations on several large language models (LLMs) accessible through Amazon Bedrock, using humans to determine which was the most accurate, in addition to scoring comprehensiveness, relevance, and clarity. To further assess accuracy, AWS engineers used cosine similarity, a method of using mathematics to measure the similarity of two sets of information. After extensive evaluation using human and mathematical methods, Alnylam chose Anthropic’s Claude, which summarizes information, maps it to the database, and categorizes it based on rules on which the LLM has trained. The solution frames real-time responses to the adverse-event reporter, providing critical additional information about the complaint faster, while improving decision-making internally, and cutting down time for training new adverse-event investigators joining the company.

The data automatically gets stored in Amazon Simple Storage Service (Amazon S3), an object storage service offering scalability, data availability, security, and performance. The data from the complaint is extracted, converted into a readable format, and sent to the LLM to interpret, infer, and generate the output. The system checks that all fields populate correctly—through a confidence score that checks for comprehensiveness, accuracy, relevancy, and clarity—and automatically creates a record in Alnylam’s quality system.

The model also generates relevant follow-up questions so that Alnylam can better understand patient concerns and the criticality of the defect, streamlining feedback and removing the delay across time zones. “It’s likely that when we come in to work the next day, we have already generated a record that includes important answers to follow-up questions,” says Dvorscak. “This has changed us from operating in a reactive mode to a proactive mode and is designed with patient data privacy and security in mind. As we scale, the number of incoming reports will grow exponentially, and this capability will save us significant time and resources.” As the solution moves out of its pilot phase, it will require human attention only for exceptions and rely on cosine similarity to score model accuracy, saving additional time.

For AskALNY – the internal employee-productivity solution – the platform resulted in gains in productivity, providing answers to employees in 30 seconds when they previously took more than 15 minutes. At the suggestion of the AWS team, Alnylam built further efficiencies into AskALNY using retrieval-augmented generation, a process that generates more accurate responses by prioritizing the LLM’s use of new information over training data.