AWS Public Sector Blog
The Mercatus Center solves unstructured data challenges using generative AI from AWS
Each year across the United States, new government policies take effect that impact the lives of millions of people, however, those policies are only as good as the data that informs them. Too often, policymakers lack access to relevant data that could better inform the development of these new policies.
The Mercatus Center, a university-based research center affiliated with George Mason University, saw an opportunity to fix this problem. Seeking to make access to economic policy documents easier, Mercatus created a generative artificial intelligence (AI) solution. The research center developed a platform, QuantGov, with assistance from Amazon Web Services (AWS) resources, including Amazon Bedrock, AWS Lambda, Amazon API Gateway, and Amazon Relational Database Service (Amazon RDS). Accessible through an interactive AI assistant, the platform provides scalable, accurate, and no-cost access to a treasure trove of data, helping policymakers craft informed public policy that will positively impact people’s lives.
Organizing unstructured data using generative AI from AWS
Before policymakers can write a new regulation, they need to understand what other policies and documentation exist that relate to the same topic. In the past, performing this research was overwhelming due to the sheer amount of policy documents available. In addition, the vast majority of these documents are unstructured. Without a good way to sift through the mountains of documents and data, policymakers are left making partially informed decisions.
After seeking a solution to its unstructured data problem for a number of years, the Mercatus team immediately recognized the value of collaborating with AWS. They were able to quickly test and iterate new features in the AWS cloud, setting up a pipeline of collected data, cleaned data, and derived results in a short period of time.
“With a click of a button, we could collect an entire state’s regulatory code, run it through multiple ML algorithms, organize the storage in a relational database, and analyze it,” noted, Stephen Strosko, AI Engineer at the Mercatus Center. “We were able to automate a large portion of that process, which would have been impossible without working in the AWS Cloud.”
However, scalability was not the benefit of the Mercatus team using AWS. “When it comes to writing law, 60 percent accuracy for the data doesn’t cut it,” stated, Thurston Powers, AI engineer at the Mercatus Center. “It needs to be specific and citable. When we began working with large language models (LLMs), we felt like they were the missing tool for unstructured data.”
When Mercatus started using generative AI, the team partnered with an AWS team led by Greg Grieff, AWS senior solutions architect. “Whenever we encountered a problem, Greg put us into contact with the right people at AWS—and fast,” says Strosko. “For example, I was running into a throttling exception with Amazon Bedrock and getting an error message that I was sending too much to the LLM. I knew that the error code was probably covering up something else. Greg got me in touch with somebody who had helped develop Bedrock, and I was able to get the error figured out very quickly.”
Using AWS networks and solutions
Working in Amazon Bedrock to develop the QuantGov platform, Powers and Strosko were given access to the foundational model (FM) Claude from Anthropic, one of many LLMs that are accessible through AWS. Claude strongly appealed to the team because of its unique academic and scholarly responses to queries. “The moment Anthropic released Claude 3.1, we had access to it on AWS,” says Strosko. “That was huge for us. If we don’t have that quick access, it’s a long process to get permission from Anthropic directly, and it’s also a process to then adapt it to our pipelines. The partnership between AWS and Anthropic made that possible.”
For a nonprofit with a limited budget, the Mercatus Center discovered another huge advantage in AWS: cost savings. The team used Amazon DynamoDB, an inexpensive, scalable database, to store individual chat conversations; while Lambda provided a serverless, computing platform. “If we had to go out and buy servers and keep them onsite, using them maybe five percent of the time, we would be paying thousands in upfront costs,” explains Powers. “For the very quick compute processing we’re doing, AWS offered a solution that costs literally fractions of pennies.”
Retrieval-Augmented Generation (RAG) architecture was important to the development of a key feature of the Mercatus Center’s chatbot—regulatory text citations. Using a RAG framework, the chatbot is able to retrieve relevant regulatory text and specific citations that provide details about the information that was used to generate the chatbot’s response.
“At the end of every output, the chatbot cites the specific information we’re using to answer the query,” says Strosko. “People can get to the source information very quickly. That’s why the RAGs are really important from an academic and a policy perspective.”
Saving budget and freeing up staff availability
Today, the Mercatus Center delivers an enterprise-level experience to its growing user base, including all 50 US states and some federal agencies, as well as government agencies in Canada, the United Kingdom, Australia, and India.
The scalability and speed of the new AWS generative AI solution translate into cost savings across the entire organization. One example is legislative bill tracking, which is a basic and important Mercatus Center function. Powers and Strosko estimate that generative AI saves Mercatus a whopping $180,000 a year in labor costs on bill tracking alone. “As a small nonprofit, being able to save that much money on one project, and more importantly free up the time for those individuals to work on high-value tasks, is really something,” says Strosko. “After seeing the savings there, it is easy to understand why folks in the organization are excited.”
By creating a platform that can sift through unstructured data to deliver policy insights, the Mercatus Center is now capable of supporting a growing list of government entities. With a successful pipeline built, the team plans to expand its reach and widen its positive impact. The Mercatus partnership with AWS demonstrates the degree of innovation that is possible when a clearly defined need is matched with an effective technology stack and top-of-line customer service.
Is your organization ready for generative AI? Take the Educause Generative AI Readiness Assessment to gather an understanding of your current state and potential applications.