What Are AI Hallucinations?

An AI hallucination is an incorrect or misleading output generated by a foundation model (FM) in response to a user query. FMs work by predicting a response to a user query. Predictions come from their training data set and involve complex mathematics related to the frequency of data elements that appear together in the training data set. Limitations in the FM’s algorithm can result in an AI hallucination—incomplete, unrelated, or just false output. AI hallucinations reduce user trust in AI systems and limit AI adoption. Organizations must closely monitor their LLM outputs and performance to reduce hallucinations.

Not all AI hallucinations come in the same format. Identifying the type of hallucination that occurs is the first step toward developing a strategy to mitigate them.

Factual errors

The most common form of AI hallucination is a simple factual error. When training foundational models, businesses use millions of sources, some of which may contradict. When you ask a question to a model that has one correct answer, and it produces something else, this is a factual error.

These errors include incorrect scientific facts, biographical details of someone’s life, or slightly inaccurate historical details. For example, a model may state that Benjamin Franklin was the first president of the United States. In reality, he was a founding father but never acted as president.

Misclassification

Another form of AI hallucination is misclassification when choosing between yes/no or positive/negative. For example, an AI model may classify a cardboard box on the street as a person, causing a self-driving car to halt. This generation of false or misleading information often results from a lack of extensive training data, as the model may not have encountered similar examples to draw upon.

Nonsense and contradictions

Some AI tools may generate text that contradicts itself. In one sentence, a model’s output may claim something only to then state a contradicting fact in the next sentence. Some language models may simply produce writing that does not make total sense. Without an extensive understanding of a topic’s context, an AI model may be unable to produce coherent responses.

Another case of nonsense responses is when generative AI tools produce a response that does not match the input. For example, if you ask a tool to produce a birthday card for your brother but it starts with “Dear Dad,” this would be a prompt contradiction.

Various factors can contribute to AI hallucinations. Here are some of the most common causes of FM hallucinations.

Poor training data

When a model has limited or insufficient training data, it may be unable to effectively understand and respond to a request. For example, if you present an AI tool with a request it hasn’t encountered before, it could lack the training to respond accurately and effectively.

Low-quality training data means that an FM will try to produce a response with gaps in its information. These gaps can cause the model to make incorrect assumptions, creating AI hallucinations.

Another potential problem with training data is that it could hold inherent bias. If the training data is biased, AI-generated content will also include those biases.

Overfitting

Overfitting occurs when AI systems excel at finding patterns in initial data but cannot extrapolate this understanding to new data. The patterns may be coincidental and not logically related. Without understanding the logic behind the correct answer, it won’t be able to deal with new data that isn’t a replica of training information. Similarly, if new datasets don’t directly resemble the initial data, a generative AI model may be unable to create accurate responses.

Limited context window

Some AI hallucinations occur when a generative AI model does not fully process every word in the prompt. For long prompts, some systems may only consider a select number of words. Without fully understanding what the prompt is asking, the AI model generates irrelevant information that seems like a hallucination.

Language ambiguity

If an AI language has to deal with human language, then the idiosyncrasies of the person typing in the prompts may confuse the AI tool. Aspects of language like sarcasm, double meanings, idioms, or slang can confuse AI, making it fail to interpret the request correctly.

When AI technology produces hallucinations, and the person using the tool doesn’t realize it has produced false information, numerous consequences can ensue. Here are some potential problems that AI hallucinations can cause.

Misguided human interactions

If a professional uses an AI tool to gather information, then an AI hallucination may result in them taking the wrong action. For example, suppose a support agent uses an internal chatbot to find answers to a customer inquiry, but the chatbot recommends the wrong solution. In that case, they will pass this false information on.

Spread of misinformation

When journalists use generative AI tools to create news stories without fact-checking them, they release misinformation to the public. People reading these stories may then read inaccurate information, further spreading it.

Alternatively, some AI tools may use training data from these public sources, creating a situation where the baseline data that an FM uses contains incorrect data.

Security risks

When businesses use an AI model to produce code, they risk hallucinations that cause errors or induce vulnerabilities. For example, an AI tool could include a command to install a specific package. Without checking the validity of that package, a developer could run the code and then download malware or ransomware onto the system.

Another potential security risk is when AI models hallucinate threats, causing cyber security employees to take action and waste their own time. This could lead to security teams needing more time or available resources when a real security risk presents itself.

If you can identify an AI hallucination and pinpoint why it is occurring, you can then implement strategies to mitigate future hallucinations. Here are several strategies you can use to prevent AI hallucinations.

Focus on high-quality training data

A large proportion of AI hallucinations occur due to low-quality, limited, or biased training data. By creating a broad, diverse, unbiased, and high-quality database for AI models to train on, you are more likely to get an accurate response.

Retrieval-augmented generation (RAG)

Retrieval-augmented generation is a method that optimizes the output of a large language model. Instead of presenting false information if an AI tool doesn’t know the answer, RAG will draw upon new, relevant information to overcome any gaps it may have.

RAG takes a user’s input and retrieves data from new sources to complement its response. RAG can produce a more accurate response by using both the new data that users point toward in their prompt and the LLM’s original training data.

Prompt engineering strategies

The more specific, direct, and descriptive your prompts are, the more accurate your response will be. Where possible, try to write clear prompts that don’t leave any room for assumptions. You can provide additional context, examples, or background information to give your AI tool as much help as possible.

For example, instead of writing, “Write a poem,” you could write, “Write a sonnet with a traditional rhyme scheme that discusses a tree at midnight.”

Generative AI models can produce a more precise response with the extra context.

Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models and a broad set of capabilities to build generative AI applications while simplifying development and maintaining privacy and security.

Guardrails for Amazon Bedrock evaluates user inputs and FM responses based on use case-specific policies and provides an additional layer of safeguards regardless of the underlying FM. You can detect and prevent content that falls into restricted categories. For example, an ecommerce site can design its online assistant to avoid using inappropriate language such as hate speech or insults.

Amazon Kendra is another highly accurate enterprise search service powered by machine learning. It provides an optimized Kendra Retrieve API that you can use with Amazon Kendra’s high-accuracy semantic ranker as an enterprise retriever for your RAG efforts in reducing AI hallucinations.

Amazon also offers options for organizations that want to build more custom generative AI solutions. Amazon SageMaker JumpStart is an ML hub with FMs, built-in algorithms, and prebuilt ML solutions that you can use for better quality control of fully customized FMs.

Get started with AI development on AWS by creating a free account today.