Measuring Hallucinations: How Do You Test the Factuality of an LLM?

What is a hallucination in an LLM?

A hallucination occurs when a Large Language Model (LLM) generates text that sounds syntactically and semantically plausible, but is factually incorrect or does not logically follow from the provided context. In the context of enterprise applications, hallucinations represent the biggest obstacle to safe and reliable deployment.

Key Measurement Methods

Structurally measuring these errors requires automated evaluation pipelines. The most effective methods include:

• LLM-as-a-Judge: This involves prompting a powerful, closed-source model (such as GPT-4 or Claude 3.5 Opus) with a strict rubric to evaluate the output of the target model for accuracy.
• Self-Check Methods: The LLM is asked to generate the same answer multiple times (high temperature). Consistency often indicates higher reliability, while large deviations point to potential hallucinations.
• N-LI (Natural Language Inference): A specific classification model checks whether the hypothesis (the generated claim) directly and logically follows from the premise (the source text).

Groundedness & Citations

When building Retrieval-Augmented Generation (RAG) systems, the focus shifts from general world knowledge to groundedness: the degree to which the answer is anchored in the documents you provide.

Example Test Setup

A robust benchmark for factuality can be set up as follows:

1. Dataset creation: Compile 100 factual questions linked to verified source texts.
2. Inference: Have the LLMs under test generate answers purely based on the provided source text (temperature = 0).
3. Claim extraction: Use a small model to break down the output into atomic claims (e.g., "The moon is gray", "The moon is made of cheese").
4. Verification: Run an LLM-as-a-Judge pipeline via the API environment to validate for each claim whether it appears in the source text.

Do you have questions about setting up your own testing pipeline? Share your approach or ask for feedback in our developer community.

Evaluation Table: Benchmark Results

In the structure below, we periodically publish the scores of various open-source and commercial models based on our standardized RAG factuality benchmark. (Note: the table currently shows the structure with placeholders).