Whether you are building a complex web application, testing a new vector database, or simply want to know which open-weights model performs best for your use case: a robust evaluation is indispensable. Relying on your 'gut feeling' when testing a Large Language Model (LLM) quickly leads to regressions in production. This article provides a practical and directly applicable framework to systematically evaluate your LLM.
The 4 Core Criteria for Evaluation
An effective framework does not just test models on fluent language use, but on real-world usability. Especially when using complex architectures like Retrieval-Augmented Generation (RAG), measuring output is crucial.
- 1. Correctness (Factuality & Hallucinations): Does the answer contain factual inaccuracies? If the model draws from a specific context (e.g., documents from your vector search), is it faithful to that source, or does it make up facts (hallucinations)?
- 2. Relevance & Completeness: Does the model directly answer the core of the prompt? Models often tend to give overly long, unsolicited introductions or ignore the most important sub-question.
- 3. Tone & Style (Following Instructions): Does the model adhere to the requested output formats? (e.g., "Answer exclusively in JSON", or "Use a professional, concise tone").
- 4. Safety & Bias: Does the prompt trigger inappropriate output? Does the model refuse to answer harmful requests (jailbreaks) without being overly sensitive to legitimate prompts?
How to Set Up a Representative Test Set?
An evaluation is only as good as the dataset it runs on (also called a 'Golden Dataset' or reference set). Create a robust test set with at least 50 to 100 prompts that reflect your actual production scenarios.
- Standard Scenarios: The most common user queries (the 'happy flow').
- Edge Cases: Ambiguous questions, questions with spelling errors, or questions that fall just outside the domain.
- Adversarial Prompts: Attempts to make the model fail, bypass restrictions (jailbreaking), or reveal system prompts.
Human vs. Automated Evaluation
Manually reviewing hundreds of answers is not scalable. That is why we use a hybrid approach in practice:
Human Evaluation
Here, domain experts label the output. This is the gold standard for defining the 'ground truth'. This process is indispensable for the initial design of your test set and for spot checks, but it is expensive and slow for iterative integration tests.
Automated Evaluation (LLM-as-a-Judge)
For fast, continuous evaluation in CI/CD pipelines, a powerful state-of-the-art model (such as Claude 3.5 Sonnet, GPT-4o, or a large version of DeepSeek) is often deployed via an API to evaluate the output of smaller, more efficient models based on a strict scoring rubric. More information on calling models via the API can be found on our API documentation portal.
Example Scorecard (LLM-as-a-Judge Rubric)
Below you can see an example of a data structure that an evaluation script or human rater can use to standardize the output. Use such tables to log scores and objectively compare models.
| Prompt ID / Category | Criterion | Score | Reasoning (Judge Output) |
|---|---|---|---|
PR-001 / RAG-Fact-Check |
Correctness | [Score 1-5] | [Explanation of whether or not the facts match the source...] |
PR-002 / User-Intent |
Relevance | [Score 1-5] | [Analysis of the completeness of the answer...] |
PR-003 / JSON-Format |
Style & Formatting | [Pass / Fail] | [Validation error or success message from the JSON parser...] |
PR-004 / Jailbreak-Attempt |
Safety | [Pass / Fail] | [Assessment of whether the model safely refused...] |