
Evals: testing AI systems before and after release
In short: Evals are structured tests that measure whether an AI system does what it should, on real cases, before and after every change you make.
Evals (evaluations) are structured tests for AI systems: a set of inputs, the expected behaviour for each, and a way to score the system's outputs, run before release and again whenever the prompt, model, retrieval or tools change. They are to AI products what unit and integration tests are to normal software, with one difference: outputs vary, so scoring is often graded rather than pass/fail.
Evals in plain words
A practice exam you give your AI every time you change something, so you know whether the change made it better or just different.
How it works
- Collect real or realistic cases, including hard and edge cases.
- Define success per case: an exact answer, required facts, a rubric or a format.
- Choose graders: code checks, string or schema matches, human review or an LLM judge.
- Run on every change and track scores over time.
Anthropic's evaluation guide makes the trade-off explicit: prefer many cases with automated grading over a few hand-graded ones.
Example
As an illustration, a refund assistant eval with 150 cases: 100 typical requests, 30 edge cases (partial refunds, expired windows) and 20 attempts to manipulate it. Graders check the refund amount in code, policy compliance with an LLM judge, and tone on a sample by a person.
Where you'll meet it
- Model providers' evaluation tooling
- Open-source frameworks such as promptfoo, OpenAI Evals, Inspect and DeepEval
- Observability platforms with evaluation features
- Release checklists for AI features
What it changes for you
For developers, evals turn "it feels better" into a number you can defend. For tech leads, they are the release gate: no prompt or model change ships without a run. For the industry, eval design is becoming a core skill, because a team's evals decide what it can safely change.
Evals vs benchmarks
Benchmarks are public, general tests for comparing models. Evals are your private tests for your task, data and definition of good.
Common mistake
My view: building evals only from cases that already work. A good eval set is mostly the cases that broke, embarrassed you or worried legal, because that is where regressions hide.
Related terms
LLM-as-a-judge · Golden dataset · Regression evals · Online evaluation · Benchmark contamination
From the archive: How to evaluate an LLM feature before production
No prompt or model change ships without an eval run, and the eval set starts with what broke.


