The eval needs a second job after the team starts optimising

I would happily replace a vague product request with a concrete evaluation. I would keep the reason the request exists, and I would stop treating the development score as sufficient release evidence once everyone starts optimising against it.

Daniel McKinnon’s guide to GenAI evals makes the practical case for giving a modelling team measurable examples rather than another ambiguous specification. It recommends small, precise checks and continued analysis of failures. That is a useful operating method. The tension begins when the same examples become both the target of repeated improvement and the entire proof that improvement reached the user.

The target does not become worthless because engineers can see it. Feedback is how development works. It acquires a different evidentiary role: it tells us how the system performs on material the team has been actively studying.

Keep the user problem beside the score

Imagine a hypothetical assistant that drafts replies to support tickets. The initial evaluation checks whether the reply mentions the customer’s issue, includes a next step and stays under a length limit. Those checks make a much better handoff than “write helpful replies.”

But a response can satisfy all three and still promise something the support team cannot do. It might offer a refund outside the authorised process or give a next step that depends on information the customer never supplied. The first evaluation did useful work. It did not exhaust the meaning of helpfulness.

I would preserve a short explanation of the intended outcome: the customer should understand the next permitted action without receiving an unsupported commitment. That sentence helps the team decide what to add when the existing score stops distinguishing useful replies from polished failures.

It also helps prevent a strange product negotiation in which an obvious failure is defended because no current test forbids it. The test suite should serve the objective, and the objective must remain available for interpretation.

Separate development feedback from release evidence

My proposed release check would use fresh examples drawn under a documented sampling rule, with sensitive information handled appropriately. Keep them out of routine tuning until that release assessment is complete. Inspect the important failures, then retire or rotate the cases as they enter the development loop.

I would not call that a guarantee against overfitting. A small sample can miss an important category, a judge can be wrong, and supposedly fresh cases can resemble training examples. It is an additional view, with its own limitations, rather than a purity certificate.

The crucial comparison is whether gains on the development suite appear in the fresh assessment and in permitted observations of actual use. If the development score rises while unsupported commitments persist, the team has a specific problem to investigate. That result is more useful than another argument over whether the overall percentage looks high enough.

McKinnon also advises checking examples directly rather than trusting a borrowed evaluation on reputation. I would apply the same suspicion to my own suite after repeated use. Familiarity can make a test feel authoritative long after its coverage has become narrow.

This is a proposed product review method, not evidence that the author’s team gamed a benchmark. The original advice improves the clarity of the handoff. My addition concerns the evidence required after that handoff succeeds and the team learns the target well.

Use evals to tell the team what to improve, then require fresh evidence that the improvement still solves the user’s problem.