Fast evals for better coding agent decisions

A fast eval is a check a reviewer can run in a few minutes to prove an agent change did what its receipt says it did. It is the cheapest way out of the merge jam that hits every team the same way: agent branches all go green in CI, and the reviewer cannot say what any of them actually did because the replay recipe scrolled off the top of the chat. The fix is a small receipt check, not another review meeting. If you want one number to watch, measure how often reviewers replay a session to understand a change. Everything below is about driving that number down.

See why your queue is slow

Review queues back up because checking is slow, not because writing code is slow. Agents made generation almost free and left verification exactly where it was. So the queue grows while everyone feels faster.

Another prompt template will not clear it. A faster way to check finished work will.

There is a trap worth naming. When green CI becomes the goal, green CI stops measuring the thing you care about, which is whether a person can explain the change. That is Goodhart's law in your pull request queue. The real lever is traceability: can a reviewer trace what happened without you in the room.

Four checks that take minutes, not meetings

Each common failure pairs with one fast eval: a named artifact a reviewer compares against the diff, without ever opening the session.

The first failure is recursive handoff blur. Chained agents hand back summaries that leave out child-owned paths, and the parent approves a mystery diff. The fix is a child receipt block. Every child returns the paths it touched, the commands it ran, and the tests that prove the regression guards hold. The eval is one comparison, receipt against diff, and parents stop approving on tone.

The second is review queue theater. CI is green, yet reviewers keep asking "why this approach?" and find no written answer anywhere. The fix is a decision stub. The PR template forces three lines: constraints considered, alternatives rejected, and the proof you verified it. The debate moves from vibes to written tradeoffs, which is a much shorter meeting than archaeology.

The third is Cursor scope fog. The .mdc rule language reads as precise until two reviewers argue about what it meant. The Cursor agent docs give you the mechanics; the contract is still yours to write. The fix is a scope ledger. The parent chat carries five lines: goal, allowed paths, forbidden paths, verification command, merge owner. Review turns into checking the ledger against the diff.

The fourth is Claude permission creep. On a shared laptop, bash approvals in Claude Code, Anthropic's coding agent, become muscle memory. The fix is a supremacy clause at the top of CLAUDE.md. State which hooks win, which folders need human eyes, and where temporary overrides live. Sessions stop inventing policy mid-run because precedence is written down.

Here is a starting rule file you can paste and adapt:

---
description: Delegation boundary snapshot (adapt globs to your repo)
globs:
  - "**/*"
alwaysApply: false
---

- Cursor: keep scopes explicit in `.mdc`; forbid undeclared MCP domains.
- Claude Code: cite `CLAUDE.md` precedence before expanding bash scope.
- Codex: ensure `AGENTS.md` carries replay-friendly verification notes for CLI runs.

Think of a fast eval as a turnstile, not a tribunal. One person, one receipt, one push through. A change that cannot clear a turnstile was never ready for the merge train.

Run the four gates before you merge

Four gates cover most agent merges. Read this table as the whole job, not a checklist of nice-to-haves.

If your repo cannot state its boundaries plainly, agents will guess. Guessing scales badly.

Common questions

• What makes a fast eval different from CI?

  CI proves the commands passed. A fast eval proves the receipt matches the diff. CI-only review drifts because the check becomes the target, while a receipt comparison stays tied to what actually changed: the paths touched, the commands run, and the tests that guard against regressions. You run both, and they answer different questions.

• How does a fast eval speed up review of coding agent work?

  It swaps session archaeology for a comparison that takes minutes. The reviewer checks the child receipt block against the diff, reads the decision stub for rejected alternatives, and runs the one verification command named in the scope ledger. Nobody replays a chat log or stands behind a terminal waiting for output.

• What belongs in a child agent receipt?

  Three things: the paths it touched, the commands it ran, and the tests that prove the regression guards hold. If a summary cannot produce those three, the parent is holding a summary and not a receipt. At that point the mystery diff is right back on the merge train and you have learned nothing.

• Who owns the eval when agents are chained?

  The merge owner named in the scope ledger. The ledger carries five lines: goal, allowed paths, forbidden paths, verification command, and merge owner. Naming the owner there keeps accountability alive across the handoff, even when work forked across several sessions. That name is who reviewers ping when something looks off.

Try it on one stuck PR

Pick one named fix and turn it into a real artifact this week: a .mdc rule, an AGENTS.md note, or a child receipt template before your next automated run. Our training runs this exact drill on a live repo with your own branches, so bring one stuck PR and leave with a receipt format the team keeps.

Further reading

• NIST AI Risk Management Framework
• Model Context Protocol specification
• Google Search Central on helpful, people-first content
• Google Search Central on generative AI content
• OpenAI Skills repository