Self-Verifying AI Agents: The One Check the Harness Cannot Run on Itself
Coding-agent harnesses learned to check their own work — tests, typecheck, a second reviewing agent, looped until green. Every one of those checks grades the code. None of them grades the record the next session will trust, and that check cannot come from inside the loop.
The agentic loop earned the autonomy it now has by learning to check its own work. A modern coding harness — Claude Code running overnight, a Cursor agent working through a backlog — does not just take action; it runs prompt → gather context → take action → verify results, and the verify step has grown teeth: run the tests, run the typechecker, replay the UI in a real browser, hand the diff to a second agent for a review with fresh eyes, and loop until every signal is green. This works, and it works for a specific reason: the strongest checks in that loop are deterministic. A test suite either passes or fails. The model cannot talk an exit code into agreeing with it.
This article is not a case against that loop — the loop is right, and you should build more of it. It is about the one artifact the loop structurally cannot verify no matter how many checks it grows: its own record of where the project stands. In State Drift we named the failure — the recorded state quietly diverging from what git contains. In Pre-Push Hooks we wired the fix into the session's two boundaries. This piece closes the argument: why that fix cannot come from inside the harness at all.
Every check in the loop grades the code
Lay the verify step's contents side by side and a pattern appears. Tests verify behavior. The typechecker verifies types. Lint verifies style. The browser replay verifies rendering. The second-agent review verifies the design and correctness of the diff. Each check is aimed at a different property of the same object: the code the agent just wrote.
But a session produces a second artifact alongside the code: the record — the status file, the "phase shipped" note, the last_commit line, the queued next step. The record is what the next session reads first; it is what a fresh agent orients against before it opens a single source file. And unlike the code, the record faces no check at all. It is generated prose, written by the same probabilistic process that wrote the code — with none of the test coverage.
The loop verifies what the model did. Nothing in it verifies what the model says it did — and the second thing is what the next session trusts.
A second agent is a second opinion
The tempting fix is more of what already works: add another agent whose job is auditing the record. This is where the categorical line sits. Review agents work on code because code gives the reviewer ground truths to lean on — it can run the tests, read the types, execute the diff. The record has exactly one ground truth: the repository itself. And a model reading last_commit: 9f31c2e does not resolve that hash against git; it judges whether the close reads plausibly. A fabricated close reads exactly as plausibly as an honest one, because both are drawn from the same distribution.
That is why a probabilistic check on a probabilistic record adds a second opinion, not a ground truth. The deterministic checks in the loop are trusted for one property: same input, same verdict, an exit code instead of a judgment. The record deserves the same property — and the check that provides it has to resolve every claim against git, mechanically, with no model anywhere in the path.
A fabricated close, fully green
Here is the failure with every harness check passing. A small delivery service, gated by casp: session 001 shipped webhook signature verification and closed honestly; session 002 builds the retry queue. The work is real, and the code checks are green:
$ node --test [… per-test lines elided …] # tests 4 # suites 0 # pass 4 # fail 0 (exit 0 — the code is green)
Then the agent closes the session the way agents close sessions: it bumps the state file — marks the phase shipped, records the last commit, names its session log — and commits the bump. Except this close is fabricated. The session log it names was never written, and the commit hash it records does not exist in this repository. Nothing about the close looks wrong; it reads exactly like an honest one.
The tests still pass. The typechecker has nothing to object to — the false claims live in a state file, not in code. A review agent reading the diff sees a routine state bump. Every check in the loop waves it through. Then the record meets the one check that does not read prose:
$ casp check casp:check · 13 PASS · 0 WARN · 3 FAIL ────────────────────────────────────────────────────────────────────── [… 9 PASS lines elided …] FAIL last_session_id does not map to a session log · expected session-logs/26-07-13-002-retry-queue.md → write the session log (try `npx @justethales/casp new log --slug <slug>`) OR fix last_session_id FAIL last_commit not found in git · state=9f31c2e does not exist in this repo → set state.last_commit to a real SHA (HEAD = 59383d3) [… 1 PASS line elided …] FAIL shipped prompt's session_log file is missing · docs/plan/sessions/PHASE-2-RETRY-QUEUE.md -> session-logs/26-07-13-002-retry-queue.md → either write the missing log(s) OR fix the pointer (repo-relative paths, comma-separated) [… 3 PASS lines elided …] ✗ 3 drifts detected. Push blocked — fix before push. (exit 1)
Each FAIL names a claim, the evidence git gave back, and the mechanical fix. casp check did not form an opinion about the close; it resolved three of its claims against the repository and got three refusals. Same input, same verdict, every run — exit 1, push blocked, and the fiction never becomes the next session's starting point.
The floor belongs inside the loop
None of this asks you to bolt a manual step onto an autonomous loop — that fails for the reasons the previous article covered: verification that exists as instructions rather than mechanism eventually gets skipped. The floor slots into the loop the same way the other deterministic signals do. casp check --json emits the machine shape of the same verdict:
$ casp check --json { "schema_version": 1, "casp_version": "0.7.0", "verdict": "drift", "exit_code": 1, "summary": { "pass": 13, "warn": 0, "fail": 3 }, "findings": [ [… 10 findings elided (9 pass · 1 fail) …] { "id": "last_commit.git", "severity": "fail", "label": "last_commit not found in git", "detail": "state=9f31c2e does not exist in this repo", "fix": "set state.last_commit to a real SHA (HEAD = 59383d3)" }, [… 5 findings elided (4 pass · 1 fail) …] ] }
An agent that already reads a failing test and fixes the code can read a failing finding and fix the record — every finding carries its own fix hint. So the verify step of your harness gains one more row: tests, typecheck, review, and beneath them casp check — the one row where the checker is not the same kind of process as the thing it checks. And at the session's edges, the same validator gates mechanically: casp next refuses to start on drift, and the pre-push hook blocks a drifted record from leaving the machine.
What stays outside the model
The floor has to keep the property that makes it a floor. No LLM participates in the verdict — not even an advisory one, because an advisory opinion from a model is precisely the thing this check exists not to be. It makes no network call, sends no telemetry, and needs no account. And it is harness-agnostic by construction: it reads two local artifacts — the state files and git — so the same floor holds under today's harnesses and under whatever ships next, because it never depended on any of them. Harnesses will keep absorbing the verification surface, and they should. The check on the harness's own record is the one piece that cannot move inside: wherever the loop ends, the floor sits below it.
CASP still does exactly one job. It does not orchestrate agents, review code, or manage tasks — the loop already has those covered. It proves the recorded state matches git, deterministically, and blocks the push when it does not.
Start in two minutes
# install npm i -g @justethales/casp # in your repo casp init # seed the state files casp check # prove state == git (exit 0/1) casp install-hook # gate every push on drift
@justethales/casp · MIT · git-native · zero telemetry · source on GitHub
The model holds the context. CASP proves the state is true — against git.