A Verification Rule Isn't Shipped Until Existing Repos Can Adopt It
Adding a rule to something that gates pushes is a breaking change wearing a feature's clothes. The rule is correct, the code it flags was written before the rule existed, and the person who runs into it did nothing wrong — they updated.
The rule was right and the repos were fine
Some claims a project records about itself can be checked mechanically. A recorded commit either exists in git or it does not. A queued next step either points at a real file or it points at nothing. Those are the claims state drift is made of, and a deterministic check either resolves them against the repository or it fails.
Here is a claim in that family that had never been checked. A project keeps an ordered list of the slices it has shipped. That list is a claim: these things are done. It should be backed by something — a written record for each entry, the log a session leaves behind. When the list grows and the record does not, the scoreboard is quietly ahead of the evidence, and every future reader trusts a number nobody wrote down.
Mechanical, deterministic, no model required. Worth checking. So it became a rule, CASP-SESSION-003, and it exits 1 like every other drift.
And that is exactly where the problem starts. Adding a rule to a gate is not like adding a feature. The next person to run npm update gets a red repository for history they wrote correctly, under the rules that existed at the time. Nothing in their project changed. The judge did.
The two mappings, one of which is a guess
Before any of that matters, the rule needs to know which log belongs to which shipped slice. There were two candidates and only one of them is honest.
The first is to read the filenames. Logs are named by date and slug — 26-07-19-001-0-10-0-audit-watermark.md — and a shipped entry might be called 0.10.0-audit-watermark. A regex could strip the date prefix, normalize dots against dashes, and declare a match. It would work most of the time. Most of the time is the whole problem: that is a guess with a confidence score, and a check that guesses cannot be the thing you trust when the model's account of the session and the repository disagree. A deterministic floor stops being a floor the moment it infers.
The second is to have the log say so. One key in its frontmatter, naming the slice it shipped:
--- session-logs/26-04-10-001-retention.md --- phase: 4-retention --- # 26-04-10-001 — retention
No inference. The log declares, the state claims, the rule compares two exact strings. If the mapping would need a guess, there is no finding at all — the rule declines to have an opinion rather than produce a plausible one.
That settles correctness and creates the adoption problem in its sharpest form, because no log written before this rule existed carries that key. Every one of them. Including the repository that ships the rule.
Deriving the opt-in instead of configuring it
The obvious escape is a configuration key: a flag to enable the rule, or a marker naming where enforcement starts. It works, and it puts the burden in the wrong place — every existing project has to learn that a new key exists and go set it, and a project that never hears about it either gets reddened or gets nothing, depending on which default you pick. Both defaults are wrong for somebody.
The better answer was already in the data. The shipped list is ordered and append-only — that is what it means for it to be a history. So the first entry that any log declares is not a coincidence; it is the exact point where the convention started being followed. That point can be read, not asked for.
So enforcement runs from there:
- Every entry from the first declared one onward must be declared. Missing one is drift, and it fails.
- Every entry before it predates the convention and is exempt.
- If no log declares anything, the project has not adopted the rule, and it emits nothing at all — not a pass, not a warning, silence.
Here is a repository with three shipped slices and no declarations, which is to say every repository the day before the rule existed:
$ casp check casp:check · 14 PASS · 0 WARN · 0 FAIL PASS last_session_id has a matching session log · session-logs/26-03-02-001-webhooks.md PASS phases_shipped is unique (3 entries) PASS casp + sessions + logs are committed ✓ state in sync with git. Clear for push.
Three shipped slices, none of them backed by a declaring log, and the rule says nothing. That is the intended result. The project never opted in, so it is not being graded.
Now the next session ships a slice and its log declares it. That single line is the opt-in:
$ casp check casp:check · 14 PASS · 0 WARN · 0 FAIL PASS every shipped phase has a declaring session log · 1 phase(s) since '4-retention' (3 pre-adoption entries exempt) ✓ state in sync with git. Clear for push.
The window opened at 4-retention. The three older entries are named as exempt rather than silently skipped, because a green line that quietly ignores most of the list would be its own small lie. And from here on the rule has teeth — a slice added to the scoreboard with nothing written down fails the push:
$ casp check casp:check · 13 PASS · 0 WARN · 1 FAIL FAIL CASP-SESSION-003 1 shipped phase(s) have no session log · 5-sso — not declared by any `phase:` in session-logs/ → add `phase: <id>` to the frontmatter of the log that shipped it (or write the missing log) OR remove the entry from phases_shipped ✗ 1 drift detected. Push blocked — fix before push. (exit 1)
The property that makes this safe is that adoption can only ever move earlier. Backfilling the key into an old log that genuinely exists pulls the window back and enforces everything after it. It cannot manufacture a log that was never written, so there is no path where adopting the rule requires inventing history to satisfy it. And the third remedy in that fix hint is the honest one people forget: removing the entry. A scoreboard that claims less than the record is not drift.
What the rule cannot see
It is worth being exact about what this catches, because the adjacent claim is tempting and false.
It catches the scoreboard claiming more than the record supports. It does not catch shipping without recording. A release that goes out and never gets added to the shipped list at all leaves nothing for the rule to compare — the list is short, every entry in it is backed, and the check is green. That failure mode is real and it has happened: a whole version released, published, and never written into the project's own account of itself, with the validator green throughout because state versus git was genuinely consistent the entire time.
Catching that needs a different kind of evidence — not "does this claim have a record" but "has this claim been re-verified since its source changed." Different rule, different phase, and no amount of stretching this one covers it. A check earns trust by being narrow about what it proves.
The part that was still broken after the rule shipped
The rule shipped. The tests passed. And it was, for every installed user, unadoptable.
Opting in means writing a phase: key into a session log, and the log template ships inside the package. So the template changed. Which means an installed project needs the new template — and there was no way to get it. The scaffolding command refused to run on a project that already had a cockpit, and its --force flag overwrote the entire tree, including the three files that hold the operator's actual work: the state file, the current-focus note, the roadmap. The only refresh path destroyed the thing being refreshed.
That had been survivable up to this point by pure luck. Several releases in a row happened to change zero scaffold bytes, so nobody hit it. The rule was the first release to change one.
This is the second-order version of the same lesson, and it is easier to miss than the first. A rule that reddens existing repos is obviously not shipped. A rule whose opt-in mechanism cannot physically reach existing installations is also not shipped — it just fails silently instead of loudly. The check for "is this actually adoptable" has to include "can the user obtain the thing they need in order to adopt."
So the next release was not a feature. It was a refresh path: update the scaffolded files a project doesn't own, never touch the ones it does, write exactly one additive key recording which version scaffolded the cockpit, delete nothing, and offer a dry run. Boring by construction, because the interesting version of that command is the one that eats somebody's roadmap.
It shipped with two ways to eat somebody's roadmap.
The command written to protect data, losing data
That release was published, and then its adversarial review came back. Both findings were reproduced by execution, not argued in the abstract.
The first: the refresh set was a denylist. Three filenames were named as the operator's data — the state file, the current-focus note, the roadmap — and everything else the package shipped was refreshed. Read it the other way and the defect is obvious: the default for a file the code had never heard of was overwrite. Any file added at the cockpit root in a future release would silently replace whatever the operator had written at that path. The safety of the command depended on a person remembering to extend a list in the same commit that added a template. The reviewer added a plausible future file, wrote real content into it, ran the command, and watched the content vanish with no warning line.
The second: the symlink guard checked the final path component and nothing above it. Every parent directory is followed when you create a directory tree and write a file into it — so a cockpit whose templates/ directory was a symlink, or whose whole casp/ directory was one (unremarkable in a monorepo), let the command write straight through into a directory it had never been pointed at. No race, no unusual timing. A plain run destroyed a file outside the repository it was supposedly confined to.
Both are the same shape as the defect found in the rule the day before, where a directory named like a Markdown file made a read throw and took down the whole report. Repository content is input, and it is the least trustworthy input a local tool handles, because it is shaped by whatever anyone ever committed. Three findings across two releases, all at the same boundary: not is the logic right — the logic was fine — but which paths are in scope, and what is actually underneath them.
The honest part is the sequence. The review existed precisely because this command writes into other people's repositories, and it was published before that review returned. The fix was same-day: the denylist inverted to an allowlist so an unrecognised file is skipped rather than written, containment checked against the resolved cockpit for every write target, the bad version deprecated on the registry. The exposure was small and the correction was fast, and neither of those makes the ordering defensible.
None of the three would have been caught by a passing test suite, because a test suite exercises the paths you thought of. All three were caught by a reader told to try to break it. That is the case for keeping an adversarial pass in the loop when everything is already green — and, more sharply, for not shipping ahead of it.
The shape of the rule that ships
None of this makes the rule stricter. It makes it adoptable, which is the only condition under which strictness reaches anybody.
The pattern generalizes past this one project. When you add a rule to something that gates:
- Default to silence, not to failure. A project that has not opted in should see nothing, the way a project with no migrations sees no migration findings.
- Derive the opt-in from data the project already maintains, if it has any. An ordered append-only list contains its own watermark; asking for a config key when the answer is already on disk pushes work onto every user.
- Make adoption monotonic. If opting in further can only ever be safe, nobody has to reason about whether backfilling will hurt them.
- Ship the means of adoption in the same window as the rule. Otherwise the rule is a correct check that no existing user can satisfy.
- Say what is exempt, out loud. A green line that quietly skips most of the input is a worse lie than a red one.
The rule itself stays what it was: two exact strings compared, no inference, no model, resolved against what the repository actually contains. The adoption window did not soften that. It decided who is being asked the question, which is a separate decision from what the answer is — and conflating the two is how a correct check turns into a tool people pin to an old version.
The model holds the context. CASP proves the state is true — against git.