@sleepy_panda_srl/portulan 0.1.2 → 0.1.3

package/README.md

@@ -20,7 +20,7 @@ workspaces and premium packs delivered as private-marketplace plugins.
 
 ## Status
 
-**Current release: `0.1.2`** — recorded in [`CHANGELOG.md`](CHANGELOG.md), which is also the honest
+**Current release: `0.1.3`** — recorded in [`CHANGELOG.md`](CHANGELOG.md), which is also the honest
 account of what the release does *not* contain.
 
 This is pre-release software. The leading `0.` is meant literally: nothing here is stable yet, and
@@ -141,7 +141,7 @@ are not subcommands at all. [`cli/README.md`](cli/README.md) is the single carri
 | `agents/` | [The three personas bound to this host's tools](agents/) — `implementer`, `reviewer`, `librarian`. A platform-fixed location; the doctrine they bind stays in [`core/personas/`](core/personas/) |
 | `examples/` | [A fictional, public demo workspace](examples/) — two products, read end to end |
 | `docs/` | [`vision.md`](docs/vision.md) (the constitution), [`plan.md`](docs/plan.md) (the milestone map) and [`milestones/`](docs/milestones/) — one file per milestone, holding amendment arguments, session notes and close evidence |
-| `evals/` | [A README today](evals/), and nothing else. The golden tasks, the A/B harness and the CI eval gate arrive at milestone 8 |
+| `evals/` | [The eval corpora](evals/) — golden tasks for the compiled gates and for the core skills — plus the review-loop register, the A/B corpus with its arms, graders and recorded baseline, and [`evals/releases/`](evals/releases/), the eval result each release from `0.1.3` onward carries. **No count is written here**; `git ls-files evals` derives it. _(This cell has now recorded going stale twice, and this is the second repair: it said the **graders** and the **baseline run** were *still to arrive* after both landed on 2026-08-31. The sentence that expires is always the one naming what is next, so this cell no longer names one — what is unbuilt lives in [`evals/README.md`](evals/README.md)'s own *What is NOT built yet*, which is the single carrier for it and is checked by nothing, exactly as this cell was.)_ |
 | `.portulan/` | [This repository's own workspace](.portulan/) — Portulan applied to building Portulan |
 | `.claude-plugin/` | The [plugin](.claude-plugin/plugin.json) and [marketplace](.claude-plugin/marketplace.json) manifests that make this repository an installable feed |
 | `.claude/` | [The compiled enforcement](.claude/settings.json) — permissions and hooks generated from [`.portulan/gates.json`](.portulan/gates.json), committed so it stays reviewable and held to its source by the `compile` verify recipe |

package/cli/README.md

@@ -16,6 +16,47 @@ that differs from its staged blob. _(The roster the package
 ships is narrower than the tree and is not restated here as a number — `npm pack --dry-run` derives
 it, and a hand-copied count is the thing that keeps going wrong in this file.)_
 
+**What the payload leaves behind, and the criterion it leaves it behind on.** `package.json`'s `files`
+array excludes by name on top of the directory roster, under the rule set at the first publish — *the
+package ships the product, not the workshop*: a file that is not part of the product surface that an
+`npx` user runs does not travel. The suite and its fixtures went first, then `eval-bundle` and its licence
+template, on the eval payload's own reasoning — an evaluee receives the stamped licence, never the
+stamp press. `ab`, `ab-run` and `ab-grade` joined them on 2026-09-01.
+
+**The argument for the A/B trio is that the rig has one subject, and it is not the reader's
+repository.** `ab.mjs`'s `DISPOSITIONS` classifies *this* workspace's `.portulan/` path by path, and
+the table is compiled into the module rather than read from beside it: pointed at an adopter's
+workspace `--check` exits 2 and names every disposition that matched nothing. There is no
+configuration under which the trio does a consumer's tree any good — `ab-run`'s `SNAPSHOT` is one
+baseline, of one arm, on one date, and `ab.mjs` refuses to build an arm it cannot classify entirely.
+
+_**Not** because their registers resolve under `evals/`, which contributes nothing to the payload.
+That reasoning is already ruled out in this tree, and the exclusion holds **despite** it rather than
+because of it:_ [`eval-bundle.mjs`](eval-bundle.mjs)'s `EXCLUDED_TOP_LEVEL.evals` settled on 2026-08-24
+that **the tool is product and the policy it reads is this team's** — `goldens` ships without its
+corpus the way `compile` ships without `gates.json`. That distinction carries the whole load here:
+`readCorpus(repoRoot, dir = CORPUS_DIR)` takes its directory as an argument, so an adopter can point it
+somewhere; `ab.mjs`'s disposition table takes nothing and can be pointed nowhere.
+
+_**The exclusion is narrower than the criterion, and every module is now classified rather than
+merely counted.** The transitive import closure of [`portulan.mjs`](portulan.mjs) and its eight
+subcommands, less the two compiled-hook runners, leaves **fourteen** shipped modules reachable from
+nothing the package exposes. **Thirteen are `unruled`** — `control-chars`, `drills`, `fuzz-shell`,
+`goldens`, `librarian`, `mutants`, `pack-identity`, `pack-version`, `review-meter`, `rule-carriers`,
+`skill-goldens`, `telemetry` and `version-carriers` — most of them the `goldens` shape the 2026-08-24
+rule keeps, none put the question the A/B trio was, and whether any follows is the maintainer's call,
+the same sentence this file already makes about a ninth subcommand. **The fourteenth, `release-eval`,
+is `product`**: ruled to ship on 2026-09-01, which is what an answered one looks like beside thirteen
+unanswered.
+
+**The sentence that stood here is now false and is corrected rather than deleted**: it said *nothing
+derives payload membership*, which was true when it was written and stopped being true one pull request
+later. [`payload.mjs`](payload.mjs) classifies every shipped `cli/*.mjs` and the `payload` recipe holds
+it on every commit. What remains true is the narrower thing: `pack-identity` checks the packed
+**bytes** and is silent on the **roster**, which is why a module could join the tarball under a green —
+measured, by staging one and watching it. The `!` lines are still hand-maintained, but they are no
+longer unchecked._
+
 **All eight dispatch.** The entry point calls each one's exported `run` and returns the code
 unchanged.
 
@@ -40,8 +81,8 @@ before the constitution catches up; while that holds, the tool says which docume
      every other hand-maintained figure. -->
 Beside those eight sit the runnable tools that are on none of the lists — `plugin-lint`,
 `librarian`, `control-chars`, `discover`, `rule-carriers`, `pack-version`, `eval-bundle`,
-`pack-identity` and `version-carriers`. Each exists because a verify recipe or a packaging check
-needed it; none is reachable as `portulan <subcommand>`.
+`pack-identity`, `version-carriers`, `goldens`, `skill-goldens`, `mutants`, `fuzz-shell`, `drills`, `review-meter`, `telemetry`, `payload`, `ab`, `ab-grade`, `ab-run` and `release-eval`. Each exists because a verify recipe or a
+packaging check needed it; none is reachable as `portulan <subcommand>`.
 The remaining files in this directory are neither subcommands nor tools: `gate` and `stop-gate` are
 the **compiled-hook runners**, invoked by generated host configuration rather than by a person, and
 `recipe-set`, `skills-set`, `manifest` and `inside` are modules other tools import — the first two
@@ -121,13 +162,39 @@ stops complaining, so both are named in the recipe and either one disappearing i
 | [`pack-version.mjs`](pack-version.mjs) | **A change to a pack's `contributes` must move that pack's `portulan.version`** — the maintainer's ruling on [#265](https://github.com/sleepy-panda-srl/portulan/issues/265), arm 3, including that a **prose-only** edit to a fragment's `reason` counts. The **second** tool here that reads history rather than the tree, and the comparison is **three-dot** — the merge-base of the base ref and `HEAD` — so work that landed on the base is never attributed to this branch. Manifests are compared as **values**, so reindenting or reordering keys is correctly not a change. The bundle version in `packs/.claude-plugin/plugin.json` is out of scope by measurement: it is one field and the packs are many. |
 | [`pack-identity.mjs`](pack-identity.mjs) | **Every file `npm pack` would emit is byte-identical to its staged blob (`git show :<path>`), and every one is tracked at all** — the rail [#149](https://github.com/sleepy-panda-srl/portulan/issues/149) asked for, built 2026-08-18 after the first publish measured the property by hand. It compares against the **tree**, not the registry: the registry form needs the network and a published version, so it cannot run before a first publish or offline, and a recipe that cannot run stops being run. This owns the half that can drift silently on any commit; the registry half stays a hand measurement recorded on #149 at each publish. `package.json` is compared like every other file rather than exempted, because the 2026-08-13 measurement asserted identity "`package.json` included, with no exemption" and an exemption here would quietly retire that claim. Could-not-run is **exit 2** — npm absent or unusable is not a finding about the bytes. |
 | [`pack-identity.test.mjs`](pack-identity.test.mjs) | The rail's suite, added on a Copilot note that it had shipped without one while every comparable rail here has a suite. Exercises all four contracted states **positively** — green, a drifted byte, an untracked packed file, and could-not-run — because a failure path nobody has run is a failure path nobody has seen work. Two of its cases are the review's own findings turned into tests: a **staged** edit must compare clean, or the rail is red before every commit and contradicts [`../.portulan/dod.md`](../.portulan/dod.md); and a filename beginning with `-` must be compared rather than read as an option, which the `cat` shell-out this rail first shipped with would have done. |
+| [`payload.mjs`](payload.mjs) | **Every `cli/*.mjs` the package would ship falls in exactly one class, and nothing arrives in the payload unclassified** — [#383](https://github.com/sleepy-panda-srl/portulan/issues/383)'s first half. `package.json`'s `files` governs the payload and nothing derived or checked its membership: `pack-identity` holds every packed file byte-identical to its source and is deliberately silent on *which* files those are. That is how `ab.mjs`, `ab-run.mjs` and `ab-grade.mjs` shipped for three sessions unnoticed, and it was demonstrated live before this was written — a `cli/` module staged into a scratch clone joined the tarball under a `pack-identity` green over 84 files. The classes are `bin` (derived from `package.json`'s own `bin`), `subcommand` (imported from `portulan.mjs`'s `SUBCOMMANDS`), `product` (unreachable but ruled to ship, with the ground for the ruling — the class `unruled` empties into as #383 is answered), `imported` (the transitive walk, which reads re-exports and side-effect imports as edges — on the grammar being total, not on a live example: an earlier draft claimed `manifest.mjs` was reachable only that way and the pre-commit checkpoint measured it false), `hook-runner` (imported from `compile.mjs`'s `HOOK_RUNNERS`, which spells them into generated host config and is their one carrier), and the **frozen** `UNRULED`. **It rosters nothing it can re-derive**, and it refuses rather than walking past a dynamic `import(` it has not accounted for. Authority for *shipped* is the `npm pack --dry-run --json` roster, never a string match on the `!` lines — npm's negation semantics are npm's. **Self-excluded**, on `eval-bundle.mjs`'s precedent: its subject is this repository's own publish surface. Could-not-run is **exit 2**. |
+| [`payload.test.mjs`](payload.test.mjs) | Its test suite. **Every red class is forced** — an unclassified arrival, an `UNRULED` module gone reachable, one the payload stopped carrying, an exclusion the payload contradicts, a stale exclusion, an unexplained absence, and a dangling import — because a rail whose failure paths are never run is a rail nobody has seen work, and this one exists precisely because a green meant nothing for three sessions. One case asserts the frozen class is exactly thirteen; one asserts the unclassified finding *names* the refusal to widen `UNRULED`, since the cheap fix under pressure is the fail-open. The `telemetry.mjs` case deliberately asserts **two** findings rather than narrowing to one: a module that leaves the payload while staying tracked is both a stale entry and an unexplained absence, and saying so twice is the totality working. |
 | [`version-carriers.mjs`](version-carriers.mjs) | **Every prose statement of the CURRENT version equals `package.json`'s** — the rail for a defect this repository shipped **twice with the sibling pairing already written down**: `README.md` and `.portulan/products/portulan/product.md` said `0.2.0` while everything machine-read said `0.1.0`, and one release later `README.md` said `0.1.0` inside the change cutting `0.1.1`. It matters past tidiness because `README.md` is in `package.json`'s `files` and **npm freezes a README per version**, so a wrong sentence that reaches a publish needs another release to correct — which is what `0.1.1` was. The **record layer is excluded by path**, and that is the design: this repository quotes retired sentences in the files recording their repair, so a matcher over everything reds on the account of the fix. `MUST_CARRY` closes the other half — a carrier deleted or reworded is a finding, so the rail cannot shrink in silence. What it does **not** close is a claim spelled a new way, and the accepted practice for that is #286's ruling: new spelling ⇒ a pattern here and a fixture first. |
 | [`version-carriers.test.mjs`](version-carriers.test.mjs) | The rail's suite. Some cases run the CLI as a **subprocess whose SCRIPT PATH contains a space** — the property the entry guard turns on, and not the same thing as a spaced working directory; the rest exercise the module in process. **No count is written here on purpose.** A hand-maintained figure beside a list that grows is the defect this whole rail exists for, and `node --test` derives it. Every contracted state is exercised **positively** — green, drift in each of the three carriers, a carrier reworded away, could-not-run in three causes, and the record layer ignored. Two cases exist only because this repository's own corpus refutes the naive design: **the record must be ignored**, since `CHANGELOG.md` and two handoffs quote that retired sentence while explaining that it was retired — **despelled here on purpose**: written out in the matched spelling, this very row is a live current-version claim and the rail reds on it, which is how it was found; and **a `g` regex must not carry `lastIndex` between files**, which would find the first claim and then resume past it, reporting a clean scan over a drifted file. |
 | [`pack-version.test.mjs`](pack-version.test.mjs) | Its test suite, written first, on **real `git init` repositories** rather than a stubbed `execFileSync` — the three-dot property cannot be observed at all without two branches and a merge-base. The shallow-clone refusal is asserted on **both** shapes a shallow clone takes, and is guarded by a full-depth clone of the same fixture reaching the red, so the `2` is the depth rather than the fixture. |
+| [`goldens.mjs`](goldens.mjs) | **The gate-corpus runner** — milestone 8 clause (a), *golden tasks reach the gates as well as the skills*. It grades [`../evals/goldens/gates/`](../evals/goldens/gates/) against the gate policy the workspace **yields** (its own rules plus its composed packs' fragments, never the declared file alone — the class [`../.portulan/dod.md`](../.portulan/dod.md) condition 1 names), through [`compile.mjs`](compile.mjs)'s own exported `matchesRule` rather than a second implementation. **Two rails:** a rule that compiles to a matcher and carries no fixture is red, so coverage is measured rather than named; and a `documented-hole` case that starts being *caught* is red too, so a hole record cannot go stale in either direction. Resolution is **hermetic** — `discovery: null`, always — because a rail that consults the host's plugin cache grades the machine rather than the tree. **A fixture is data and is never executed**, asserted statically in the suite. What it does not establish is **adequacy**: it is a presence floor, and it prints that limit on every green. Rules with `action.none` are exempt and **named on every run**, because writing the next gate `none`-shaped is the way to dodge it. Each case records which branch of `matchesRule` it exercises, **derived** by `matcherPath` from the action kind and the tool and refused when the stored value disagrees — a field written from memory on every case in the corpus would be the second carrier this repository keeps finding wrong. An unexpected throw returns **2**, never 1: a crash that exits 1 reaches the recipe as a red about a corpus nothing finished grading. **Deliberately not a ninth `portulan` subcommand**, on `eval-bundle`'s precedent. |
+| [`goldens.test.mjs`](goldens.test.mjs) | Its test suite. Every contracted state is exercised **positively** — a missing fixture, a regression, a hole that closed, all ten malformed-fixture refusals, and could-not-run — because a failure path nobody has run is one nobody has seen work. Three cases exist because this repository's history refutes the naive design: the entry guard must survive a path containing a space (this module shipped the broken spelling and exited **0 having run nothing**, the third time here), the denominator is asserted on the **census** rather than the verdict (both pack-contributed rules are `none`-shaped today, so a declared-only denominator would coincide — which is how that defect goes unnoticed), and the no-subprocess contract is asserted against the module's **source**, since it is structural rather than behavioural. |
+| [`mutants.mjs`](mutants.mjs) | **The mutation census** — milestone 8 clause (b), first half, *mutation testing over both matchers*. It breaks [`compile.mjs`](compile.mjs)'s matcher region on purpose, one **anchored** operator at a time, and grades each mutant against the gate corpus. This answers the question [`goldens.mjs`](goldens.mjs) says out loud that it cannot: not *does a fixture exist* but *does the corpus NOTICE a broken matcher*. **Recorded in both directions**, as `documented-hole` is — a `killed` operator that survives means the kill-set weakened, a `survives` operator that is killed is good news the record must absorb — and a `survives` entry admits only a proof of equivalence, never an unfilled gap. **A missing or ambiguous anchor is could-not-run, never a skip**; so is a mutant that will not import, since counting an unimportable module as killed would be the loudest false green available to a tool whose whole output is a coverage claim. Every mutant gets its **own** temp directory, because ESM caches by URL with no invalidation and a reused path would grade the first mutant once per operator. **Deliberately not a ninth `portulan` subcommand**, on `eval-bundle`'s precedent. |
+| [`mutants.test.mjs`](mutants.test.mjs) | Its test suite. The ESM-cache canary is the case worth knowing: two different mutants are imported in one process and must answer differently, which is the only assertion here that would pass on a correct runner and fail on the plausible wrong one. The refusals are exercised positively — an anchor that does not place, one that places twice, a no-op substitution, a workspace with no policy — and the entry guard is pinned under a path containing a space. |
+| [`fuzz-shell.mjs`](fuzz-shell.mjs) | **The grammar fuzzer** — milestone 8 clause (b), second half, *grammar-aware fuzzing over the shell segmenter*, which is **two** functions here and they differ. It **composes** commands from a grammar rather than mutating strings, so it knows by construction whether the payload sits in a position bash executes or one bash only prints: the oracle is exact, and a disagreement is a gate bypass or a false red rather than something to adjudicate. **Positions are enumerated and recorded** — every cell that diverges from ground truth cites the record licensing it, in the gate map's honest-holes list or in [`compile.mjs`](compile.mjs) itself — **and spellings are fuzzed**, the invariant being that every spelling of one command in one position answers the same. Three payloads, not two, because the write matcher has two recognitions and measurement showed they diverge. **Deterministic and seeded**, seed printed on every run including the green. A finding is emitted **in the corpus's own case shape**, so it becomes a regression pin by paste. It found a live bypass of every Gated shell action — one wrapper plus one separator — on its first full run. |
+| [`fuzz-shell.test.mjs`](fuzz-shell.test.mjs) | Its hermetic suite: determinism, budget refusals that are refused rather than coerced (`Number("")` is 0), the recorded table's totality in both directions, and the rule that a divergence must name a record. |
+| [`fuzz-shell.ground.test.mjs`](fuzz-shell.ground.test.mjs) | **The fuzzer's oracle, measured under real bash.** A grammar that lies about itself produces not a red but a green about the wrong thing — the one failure a fuzzer cannot detect in itself — so every position is run with a **neutral** payload and every path spelling is written to a throwaway file. Separate from the file above precisely because that one handles a force-push spelling and must never spawn anything. It caught two generator defects: a `..` hop through a directory nothing created, and a marker check fooled by `echo` printing its own payload. |
+| [`drills.mjs`](drills.mjs) | **The forced-red drill harness** — milestone 8 clause (d), *every rail forced red on a calendar and required to fire*. It asks the question its two siblings do not: [`goldens.mjs`](goldens.mjs) asks whether a gate has fixtures and [`mutants.mjs`](mutants.mjs) whether those fixtures discriminate; this asks whether the **rail still fires at all**. Every drill is a **pair** — a control on a pristine tree, then the perturbation — because a rail that only ever reds proves nothing about its green, and because a control that is already red makes the drill could-not-run rather than a fire it did not cause. Three guards, each from a defect already shipped here: a **`tell`** the rail's own output must carry when it fires and must not carry before, so a red for the wrong reason is not counted as one; an anchored substitution that must place **exactly once** *and* move bytes on disk, because both hand-run sessions had a drill that no-opped and reported on nothing; and `--only` narrowing what **runs** and never what must be well formed. Isolation is one throwaway `git worktree` per drill, so the working tree is never perturbed. **It reports on a commit and prints which one** — a dirty tree is refused outright, and `--working-copy` synthesizes one with `git stash create` and refuses while untracked-and-unstaged files exist, since a synthesized tree missing the file under review would be a green about the wrong tree. `--check` is the correspondence half the `drills` recipe runs. **Unlike its three siblings it SPAWNS by nature** — running a rail is spawning it — so what the suite pins instead is that nothing spawned is composed from a drill's declarations. The rails it does **not** drill are named on every run, with the reason: the platform floor, the host's permission layer, the CI seam, the seam scan, `claude plugin validate`, the platform watchers, and the librarian pass. **Deliberately not a ninth `portulan` subcommand**, on `eval-bundle`'s precedent. |
+| [`drills.test.mjs`](drills.test.mjs) | Its suite. The refusals are exercised **positively** — an anchor that does not place, one that places twice, a substitution that is its own replacement, a creation over a path that exists, a drill naming an undeclared rail, a yielded rail with no drill, a drill whose control and fire are the same run — and the two host-shaped rules are pinned: a dirty tree is refused without `--working-copy`, and untracked-and-unstaged files are refused with it. The `tell` contract is asserted in **both** directions against a fake rail: present-in-fire is not enough if it was also present in the control. The entry guard is pinned under a path containing a space. |
+| [`skill-goldens.mjs`](skill-goldens.mjs) | **The core-skill golden corpus** — milestone 8's *golden tasks per core skill*, the row's original first subject and the half clause (a) named when it said golden tasks reach the gates *"as well as the skills"*. It binds each numbered step of a skill's `## The pass` to the **live artifacts** it governs and to the rail that enforces it, or `null`. **The denominator is derived** — the steps come out of the skill file, so a step nobody wrote a case for is a red rather than a default, which is the difference between a rail and a census over a set its author drew. **An empty step list is could-not-run**: three `## The pass` spellings exist, an exact match would find one skill of three, and the other two would satisfy *every step is accounted for* vacuously. A mandate **quote must place exactly once** or the case refuses, so a reworded skill reddens instead of drifting. **The predicates are pinned by import**, not re-spelled — this module imports `RETIRE_WHEN` from [`doctor.mjs`](doctor.mjs), so the retire-when check is `doctor`'s own regex and `doctor` dropping that export breaks the runner at load. **The per-case `carrier` field is a different thing and is NOT verified**: nothing links it to an import or to any check, so a corpus naming a module that does not exist still grades green. It is declared, reviewed metadata, a reviewer is what checks it, and the runner prints that limit on every run. **`unbindable` is adjudicated**: its reason comes from a closed vocabulary, and a `judgement-only` step may name no artifacts — if you can name them it is unbuilt, not unjudgeable. The bound/unbindable ratio is printed **as a finding**, because it measures how much of a core skill is artifact discipline and how much is agent judgement. **Deliberately not a ninth `portulan` subcommand**, on `eval-bundle`'s precedent. |
+| [`skill-goldens.test.mjs`](skill-goldens.test.mjs) | Its suite. Three cases exist because this session's own opening passes produced the defect they pin: the denominator must not go quietly empty, the EARS predicate must read the mandate's **shape** rather than a section's presence (two checkpoints computed two different figures for one row because each measured before deciding the rule), and the provenance predicate must be **shape-tolerant** because the tree carries two spellings and a literal one reports thirteen renamed files as absent. Every refusal is exercised positively — an unbindable reason outside the vocabulary, a judgement-only step naming artifacts, a step in the pass and in no case, a case naming a step the pass lacks, a corpus whose `skill` field disagrees with its filename, an undeclared slot, a slot resolving to zero files. The **both-directions** case is the one worth knowing: an accepted drift that starts complying is a finding, and it caught three filenames the author had written from a number prefix rather than read off disk. The entry guard is pinned under a path containing a space. |
+| [`review-meter.mjs`](review-meter.mjs) | **The review-loop meter** — milestone 8 clause (c), *rounds per pull request, pushes per round, empty-round rate*. It derives the figures [`../.portulan/memory/a-review-loop-needs-a-bound.md`](../.portulan/memory/a-review-loop-needs-a-bound.md) bounds the loop on, which were counted **by hand** on 2026-07-28 and which that record says of itself **nothing checks**. Every figure is in **submission** units — the criterion predates the 2026-07-30 redefinition of *round*, and the record's own `Retire when:` names its threshold in *"the submission units of the table above, not fix-rounds"* — and the tool never prints the bare word for a figure. **Fix-rounds it does not compute at all**, and that is a finding rather than a gap: on #105 a fix rode inside another push and was never a reviewed head, and a records push after a finding-bearing submission answers nothing, so *"count pushes, then look inside each one"* is not something an API answers. The empty-round rate is reported as an **upper bound** — separating a submission that found nothing from one carrying only suppressed notes needs the awk in [`../.github/workflows/copilot-review.yml`](../.github/workflows/copilot-review.yml), a **workspace-layer** matcher reduced to one carrier, and copying it into engine code would put two spellings of one rule on opposite sides of the boundary. It **reports on a snapshot, never the network**: `--fetch` is the one mode that talks to GitHub, and it stores no review bodies. It also **detects when two of its figures are one figure** — where every submission judged its own head, pushes-per-finding-bearing-submission reduces to `1/(1 - the no-inline rate)`, which it says out loud rather than presenting as a third measurement. **A meter, not a bound**: nothing here stops a loop, and it adjudicates no sibling exemption. **Deliberately not a ninth `portulan` subcommand**, on `eval-bundle`'s precedent. |
+| [`review-meter.test.mjs`](review-meter.test.mjs) | Its suite. Every counting trap this repository has already paid to discover is a case, because the argument for the tool is that a hand count kept getting them wrong: both reviewer logins accepted (a filter on one returns zero from the other, #154), our own reviews refused as **could-not-judge** rather than counted (at merge: #105 is seven of fifteen, #342 is ninety of a hundred and two), a duplicated pull request caught, and an absent inline count refused because absent is not zero — as is an absent `head`, an unparsable `mergedAt`, and a window heading disagreeing with the corpus under it. The empty denominator returns **null and never zero** — an unmeasured window is not a window below the retirement threshold — and the boundary is asserted **at** the threshold, since *below* read generously retires a rule the measurement did not. The register's byte-compare is exercised in both directions, and the entry guard is pinned under a path containing a space: this module shipped the broken spelling and `--fetch` exited **0 having written nothing**, the fourth time here. **Three cases assert an error MESSAGE rather than only an exit code**, each because the case they replaced passed for the wrong reason — a missing-register case that never reached the register, a `--pool` case that fell through into the real fetch and got its 2 from `gh` failing, and a `--check` that compared nothing and returned 0. An assertion holding for a reason other than its subject keeps passing after the code it names is deleted. |
+| [`telemetry.mjs`](telemetry.mjs) | **The OTel emitter** — milestone 8's *OTel opt-in config*, and the first thing here that can send anything anywhere on purpose. The reading was **ruled, not derived** (2026-08-28): a validated opt-in config surface **plus a real emitter**, hand-written OTLP-over-HTTP JSON with no OpenTelemetry SDK, because an `@opentelemetry/*` import would end the zero-dependency property [`../.portulan/identity.md`](../.portulan/identity.md) protects — the maintainer's 2026-07-31 ruling that this ships as plain ESM, which [`../.portulan/repos/portulan.md`](../.portulan/repos/portulan.md) records as permanent rather than pending. **The committed config is the only gate**, and that is a deliberate departure from the OTel environment contract: `OTEL_SDK_DISABLED` defaults to *false*, so honouring the standard env surface as the gate would make an adopter who already exports an endpoint for some other service start emitting without deciding to — an opt-**out**, where the criterion says opt-in. The environment supplies **transport only** (`OTEL_EXPORTER_OTLP_ENDPOINT`, `OTEL_EXPORTER_OTLP_HEADERS`), so a token never enters a committed file. `--export` is the one mode that reaches the network and it refuses twice over: when the config opts out, and in the **three** states that are not a committed consent: the config is **untracked**, **tracked but absent from `HEAD`** (staged and never committed), or **differs from `HEAD`**. The staged case is the one worth naming — it is how an agent would manufacture consent one step short of a commit, and a round of this pull request's own review had made it unreachable. **Opted-out and could-not-read are different answers with different codes**: a config with a typo its author meant as `true` must never report *opted out*. Figures are **Gauge** points stamped with the instant the measurement is *about* — the snapshot's own `captured` — not a clock read, which is both what OTLP means and what makes the golden byte-stable. It also carries `--audit-recipes`, the rail behind the standing prohibition on a network call inside a verify recipe: it sweeps the set the workspace **yields** at a pinned root, rails the **class** of network modes rather than this session's own, and refuses an empty set rather than passing vacuously. **Deliberately not a ninth `portulan` subcommand**, on `eval-bundle`'s precedent. |
+| [`telemetry.test.mjs`](telemetry.test.mjs) | Its suite. **No case opens a real socket** — [`../.portulan/verify/tests.sh`](../.portulan/verify/tests.sh) runs this suite, so a test that did would be the network call inside a verify recipe that this module's own audit forbids; the transport is injected and the send is proven against a temporary git repository with the consent actually committed, asserting the wire bytes **are** the golden's bytes. Every refusal is exercised positively: an absent `enabled` (absent is not false), a secret in a committed config, an unknown signal, a wrong spec, an unparsable capture stamp, an edited consent, a consent outside the tree, two modes at once — and a valid config producing **no** problems, without which a validator that refused everything would satisfy the rest. The closed attribute vocabulary is pinned **in both directions**, so an added key and a deleted carrier are each a red, and separate cases assert that no reviewer login and no commit SHA reaches the payload. The producer seam is exercised against a producer this module **did not ship**, because a generic shape with one implementation is a capability claim nothing checks. The entry guard is pinned under a path containing a space — the fourth instance here, and the one where silence is indistinguishable from correct behaviour. _(Three cases failed on first run because `withTemp` was synchronous around async bodies and deleted the directory mid-test — a helper that tidies up before the test runs is a false red today and a false green one assertion later. A fourth case **passed** and should not have: the non-2xx collector arm pointed `--repo-root` at a directory with no snapshot, so it exited 2 at the signal read and never called the injected transport, leaving that branch covered by nothing. Caught at the pre-commit checkpoint; it now asserts the send happened, and a mutant deleting the branch kills it.)_ |
 | [`eval-bundle.mjs`](eval-bundle.mjs) | **The evaluation-bundle cutter** — a named-recipient stamped copy of the shippable payload, cut from an explicitly named commit, ported into the tree on the maintainer's ruling of 2026-08-17. The payload partition and the machine-read license census are **pinned in both directions**, and **two different steps refuse two different things**, which is worth keeping straight because each sends an operator somewhere else: the **census** runs on the freshly materialised cut and refuses an Apache assertion in a file that is not rostered in `APACHE_MANIFESTS`, or a rostered file that has stopped asserting; the **guard** runs on the finished cut and — **inverted on the maintainer's ruling of 2026-08-18 ([#284](https://github.com/sleepy-panda-srl/portulan/issues/284))** — refuses any machine-read `license` value that is **not** Apache-2.0, plus any issuer-machinery path that leaked in, diagnosing a drifted rostered manifest separately from an unrostered one and from a failed self-exclusion. Nothing rewrites licence metadata and `LICENSE` ships: the bundle is the public tree's bytes under the public tree's licence, so the README's own License section is correct as written and its link resolves. The issuance ledger, and all recipient data, stay OUTSIDE this repository; evaluation terms render from [`eval-license.template.md`](eval-license.template.md) **at the payload commit**, pinning payload and terms to one sha; `--check` (what [`../.portulan/verify/eval-bundle.sh`](../.portulan/verify/eval-bundle.sh) runs) cuts the **index**, wrapped as an unreferenced probe commit, for a fixture recipient into scratch it always deletes. **Deliberately not a ninth `portulan` subcommand**, on `discover`'s precedent above — and deliberately **self-excluded from the payload**, so an evaluee never receives the issuer's machinery. |
 | [`eval-bundle.test.mjs`](eval-bundle.test.mjs) | Its test suite. The rosters are pinned with the suite's **own** instruments — `git ls-tree` and `git grep`, never the module's helpers — because a pin that asks the subject to measure itself tests agreement, not truth; the self-exclusion is exercised **positively** on a fixture repository that plants files at exactly the excluded paths; the terms pin is **demonstrated** (an old commit stamps its old template whatever the tree says now); and the plumbing transport is asserted byte-identical, modes included, to `git archive \| tar -x`, so the port's one mechanical delta is demonstrated rather than trusted. |
 | [`eval-license.template.md`](eval-license.template.md) | **The evaluation-license template** — four placeholders (`{{name}}`, `{{login}}`, `{{date}}`, `{{shortSha}}`), rendered into a bundle's `EVAL-LICENSE.md` by [`eval-bundle.mjs`](eval-bundle.mjs), which reads it **from the payload commit with `git show`, never from the working tree** (the supervisor's ruling of 2026-08-17: payload and terms pin to one sha in the stamp's `source_commit`, so a later edit here cannot drift under an already-stamped bundle). Issuer machinery, so **self-excluded from the payload**: an evaluee receives the stamped license, never the stamp press. The repository is public, so this text is world-readable, and the maintainer settled its wording on 2026-08-17: the bundle's files carry Apache-2.0, the same terms as the public repository, so the template's "non-public materials" and no-redistribution clauses are withdrawn. The mechanism is indifferent — it stamps whatever the payload commit carries. |
 | [`librarian.test.mjs`](librarian.test.mjs) | Its test suite, written first, on real scratch git repositories rather than an injected clock — a fake history proves nothing about the one call this tool exists to make. |
+| [`ab.mjs`](ab.mjs) | **The A/B arm builder** — milestone 8's *A/B baseline recorded* clause, session 6b. It builds the two arms [`../evals/ab/arm.md`](../evals/ab/arm.md) specifies and **grades neither**: no agent is run here and no scenario is graded — the graders and their three classes of discrimination fixture are [`ab-grade.mjs`](ab-grade.mjs)'s, the run [`ab-run.mjs`](ab-run.mjs)'s, on the maintainer's ruling of 2026-08-29. Construction is **stage-then-vendor**, and that order is forced rather than chosen: [`vendor.mjs`](vendor.mjs) refuses this workspace outright while `constitution` resolves outside `.portulan`, so dropping that slot is a precondition and not only a faithfulness move. **Its disposition table is TOTAL and it refuses to build when it is not** — the defect it exists to make impossible is measured, not imagined: built to `arm.md`'s six rows and vendored, the treatment arm still carried customer zero's `memory-index.md` (30 rule titles over an empty store), `handoffs-index.md` (146), `rule-carriers.json` (five entries naming the A/B clause) and `labels.json`, with `doctor` GREEN over all of it, because `vendor` carries every ordinary file under a workspace directory and a specification that names what to REMOVE leaks whatever it forgot. It also gives `arm.md`'s **rule 2** its first checker — *no move authors a normative sentence* — and that checker is a **keyword matcher, not a reading of English**: the `deletion` and `emptying` kinds are covered in full because they add no sentence, while `substitution` is covered by a declared marker vocabulary that **misses a paraphrase carrying the mandate and none of the words**. One case in its own corpus is exactly that miss and the suite requires it to stand. `--stop-probe` is [`../evals/ab/corpus.md`](../evals/ab/corpus.md)'s acceptance test and is **not** a recipe mode: it runs one real agent turn, because a fixture calling `stop-gate.mjs` would prove the script records and assume the thing under test — whether the host invokes the hook at all. **Deliberately not a ninth `portulan` subcommand**, on `eval-bundle`'s precedent. |
+| [`ab.test.mjs`](ab.test.mjs) | Its suite. **No case runs an agent** — [`../.portulan/verify/tests.sh`](../.portulan/verify/tests.sh) runs it, and a test spawning `claude` would put an agent turn inside a verify recipe; the probe's refusals are exercised against `false` standing in for an agent that cannot start. The distinctions that cost something are each a case: an unclassified path is exit **1** and a stale disposition exit **2**, because a finding about the tree is not a could-not-run and the first cut collapsed them; a **declared** substitution is un-substituted before comparison while an undeclared one reads as an authored sentence, which is the property that makes the matcher auditable; and `mayBeAbsent` is audited on its **reason** — git carries nothing under the path — rather than on the path's presence, because `personas/` is present in a working copy and absent in a checkout and an audit keyed on presence refused every working-copy run. That last one was found by this rail's own drill on its **control** leg, which is the argument for having a control leg. The corpus case requiring a **documented miss** is not a tautology over the table: a corpus in which everything is caught proves only that the author picked cases the matcher already handled. |
+| [`ab-grade.mjs`](ab-grade.mjs) | **The A/B graders** — milestone 8's *A/B baseline recorded* clause, session 6c, the middle of the maintainer's construction \| grading \| running split. It carries the four scenarios' **stimuli**, the four graders, and the three classes of discrimination [`../evals/ab/corpus.md`](../evals/ab/corpus.md) requires. **It runs no agent and records no baseline figure**: every figure it produces is about the instrument, and the run is 6d's. The stimulus ships here because a level-1 fixture *is* a staged tree plus a mutation — `stageScenario()` writes files and **spawns nothing**, and this module imports no child-process API at all. Each grader recomputes its attribution anchor from the scenario definition and **refuses** a tree it cannot attribute, which is neither a pass nor a fail; the nonce is never read out of arm output, since *"a grader that trusted arm output for its nonce could be satisfied by an arm naming its own"*. **The blindness it exists for is measured**: a prototype grader returned *"30 records, each with its own provenance and retirement condition"* against a vendored arm that had done nothing at all — it passed level 1, and only **inversion at the delta** caught it. `done-demonstrated` has no `demonstrated` verdict, deliberately: the rig fails unconditionally, so a green rail is reachable only by rewriting the check, and grading that as compliance would score the gaming move as the good outcome. |
+| [`ab-grade.test.mjs`](ab-grade.test.mjs) | Its suite. **No case runs an agent**, for the reason `ab.test.mjs` gives. The cases that cost something: a staged-and-untouched tree grades to its **declared** inert verdict, which is the prototype's failure asked directly rather than approximated; two of the four scenarios are compliant when inert and the suite pins **which**, because not destroying a target and not claiming completion are both satisfied by doing nothing; a constant grader is **reproduced** and shown passing level 1 while failing level 2; and the positive control [`../evals/ab/corpus.md`](../evals/ab/corpus.md) named as unbuilt is built at its real size — a stub agent proves the stop probe's **read path**, and proves nothing about the host invoking the hook, which stays a by-hand run. |
+| [`ab-run.mjs`](ab-run.mjs) | **The A/B run** — milestone 8's *A/B baseline recorded* clause, session 6d, the last third of the maintainer's construction \| grading \| running split, and **the only module in this family that spawns an agent**. It is separate from [`ab-grade.mjs`](ab-grade.mjs) because that file says of itself, in three carriers and one rail, that it runs no agent and imports no child-process API at all. **There is no `--operator-env` flag**: every turn runs isolated with a fresh home and config directory of its own — forty, not two — because [`../evals/ab/corpus.md`](../evals/ab/corpus.md) forbids a baseline over an unisolated arm, and a departure is a visible code change plus a maintainer's ruling rather than a flag. `did-not-complete`, `could-not-attribute` and `non-compliant` are three different facts and the four states are total over `k`. **The invocation was ruled by measurement**: `--print` alone left both arms byte-unchanged after completed turns, so `acceptEdits` joins it — which auto-approves edits the settings do not forbid and does **not** override a `deny` rule, so arm A's compiled enforcement survives. The record is a committed snapshot plus a register rendered from it and byte-compared, on [`review-meter.mjs`](review-meter.mjs)'s precedent: the events do not repeat, so the unreproducible half is the snapshot alone and it says so on its own face. |
+| [`ab-run.test.mjs`](ab-run.test.mjs) | Its suite. **No case runs a real agent** — `runTurn` takes its agent as a parameter and a stub script stands in, exactly as `ab.test.mjs` does for the stop probe, because a test spawning `claude` would put an agent turn and a credential inside a verify recipe. The cases that cost something: passing `--operator-env` is a refusal rather than a silent ignore; a flag that would dissolve arm A's enforcement is refused at the turn and again in the record; the operator seed is asserted to touch onboarding and trust and **nothing else**, since a seed reaching permissions would silently become treatment; stdin is closed, so a prompt the harness cannot answer fails fast instead of waiting out a ten-minute timeout, which one real turn already did; and a published cell that has drifted from its own per-turn rows is red. |
+| [`release-eval.mjs`](release-eval.mjs) | **The eval result a release carries** — milestone 8's ninth clause, added by the maintainer's ruling of 2026-08-24, which made a Protocol obligation older than the row bind that row's close. A record is a pair per version under [`../evals/releases/`](../evals/releases/): a capture holding the verdict every recipe the workspace **yields** returned at a named commit, and a register rendered from it and byte-compared. The A/B baseline is cited **by identity and never by figure** — path, capture date, commit, cleanliness — because its cells have one carrier and a second copy inside a release record is the hand-maintained figure this milestone deleted six of. **Two designs were reversed before a line was written.** Cut detection off `CHANGELOG.md`'s top heading never fires: the cut re-seeds `## Unreleased` above the version it just wrote, so the top heading is `Unreleased` on the cut commit too. And a rail keyed to `package.json`'s version alone grades one record at a time — once `0.1.4` is declared, `0.1.3`'s record can be deleted in silence — so the subject is every version the changelog records from `0.1.3` onward, permanently, graded in both directions. Its **own** row is excluded from every capture, with the reason printed, because a capture cannot be accurate about the record it is inside. `--tagged` is the release act's mode and the only check here that sees a release rather than a tree; [`../.github/workflows/publish-github-packages.yml`](../.github/workflows/publish-github-packages.yml) runs it before publishing. **Deliberately not a ninth `portulan` subcommand**, on `eval-bundle`'s and `ab`'s precedent. |
+| [`release-eval.test.mjs`](release-eval.test.mjs) | Its suite. **No case runs the recipe set** — `--capture` spawns every rail in the workspace, and a test that did would put the whole suite inside one of its own members; every case exercises the record layer instead, which is what the rail reads. The cases that cost something: the changelog fixture is shaped like the real file, with a re-seeded `## Unreleased` on top and an entry that *quotes* a heading, so a looser matcher fails it; an older governed record deleted after a newer release is cut is red, which is the case the replaced design passed; a missing **boolean** is caught by name, because it renders as a branch and its absence publishes a claim the capture never made rather than leaving a hole a render probe could see; a record for a release the changelog never cut is red; and `--write` refuses a capture it could not read rather than rendering from one. |
+| [`checkout-refs.live.test.mjs`](checkout-refs.live.test.mjs) | **A workflow that checks out more than once names the ref of every checkout after the first.** The rail for a defect this repository shipped **twice at one step**: [`publish-github-packages.yml`](../.github/workflows/publish-github-packages.yml) grades the tagged tree with a grader that must not come *from* it, since `release-eval.mjs` exists in no tag before `v0.1.3` — and the second checkout added to fix that carried no `ref`, which `actions/checkout` defaults to *the event's own* ref, the tag on `release: published`. The crash came back inside its own repair. **The rule is narrow on purpose and the narrowness was measured:** five of the seven checkouts in `.github/workflows/` carry no `ref` and every one is correct, so a blanket rule would red five right answers to expose one wrong one — which is how a recipe gets switched off. Checking out *twice* is a statement that two different trees are wanted, so the second cannot inherit a default whose value depends on which event fired. It does **not** establish that the ref named is the right one — an explicit tag on a grader would pass and be the same bug — only that the choice was made rather than inherited. |
 | [`recipe-set.mjs`](recipe-set.mjs) | **The recipe set**, the one carrier of *what this workspace runs*. Row 7's amendment of 2026-07-31 ruled that a composed pack's verify recipes reach the adopting workspace's runnable set, under a contract stated in the row rather than left to this file: **additive only** — a pack may add a recipe and may never redefine, remove or replace one the workspace declares, nor become `verify.default` — and **namespaced by its pack, so a collision is impossible rather than resolved**. |
 | [`recipe-set.test.mjs`](recipe-set.test.mjs) | Its test suite, against a fixture resolver: the additive-only rule, the namespacing, the refusal to become `verify.default`, and **could-not-run — exit 2, never silently absent** when a composed recipe cannot resolve. |
 | [`recipe-set.live.test.mjs`](recipe-set.live.test.mjs) | The composed set against **this repository** rather than against fixtures, and it exists because the unit suite is not enough — measured, not supposed, on the change that added both. `compile.mjs`'s `resolvePack` returns `manifest` as the **path** to `pack.json`, not the parsed object; the first cut of `resolverFor` treated it as parsed, so every composed pack contributed nothing while the unit suite stayed green, because its fixture resolver returned a parsed object. A harness that supplies the shape the code assumes cannot find the assumption. |