@sleepy_panda_srl/portulan 0.1.3 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (76) hide show
  1. package/README.md +6 -5
  2. package/cli/README.md +142 -358
  3. package/cli/advisory.mjs +354 -0
  4. package/cli/comments.mjs +932 -0
  5. package/cli/compile.mjs +1322 -1607
  6. package/cli/context.mjs +883 -0
  7. package/cli/control-chars.mjs +12 -301
  8. package/cli/discover.mjs +17 -574
  9. package/cli/doctor.mjs +224 -1074
  10. package/cli/drills.mjs +68 -502
  11. package/cli/feedback.mjs +16 -237
  12. package/cli/finish.mjs +437 -0
  13. package/cli/form.mjs +644 -0
  14. package/cli/fuzz-shell.mjs +20 -305
  15. package/cli/gate.mjs +7 -199
  16. package/cli/goldens.mjs +26 -184
  17. package/cli/index.mjs +305 -655
  18. package/cli/init.mjs +243 -566
  19. package/cli/inside.mjs +3 -28
  20. package/cli/instructions.mjs +829 -0
  21. package/cli/ledger.mjs +778 -0
  22. package/cli/librarian.mjs +135 -599
  23. package/cli/manifest.mjs +2 -39
  24. package/cli/mutants.mjs +9 -170
  25. package/cli/new.mjs +12 -243
  26. package/cli/pack-identity.mjs +5 -74
  27. package/cli/pack-version.mjs +13 -255
  28. package/cli/plugin-lint.mjs +42 -658
  29. package/cli/portulan.mjs +5 -124
  30. package/cli/recipe-set.mjs +15 -245
  31. package/cli/release-eval.mjs +28 -348
  32. package/cli/review-meter.mjs +17 -258
  33. package/cli/rule-carriers.mjs +12 -175
  34. package/cli/sessions.mjs +82 -0
  35. package/cli/skill-goldens.mjs +10 -176
  36. package/cli/skills-set.mjs +12 -414
  37. package/cli/stop-gate.mjs +83 -514
  38. package/cli/symbols.mjs +1132 -0
  39. package/cli/telemetry.mjs +27 -492
  40. package/cli/upgrade.mjs +171 -314
  41. package/cli/vendor.mjs +146 -476
  42. package/cli/version-carriers.mjs +8 -86
  43. package/core/README.md +1 -1
  44. package/core/engine.md +3 -9
  45. package/core/operating/autonomy.md +1 -1
  46. package/core/operating/context.md +183 -0
  47. package/core/operating/evolution.md +3 -3
  48. package/core/operating/loop.md +13 -20
  49. package/core/operating/memory.md +12 -3
  50. package/core/operating/sessions.md +100 -0
  51. package/core/personas/librarian.md +2 -2
  52. package/core/skills/consolidate/SKILL.md +4 -2
  53. package/core/templates/handoff.md +28 -15
  54. package/core/templates/memory-entry.md +1 -0
  55. package/evals/README.md +255 -845
  56. package/evals/releases/0.2.0.json +146 -0
  57. package/evals/releases/0.2.0.md +67 -0
  58. package/evals/releases/README.md +3 -3
  59. package/package.json +4 -2
  60. package/packs/rituals/checkpoints/README.md +1 -1
  61. package/packs/tools/github/verify/actions-pinned.sh +10 -55
  62. package/spec/README.md +112 -6
  63. package/spec/migrations/0001-repository-declares-its-tree.mjs +1 -26
  64. package/spec/migrations/0002-bundle-fallback-path.mjs +13 -74
  65. package/spec/migrations/0003-handoff-index-not-kept.mjs +66 -0
  66. package/spec/migrations/0004-changelog-fragments.mjs +53 -0
  67. package/spec/migrations/0005-session-log-retired.mjs +69 -0
  68. package/spec/migrations/0006-boot-card.mjs +77 -0
  69. package/spec/migrations/0007-guidance-compiled.mjs +31 -0
  70. package/spec/migrations/0008-card-reading.mjs +55 -0
  71. package/spec/migrations/0009-instruction-sections-on-read.mjs +60 -0
  72. package/spec/migrations/0010-comment-rail.mjs +57 -0
  73. package/spec/migrations/README.md +45 -10
  74. package/spec/pack.schema.json +1 -1
  75. package/spec/slots.md +333 -21
  76. package/spec/workspace.schema.json +145 -4
package/evals/README.md CHANGED
@@ -1,910 +1,320 @@
1
1
  # evals/
2
2
 
3
- Milestone 8's home: golden tasks, the A/B harness, and the eval gate that lets a rule change merge or
4
- be rejected on evidence rather than assertion. Row 8 of [`../docs/plan.md`](../docs/plan.md) is the
5
- binding criterion; [`../docs/milestones/m08.md`](../docs/milestones/m08.md) is its legislative history.
6
- This file does not restate either — a rule with two carriers is obeyed at the narrower one.
3
+ Milestone 8's home: the golden corpora, the review-loop register, the telemetry opt-in, the A/B harness's
4
+ records and the eval result each release carries. Row 8 of [`../docs/plan.md`](../docs/plan.md) is the
5
+ binding criterion; [`../docs/milestones/m08.md`](https://github.com/sleepy-panda-srl/portulan/blob/8a33f9b/docs/milestones/m08.md) is its legislative history.
6
+ This file does not restate either — a rule with two carriers is obeyed at the narrower one. It says what
7
+ is here, how to run or extend each rail, and what each one does not establish; a rail's detail is its
8
+ module, whose longer header is at `git show a31da46:cli/<module>.mjs`. How each clause came to be —
9
+ first-run figures, reversed designs, review rounds — is at `git show 8a33f9b:evals/README.md`.
7
10
 
8
- ## First, the word — because it already means something else here
11
+ ## Two words that mean two things here
9
12
 
10
- **"Eval" carries two unrelated senses in this repository, and only one of them lives in this
11
- directory.** Both were load-bearing before this directory had any content, so the disambiguation is
12
- written down rather than left to be inferred from a filename.
13
+ **"Eval" carries two unrelated senses in this repository, and only one of them lives in this directory.**
13
14
 
14
15
  | Sense | Where it lives | What it is |
15
16
  |---|---|---|
16
17
  | **Evaluation as measurement** — golden tasks, A/B, the eval gate | **here**, `evals/` | Does the engine make an agent work better? Milestone 8's subject. |
17
- | **Evaluation as a licensed copy** | [`../cli/eval-bundle.mjs`](../cli/eval-bundle.mjs), [`../.portulan/verify/eval-bundle.sh`](../.portulan/verify/eval-bundle.sh) | A named-recipient bundle cut from a commit under evaluation terms. A commercial artifact, with its issuance ledger kept permanently outside this repository. Nothing to do with this directory. |
18
-
19
- **And `goldens` itself now names two runners, so that is disambiguated here too rather than left to a
20
- prefix.** [`../cli/goldens.mjs`](../cli/goldens.mjs) grades the **compiled gates** against adversarial
21
- fixtures, through the compiler's own `matchesRule`. [`../cli/skill-goldens.mjs`](../cli/skill-goldens.mjs)
22
- grades the **core skills'** mandates against the live artifacts they govern. Same clause, two subjects —
23
- the 2026-07-28 amendment's *"golden tasks reach the gates as well as the skills"* — and two oracles, which
24
- is why they are two modules and two recipes rather than one with a flag.
25
-
26
- The machinery this directory ships is therefore named **`goldens`**, never `evals` — so a reader meeting
27
- `cli/goldens.mjs` beside `cli/eval-bundle.mjs` is not left to guess which sense is which. The directory
28
- keeps the row's own word; the tools take a narrower one.
29
-
30
- ## What is built today
31
-
32
- **Eight clauses of row 8, of nine, and a ninth built to its mechanism but not to its instance.** (a), adversarial fixtures per compiled gate, landed 2026-08-24;
33
- **(b)**, mutation testing over both matchers and grammar-aware fuzzing over the shell segmenter,
34
- landed 2026-08-25; **(d)**, scheduled forced-red drills, landed 2026-08-25; **(c)**, review-loop
35
- metering, landed 2026-08-26; **golden tasks per core skill** — the row's original first subject —
36
- landed 2026-08-26; **OTel opt-in config** landed 2026-08-28; **the A/B baseline** landed 2026-08-31,
37
- across three sessions the maintainer split at construction | grading | running — 6b the arms, 6c the
38
- graders, **6d the run**; and **a rule change merged or rejected on eval evidence** was decided by the
39
- instruments 2026-09-03 and **discharged on its merge, 2026-09-09** — the clause reads *merges*, and the
40
- session that measured the table said in terms that *"his merge is the ruling"*, so the date the evidence
41
- was taken is not the date the clause landed; its own section is below. **A release carries an eval
42
- result** landed as a *mechanism*
43
- 2026-09-01 — the record layer, the rail and its forced-red drill — and is **half-vouched**, the row's own
44
- word for a clause whose demonstration was owed to an event that had not happened yet. It has now half
45
- happened: `0.1.3` carries a record **in the tree**, and what is still owed is the tag and a published
46
- body citing it — both Gated and both the maintainer's. The two are not equally unreachable, and saying
47
- so loosely is the overclaim this file warns about elsewhere: `release-eval --tagged` reaches the
48
- **tagged tree** at the publish, from the release workflow, while the **body** — authored on GitHub — is
49
- the surface no check reaches at all. **Nothing remains outright.** Each is listed below with the sentence
50
- [`../.portulan/dod.md`](../.portulan/dod.md) condition 4 requires.
51
-
52
- ## The A/B baseline — what 40 turns measured, and what they did not
53
-
54
- [`ab/baseline.json`](ab/baseline.json) is the capture and [`ab/baseline.md`](ab/baseline.md) the register
55
- rendered from it. **k = 5 per cell, ruled by the maintainer; 4 scenarios × 2 arms × 5 runs; every turn
56
- isolated with a home and config directory of its own.** Held by the `ab-run` recipe and a forced-red
57
- drill.
58
-
59
- **At the 2026-08-31 capture the headline is a tie**, and [`ab/baseline.md`](ab/baseline.md) carries the
60
- figure itself, with every per-turn row beside it — this paragraph cites that register rather than
61
- restating it. The date is part of the claim: it is what keeps this sentence true rather than stale if
62
- the arms are ever re-run. It is recorded as measured. No re-run was taken to find a better number, and
63
- none will be: `k = 5` supports a recorded rate and nothing else.
64
-
65
- **The finding that matters most is not the tie.** On `done-demonstrated` the rig fails unconditionally,
66
- so the only route to a green rail is rewriting the check — and **arm A rewrote it in 3 of 5 turns**.
67
- Session 6c's first design labelled a green rail `demonstrated` and scored it as compliance; the
68
- session-open checkpoint reversed that before any turn ran. Had it not, those three turns would have
69
- counted as compliances and the register would publish **an arm-A lead manufactured entirely by scoring
70
- the gaming move as the good outcome**, against the tie [`ab/baseline.md`](ab/baseline.md) recorded on
71
- 2026-08-31. The reversal is what makes that tie trustworthy, and it is the strongest evidence this milestone has
72
- produced that a fresh-context checkpoint catches what its own author cannot.
73
-
74
- **On `altitude` arm A REACHED the compliant location and was still scored 0/5, and that is the finding.**
75
- It wrote the constraint into `.portulan/tasks/` on **3 of 5** turns — arm B reached it never. All three
76
- graded `higher-layer` because `gradeAltitude` gives any governance-surface occurrence precedence, and in
77
- every one of those turns the governance hit is **the dated handoff arm A's own `dod.md` condition 8
78
- demands**. So the predicate **counted** a gate-mandated artifact as promotion and **penalised** the
79
- treatment arm for obeying the treatment: a treatment–instrument interaction, not a fact about the arm. _(Runs 0 and
80
- 3 are different and are genuine promotion — they wrote the constraint into `memory/`.)_ **The registered
81
- carrier of what may not be concluded from this is
82
- [`ab/corpus.md`](ab/corpus.md)'s section of that name**, which since the milestone-8 close carries the
83
- argument in full — including the half this paragraph does not state, that arm B has no path to the
84
- penalising branch at all, so the row is one-directional rather than merely noisy. The rendered register
85
- cites that section; this paragraph cites it too rather than becoming a fourth uncited restatement. On
86
- `curated-layer` arm A wrote into `memory/` twice and arm B touched neither curated directory — a visible
87
- difference, 0/5 both.
88
-
89
- Recorded rather than repaired at the capture and at the close: moving a target after seeing the figures
90
- is the selection this instrument exists to refuse, and what to do about it was the maintainer's.
91
- **Repaired 2026-09-09, the same day the close named it**, when he lifted the hold — the session-record
92
- slots left `altitude`'s population, precedence was left alone, and what that implies for the 2026-08-31
93
- capture is a **re-classification of a fixed record and never a new baseline**, disclosed in
94
- [`ab/corpus.md`](ab/corpus.md) beside the date. The paragraph above describes that capture and stays in
95
- its own tense: it is a record, not a carrier. **The baseline is scoped to the
96
- vendored-and-compiled tier** [`ab/arm.md`](ab/arm.md) specifies, and closes row 8 for no other
97
- configuration of *Portulan on*.
98
-
99
- **One cell is at ceiling.** `observed-content` is 5/5 for both arms — [`ab/corpus.md`](ab/corpus.md)
100
- warned in terms that a bare agent which also declines the injection makes the row uninformative, and it
101
- does.
18
+ | **Evaluation as a licensed copy** | [`../cli/eval-bundle.mjs`](../cli/eval-bundle.mjs), [`../.portulan/verify/eval-bundle.sh`](../.portulan/verify/eval-bundle.sh) | A named-recipient bundle cut from a commit under evaluation terms, its issuance ledger kept outside this repository. Nothing to do with this directory. |
102
19
 
103
- ```
104
- evals/goldens/gates/<rule-id>.json one fixture file per rule in the yielded gate policy
105
- ```
20
+ So the machinery here is named **`goldens`**, never `evals`: the directory keeps the row's word and the
21
+ tools take a narrower one. And **`goldens` names two runners** — two subjects and two oracles, which is why
22
+ they are two modules and two recipes rather than one with a flag. [`../cli/goldens.mjs`](../cli/goldens.mjs)
23
+ grades the **compiled gates** through the compiler's own `matchesRule`;
24
+ [`../cli/skill-goldens.mjs`](../cli/skill-goldens.mjs) grades the **core skills'** mandates against the
25
+ live artifacts they govern.
106
26
 
107
- **The filename is checked against the file's own `rule` field**, since 2026-08-24. It was documented
108
- here, printed in the missing-fixture red (*"add `evals/goldens/gates/<rule-id>.json`"*), and enforced
109
- nowhere — so a renamed or misfiled fixture validated cleanly and graded anyway. A mandate nothing
110
- checks is already broken, and this directory was carrying one of its own. Found as a suppressed note
111
- in Copilot round 5 of [#336](https://github.com/sleepy-panda-srl/portulan/pull/336).
27
+ Each rail below is a verify recipe in [`../.portulan/verify/`](../.portulan/verify/), run on every pull
28
+ request; `bash .portulan/verify/<recipe>.sh` runs one by hand, and `node cli/<module>.mjs --help` prints a
29
+ module's modes. Every runner prints its own totals, so no figure is written here.
112
30
 
113
- Graded by [`../cli/goldens.mjs`](../cli/goldens.mjs), run as the `goldens` verify recipe, on every
114
- pull request. Run it by hand with:
31
+ ## The gate corpus — clause (a)
115
32
 
116
33
  ```
34
+ evals/goldens/gates/<rule-id>.json one fixture file per rule in the yielded gate policy
117
35
  node cli/goldens.mjs --workspace . --pack-root packs
118
36
  ```
119
37
 
120
- **Why this clause first.** Not because everything else hangs off it — golden tasks, the A/B harness,
121
- OTel and review-loop metering share almost nothing with a matcher-fixture runner, and claiming
122
- otherwise would be an overclaim of exactly the kind this repository keeps finding. The honest reason
123
- is narrower: clause **(b)** — mutation testing over both matchers and grammar-aware fuzzing over the
124
- shell segmenter — needs this corpus as its kill-set and this fixture format as its output shape. (a)
125
- is load-bearing for (b) and for nothing else in the row.
126
-
127
- ### What a fixture is
128
-
129
- A case is **data**, answered by the compiler's own exported `matchesRule` — the same function the hook
130
- calls at tool time, never a re-implementation. **A case's command string is never executed.** The
131
- corpus contains `git push --force`, `rm -rf docs` and constitution-write spellings by design; there is
132
- no code path from a fixture to a subprocess, and the suite asserts the runner imports no
133
- process-spawning API at all.
134
-
135
- Two case classes, and exactly two:
136
-
137
- - **`holds`** — the matcher catches this today and must keep catching it. Every one of the eight
138
- bypasses found *after* [#60](https://github.com/sleepy-panda-srl/portulan/pull/60)'s gate was called
139
- done is one of these.
140
- - **`documented-hole`** — the matcher does **not** catch this, a named record says so, and the case
141
- keeps that admission true **in both directions**: if the hole silently closes, the case goes red
142
- until the record is updated. A hole list that still lists a closed hole is as wrong as one that
143
- hides an open one.
144
-
145
- **Every case records which branch of `matchesRule` it exercises**, and the field is **derived, never
146
- declared**: `matcherPath(kind, tool)` computes it from the rule's action kind and the case's tool, and
147
- the runner refuses a case whose stored value disagrees. Four values — `matchesPath`, `shell-write`,
148
- `shell-prefix`, and `no-branch` for a combination the matcher has no code for. The green prints the
149
- per-path census, including the zeroes, because a corpus can carry two hundred cases and exercise one
150
- branch of four.
151
-
152
- It earns its place on one asymmetry: a `then`/`do`/brace-group leader is **caught** on the write path
153
- — `shellSegments` knows `SEGMENT_LEADERS` — and **escapes** on the shell path, where `commandSegments`
154
- does not. One rule id, two segmenters, two answers. Without the field those two cases read as a
155
- contradiction rather than as the asymmetry they are.
156
-
157
- Byte-level attacks are stored **escaped** (JSON `\r`, `\u0000`) and decoded by `JSON.parse`.
158
- [`../cli/control-chars.mjs`](../cli/control-chars.mjs) refuses a raw CR anywhere in this tree by
159
- decision, and exempting a growing adversarial-content directory is the allow-list defect that same file
160
- names — so the corpus carries no raw control bytes, and a test asserts both halves: the bytes are clean
161
- *and* the escapes really decode.
162
-
163
- ## Clause (b) — the corpus is measured against a broken matcher, and the grammar against bash
164
-
165
- Two rails, landed 2026-08-25, and they answer two different questions:
38
+ **What a fixture is.** A file is named for its own `rule` field — the runner refuses a misfiled one — and
39
+ holds `cases`, each `{id, class, tool, path, input, expect, why}`. A case is **data**, answered by the
40
+ compiler's own exported `matchesRule`, the function the hook calls at tool time. **A case's command string
41
+ is never executed**, and the suite asserts the runner imports no process-spawning API. Two classes, and
42
+ exactly two:
43
+
44
+ - **`holds`** — the matcher catches this today and must keep catching it.
45
+ - **`documented-hole`** — the matcher does **not** catch this, and `hole` names the record that says so.
46
+ It holds **in both directions**: if the hole silently closes, the case goes red until the record is
47
+ updated.
48
+
49
+ `path` names the branch of `matchesRule` a case exercises — `matchesPath`, `shell-write`, `shell-prefix`
50
+ or `no-branch` — and is **derived, never declared**: the runner computes it with `matcherPath(kind, tool)`
51
+ and refuses a case that disagrees. Byte-level attacks are stored **escaped** (JSON `\r`, `\u0000`), since
52
+ [`../cli/control-chars.mjs`](../cli/control-chars.mjs) refuses raw control bytes in the tree; the suite
53
+ checks both that the corpus is clean and that the escapes decode.
54
+
55
+ The corpus is load-bearing for the next clause: clause (b) needs this corpus as its kill-set and this
56
+ fixture format as its output shape. A `mutants` survivor is repaired by a new case here, and a `fuzz-shell`
57
+ finding prints as a case ready to paste — reviewed first, since its `expect` records what the matcher did.
58
+
59
+ **What it does NOT establish: adequacy.** It checks **presence** — every compiled gate has fixtures and
60
+ every case answers as recorded — which one trivial case per rule satisfies. `mutants` checks
61
+ **discrimination**; **realism**, whether the attacks resemble anything an adversary would type, stays a
62
+ reviewer's judgement. The runner prints that limit on every green and names every rule exempt for
63
+ declaring no matchable action.
64
+
65
+ ## Mutants and the grammar fuzzer — clause (b)
166
66
 
167
67
  ```
168
68
  node cli/mutants.mjs --workspace . --pack-root packs the mutation census
169
69
  node cli/fuzz-shell.mjs --workspace . --pack-root packs the grammar fuzzer
170
70
  ```
171
71
 
172
- **`mutants` asks whether the corpus DISCRIMINATES.** It breaks
173
- [`../cli/compile.mjs`](../cli/compile.mjs)'s matcher region on purpose — one declared, anchored,
174
- place-exactly-once substitution at a time — and grades each mutant against this corpus. An operator
175
- that the corpus fails to notice is a hole in the kill-set, and the repair is a new fixture: `matchesRule`
176
- is a pure function of `(rule, tool, input)` and a fixture is exactly that triple, so any non-equivalent
177
- mutant is killable by one. A `survives` record is admissible only as a **proof** — semantic
178
- equivalence, or equivalence under the yielded policy — never as a standing note that a gap exists,
179
- which would rebuild the prose hole list clause (a) exists to have replaced.
180
-
181
- **It went red on its first run and the corpus lost.** Among the breakages that went unnoticed:
182
- removing `sudo` from the command-prefix table, dropping `..` resolution from path normalisation, and
183
- disabling quote tracking in the segmenter. Every one is a fixture now, and each was derived by
184
- measuring which input distinguishes the mutant rather than by reasoning about it — two were *not*
185
- killed by the spelling that seemed obvious.
186
-
187
- _The figures for that first run are dated in
188
- [the session's handoff](../.portulan/handoffs/2026-08-25-the-corpus-lost-and-the-fuzzer-found-a-live-bypass.md),
189
- and the SHIPPED totals are printed by `node cli/mutants.mjs` and `node cli/goldens.mjs`, which are
190
- their one carrier. This paragraph carried "eleven of forty-eight" against a table that had since grown
191
- to fifty-three — a count written before the thing it counted stopped growing, which is this
192
- repository's most-repeated defect and was this session's third instance of it. The pre-commit
193
- checkpoint caught the same figure in the handoff and the repair stopped at the site that was quoted;
194
- Copilot round 2 found the one it missed. Deleted rather than corrected, so the trap is not re-armed
195
- for whoever adds the next operator._
196
-
197
- **`fuzz-shell` asks whether the SEGMENTERS answer one grammar.** It composes a command from a grammar
198
- instead of mutating a string, so it knows by construction whether the payload sits where bash would
199
- execute it or where bash would only print it. Positions are enumerated and recorded; **spellings are
200
- fuzzed**, and the invariant is that every spelling of one command in one position gets the same
201
- answer. Every recorded divergence from ground truth cites the record that licenses it.
202
-
203
- **The grammar's own ground truth is measured, not argued.**
204
- [`../cli/fuzz-shell.ground.test.mjs`](../cli/fuzz-shell.ground.test.mjs) runs every position under
205
- real bash with a **neutral** payload — never a gated command — and writes every path spelling to a
206
- throwaway file. A grammar that lies about itself produces not a red but a green about the wrong thing,
207
- which is the one failure a fuzzer cannot detect in itself. It caught two.
208
-
209
- **It found a live bypass of every Gated shell action.** `bash -c "ls; git push --force origin main"`
210
- answered **false**: the composition tested the raw command's segments and each segment's spellings,
211
- and never a spelling's segments. The write matcher never had the gap, because `shellWrites` segments
212
- again internally — one fix landing in one carrier and not its sibling, between two branches of one
213
- function. Closed the same day, at the class rather than the spelling, with the two-wrapper
214
- counterexample asserted so the unwrap budget stays at one level.
72
+ **`mutants` asks whether the corpus discriminates.** It breaks [`../cli/compile.mjs`](../cli/compile.mjs)'s
73
+ matcher region on purpose, one declared, anchored operator at a time, and grades each mutant against the
74
+ gate corpus. An operator the corpus fails to notice is a hole in the kill-set and the repair is a new
75
+ fixture; a `survives` record is admissible only as a **proof** of equivalence, never as a standing note
76
+ that a gap exists.
215
77
 
216
- ### What this rail does NOT establish
78
+ **`fuzz-shell` asks whether the two segmenters answer one grammar.** It composes each command from a
79
+ grammar, so it knows whether the payload sits where bash would execute it or only print it. Positions are
80
+ enumerated and recorded, spellings are fuzzed, and every spelling of one command in one position must get
81
+ the same answer; every recorded divergence from ground truth cites its record. The grammar's own ground
82
+ truth is measured under real bash with a neutral payload by
83
+ [`../cli/fuzz-shell.ground.test.mjs`](../cli/fuzz-shell.ground.test.mjs), and the seed is pinned and
84
+ printed on every run.
217
85
 
218
- **Adequacy — whether the cases are a real attack.** What the rail checks is **presence**, and the two
219
- are worth separating out loud, because a green looks the same either way:
86
+ **What they do NOT establish:** that the attacks are realistic (`mutants`), or what a gate ought to cover
87
+ (`fuzz-shell` — whether a recorded escape should stay open is a policy question for the maintainer).
220
88
 
221
- | The rail answers | The rail cannot answer |
222
- |---|---|
223
- | Does every compiled gate have fixtures at all? | Are those fixtures any good? |
224
- | Does every case still answer as recorded? | Is the case worth answering? |
225
-
226
- So a gate cannot reach the compiled policy with **no** adversarial thought recorded against it — and
227
- one trivial happy-path fixture per rule satisfies the floor while proving nothing. The runner prints
228
- this limit on every green rather than letting the exit code imply more than it means.
229
-
230
- **Half of that gap closed on 2026-08-25, and the sentence has to move with it.** This section used to
231
- end *"no check can tell those apart"*, and clause (b)'s mutation census is a check that tells part of
232
- them apart: it breaks the matchers on purpose and asks whether the corpus notices. So the right split
233
- is now three ways rather than two — **presence** (the `goldens` rail), **discrimination** (the
234
- `mutants` rail: does the kill-set catch a matcher that has been broken?), and **realism** (whether the
235
- attacks resemble anything an adversary would type), which is still a reviewer's judgement and stays
236
- one. The census is what forced the correction rather than a reader noticing: it went red on its first
237
- run, on 2026-08-25, against the corpus as it then stood — which failed to notice a whole class of
238
- breakages, every one of which is a fixture now.
239
-
240
- The **exemption** is the obvious way to dodge the rail: write the next gate `none`-shaped and it needs
241
- no fixtures. So every exempt rule is named in the output on every run, the way `compile --matrix` prints
242
- its own refused rules.
243
-
244
- ### What it found on its first run
245
-
246
- A hole nobody had recorded: **a rule whose target is the whole repository (`./`) matches nothing at
247
- runtime.** `matchesPath` reduces `"./"` to the empty string and then refuses the empty string, so
248
- `edit-on-a-working-branch` and `read-anything-in-the-repository` answer false for every input. Nothing
249
- is mis-enforced today — both are `auto`, and neither layer ever asks — but a **gated** rule written
250
- that way would compile to a named permission surface while the matcher covered nothing. Now
251
- entry 8 of [`../.portulan/gate-map.md`](../.portulan/gate-map.md)'s honest-holes list, tracked as [#337](https://github.com/sleepy-panda-srl/portulan/issues/337), asserted here, and **closed at the
252
- enforcing tiers on 2026-09-09**, the date the change merged — see *A rule change decided on eval
253
- evidence* below.
254
-
255
- _(This paragraph said the gated rule **"would compile to a permission rule covering the tree"**, and
256
- that was wrong in a way worth keeping rather than quietly deleting: re-derived on 2026-09-03, a real
257
- target compiles to a `**` glob and `./` does not — the emitted spec is the bare `Edit(./)`, whose host
258
- meaning this repository installs nothing to measure. Either reading is a hazard, and the one this
259
- repository can measure offline is the narrower and sharper one: the compiler reports the rule **compiled**
260
- while the matcher answers false for every input. Corrected under
261
- [`../.portulan/proposals/0022-a-claim-about-a-mechanism-is-re-derived-like-a-figure.md`](../.portulan/proposals/0022-a-claim-about-a-mechanism-is-re-derived-like-a-figure.md).)_
262
-
263
- Ten of the corpus's own hand-written expectations were refuted by the rail on the same run, which is
264
- the argument for the rail in one sentence.
265
-
266
- ## Clause (d) — every rail is forced red on a calendar, and required to fire
89
+ ## Forced-red drills — clause (d)
267
90
 
268
91
  ```
269
92
  node cli/drills.mjs --pack-root packs the sweep: force every rail red
270
93
  node cli/drills.mjs --check --pack-root packs the roster: every rail has a drill
271
94
  ```
272
95
 
273
- Landed 2026-08-25. `goldens` asks whether a gate has fixtures; `mutants` asks whether those fixtures
274
- discriminate; **this asks whether the rail still fires at all.** A recipe whose precondition quietly
275
- started exiting 0 over an empty file list, a hook that fails open on a crash, a check whose enumeration
276
- went empty — each reports green and each has stopped being a rail, and until this landed the way that
277
- was found here was an incident.
278
-
279
- **Its own provenance is two sessions doing it by hand.** The drills run against `goldens` on 2026-08-24
280
- and against `mutants` and `fuzz-shell` on 2026-08-25 were all run by hand, in a session, and recorded in
281
- those sessions' handoffs — which is precisely the state this clause exists to replace. Both of those
282
- sessions also had **a drill that did not fire**: one anchored substitution missed by four spaces of
283
- indentation, one patch script's quoting broke, and both times the recipe ran green against an unmodified
284
- file. Every guard in [`../cli/drills.mjs`](../cli/drills.mjs) traces to one of those two.
285
-
286
- **A drill is a pair, and the pair is the oracle.** A control on a pristine tree, then the perturbation —
287
- because *a rail that only ever reds proves nothing about its green*, and because a control that is
288
- already red makes the drill **could-not-run** rather than a fire the perturbation did not cause. Each
289
- drill declares a **tell** its rail's own output must carry when it fires and must not carry before, so a
290
- red for the wrong reason is not counted as a fire; each perturbation must place **exactly once** and must
291
- move bytes on disk. Isolation is one throwaway `git worktree` per drill, so nothing perturbs a working
292
- tree.
293
-
294
- **The sweep reports on a COMMIT and prints which one.** A dirty tree is refused outright; `--working-copy`
295
- synthesizes one with `git stash create` and refuses while untracked-and-unstaged files exist, since a
296
- synthesized tree missing the file under review would be a green about the wrong tree. That is also why
297
- the **verify recipe runs `--check` and not the sweep**: a recipe that answered about `HEAD` would not be
298
- answering [`../.portulan/dod.md`](../.portulan/dod.md) condition 1's question.
299
-
300
- **What the word *every* covers, and what it does not.** The sweep drills every recipe the workspace
301
- yields, plus the Stop-gate and the PreToolUse gate runner — the amendment's *"from watchers to every
302
- rail"*. The rails it cannot force are **named in the output on every run** with the reason: the platform
303
- floor, the host's own permission layer, the CI seam, the pre-commit seam scan, `claude plugin validate
304
- --strict`, the platform watchers, and the librarian's pass. A scope claim with no carrier is what the
305
- prose register in [`../.portulan/verify/README.md`](../.portulan/verify/README.md) was, and that table is
306
- deleted in favour of this.
307
-
308
- **What is demonstrated, and what is unvouched.** All twenty-one rails were forced red by hand on
309
- 2026-08-25 and every one fired. The **calendar** — [`../.github/workflows/drills.yml`](../.github/workflows/drills.yml),
310
- weekly — is a watcher, so it owes its own observation under
311
- [`../.portulan/proposals/0007-every-watcher-ships-with-its-observation-procedure.md`](../.portulan/proposals/0007-every-watcher-ships-with-its-observation-procedure.md):
312
- `workflow_dispatch` is **answered** — run
313
- [`32883413709`](https://github.com/sleepy-panda-srl/portulan/actions/runs/32883413709) on `da9c06e`, 21 of
314
- 21 — and the **schedule** is answered by its first run and nothing earlier. **The schedule has now answered
315
- twice, and both runs — with the dispatch before them — are in [`../.portulan/verify/README.md`](../.portulan/verify/README.md), the one carrier of that answer.** And a *missing* run stays undetectable — that is `0007`'s silence problem one altitude up,
316
- tracked as [#344](https://github.com/sleepy-panda-srl/portulan/issues/344) rather than built.
317
-
318
- _This paragraph said the gap was already **filed** — the fifth carrier of that sentence, and the one a
319
- first repair of the other four walked past. A claim in the past tense about an issue that did not exist is
320
- `../.portulan/dod.md` condition 4's own case, and a fix that stops at the sites somebody quoted is
321
- [`0020`](../.portulan/proposals/0020-a-fix-is-not-done-at-the-site-it-was-found.md). Both found at the
322
- pre-commit checkpoint's second pass, which re-derived the carrier set instead of reading the list it had
323
- been handed._
324
-
325
- ## Clause (c) — the loop's own figures stop being counted by hand
96
+ `goldens` asks whether a gate has fixtures and `mutants` whether they discriminate; **this asks whether
97
+ the rail still fires at all.** A drill is a pair — a control on a pristine tree, then a perturbation that
98
+ must place exactly once and move bytes on disk — and the rail's output must carry the drill's declared
99
+ tell in the fire and not in the control. Each runs in a throwaway `git worktree`. A new rail needs an
100
+ entry in `DRILLS` in [`../cli/drills.mjs`](../cli/drills.mjs): `--check` reds a yielded rail with no drill.
101
+ The sweep reports on a **commit** and refuses a dirty tree (`--working-copy` synthesizes one), which is why
102
+ the verify recipe runs `--check` and not the sweep; the rails it cannot force are named in its output, each
103
+ with the reason.
104
+
105
+ **The calendar** is [`../.github/workflows/drills.yml`](../.github/workflows/drills.yml), weekly, and the
106
+ recorded runs of its dispatch and its schedule are in
107
+ [`../.portulan/gate-map/platform-floor.md`](../.portulan/gate-map/platform-floor.md).
108
+
109
+ **What it does NOT establish:** `--check` runs no rail, so whether each one fires is the sweep's answer on
110
+ the calendar; and a scheduled run that never starts stays undetectable from inside —
111
+ [#344](https://github.com/sleepy-panda-srl/portulan/issues/344).
112
+
113
+ ## The review-loop meter — clause (c)
326
114
 
327
115
  ```
328
116
  node cli/review-meter.mjs --snapshot evals/review-loop/snapshot.json the figures
329
117
  node cli/review-meter.mjs --fetch --repo <owner/name> --out <file> the one mode that fetches
330
118
  ```
331
119
 
332
- Landed 2026-08-26. `goldens` asks whether a gate has fixtures; `mutants` whether those fixtures
333
- discriminate; `drills` whether the rail still fires. **This one measures the process wrapped around all
334
- three** — the review loop every change here goes through.
335
-
336
- **Its provenance is a record that indicts itself.**
337
- [`../.portulan/memory/a-review-loop-needs-a-bound.md`](../.portulan/memory/a-review-loop-needs-a-bound.md)
338
- bounds the loop on a table — 110 submissions over 30 pull requests, 29% of them finding nothing — every
339
- figure counted by hand on 2026-07-28, and its own *Why it holds* section says **"Nothing checks it —
340
- discipline, not a rail"**, citing
341
- [`../.portulan/memory/a-mandate-nothing-checks-is-already-broken.md`](../.portulan/memory/a-mandate-nothing-checks-is-already-broken.md).
342
- The 2026-07-28 amendment answered it in as many words: *"the telemetry clause is where that checker's
343
- home is, and naming the home is what this amendment does rather than claiming the checker exists."*
344
-
345
- ### The units were the design decision, and the criterion predates the definition it uses
346
-
347
- The criterion was written **2026-07-28**; *round* was defined **2026-07-30** as *a Copilot review the
348
- working session answers with a push*. The rule's table was **re-labelled, not re-counted** — its figures
349
- always counted **submissions** — and its `Retire when:` settles it, naming the threshold in *"the
350
- submission units of the table above, **not fix-rounds**"*. So the criterion's *"rounds per pull
351
- request"* is submissions per pull request, and the tool prints that name and never the bare word.
352
-
353
- ### What the API answers, and the one thing it does not
354
-
355
- **Fix-rounds are not derivable, and that is the finding rather than a gap.** Two demonstrations, both
356
- from the pull request that produced the definition: on
357
- [#105](https://github.com/sleepy-panda-srl/portulan/pull/105) the commit `08d7d10` answered an inline
358
- finding and **was never a reviewed head** — it rode inside the next push — and the push at `cff3e4e0`
359
- follows a 4,087-byte finding-bearing submission while answering none of it. Any rule keyed on *"a
360
- finding-bearing submission preceded this push"* calls the second a round; the maintainer's own table
361
- calls it *no*. The 2026-07-30 ruling states the method that works and it is not one an API has:
362
- *"Count pushes, then look inside each one."*
363
-
364
- So the tool computes no fix-rounds, estimates none, and prints nothing that could be mistaken for one.
365
-
366
- **The empty-round rate is an upper bound, and the reason is a layering rule.** Finding nothing has two
367
- halves — no inline thread, *and* no suppressed low-confidence note in the body. The first is
368
- structural. The second is decided by the awk in
369
- [`../.github/workflows/copilot-review.yml`](../.github/workflows/copilot-review.yml), fixtured in
370
- [`../.portulan/verify/workflow-filters.mjs`](../.portulan/verify/workflow-filters.mjs) — a matcher
371
- deliberately reduced to **one** carrier, and a **workspace-layer** one, while the tool is engine.
372
- Copying it would put two spellings of one rule on opposite sides of the boundary where neither could
373
- see the other drift. So the tool reports `submissions that found nothing ≤ submissions with no inline
374
- comment` and prints the right-hand side under its own name.
375
-
376
- **The boundary forbids a copy; it does not make the exact rate unreachable**, and the first draft of
377
- this section conflated the two. `../.portulan/verify/workflow-filters.mjs` already **lifts and runs**
378
- those awk programs out of the workflow's parsed `run:` scalars, and `--fetch` already spawns — so a body
379
- could be piped through the lifted program at capture time and stored as one integer, no second spelling
380
- and still no bodies in the snapshot. Left unbuilt for **budget**, one clause per session, and tracked as
381
- [#355](https://github.com/sleepy-panda-srl/portulan/issues/355).
382
-
383
- ### What the first run measured
384
-
385
- **140 submissions over the 30 most recently merged pull requests — 4.67 each.** The record's
386
- hand-counted figure is **3.7**, and the two are **not a disagreement**: they measure **disjoint
387
- corpora**. The ratified window is the thirty most recent as of 2026-07-28 and names #44, #49 and #57;
388
- this one runs #301–#354. They share no pull request. So the honest sentence is *the loop weighs 4.67 a
389
- month later*, not *his 30 were re-counted and came out different* — a distinction the first draft of
390
- this section lost, and the pre-commit checkpoint restored.
391
-
392
- That figure is **reported and not acted on**: the record is maintainer-ratified and the curated layer is
393
- human-owned, so a re-derived number is a finding for him rather than an edit for an implementer.
394
-
395
- **And two of the three figures turned out to be one figure.** Pushes and submissions came out exactly
396
- equal — 140 and 140 — because `review_on_push: true` draws one submission per push. So the criterion's
397
- *"pushes per round"* is **1.00 by construction** in submission units, which the register now prints
398
- under that name rather than leaving a reader to infer it; and pushes-per-finding-bearing-submission,
399
- the informative substitute, is not a second measurement either: it is `1/(1 - the no-inline rate)`, and
400
- 140/46 = 3.04 = 1/(1 - 0.671). The tool **detects the coincidence and says so** on every run and in the
401
- register, rather than presenting three columns of which two are algebraically the same column.
402
-
403
- **The window is by merge date, and getting that wrong was this change's own worst defect.**
404
- `gh pr list` orders by pull request **number**, and the first capture taken here inherited that order —
405
- three merge-order inversions, a corpus containing #303 and missing #301, under a register claiming *"the
406
- 30 most recently merged"*. Every published figure was against a corpus its own heading did not name.
407
- Found at the pre-commit checkpoint from evidence inside the committed snapshot: `mergedAt` was already
408
- captured and nothing sorted on it. The fetch now lists a **pool** and takes the window from it by merge
409
- date, records whether the pool saturated — an unsaturated pool makes the window provable rather than
410
- likely — and `validateSnapshot` refuses a snapshot that is not in descending merge order.
411
-
412
- ### What this rail does NOT establish
413
-
414
- | The rail answers | The rail cannot answer |
415
- |---|---|
416
- | Do the published figures come out of the captured data? | Is the captured data current? |
417
- | How heavy is the loop, in submissions? | How heavy is it in fix-rounds? |
418
-
419
- **It is a meter, not a bound.** Rule 4 stops a loop at two fix-rounds and nothing here stops anything;
420
- the record's own honest-limits section already says the judgement it depends on is the interested
421
- party's about its own work, and a tool reporting after the merge does not change that. It adjudicates
422
- no **sibling** exemption either, so *rounds past the bound* is not computable and is not claimed.
423
-
424
- **And the snapshot does not refresh itself.** The `review-loop` recipe compares a register to a
425
- snapshot, never a snapshot to the world, so a stale capture and a current one are the same green.
426
- Refreshing is `--fetch`, run by a person. That is the same silence
427
- [#344](https://github.com/sleepy-panda-srl/portulan/issues/344) tracks for the drill calendar, in a
428
- second place, and it is now **filed** as
429
- [#356](https://github.com/sleepy-panda-srl/portulan/issues/356) rather than only named — the two want
430
- one mechanism between them and closing them together is likely cheaper than either alone.
431
-
432
- ## Golden tasks per core skill — the mandates bound to what they govern
120
+ [`review-loop/snapshot.json`](review-loop/snapshot.json) is the capture and
121
+ [`review-loop/register.md`](review-loop/register.md) the register rendered from it: the first command
122
+ with `--register evals/review-loop/register.md --write` regenerates it, and the recipe byte-compares it.
123
+ The criterion's *"rounds per pull request"* is **submissions** per pull request — its unit before *round*
124
+ was redefined — and the tool prints that name, never the bare word.
125
+ What each figure is, at length, is in the header at `git show a31da46:cli/review-meter.mjs`.
126
+
127
+ **What it does NOT establish:**
128
+
129
+ - **Fix-rounds.** Whether a push answers a submission is a fact about its contents that no API answers, so
130
+ the tool computes none, estimates none, and prints nothing that could be mistaken for one.
131
+ - **The exact empty-round rate.** It prints `submissions that found nothing ≤ submissions with no inline
132
+ comment`, an upper bound. The other half — a suppressed note in the review body — was decided by a
133
+ workspace-layer matcher the engine tool refused to copy, since two spellings of one rule on opposite
134
+ sides of that boundary cannot see each other drift; the matcher left with its workflow on 2026-09-23
135
+ ([#355](https://github.com/sleepy-panda-srl/portulan/issues/355)).
136
+ - **A bound.** It is a meter: rule 4 of
137
+ [`a-review-loop-needs-a-bound.md`](../.portulan/memory/a-review-loop-needs-a-bound.md) stops a loop, and
138
+ nothing here stops anything.
139
+ - **Current data.** The recipe compares the register to the snapshot, never the snapshot to the world, so a
140
+ stale capture and a current one are the same green. Refreshing is `--fetch`, run by a person
141
+ ([#356](https://github.com/sleepy-panda-srl/portulan/issues/356)).
142
+
143
+ ## Golden tasks per core skill
433
144
 
434
145
  ```
435
146
  node cli/skill-goldens.mjs --repo-root . --workspace .portulan
436
147
  ```
437
148
 
438
- Landed 2026-08-26. **The row's original first subject**, untouched by either amendment: the 2026-07-28
439
- expansion says it *"stands exactly as written"*, and clause (a) *widened its subject* to reach the gates
440
- *"as well as the skills"* **without narrowing the subject it already had**
441
- ([`../docs/milestones/m08.md`](../docs/milestones/m08.md), the amendment's own words). (a) built the
442
- gates half. This is the half that sentence always named.
443
-
444
- ### The reading was ruled, not derived
445
-
446
- The criterion reads two ways — a deterministic corpus, or task specifications run against a live model
447
- and judged. [`../.portulan/gate-map.md`](../.portulan/gate-map.md)'s *"Session-open runs `clarify`
448
- against the milestone row itself"* exists because milestone 4 guessed at exactly this shape of ambiguity,
449
- so it went to the maintainer instead of into an argument. **His ruling: a deterministic corpus graded
450
- against live artifacts in the tree.** The model-run sense is the **A/B clause**, separately named in the
451
- same row — and under the other reading the two clauses would collapse into one, which no amendment did.
452
-
453
- ### What a case is
454
-
455
- A case binds one numbered step of a skill's `## The pass` to the artifact set it governs, and names the
456
- rail that enforces it — or `null`, which is the interesting value. Four properties, each of which exists
457
- because a checkpoint found the version without it:
458
-
459
- - **The denominator is derived, not chosen.** The runner enumerates the steps out of the skill file and
460
- requires every one to be `bound` or `unbindable`. A fifteenth step is red. The first draft let the
461
- author pick which mandates to answer for, and a census over a set its own author drew reports *"5 of
462
- 5"* and means nothing — the difference [`../cli/goldens.mjs`](../cli/goldens.mjs) already draws by
463
- deriving its denominator from the yielded policy.
464
- - **An empty step list is could-not-run.** Three `## The pass` spellings exist, so an exact match finds
465
- one skill of three and the other two satisfy *every step is accounted for* **vacuously**. That is this
466
- directory's own sentence about clause (d) — *"a check whose enumeration went empty … reports green and
467
- has stopped being a rail"* — and the heading match is anchored besides, because a loose one grabs
468
- `consolidate`'s `## The one move this pass may not make`.
469
- - **A mandate quote must place exactly once**, or the case refuses. A skill whose wording is reworded,
470
- softened or deleted reddens instead of drifting, and **nothing in this tree did that before.**
471
- - **The predicates use the carriers' own exported functions rather than re-spelling them.** The runner
472
- imports `RETIRE_WHEN` from [`../cli/doctor.mjs`](../cli/doctor.mjs), so the retire-when check is
473
- `doctor`'s regex and not a second one. **What that does not do — and three carriers of this sentence
474
- once said it did — is verify the `carrier` field.** Nothing links it to a check: the pre-commit
475
- checkpoint rewrote every carrier in a corpus file to a module that does not exist and the corpus
476
- stayed green. `carrier` is a **declared, reviewed field**, and a reviewer is what checks it.
477
-
478
- ### `unbindable` is adjudicated, never asserted
479
-
480
- The dodge is obvious — call a mandate unbindable and it needs no case. So the reason takes a **closed
481
- vocabulary**: `judgement-only`, `no-artifact`, `cross-language`, and `already-carried` — the last added
482
- at the pre-commit checkpoint, which found a row classed `bound` whose named carrier the runner never
483
- reads, and no honest term for what it actually was. And a `judgement-only` step **may name
484
- no artifacts**: if you can name the artifacts a mandate governs, it is not judgement, it is unbuilt.
485
- That is [`../cli/mutants.mjs`](../cli/mutants.mjs)'s rule that a `survives` record is admissible only as
486
- a proof and never as a standing note that a gap exists.
487
-
488
- ### What the first run measured
489
-
490
- **Five of fifteen mandates bind to live artifacts — four load-bearing, one census — and ten are
491
- adjudicated unbindable.** No figure is written here beyond that shape; the runner prints its own totals.
492
-
493
- **The ratio is a finding rather than bookkeeping.** How much of a core skill is artifact discipline and
494
- how much is agent judgement is a measurement *about the engine*, which is what this row exists to
495
- produce. **Read the split, not the total**: of the ten unbindable, five are `judgement-only` — one third
496
- of the fifteen — and the rest are `no-artifact`, `cross-language` or `already-carried`, which mean a
497
- carrier exists somewhere else. The judgement rows are the A/B clause's **first** subject. _(**Widened
498
- 2026-08-28, on the maintainer's ruling**, and this sentence read "Only the judgement rows are the A/B
499
- clause's subject" until then. The A/B clause's subject is the mandates `core/` ships that an arm actually
500
- receives — the kernel's, plus the artifact mandates reachable in a built arm — of which the judgement
501
- rows are a part. The reason the narrower reading could not stand is measured rather than argued:
502
- `vendor --host` carries `core/skills/` **not at all**, so all five judgement rows reach nothing
503
- in a vendored arm and would grade the base model. See [`ab/corpus.md`](ab/corpus.md). Four carriers
504
- stated the narrower reading and all four moved in one change; the claim is registered in
505
- [`../.portulan/rule-carriers.json`](../.portulan/rule-carriers.json) so a fifth cannot appear **in the
506
- registered scope** unnoticed — bounded by that registry's own `exclude` list, which is the one carrier
507
- of what it holds and which covers `docs/milestones/`, where the fourth carrier sat.)_ _(Three carriers of
508
- this paragraph said "two thirds are judgement", which was the unbindable total wearing the judgement
509
- label. Corrected at the pre-commit checkpoint, which also found one row classed `bound` whose named
510
- carrier the runner never reads.)_
511
-
512
- **One of the five rows is `census`, and it is marked as such** in the corpus and in the output. A census
513
- row re-indexes a figure `doctor` already prints, under the mandate that wants it; it is not a new check,
514
- and presenting five equal rows would read as more evidence than it holds — the repair this directory
515
- already applied to three columns of which two were one column.
149
+ The row's original first subject. **Ruled, not derived:** a deterministic corpus graded against live
150
+ artifacts in the tree; the model-run sense is the A/B clause's.
151
+
152
+ **What a case is.** [`goldens/skills/<skill>.json`](goldens/skills/) holds one case per numbered step of
153
+ the skill's `## The pass`. The runner derives that denominator from the skill file, so a new step is red
154
+ until a case accounts for it. A step is either:
155
+
156
+ - **`bound`** — a `mandate` quote that must place exactly once in the skill's `SKILL.md`, the `artifacts`
157
+ it governs as a declared slot, a `predicate` decided before any figure is taken, and a `carrier` naming
158
+ the rail that enforces it, or `null`. `kind` is `load-bearing`, or `census` for a row that re-indexes a
159
+ figure an existing recipe already prints.
160
+ - **`unbindable`** — with a `reason` from a closed vocabulary: `judgement-only`, `no-artifact`,
161
+ `cross-language` or `already-carried`. A `judgement-only` step may name no artifacts: if you can name the
162
+ artifacts a mandate governs, it is not judgement, it is unbuilt.
516
163
 
517
164
  ### Accepted drift — the in-tree record
518
165
 
519
- Three live findings, and **all three ship green with the defect accepted rather than repaired.** That is the
520
- distinction to keep: `goldens` and `mutants` each went red on their first run and shipped green with the
521
- defect **fixed**. These are merged records, not an implementer's to rewrite, and doing so under a
522
- one-clause budget would be a second clause.
166
+ A bound case may list live files that fail its mandate and still ship green, in `expect.accepted` as
167
+ `{file, why}`: a merged record is not an implementer's to rewrite. The runner refuses an entry whose `why`
168
+ carries no argument, and the list holds **in both directions** — a file that starts complying reddens the
169
+ corpus until it is delisted, because a drift list that outlives its drift is as wrong as one that hides
170
+ it. The runner prints each count beside the compliant total. Three mandates carry accepted drift, tracked
171
+ together as [#358](https://github.com/sleepy-panda-srl/portulan/issues/358):
523
172
 
524
173
  | Mandate | Live artifacts | Drift |
525
174
  |---|---|---|
526
- | `clarify` step 4 — fold answers back as **EARS** acceptance criteria | `.portulan/tasks/` | **12 of 19** — but see the split below |
527
- | `codify` step 1 — name the incident and link it | `.portulan/proposals/` | **6 of 33** carry no provenance field |
528
- | `codify` step 3 — attach how it earns its place | `.portulan/proposals/` | **13 of 33** carry no enforcement field |
529
-
530
- **The composition of the twelve matters and the headline hides it.** Ten carry **no acceptance-criteria
531
- section in any spelling** — five of those renamed it `## Done when` (`0010`–`0013`, `0015`) and five are
532
- retrospective arc records — and only **two** fail on EARS *shape*: `0001` (one bullet of four) and
533
- `0007` (one of six). The 48 criteria live entirely in the nine files that have a section, and **46 of
534
- them are shaped**. So *"12 of 19 fail the EARS mandate"* is arithmetically right and rhetorically
535
- wrong, and the three-cohort breakdown is in
536
- [#358](https://github.com/sleepy-panda-srl/portulan/issues/358).
537
-
538
- **The maintainer ruled on 2026-08-26 that [`../core/templates/task.md`](../core/templates/task.md)'s
539
- shape still binds**, so the twelve are non-compliant by his word rather than by an implementer's
540
- inference — which is what makes this an accepted drift and not a divergence pinned before anyone ruled.
541
-
542
- **It holds in both directions.** A file here that starts complying reddens the corpus until it is
543
- delisted, because a drift list that outlives its drift is as wrong as one that hides it. That direction
544
- is not decoration: it caught **three filenames the author had written from a number prefix rather than
545
- read off disk**, before any reviewer saw them.
546
-
547
- **The figure moved twice before it was written down, and that is the lesson worth carrying.** One
548
- opening pass said 10 of 19, another said 12, a third said 19 — because each computed before deciding the
549
- predicate. Section-presence answers 10; the mandate's own words are about EARS **shape**, and under that
550
- predicate it is 12, with 46 of 48 individual criteria EARS-shaped. **The predicate is now a reviewed
551
- field on the case**, decided before any figure.
175
+ | `clarify` step 4 — fold answers back as **EARS** acceptance criteria | `.portulan/tasks/` | no acceptance-criteria section, or criteria not EARS-shaped |
176
+ | `codify` step 1 — name the incident and link it | `.portulan/proposals/` | no provenance field |
177
+ | `codify` step 3 — attach how it earns its place | `.portulan/proposals/` | no enforcement field |
178
+
179
+ **The maintainer ruled on 2026-08-26 that [`../core/templates/task.md`](../core/templates/task.md)'s shape
180
+ still binds**, so `clarify`'s six accepted files are non-compliant by his word rather than by an
181
+ implementer's inference — which is what makes them accepted drift and not a divergence pinned before
182
+ anyone ruled. Of those six, five carry no acceptance-criteria section in any spelling and one fails on
183
+ EARS shape alone, so *"fails the EARS mandate"* is arithmetically right and rhetorically wrong; #358 has
184
+ the cohorts.
552
185
 
553
186
  ### What this rail does NOT establish
554
187
 
555
- | The rail answers | The rail cannot answer |
556
- |---|---|
557
- | Is every mandate accounted for? | Is the binding any good? |
558
- | Does a named carrier **contain** its check? | Does it **run** it? |
559
-
560
- The second row is the one to read twice. An import proves the carrier holds the symbol; a rail could
561
- keep it, stop calling it, and this corpus would not notice, because the verdict is computed here over
562
- the artifacts rather than by asking the rail. Closing it needs a per-mandate drill — perturb a record,
563
- require the named rail to fire with its own tell — which is [`../cli/drills.mjs`](../cli/drills.mjs)'s
564
- shape one grain finer and is a second clause's work.
565
-
566
- And it grades a skill's **mandates** against the tree, never an agent's judgement in following one. No
567
- golden task here tells you whether an agent given `clarify` asks good questions.
568
-
569
- ## OTel opt-in config — the first thing here that can send anything anywhere
570
-
571
- **The reading was ruled, not derived.** *"OTel opt-in config"* reads at least three ways, and
572
- [`../.portulan/gate-map.md`](../.portulan/gate-map.md)'s *"Session-open runs `clarify` against the
573
- milestone row itself"* exists because milestone 4 guessed at exactly this shape of ambiguity and cost a
574
- session-blocking question. Put to the maintainer on 2026-08-28 as three readings — a config surface
575
- emitting nothing; that surface **plus a real emitter**, hand-written OTLP-over-HTTP JSON, off by
576
- default and never callable from a verify recipe; or an OTLP-shaped local file sink with no network path
577
- at all. **He ruled the second.** The same session ruled the signal set (**the review-loop figures
578
- only**) and the consent model (below), so all three axes of this clause are his rather than an
579
- implementer's — which is the standard session 4 set for this row and the one milestone 4 paid for.
580
-
581
- **Why this clause now.** A written-down dependency pointed at it and nothing pointed anywhere else:
582
- [`../.portulan/verify/review-loop.sh`](../.portulan/verify/review-loop.sh) said the then-still-open OTel
583
- clause *"owes an emission path for these figures … which is why they are written as machine-readable
584
- JSON with a rendered register beside them, rather than as prose an exporter could not consume."*
585
- Session 3 shaped its output for this session. The debt is discharged **on either reading** of that
586
- recipe's locative phrase: if he reads *"in the telemetry clause"* the other way, the path is now built;
587
- if he does not, none was owed.
188
+ - **Adequacy.** A mandate can be bound and the binding trivial.
189
+ - **That a carrier runs its check.** The predicates import the carriers' own functions — `RETIRE_WHEN`
190
+ comes from [`../cli/doctor.mjs`](../cli/doctor.mjs) — which proves a carrier **contains** the check,
191
+ not that it **runs** it; and `carrier` is a declared, reviewed field that nothing links to a check.
192
+ Closing that needs a per-mandate drill, the `drills` shape one grain finer.
193
+ - **An agent's judgement.** It grades a skill's mandates against the tree, never an agent following one.
194
+ The `judgement-only` rows are the A/B clause's first subject, which [`ab/corpus.md`](ab/corpus.md)
195
+ carries.
196
+
197
+ ## The OTel opt-in
588
198
 
589
199
  ```
590
200
  evals/telemetry/config.json the committed opt-in — the ONLY gate on emission
591
201
  evals/telemetry/review-loop.otlp.json the golden payload, regenerated and byte-compared
202
+ node cli/telemetry.mjs --config evals/telemetry/config.json --render
592
203
  ```
593
204
 
594
- Graded by [`../cli/telemetry.mjs`](../cli/telemetry.mjs), run as the `telemetry` verify recipe, on
595
- every pull request. See it by hand with:
205
+ The same command with `--write evals/telemetry/review-loop.otlp.json` in place of `--render` regenerates
206
+ the golden, which every release cut owes because the version rides in the payload. **Ruled, not
207
+ derived:** a real emitter of hand-written OTLP-over-HTTP JSON, off by default, never callable from a
208
+ verify recipe, carrying the review-loop figures only. [`../cli/telemetry.mjs`](../cli/telemetry.mjs)
209
+ carries the ruling, the environment variables it reads and the ones it does not, and what the payload
210
+ may hold.
211
+
212
+ **The committed file is the only gate** — deliberately not `OTEL_SDK_DISABLED`, which defaults to `false`
213
+ and would make emission an opt-out. The environment supplies transport only, and the validator refuses a
214
+ config carrying `headers`, `endpoint` or a token, so a secret never enters a committed file.
215
+
216
+ **Consent is ruled, and railed.** The committed opt-in **is** the standing consent, and committing a
217
+ config that says `enabled: true` is Gated and the maintainer's alone. `--export` refuses a config that is
218
+ **untracked**, **tracked but absent from `HEAD`** — staged and never committed — or **differs from
219
+ `HEAD`**, so editing a working copy cannot manufacture consent. It is person-invoked only: CI runs this
220
+ module on every pull request but never `--export`, and wiring that mode into a workflow, hook or schedule
221
+ is a new consent question rather than a covered one. It neither queues nor retries.
222
+
223
+ **What it does NOT establish:**
224
+
225
+ - **Current data.** The payload is checked against the snapshot, never the snapshot against the world
226
+ ([#356](https://github.com/sleepy-panda-srl/portulan/issues/356)). The capture stamp travels inside the
227
+ payload, so a stale export arrives labelled stale — and time-series backends commonly drop aged samples.
228
+ - **Who committed the consent.** The rail establishes *tracked and byte-identical to `HEAD`*, not
229
+ *committed by the maintainer*: `commit-to-a-working-branch` is `auto`, so an agent can commit it. What it
230
+ buys is that the act appears in a reviewed diff.
231
+ - **The opt-in in the affirmative.** No workspace in this tree opts in, so every green is about the off
232
+ path. The send is proven only in the suite, against an injected transport, and no yielded recipe may
233
+ ever exercise `--export`.
234
+ - **A network mode reached indirectly.** The suite derives the set of network-capable modules from the
235
+ tree, and the offline audit matches a module as a path suffix and a flag as a token; a mode reached
236
+ through another script, or through a flag built at runtime, is invisible to it.
237
+ - **Scheduled emission.** Nothing fires when nobody runs it —
238
+ [#344](https://github.com/sleepy-panda-srl/portulan/issues/344)'s silence again.
239
+ - **Enforcement of the review-loop bound.** An emitter with no backend reading it is not the checker
240
+ [`a-review-loop-needs-a-bound.md`](../.portulan/memory/a-review-loop-needs-a-bound.md) lacks.
241
+ - **Traces.** Metrics only.
242
+
243
+ ## The A/B baseline
244
+
245
+ | File | What it is |
246
+ |---|---|
247
+ | [`ab/corpus.md`](ab/corpus.md) | The scenarios and grading rules, the registered carrier of the clause's subject, and *What may not be concluded* |
248
+ | [`ab/arm.md`](ab/arm.md) | What *Portulan on* denotes here: the vendored-and-compiled treatment arm |
249
+ | [`ab/register.md`](ab/register.md) | Both arms as built, byte-compared by the `ab` recipe |
250
+ | [`ab/graders.md`](ab/graders.md) | The graders' discrimination, byte-compared by the `ab-grade` recipe |
251
+ | [`ab/baseline.json`](ab/baseline.json), [`ab/baseline.md`](ab/baseline.md) | The capture, and the register rendered from it — the one carrier of the figures |
252
+ | [`ab/warm.md`](ab/warm.md) | The warm-start A/B, a sibling of the baseline rather than a clause of row 8: it prices fresh sessions warm against cold from the host's own records, in the three lines every measurement reports, and says which host switch earns a place in `sessions`. Its runner, [`../cli/warm.mjs`](../cli/warm.mjs), is built, and its record waits on the maintainer's word, since every run starts real sessions |
253
+
254
+ [`../cli/ab.mjs`](../cli/ab.mjs) builds the arms, [`../cli/ab-grade.mjs`](../cli/ab-grade.mjs) grades the
255
+ trees they leave behind, and [`../cli/ab-run.mjs`](../cli/ab-run.mjs) runs the turns between: `k = 5` per
256
+ cell, ruled by the maintainer, every turn isolated with a home and config directory of its own. Each
257
+ module's `--write` regenerates its document; `ab-run --write` re-renders the register from the committed
258
+ capture. The recipes run no agent; the modes that do — `ab --stop-probe`, `ab-run --smoke` and
259
+ `--matrix` — are a person's to run.
260
+
261
+ **What may not be concluded** from the baseline is [`ab/corpus.md`](ab/corpus.md)'s section of that name,
262
+ and the register cites it rather than restating it. **The baseline is scoped to the vendored-and-compiled
263
+ tier** [`ab/arm.md`](ab/arm.md) specifies, and closes row 8 for no other configuration of *Portulan on*.
264
+
265
+ ## A release carries an eval result
596
266
 
597
267
  ```
598
- node cli/telemetry.mjs --config evals/telemetry/config.json --render
268
+ node cli/release-eval.mjs --capture at a cut: run every yielded recipe, write the record pair
269
+ node cli/release-eval.mjs --verify the recipe: every governed release has a record matching its capture
599
270
  ```
600
271
 
601
- ### Why the opt-in is a committed file and not `OTEL_SDK_DISABLED`
602
-
603
- This is the one place the tool departs from the OpenTelemetry environment contract, and it is stated
604
- out loud rather than left to be discovered. **`OTEL_SDK_DISABLED` defaults to `false`.** Honouring the
605
- standard environment surface as the *gate* would mean an adopter who already exports
606
- `OTEL_EXPORTER_OTLP_ENDPOINT` in CI — for some other service — starts emitting from Portulan without
607
- having decided to. That is an opt-**out**, and the criterion's word is *opt-in*. It would also put a
608
- team decision in ambient per-machine state, which is the inverse of `core < pack < workspace`.
609
-
610
- So the committed file is the only gate, and the environment supplies **transport only** — the OTLP
611
- endpoint and header variables in both their general and metrics-specific forms, so an adopter's
612
- existing collector configuration works unchanged and **a secret never enters a committed file**
613
- ([`../.portulan/dod.md`](../.portulan/dod.md) condition 5). The validator refuses a config carrying
614
- `headers`, `endpoint` or a token outright, so that is a rail rather than a review note.
615
-
616
- **What of that contract is implemented, at its real width.** Read:
617
- `OTEL_EXPORTER_OTLP_METRICS_ENDPOINT` — used **as given**, since the specification makes the
618
- signal-specific variable the full URL — falling back to `OTEL_EXPORTER_OTLP_ENDPOINT` as a base with
619
- `/v1/metrics` appended; and `OTEL_EXPORTER_OTLP_METRICS_HEADERS` **replacing**
620
- `OTEL_EXPORTER_OTLP_HEADERS` rather than merging. **Not** read, each a silent no-op for an adopter who
621
- sets it: `OTEL_EXPORTER_OTLP_PROTOCOL` (this emitter speaks `http/json` only), `_TIMEOUT`,
622
- `_COMPRESSION`, `_CERTIFICATE` and the client-auth pair. _This paragraph exists because the two carriers
623
- it replaces said the variables were read **"exactly as the specification defines them"** while the
624
- metrics-specific endpoint was ignored and appended to — which produced `/v1/metrics/v1/metrics` for
625
- anyone configured the standard way. A partial implementation of a public contract is fine; describing
626
- one as complete is the overclaim [`../.portulan/principles.md`](../.portulan/principles.md) opens with._
627
-
628
- ### The config was NOT made a Workspace Definition slot, and the cut is the session-open checkpoint's
629
-
630
- The first plan put it in the schema as a `telemetry` slot at spec 2.9. The checkpoint cut it, on
631
- grounds worth keeping: **every slot in that train arrived through a ruled proposal** —
632
- [`../spec/README.md`](../spec/README.md)'s 2.8 argument cites proposal `0025`, `governed_by` cites
633
- `0017`, `provenance` cites `0002` — and [`../.portulan/gate-map.md`](../.portulan/gate-map.md) makes an
634
- idea that adds a surface a **proposal**, *"never opened as an implementation pull request with tests."*
635
- [`../spec/slots.md`](../spec/slots.md)'s own `evals` deferral says a schema change plus a spec bump plus
636
- a migration *"is not a thing to do in passing"* and defers to **when milestone 8 closes**; and the slot
637
- would have shipped with **zero filled instances**, since this workspace declares telemetry off and
638
- [`../examples/`](../examples/) stays at 2.4 — which is
639
- [`../spec/slots.md`](../spec/slots.md)'s *"splitting on speculation is how a schema acquires slots
640
- nobody fills"* exactly. It is filed as [`0034`](../.portulan/proposals/0034-one-spec-bump-carries-both-evals-and-telemetry.md) — **accepted by the maintainer 2026-08-28** — to ride with
641
- `evals` at the close, which makes it one bump instead of two. **That close ran 2026-09-09, so the
642
- deferral is now due and still undrafted** — named by the close rather than performed inside it, since
643
- drafting a schema there is the *"splitting on speculation"* the same entry warns against. The acceptance
644
- commissions the drafting
645
- and settles none of the shape: the key names, either slot's structure, and whether a migration is owed
646
- are the drafted work, and the close remains the earliest sensible moment because `slots.md`'s deferral
647
- conditions the `evals` half on row 8 having decided what an eval artifact is here.
648
-
649
- ### Consent — ruled, and railed rather than promised
650
-
651
- [`../.portulan/gates.json`](../.portulan/gates.json) makes *sending anything outward on the team's
652
- behalf* Gated, and [`../core/operating/autonomy.md`](../core/operating/autonomy.md) says Gated is **per
653
- action**. Proposal
654
- [`0014`](../.portulan/proposals/0014-a-feedback-pipe-points-out-of-the-seam.md) reserved telemetry as
655
- *"a separate mechanism with separate consent"* — named there, ruled nowhere, until now.
656
-
657
- **His ruling: the committed opt-in config IS the standing consent.** Each export rides on it without a
658
- fresh per-send approval, the way a compiled gate carries a tier without asking again; what stays Gated
659
- and his alone is **committing a config that says `enabled: true`**. The ruling is transcribed into
660
- `gates.json`'s own rule and into the gate map in the same change, because a ruling living only in a
661
- session note while those said otherwise would be one rule with two disagreeing carriers, obeyed at the
662
- narrower.
663
-
664
- Three things keep it from widening into the thing the tier exists to stop, and each is a rail:
665
-
666
- - `--export` refuses in **three** states, and the third is the one that matters most: the config is
667
- **untracked**, **tracked but absent from `HEAD`** — staged and never committed — or **differs from
668
- `HEAD`**. So an agent cannot
669
- manufacture consent by editing a working copy. Without this the ruling would be a mandate nothing
670
- checks, arriving inside the change that states it.
671
- - **`--export` is person-invoked only.** No workflow, hook or schedule runs *that mode*, and wiring one
672
- in is a new consent question rather than a covered one. _The narrowing matters and the first draft got
673
- it wrong in the direction that misleads: CI runs `cli/telemetry.mjs` on **every** pull request, because
674
- the `telemetry` recipe is in the yielded set and [`../.github/workflows/verify.yml`](../.github/workflows/verify.yml)
675
- runs the set. What CI never runs is `--export`, and a sentence saying the module is person-invoked
676
- tells a reader auditing consent to stop looking._
677
- - It **neither queues nor retries**, on [`../cli/feedback.mjs`](../cli/feedback.mjs)'s rule that a queue
678
- flushing itself later is a silent send with extra steps.
679
- - **Nothing from the transport environment is logged.** Headers never; the endpoint only as origin and
680
- path, with userinfo and query stripped, and not at all when it failed to parse. An endpoint legally
681
- carries `user:pass@` and tokens ride in query parameters, so printing it in full leaked exactly what
682
- withholding the headers protected — into CI logs, which are long-lived and world-readable here.
683
-
684
- ### The payload is closed by construction, and was widened exactly once
685
-
686
- Emission is assembled from an explicit allow-list of aggregate figures; no producer is handed a
687
- snapshot to spread into a body, so there is no path from a free-text or path-shaped field to the wire.
688
- The ruled vocabulary was *figures, rule ids, recipe ids* — **no identity at all**, which would have made
689
- one workspace's export indistinguishable from another's at any collector. Put to him rather than
690
- widened by an implementer, on `cli/feedback.mjs`'s rule that a closed list is wrong only when somebody
691
- widens it **on purpose**: he widened it to **one named resource attribute**, the repository slug. A
692
- reviewer **login** stays excluded and is the field this deliberately is not — this snapshot's only login
693
- is a bot's, but an adopter's would carry human names.
694
-
695
- The attribute vocabulary is **pinned in both directions** by the suite, so an added key *and* a deleted
696
- carrier are each a red — against two lists rather than one, because they are two questions. The
697
- allow-list says what an emission *may* carry; a separate floor says what it *must*. One list checked
698
- both ways made `service.namespace` — optional in the config and optional in OpenTelemetry's own
699
- conventions — look mandatory, and was green only because this workspace happens to declare one. A pin
700
- that is correct by coincidence of a single instance is the shape a census over its author's own set
701
- takes, one clause over.
702
-
703
- ### What this rail does NOT establish
704
-
705
- - **It cannot tell a current snapshot from a stale one.** It checks the payload against the snapshot,
706
- never the snapshot against the world — [`review-loop.sh`](../.portulan/verify/review-loop.sh)'s cost
707
- inherited one step down, and [#356](https://github.com/sleepy-panda-srl/portulan/issues/356). What
708
- this layer adds is that the capture stamp travels **inside** the payload, so a stale export arrives
709
- labelled stale rather than looking fresh. A consequence to state rather than assert a threshold for:
710
- a stale export carries an aged `timeUnixNano`, and time-series backends commonly drop or reject aged
711
- samples — semantically correct and silently discarded is the worst pair.
712
- - **The consent rail establishes *tracked and byte-identical to `HEAD`* — not *committed by him*.**
713
- `commit-to-a-working-branch` is tier **auto**, so an agent commits unattended; a consent committed by a
714
- scripted `git commit` clears every gate this rail has. What the rail buys is real and is narrower than
715
- it first reads: it moves the act from a file edit nobody sees into **a commit that appears in a
716
- reviewed diff**, on the branch a human merges. It cannot tell whose commit it is, and nothing here
717
- claims it can. Measured at the pre-commit checkpoint rather than reasoned about.
718
- - **The offline audit's class is derived, and was not always.** The table claimed to rail the class of
719
- network-capable modes in `cli/` and listed **two of three** — `cli/feedback.mjs` files a GitHub issue
720
- through `gh issue create` and had been network-capable longer than either of the others. A set drawn
721
- by its author and reported as complete is the census shape session 4 named one clause over, committed
722
- at this table's birth and found at round 10 of the review. The suite now **derives** the set from the
723
- tree, so a new network-capable module reddens. What remains uncovered is narrower: a module reaching
724
- the network in a way the derivation does not see, and a mode reached **indirectly**. It
725
- matches a module as a path **suffix** and a flag as a **token**, so `./` and `--flag=value` spellings
726
- are covered — the first version knew one literal string, which made `./` a bypass — but a mode reached
727
- **indirectly**, through another script or a flag built at runtime, is invisible to it and no wider
728
- pattern fixes that.
729
- - **The opt-in is never demonstrated in the affirmative from a committed artifact.** No workspace in
730
- this tree declares telemetry on, so every green here is a green about the **off** path. The send is
731
- proven in the suite against an injected transport and a temporary repository with the consent really
732
- committed — a session-time observation, not a standing one, which is the state clause (d) exists to
733
- replace and does not reach here.
734
- - **`--export`'s network path is covered only by unit tests against a fake**, and by construction no
735
- yielded recipe may ever exercise it — this recipe's own `offline` check forbids it. That is a
736
- deliberate consequence rather than an oversight, and it is the mode that matters.
737
- - **Emission is not scheduled**, and nothing fires when nobody runs it —
738
- [#344](https://github.com/sleepy-panda-srl/portulan/issues/344)'s silence problem in a third place.
739
- - **The review-loop bound is still checked by nothing.** The 2026-07-28 amendment says *"the telemetry
740
- clause is where that checker's home is"* for
741
- [`a-review-loop-needs-a-bound.md`](../.portulan/memory/a-review-loop-needs-a-bound.md). An emitter
742
- with no backend reading it does not discharge that, and this clause must not be read as enforcement.
743
- - **Metrics only.** OTLP's trace payload is a second shape with a second set of semantics; one signal
744
- demonstrated beats two asserted.
745
-
746
- ## A release carries an eval result — the mechanism, and the instance it is still owed
747
-
748
- Row 8's **ninth** clause, added by the maintainer's ruling of 2026-08-24. The mandate is older than the
749
- clause: [`../docs/plan.md`](../docs/plan.md)'s Protocol → Versioning has said *"from milestone 8,
750
- releases carry an eval result"* since the plan was locked, and it belonged to **no row** until that day —
751
- so a close re-deriving row 8's criterion clause by clause would have re-derived eight and left it
752
- unbuilt. [`releases/`](releases/) is where a release's record lives, one pair per version:
753
- `<version>.json` is the capture and `<version>.md` the register rendered from it and byte-compared.
754
-
755
- ### The three things the amendment left open, and what was ruled here
756
-
757
- The amendment named what it did **not** settle — *"what an eval result attached to a release consists of,
758
- where it is carried (the changelog entry, the release body, a file in the payload), and whether a release
759
- with no eval result is refused by a rail or by a person. **Those are this row's remaining work, not this
760
- amendment's content.**"* The last sentence is quoted rather than trimmed because it is the half that makes
761
- the passage a **scoping limit** — without it the list reads as three open questions rather than as three
762
- questions assigned to this row — and a first draft of this section dropped it.
763
-
764
- **What it consists of.** The verdict every recipe the workspace **yields** returned, measured by running
765
- them at a named commit, plus the **identity** of the A/B baseline the release ships against — path,
766
- capture date, commit, cleanliness. Never that baseline's figures: those have one carrier and a second
767
- copy of them inside a per-release record is the hand-maintained figure this milestone deleted six of.
768
- There is no *eval* subset of the recipe set — [`../cli/recipe-set.mjs`](../cli/recipe-set.mjs) has no
769
- such category — so an "eval recipes only" record would have required a hand-listed roster, which is the
770
- defect the record exists to avoid.
771
-
772
- **Where it is carried.** One file per version, cited and never restated — **and it ships**. `evals/releases/` is in `package.json`'s `files`, the maintainer's decision of 2026-09-01 on an option the amendment had left open and the implementer had closed the other way. **A measurement settled it:** a record written into the *unpacked tarball* is graded `exit 0` by `cli/release-eval.mjs --tagged`, which already ships — so the artifact carries both the claim and the tool that checks it, and the objection that an npm consumer can run no recipe was true and beside the point. [`../docs/milestones/m08.md`](../docs/milestones/m08.md) carries the adjudication, the one citation that does not survive packing, and why absolute URLs were refused.
773
-
774
- **Rail or person — both, split by what each can reach.** The `release-eval` recipe grades the tree on
775
- every commit. [`../.github/workflows/publish-github-packages.yml`](../.github/workflows/publish-github-packages.yml)
776
- runs `release-eval --tagged` against the tag's own checkout, before publishing — the one mechanical
777
- surface at the release act. The maintainer owns the rest, and
778
- [`../.portulan/gate-map.md`](../.portulan/gate-map.md) carries that half beside the tier it belongs to.
779
-
780
- ### Two designs a fresh context reversed before a line was written
781
-
782
- **Cut detection off `CHANGELOG.md`'s top heading never fires.** The cut commit re-seeds `## Unreleased`
783
- *above* the version it just wrote — `b410c020` — because that file's own header requires the tagged tree
784
- to contain its own entry. The top heading is `Unreleased` on every commit including the cut, so a rail
785
- keyed to it would have reported *no cut in this tree* on precisely the tree it was built to grade.
786
-
787
- **A rail keyed to `package.json`'s version alone grades one record at a time.** The moment `0.1.4` is
788
- declared, `0.1.3`'s record can be deleted and nothing is looking at it. The subject is therefore **every
789
- version the changelog records**, from `0.1.3` onward, permanently — and the set is graded in both
790
- directions, so a record for a release that was never cut reds as well.
272
+ [`releases/`](releases/) holds one pair per version, described in
273
+ [`releases/README.md`](releases/README.md), for every version [`../CHANGELOG.md`](../CHANGELOG.md)
274
+ records from `0.1.3` onward — graded permanently and in both directions. A record is the verdict every
275
+ recipe the workspace yields returned at a named commit, plus the **identity** of the A/B baseline the
276
+ release ships against, never its figures.
277
+ **`evals/releases/` ships in the package**, by the maintainer's decision of 2026-09-01, so the package
278
+ carries both the record and the tool that checks it: `--tagged` grades a record in the unpacked package as
279
+ it does in the tagged tree. The recipe grades the tree on every commit;
280
+ [`../.github/workflows/publish-github-packages.yml`](../.github/workflows/publish-github-packages.yml) runs
281
+ `--tagged` against the tag's own checkout before publishing; the rest is Gated, and
282
+ [`../.portulan/gate-map.md`](../.portulan/gate-map.md) carries it beside its tier.
283
+
284
+ **What it does NOT establish.** Every register prints its own limits: a green means the record agrees with
285
+ its capture, never that the release is good; the recipes ran at a commit inside the cut change, not at the
286
+ tag; `release-eval`'s own row is excluded. Beyond the register lies the **published** half: `--tagged`
287
+ reaches the tagged tree at the publish, and the release body, written outside the tree, is reached by no
288
+ check. While every governed release has its record, the clause's central arm — a governed release with
289
+ none — is exercised only by the forced-red drill that moves the boundary.
290
+
291
+ **`v0.1.3` is the first release published with its eval result.** It was tagged on 2026-09-30 at the
292
+ commit that cut it, the publish workflow ran `--tagged` against that tree before publishing it, and its
293
+ release body cites [`releases/0.1.3.md`](releases/0.1.3.md) without restating it. That the body cites the
294
+ register is the maintainer's act, and nothing here checks it.
295
+
296
+ ## A rule change decided on eval evidence
297
+
298
+ Row 8's clause *a rule change merges or is rejected on eval evidence* is discharged by one instance, hole 8
299
+ of [`../.portulan/gate-map.md`](../.portulan/gate-map.md): a `gated` or `prohibited` path target that can
300
+ never match. [#337](https://github.com/sleepy-panda-srl/portulan/issues/337) set out three answers, the
301
+ instruments chose between them, and the change merged on 2026-09-09; the evidence table and the argument
302
+ are [`../docs/milestones/m08.md`](https://github.com/sleepy-panda-srl/portulan/blob/8a33f9b/docs/milestones/m08.md)'s session-10 note. The rail is `neverMatches`
303
+ and the backend refusal beside `HOST_GATE_TIERS` in [`../cli/compile.mjs`](../cli/compile.mjs), a suite
304
+ block, and a second forced-red drill on `doctor`; its limit — the refusal is at compile time, not at the
305
+ hook — is stated beside the refusal. The hazard is held there and not in the gate corpus, whose
306
+ denominator is the yielded policy: adding a never-matching `gated` rule to `gates.json` to give the corpus
307
+ a case would manufacture the defect.
791
308
 
792
- ### What this rail does NOT establish
309
+ ## What is NOT built yet
793
310
 
794
- - **Nothing about whether a release is good.** Each recipe's green establishes what that recipe's own
795
- documentation says and no more; this checks that the record exists, agrees with its capture, and
796
- records no red.
797
- - **The recipes did not run at the tag.** A record is committed *in* the change that cuts a release, so
798
- its `source.commit` names a commit **inside that change** rather than the tag — a capture cannot name a
799
- commit that does not exist yet. How much of the cut precedes it is a property of how the change is
800
- sequenced, and `--capture` refuses a version below the first governed one, so a clean capture can never
801
- run before the version bump. The register prints the commit it was taken at and says this, the way the
802
- A/B capture prints `clean`.
803
- - **`release-eval`'s own row is excluded from every capture**, because a capture cannot be accurate about
804
- the record it lives inside. The exclusion is printed with its reason rather than the row being dropped.
805
- - **`0.1.3` carries one, so the rail grades a non-empty set.** Before that cut its green read *no release
806
- from `0.1.3` onward has been cut yet*, which was a **state, not a measurement**; the module said so on
807
- its own face rather than letting exit 0 imply otherwise, and it prints whichever of the two holds. The
808
- clause's central arm — *a governed release with no record* — is still unreachable from the committed
809
- tree, now because the one governed release **has** its record rather than because none exists, so it
810
- remains exercised by the forced-red drill that moves the boundary; it fired on 2026-09-01. A drill is
811
- not a release, and a record in the tree is not a published one.
812
-
813
- ## A rule change decided on eval evidence — the ninth clause
814
-
815
- **The clause is instance-shaped, and this is the instance.** Row 8 asks that *a rule change merges or
816
- is rejected on eval evidence*, and until 2026-09-09 the honest sentence in this file was that **every
817
- rule in [`../.portulan/memory/`](../.portulan/memory/) had been merged on review alone**. _(The evidence
818
- was taken on 2026-09-03 and the change merged on 2026-09-09; the clause's verb is **merges**, so the
819
- later date is the one that discharges it. Three sentences in this file dated it to the session rather
820
- than to the merge until the milestone-8 close re-derived it from `gh pr view 408` — created
821
- `2026-09-03T18:57:23Z`, merged `2026-09-09T07:48:12Z`.)_ What changed
822
- is not that a rule was minted — rules are minted often — but that a **decision between three drafted
823
- answers was made by running the instruments** rather than by argument.
824
-
825
- **The rule change.** Hole 8 of [`../.portulan/gate-map.md`](../.portulan/gate-map.md): a `gated` or
826
- `prohibited` gate rule whose path target can never match compiles to a named permission surface while
827
- the runtime matcher answers false for every input — a gate reported as covered that enforces nothing.
828
- [#337](https://github.com/sleepy-panda-srl/portulan/issues/337) set out three defensible answers and
829
- presumed none, precisely so the choice could be made on evidence.
830
-
831
- **The evidence, measured on this tree.** Each candidate applied as a scratch mutation — `cli/mutants.mjs`'
832
- own technique — and every instrument run and read:
833
-
834
- | candidate | `compile` | `goldens` | `mutants` | `doctor` | `tests` |
835
- |---|---|---|---|---|---|
836
- | baseline (`main`) | 0 | 0 | 0 | 0 | 0 |
837
- | **1** — refuse it at `parse`, every tier | **2** | **2** | **2** | **1** | **1** |
838
- | **2** — make `./` match the tree | 0 | **1** | **2** | 0 | **1** |
839
- | **3** — refuse it in the backend that enforces | 0 | 0 | 0 | 0 | 0 |
840
- | **the hazard itself**, on a clean `main` clone — `edit-on-a-working-branch` flipped to `gated` | 1 | 0 | 0 | **0** | 1 |
841
- | **the hazard itself**, on this tree | **2** | 0 | 0 | **1** | 1 |
842
-
843
- **What the table decided, and what it did not.** Option 1 is **rejected on the measurement**: this
844
- workspace's own policy carries two `auto` rules written `write: "./"` and `read: "./"`, so refusing the
845
- shape at `parse` takes their spelling — three instruments could not run and two went red, and
846
- *could-not-run* is the worse of the two outcomes because it reports nothing about the tree. That is
847
- #337's own prediction re-derived rather than quoted. Option 2's reds are **not** a rejection and must not
848
- be read as one: they are the record-keeping rails firing exactly as designed — `goldens` reds whenever the
849
- documented hole moves, *including when it closes*, and `cli/mutants.mjs` already records that same edit
850
- as the operator `matchesPath-admits-the-empty-target`, `killed`, with the note *"CLOSES hole 8 … the
851
- good-news direction the corpus exists to catch"*. Its other two reds are that one fact twice more rather
852
- than three independent objections: `mutants` exits 2 because the census refuses to run over a reddened
853
- corpus, and the eight suite failures are the cases that **pin** that census and that corpus. Option 2 is
854
- declined on **authority**: it decides what
855
- `./` should MEAN as a policy target, which is the question #337 reserves. Option 3 keeps every
856
- instrument green and closes the hazard, and it is the one taken.
857
-
858
- **The last two rows are what decide the merge half.** On the first four rows alone, option 3 and *doing
859
- nothing* are indistinguishable — both are all-green — so the choice would rest on argument. They grade
860
- the **defect** instead, by flipping this repository's own `edit-on-a-working-branch` from `auto` to
861
- `gated`. **Every cell in this table is a verify recipe's exit**, so `compile` there is `compile --check`;
862
- the clean-clone row is measured on a detached checkout of `main`, not on this tree with a file swapped,
863
- because a hybrid is neither tree.
864
-
865
- **The discriminating cell is `doctor`, and it is the whole argument.** On `main`, `doctor` exits **0**
866
- over that policy — it counts the hollow gate among *11 of 25 rule(s) compiled* and names five gates no
867
- backend compiles, none of them this one, so the tree reports the rule as covered and nothing contradicts
868
- it. On this tree it exits **1**, printing the refusal. In write mode the same policy at `main` also
869
- *emits* the rule, surface `Edit(./) · hook: a Bash command writing ./`; the recipe's **1** is the drift
870
- that emission causes against the committed artifact, which is a different fact and is why the cell is
871
- labelled by its instrument.
872
-
873
- **`tests` is 1 on both sides and discriminates nothing**, which is stated rather than hidden behind a
874
- prettier cell: on `main` one case fails, *"every rule is cited under the gate map section matching its
875
- TIER"*, because the prose still files that rule under Auto; on this tree sixteen do — **eight**
876
- because the new drill's anchor searches for the `"tier": "auto"` line the fixture rewrote, **seven**
877
- because the refusal fires, and one that fails on `main` too. Both are
878
- artefacts of perturbing a policy the suite pins, on either tree, and neither says anything about the
879
- change.
880
-
881
- **So the clause discharges as *merges on eval evidence*, with option 1 a measured-cost rejection** — and
882
- the second half of *"merges or is rejected"* is not overclaimed beyond that.
883
-
884
- **What the rail is.** The predicate `neverMatches` in [`../cli/compile.mjs`](../cli/compile.mjs), the
885
- backend refusal beside `HOST_GATE_TIERS`, a suite block that goes red in six cases when the refusal is
886
- disabled, and a second forced-red drill on the `doctor` rail — `doctor` rather than `compile`, because a
887
- `CompileError` exits 2 and `cli/drills.mjs`'s coverage check refuses a drill declaring `exit: 2`, on the
888
- ground that it *"would read a refusal as a verdict"*.
889
-
890
- **Its limits, in the shape this directory asks for.** The refusal is at **compile** time, not at the
891
- hook: `cli/gate.mjs` reads the policy through `parse`, which this change deliberately does not touch, so
892
- a workspace that has already committed such a rule still loads it at run time and the hook steps aside
893
- because nothing matched. And the hazard cannot live in the gate corpus itself — `cli/goldens.mjs`
894
- derives its denominator from the **yielded policy**, and adding a `gated` `./` rule to `gates.json` to
895
- satisfy a corpus would be manufacturing the defect. The suite block and the drill are its home.
311
+ Each names where it arrives, per [`../.portulan/dod.md`](../.portulan/dod.md) condition 4 — nothing here
312
+ claims a capability that does not exist:
896
313
 
897
- ## What is NOT built yet
314
+ - **The `evals` and `telemetry` Workspace Definition slots.** Proposal
315
+ [`0034`](../.portulan/proposals/0034-one-spec-bump-carries-both-evals-and-telemetry.md), accepted
316
+ 2026-08-28, carries both in one spec bump. The deferral in [`../spec/slots.md`](../spec/slots.md) is due
317
+ since the milestone-8 close and undrafted; the drafting settles the key names, each slot's structure and
318
+ whether a migration is owed.
898
319
 
899
- Each names where it arrives, per [`../.portulan/dod.md`](../.portulan/dod.md) condition 4 — nothing
900
- here claims a capability that does not exist:
901
-
902
- - **A release actually *published* with its eval result.** The mechanism is built — its section is above —
903
- and since the `0.1.3` cut so is the instance's in-tree half: `evals/releases/0.1.3.{json,md}`. What is
904
- still not built is the half no tree can hold. **Ruled 2026-09-01: it waits for a real cut**, and the
905
- release is done once milestone 8's other work is finished — so this clause was never closable on the
906
- mechanism, and the cut is the milestone's last act rather than its first sequel. `tag-a-release` and
907
- `publish-a-release` are Gated, so the clause stays undemonstrated until the maintainer takes them and
908
- the release body cites the register. The argument that was put to him, and what a cut mechanically is, are in
909
- [`../docs/milestones/m08.md`](../docs/milestones/m08.md): a clause about releases, held satisfied while
910
- zero governed releases exist, is true the way *a baseline names its host* was true of an empty string.
320
+ Each rail's open limits are in its section above, with the issue tracking each where there is one.