paperlint 10.0.0 → 10.1.0

package/skills/run-agent-grid/SKILL.md

@@ -0,0 +1,181 @@
+---
+name: run-agent-grid
+description: "Run a paid or counted agent experiment grid end to end — units × arms × seeds (× agent stacks), one agent CLI run per cell in a fresh working copy, scored by an outcome oracle — without the failures that silently spoil one: an unapproved or hand-picked sample, a timeout that kills a slower model, two drivers sharing a working copy, secrets in transcripts, a classifier that mislabels outcomes, a session cap that kills the run, results lost from scratch space. Use when a study design exists and you are about to spend money or count runs: the registry and approval gate, the sandbox and secret probe, the smoke per model, the locked detached driver, the overlap audit, hand labels and the commit of finished runs. Ships grid-guard.js (lock, gate, overlap audit, env probe, key scan, pre-commit). Composes with build-benchmark (the design, upstream), build-label-page (hand labels) and osf-artifact-upload (the artifact)."
+allowed-tools: [Read, Write, Edit, Grep, Glob, Bash, Agent]
+---
+
+<!-- vigiles:sha256:aef4152e680a2ccd compiled from skills/run-agent-grid/SKILL.md.spec.ts -->
+
+# run-agent-grid — running a counted agent grid without spoiling it
+
+**What a grid is here.** Units under test × conditions (arms) × seeds, repeated per agent stack
+(model + CLI). Each cell is one agent CLI run in a fresh working copy of the unit; a scorer reads
+what the run left behind, and an outcome oracle decides the cell's result. `build-benchmark` designs
+the study and the artifact; this skill is the **operational procedure for running it**, from the
+first paid call to the last committed run.
+
+Every rule below was paid for by a failure. Each is stated with that failure and with what HOLDS it:
+a refusal in `skills/run-agent-grid/scripts/grid-guard.ts` (run as the compiled `scripts/grid-guard.js`), a commit hook, a probe — or, said plainly, **prose only**,
+which means it holds only as long as someone reads this. The full catalog, one entry per failure:
+[`references/failure-catalog.md`](references/failure-catalog.md). The registry, the attempt ledger
+and a driver loop that calls the guard: [`references/registry-and-ledger.md`](references/registry-and-ledger.md).
+
+**Exit codes of every guard command:** 0 allowed or clean · 1 refused or a finding · 2 usage ·
+3 (`gate` only) the cell is already counted, skip it.
+
+## The order — each gate is passed before the next phase starts
+
+### 1. The registry: what may be measured, written as data (catalog 1, 3, 14)
+
+- Every measurement is a row in a **machine-readable registry** (`grid.json`): each unit with its
+  role (`counted`, `smoke`, `excluded`), the author's requirement on it **verbatim**, the arms, the
+  seeds, the models, the budget cap. A requirement that lives only in a chat or a design note is not
+  a requirement: a driver cannot read it, and a whole grid can run on one easy, hand-picked unit.
+- **The frame is the whole population** of the measured kind. Never "where our instrument can
+  fire": that filter shrinks the frame to trivial units. Allowed exclusions are listed per unit, each
+  with a reason, and counted. The draw is seeded, its seed and script committed **before anyone looks
+  at the drawn units**; a stratified draw is proportional to the frame.
+- **Only the committed registry allows a run.** An added model, seed or arm is a dated addendum,
+  committed before its first run. Report models as columns, never pooled.
+- **Held by:** `gate` reads the registry from `HEAD` and refuses when the working copy differs from
+  it, when the cell is not declared, when an excluded unit has no reason, when a counted run has no
+  draw seed. **Prose only:** that the frame is the whole population, and that the draw is
+  proportional; the guard sees the registry, not the population behind it.
+
+### 2. The author's approval of the drawn units (catalog 2)
+
+- The **author** approves the drawn units — in their own words, with the time and the channel
+  (chat, message, review comment) recorded in `approval`. The agent that designed or drew the sample
+  never fills this field, not even to "save a step".
+- **Held by:** `gate` refuses a counted run unless `approval.by` equals the registry's `author` and
+  `at` and `channel` are filled. **Prose only:** who typed it — the guard cannot tell the author
+  from an agent writing the author's name. Make it reviewable: the approval lands in its own commit,
+  quoting the author's message.
+
+### 3. The sandbox: secrets out, tools in, per unit (catalog 8, 9, 10)
+
+- The agent's environment **excludes secrets by name pattern**, and a names-only probe passes before
+  any model call. Transcripts capture environment dumps; a secret in the environment ends up in a
+  committed transcript.
+- The same probe proves that **what the agent needs still works**: an exclude-by-pattern policy also
+  strips variables git needs (`GIT_CONFIG_KEY_n` matches "KEY"). Prove git commits inside the sandbox.
+- **Per unit, with no model call:** the tools the agent is meant to use are visible inside the
+  sandbox, the instrument fires on the pinned state of the unit, and it accepts the reference fix.
+  A shared tools directory that is invisible inside the sandbox turns every run of an arm into a
+  run without the tool.
+- **Held by:** run inside the sandbox, as the agent would run —
+
+  ```bash
+  node ${CLAUDE_SKILL_DIR}/scripts/grid-guard.js env-probe --git
+  ```
+
+  It prints secret-looking variable NAMES (never values) and exits 1; `--allow NAME` accepts one by
+  name after you have looked at it. `gate` refuses a counted unit without `proof` (`toolVisible`,
+  `firesOnPinned`, `acceptsReference`, all `true`). **Prose only:** how the proof is taken — the
+  three facts are recorded by your proof script; the guard only refuses their absence.
+
+### 4. The smoke, per agent stack (catalog 5, 13)
+
+- Before each model's grid: **one run per arm**, each **read end to end** — the transcript, the diff,
+  the scorer's output — not only its exit code. A timeout fit for one model kills a slower one
+  mid-task, and a killed run reads as a failure of the arm.
+- From the smoke, write into the registry, per model, `smoke.timeoutSec` and `smoke.costPerRun`
+  (measured, with reserve), and commit it **before the grid**. A model can cost a multiple of its
+  planned price; the cap and the stop rule are written down before the first counted run.
+- **Held by:** `gate` refuses a counted run on a model without `smoke.timeoutSec` and
+  `smoke.costPerRun`, hands the driver `timeoutSec` in its output, and refuses when the spend in the
+  ledger plus the next run's cost would pass `budget.cap`. **Prose only:** that the smoke runs were
+  actually read.
+
+### 5. The run: locked, detached, resumable (catalog 6, 7, 12, 13)
+
+- **One lock per working-copy root.** Two drivers on one scratch path delete each other's trees;
+  one crashes, the other's runs are silently contaminated. A second concurrent driver gets **its own
+  root**.
+
+  ```bash
+  node ${CLAUDE_SKILL_DIR}/scripts/grid-guard.js lock <root> --owner <driver-id> --pid $$
+  node ${CLAUDE_SKILL_DIR}/scripts/grid-guard.js release <root> --owner <driver-id>
+  ```
+
+  The lock is a file beside the root. A live holder → refused. A holder whose process is gone on
+  the same host → taken over, and said so. A holder on another host → refused (it cannot be judged).
+  The holder must be detached — see the next point.
+- **Before every cell, ask the gate** and obey its exit code; 3 means the cell is already counted,
+  or reserved by another attempt — skip it. That makes a killed driver resumable: run it again and
+  it continues.
+
+  ```bash
+  node ${CLAUDE_SKILL_DIR}/scripts/grid-guard.js gate --registry grid.json --ledger attempts.jsonl --unit <u> --arm <a> --model <m> --seed <s> --attempt <id>
+  ```
+
+  An allowed cell is **reserved** for `<id>` in the ledger in the same step, under the ledger's
+  lock: two drivers asking at once cannot both get one cell, nor both fit under the cap, because a
+  reservation holds the cell and counts its estimated cost until the attempt's end record (same
+  id) replaces it, or until it lapses (`--reserve-sec`, default six hours).
+
+- **Never delete shared inputs from a step.** Templates, fixtures and pinned states are read-only
+  (`chmod -R a-w` them); a packing step that deletes what it packed removes what another driver
+  still reads. **Prose only**, held by file permissions where you set them.
+- **Run the driver detached** (`setsid nohup … &`, its log to a file), never as a task tracked by
+  the agent session: a session's time cap kills a long grid. A separate waiter (a scheduled wake-up,
+  a loop that polls the ledger) brings the session back. **Held by `lock`:** it refuses a holder
+  that is not the leader of its own session (Linux `/proc/<pid>/stat`; true after `setsid`) and
+  names the command to use instead; `--attached` allows a short run, said out loud; without
+  `/proc` it warns and passes.
+- The driver appends a ledger line when an attempt starts and another when it ends (same `attempt`
+  id, `root`, `start`/`end`, `status`, `cost`). Format: `references/registry-and-ledger.md`.
+
+### 6. After any incident — and before reporting — the overlap audit (catalog 6)
+
+```bash
+node ${CLAUDE_SKILL_DIR}/scripts/grid-guard.js overlap attempts.jsonl
+```
+
+Lists every pair of attempts that shared a root at the same time (an attempt with no end overlaps
+everything after its start), where at least one is counted. Each contaminated attempt is marked
+`"status": "harness-error"` — the record is **kept**, not deleted, and not counted — and its cell
+runs again as a **new** attempt. Run it after any crash, kill or harness change, and before the
+numbers are reported.
+
+### 7. Scoring and labels (catalog 4, 11)
+
+- **A check reads every stratum's expectation file.** When a new stratum's expected values live in a
+  second file, a proof or precondition check that reads only the first reports false FAILs. Load
+  expectations through one function that enumerates them, and fail when a unit's stratum has none.
+  A precondition failure is investigated before it is trusted — either way. **Prose only.**
+- **Every run whose outcome is not mechanically certain is read in full and labelled by hand.** An
+  automatic outcome classifier misses classes (a deferral phrased without a question mark); its
+  label is a pre-sort, and the disagreements between it and the hand labels are reported. The
+  procedure, with a label-file validator and an agreement report: `label-agent-runs`.
+
+### 8. Commit finished runs, often (catalog 15, 8)
+
+- Results in scratch space vanish. Commit **finished** runs — those with a completion record —
+  frequently; never a half-written run folder.
+- Every committed artifact is scanned for key-shaped strings, by a pre-commit hook:
+
+  ```bash
+  node ${CLAUDE_SKILL_DIR}/scripts/grid-guard.js precommit --runs runs --done done.json
+  ```
+
+  It reads the **staged** content (the index is what commits), refuses key-shaped strings by file and
+  line without printing them, and refuses a staged run folder that has no completion record. Before
+  an artifact leaves the repository: `node ${CLAUDE_SKILL_DIR}/scripts/grid-guard.js scan-keys <dir>`.
+- **Report the spend**: total, per model, against the cap.
+
+## When something goes wrong mid-grid
+
+Stop the driver; do not fix forward on a running grid. Write the incident down (what, when, which
+attempts), run the overlap audit, mark contaminated attempts, fix the harness, re-run the smoke for
+any stack the fix touched, then resume. A harness change is an addendum like any other.
+
+## Compose with
+
+- **build-benchmark** (upstream) — the design, the statistics and the artifact this run feeds.
+- **label-agent-runs** (downstream) — the outcome labels: codebook first, hand labels for every
+  uncertain run, a blind second rater, labels frozen after the analysis.
+- **build-label-page** — the page a person labels those runs on.
+- **osf-artifact-upload** — the artifact, after `scan-keys` is clean.
+- An **independent refutation pass** (a second agent framed to refute, not to confirm) on the
+  registry and the smoke before the first counted run.

package/skills/run-agent-grid/SKILL.md.spec.ts

@@ -0,0 +1,184 @@
+import { experimental_skill, file, prose } from "vigiles/spec";
+import { COMMANDS } from "./scripts/grid-guard-commands.ts";
+
+export default experimental_skill({
+  name: "run-agent-grid",
+  description:
+    "Run a paid or counted agent experiment grid end to end — units × arms × seeds (× agent stacks), one agent CLI run per cell in a fresh working copy, scored by an outcome oracle — without the failures that silently spoil one: an unapproved or hand-picked sample, a timeout that kills a slower model, two drivers sharing a working copy, secrets in transcripts, a classifier that mislabels outcomes, a session cap that kills the run, results lost from scratch space. Use when a study design exists and you are about to spend money or count runs: the registry and approval gate, the sandbox and secret probe, the smoke per model, the locked detached driver, the overlap audit, hand labels and the commit of finished runs. Ships grid-guard.js (lock, gate, overlap audit, env probe, key scan, pre-commit). Composes with build-benchmark (the design, upstream), build-label-page (hand labels) and osf-artifact-upload (the artifact).",
+  tools: ["Read", "Write", "Edit", "Grep", "Glob", "Bash", "Agent"],
+  body: prose`
+# run-agent-grid — running a counted agent grid without spoiling it
+
+**What a grid is here.** Units under test × conditions (arms) × seeds, repeated per agent stack
+(model + CLI). Each cell is one agent CLI run in a fresh working copy of the unit; a scorer reads
+what the run left behind, and an outcome oracle decides the cell's result. \`build-benchmark\` designs
+the study and the artifact; this skill is the **operational procedure for running it**, from the
+first paid call to the last committed run.
+
+Every rule below was paid for by a failure. Each is stated with that failure and with what HOLDS it:
+a refusal in ${file("skills/run-agent-grid/scripts/grid-guard.ts")} (run as the compiled \`scripts/grid-guard.js\`), a commit hook, a probe — or, said plainly, **prose only**,
+which means it holds only as long as someone reads this. The full catalog, one entry per failure:
+[\`references/failure-catalog.md\`](references/failure-catalog.md). The registry, the attempt ledger
+and a driver loop that calls the guard: [\`references/registry-and-ledger.md\`](references/registry-and-ledger.md).
+
+**Exit codes of every guard command:** 0 allowed or clean · 1 refused or a finding · 2 usage ·
+3 (\`${COMMANDS.gate}\` only) the cell is already counted, skip it.
+
+## The order — each gate is passed before the next phase starts
+
+### 1. The registry: what may be measured, written as data (catalog 1, 3, 14)
+
+- Every measurement is a row in a **machine-readable registry** (\`grid.json\`): each unit with its
+  role (\`counted\`, \`smoke\`, \`excluded\`), the author's requirement on it **verbatim**, the arms, the
+  seeds, the models, the budget cap. A requirement that lives only in a chat or a design note is not
+  a requirement: a driver cannot read it, and a whole grid can run on one easy, hand-picked unit.
+- **The frame is the whole population** of the measured kind. Never "where our instrument can
+  fire": that filter shrinks the frame to trivial units. Allowed exclusions are listed per unit, each
+  with a reason, and counted. The draw is seeded, its seed and script committed **before anyone looks
+  at the drawn units**; a stratified draw is proportional to the frame.
+- **Only the committed registry allows a run.** An added model, seed or arm is a dated addendum,
+  committed before its first run. Report models as columns, never pooled.
+- **Held by:** \`${COMMANDS.gate}\` reads the registry from \`HEAD\` and refuses when the working copy differs from
+  it, when the cell is not declared, when an excluded unit has no reason, when a counted run has no
+  draw seed. **Prose only:** that the frame is the whole population, and that the draw is
+  proportional; the guard sees the registry, not the population behind it.
+
+### 2. The author's approval of the drawn units (catalog 2)
+
+- The **author** approves the drawn units — in their own words, with the time and the channel
+  (chat, message, review comment) recorded in \`approval\`. The agent that designed or drew the sample
+  never fills this field, not even to "save a step".
+- **Held by:** \`${COMMANDS.gate}\` refuses a counted run unless \`approval.by\` equals the registry's \`author\` and
+  \`at\` and \`channel\` are filled. **Prose only:** who typed it — the guard cannot tell the author
+  from an agent writing the author's name. Make it reviewable: the approval lands in its own commit,
+  quoting the author's message.
+
+### 3. The sandbox: secrets out, tools in, per unit (catalog 8, 9, 10)
+
+- The agent's environment **excludes secrets by name pattern**, and a names-only probe passes before
+  any model call. Transcripts capture environment dumps; a secret in the environment ends up in a
+  committed transcript.
+- The same probe proves that **what the agent needs still works**: an exclude-by-pattern policy also
+  strips variables git needs (\`GIT_CONFIG_KEY_n\` matches "KEY"). Prove git commits inside the sandbox.
+- **Per unit, with no model call:** the tools the agent is meant to use are visible inside the
+  sandbox, the instrument fires on the pinned state of the unit, and it accepts the reference fix.
+  A shared tools directory that is invisible inside the sandbox turns every run of an arm into a
+  run without the tool.
+- **Held by:** run inside the sandbox, as the agent would run —
+
+  \`\`\`bash
+  node \${CLAUDE_SKILL_DIR}/scripts/grid-guard.js ${COMMANDS.envProbe} --git
+  \`\`\`
+
+  It prints secret-looking variable NAMES (never values) and exits 1; \`--allow NAME\` accepts one by
+  name after you have looked at it. \`${COMMANDS.gate}\` refuses a counted unit without \`proof\` (\`toolVisible\`,
+  \`firesOnPinned\`, \`acceptsReference\`, all \`true\`). **Prose only:** how the proof is taken — the
+  three facts are recorded by your proof script; the guard only refuses their absence.
+
+### 4. The smoke, per agent stack (catalog 5, 13)
+
+- Before each model's grid: **one run per arm**, each **read end to end** — the transcript, the diff,
+  the scorer's output — not only its exit code. A timeout fit for one model kills a slower one
+  mid-task, and a killed run reads as a failure of the arm.
+- From the smoke, write into the registry, per model, \`smoke.timeoutSec\` and \`smoke.costPerRun\`
+  (measured, with reserve), and commit it **before the grid**. A model can cost a multiple of its
+  planned price; the cap and the stop rule are written down before the first counted run.
+- **Held by:** \`${COMMANDS.gate}\` refuses a counted run on a model without \`smoke.timeoutSec\` and
+  \`smoke.costPerRun\`, hands the driver \`timeoutSec\` in its output, and refuses when the spend in the
+  ledger plus the next run's cost would pass \`budget.cap\`. **Prose only:** that the smoke runs were
+  actually read.
+
+### 5. The run: locked, detached, resumable (catalog 6, 7, 12, 13)
+
+- **One lock per working-copy root.** Two drivers on one scratch path delete each other's trees;
+  one crashes, the other's runs are silently contaminated. A second concurrent driver gets **its own
+  root**.
+
+  \`\`\`bash
+  node \${CLAUDE_SKILL_DIR}/scripts/grid-guard.js ${COMMANDS.lock} <root> --owner <driver-id> --pid $$
+  node \${CLAUDE_SKILL_DIR}/scripts/grid-guard.js ${COMMANDS.release} <root> --owner <driver-id>
+  \`\`\`
+
+  The lock is a file beside the root. A live holder → refused. A holder whose process is gone on
+  the same host → taken over, and said so. A holder on another host → refused (it cannot be judged).
+  The holder must be detached — see the next point.
+- **Before every cell, ask the gate** and obey its exit code; 3 means the cell is already counted,
+  or reserved by another attempt — skip it. That makes a killed driver resumable: run it again and
+  it continues.
+
+  \`\`\`bash
+  node \${CLAUDE_SKILL_DIR}/scripts/grid-guard.js ${COMMANDS.gate} --registry grid.json --ledger attempts.jsonl --unit <u> --arm <a> --model <m> --seed <s> --attempt <id>
+  \`\`\`
+
+  An allowed cell is **reserved** for \`<id>\` in the ledger in the same step, under the ledger's
+  lock: two drivers asking at once cannot both get one cell, nor both fit under the cap, because a
+  reservation holds the cell and counts its estimated cost until the attempt's end record (same
+  id) replaces it, or until it lapses (\`--reserve-sec\`, default six hours).
+
+- **Never delete shared inputs from a step.** Templates, fixtures and pinned states are read-only
+  (\`chmod -R a-w\` them); a packing step that deletes what it packed removes what another driver
+  still reads. **Prose only**, held by file permissions where you set them.
+- **Run the driver detached** (\`setsid nohup … &\`, its log to a file), never as a task tracked by
+  the agent session: a session's time cap kills a long grid. A separate waiter (a scheduled wake-up,
+  a loop that polls the ledger) brings the session back. **Held by \`${COMMANDS.lock}\`:** it refuses a holder
+  that is not the leader of its own session (Linux \`/proc/<pid>/stat\`; true after \`setsid\`) and
+  names the command to use instead; \`--attached\` allows a short run, said out loud; without
+  \`/proc\` it warns and passes.
+- The driver appends a ledger line when an attempt starts and another when it ends (same \`attempt\`
+  id, \`root\`, \`start\`/\`end\`, \`status\`, \`cost\`). Format: \`references/registry-and-ledger.md\`.
+
+### 6. After any incident — and before reporting — the overlap audit (catalog 6)
+
+\`\`\`bash
+node \${CLAUDE_SKILL_DIR}/scripts/grid-guard.js ${COMMANDS.overlap} attempts.jsonl
+\`\`\`
+
+Lists every pair of attempts that shared a root at the same time (an attempt with no end overlaps
+everything after its start), where at least one is counted. Each contaminated attempt is marked
+\`"status": "harness-error"\` — the record is **kept**, not deleted, and not counted — and its cell
+runs again as a **new** attempt. Run it after any crash, kill or harness change, and before the
+numbers are reported.
+
+### 7. Scoring and labels (catalog 4, 11)
+
+- **A check reads every stratum's expectation file.** When a new stratum's expected values live in a
+  second file, a proof or precondition check that reads only the first reports false FAILs. Load
+  expectations through one function that enumerates them, and fail when a unit's stratum has none.
+  A precondition failure is investigated before it is trusted — either way. **Prose only.**
+- **Every run whose outcome is not mechanically certain is read in full and labelled by hand.** An
+  automatic outcome classifier misses classes (a deferral phrased without a question mark); its
+  label is a pre-sort, and the disagreements between it and the hand labels are reported. The
+  procedure, with a label-file validator and an agreement report: \`label-agent-runs\`.
+
+### 8. Commit finished runs, often (catalog 15, 8)
+
+- Results in scratch space vanish. Commit **finished** runs — those with a completion record —
+  frequently; never a half-written run folder.
+- Every committed artifact is scanned for key-shaped strings, by a pre-commit hook:
+
+  \`\`\`bash
+  node \${CLAUDE_SKILL_DIR}/scripts/grid-guard.js ${COMMANDS.precommit} --runs runs --done done.json
+  \`\`\`
+
+  It reads the **staged** content (the index is what commits), refuses key-shaped strings by file and
+  line without printing them, and refuses a staged run folder that has no completion record. Before
+  an artifact leaves the repository: \`node \${CLAUDE_SKILL_DIR}/scripts/grid-guard.js ${COMMANDS.scanKeys} <dir>\`.
+- **Report the spend**: total, per model, against the cap.
+
+## When something goes wrong mid-grid
+
+Stop the driver; do not fix forward on a running grid. Write the incident down (what, when, which
+attempts), run the overlap audit, mark contaminated attempts, fix the harness, re-run the smoke for
+any stack the fix touched, then resume. A harness change is an addendum like any other.
+
+## Compose with
+
+- **build-benchmark** (upstream) — the design, the statistics and the artifact this run feeds.
+- **label-agent-runs** (downstream) — the outcome labels: codebook first, hand labels for every
+  uncertain run, a blind second rater, labels frozen after the analysis.
+- **build-label-page** — the page a person labels those runs on.
+- **osf-artifact-upload** — the artifact, after \`${COMMANDS.scanKeys}\` is clean.
+- An **independent refutation pass** (a second agent framed to refute, not to confirm) on the
+  registry and the smoke before the first counted run.
+`,
+});

package/skills/run-agent-grid/references/failure-catalog.md

@@ -0,0 +1,36 @@
+# Failure catalog — what went wrong in real grids, the rule, and what holds it
+
+Each entry: the failure as it happened (generic), the rule it produced, and what holds the rule.
+"Prose only" means nothing refuses the mistake; the rule holds while someone reads it. A new failure
+gets a new entry here, and the entry names its mechanism or says that it has none.
+
+`grid-guard` is `scripts/grid-guard.js` in this skill.
+
+| #   | failure                                                                                                                                  | rule                                                                                                                                                                                                                          | held by                                                                                                                                                                                                                                                                                     |
+| --- | ---------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| 1   | The author's requirement on which units to test lived only in prose; a large batch of paid runs used one easy, hand-picked unit.         | Every measurement is a row in a machine-readable registry (unit, role, requirement verbatim), checked before any paid run.                                                                                                    | `grid-guard gate`: refuses a unit, arm, model or seed the committed registry does not declare.                                                                                                                                                                                              |
+| 2   | The agent that designed a sample filled the author-approval field itself.                                                                | Only the author approves the drawn units, recorded with time and channel.                                                                                                                                                     | `gate` refuses a counted run unless `approval.by` = `author` with `at` and `channel`. **Prose only:** who typed it.                                                                                                                                                                         |
+| 3   | Units were filtered by "where our instrument can fire", which shrank the frame to trivial units.                                         | The frame is the whole population of the measured kind; exclusions listed per unit with a reason and counted; seeded draw committed before anyone looks; stratified only in proportion.                                       | `gate` refuses an excluded unit, an excluded unit without a reason, and a counted run without a recorded draw seed. **Prose only:** the frame and the proportion.                                                                                                                           |
+| 4   | A new stratum's expected values lived in a second file; two proof/precondition checks read only the first and reported false FAILs.      | Checks read every stratum's expectation file; a precondition failure is investigated before it is trusted, either way.                                                                                                        | **Prose only.** Load expectations through one enumerating function that fails on a stratum with none.                                                                                                                                                                                       |
+| 5   | A timeout fit for one model killed smoke runs of a slower model mid-task.                                                                | Smoke per stack — one run per arm, read end to end; timeout and cost reserve set per model from it and written down before the grid.                                                                                          | `gate` refuses a counted run on a model without `smoke.timeoutSec` / `smoke.costPerRun` and hands the driver the timeout. **Prose only:** the reading.                                                                                                                                      |
+| 6   | Two drivers ran at once on the same scratch path; one deleted the other's working tree: a crash, and several silently contaminated runs. | One lock per working-copy root; a second driver gets its own root; after any incident, audit every attempt's window for overlap; contaminated attempts marked `harness-error` (kept, not counted) and re-run as new attempts. | `grid-guard lock` / `release` (live holder refused, stale holder on this host taken over); `grid-guard overlap` over the ledger; `gate` re-runs a cell whose only attempts are harness errors.                                                                                              |
+| 7   | A packing step that deleted shared templates after packing them would have removed templates another driver still needed.                | Never delete shared inputs from a step; templates read-only.                                                                                                                                                                  | **Prose only**, held by file permissions (`chmod -R a-w`) where set.                                                                                                                                                                                                                        |
+| 8   | The agent's transcript captured environment secrets (an environment dump).                                                               | The sandbox's environment policy excludes secrets by name pattern; a names-only probe passes before any model run; every committed artifact is scanned for key patterns.                                                      | `grid-guard env-probe` (names, never values); `grid-guard precommit` as a pre-commit hook; `grid-guard scan-keys` before an artifact leaves.                                                                                                                                                |
+| 9   | The secret-excluding policy also stripped variables git needed, breaking git inside the sandbox.                                         | The probe also proves that the tools the agent needs still work.                                                                                                                                                              | `grid-guard env-probe --git`: `git init`, `add` and `commit` in a scratch repository, inside the sandbox.                                                                                                                                                                                   |
+| 10  | A shared tools directory was invisible inside the agent's sandbox.                                                                       | Per-unit sandbox proof, no model call, before the grid: tool visible, instrument fires on the pinned state, accepts the reference fix.                                                                                        | `gate` refuses a counted unit without `proof` (three `true` facts). **Prose only:** how the proof is taken.                                                                                                                                                                                 |
+| 11  | The automatic outcome classifier missed a class of outcomes (deferrals phrased without a question mark).                                 | Every run whose outcome is not mechanically certain is read in full and hand-labelled; the auto label is a pre-sort; disagreements reported.                                                                                  | `label-agent-runs`: `labels.js validate --ledger` refuses a completed run without a label; `labels.js agree` lists every auto-vs-hand disagreement. **Prose only:** the reading.                                                                                                            |
+| 12  | A long grid was killed by the session's time cap on tracked background tasks.                                                            | Run the driver detached; a separate waiter wakes the session; the driver resumes, skipping counted cells.                                                                                                                     | `grid-guard lock` refuses a holder that is not its own session leader (`/proc/<pid>/stat` session = pid, as after `setsid`), naming `setsid nohup <cmd> &`; `--attached` for a short run; warns and passes without `/proc`. `gate` exit 3 = already counted, so a restarted driver resumes. |
+| 13  | A model cost a multiple of its planned price.                                                                                            | Per-run cost measured in the smoke; hard cap in the driver; stop rule written before the grid; spend reported.                                                                                                                | `gate` refuses when ledger spend + the next run's smoke cost passes `budget.cap`. **Prose only:** the report.                                                                                                                                                                               |
+| 14  | Adding a model or seed after seeing early results invites forking paths.                                                                 | Any added model, seed or arm is a dated addendum committed before its first run; models reported as columns, never pooled.                                                                                                    | `gate` reads the registry from `HEAD` and refuses while the working copy differs. **Prose only:** columns, not pooled.                                                                                                                                                                      |
+| 15  | Results lived in scratch space that vanishes.                                                                                            | Commit finished runs (only those with a completion record) frequently; never half-written run folders.                                                                                                                        | `grid-guard precommit --runs <dir> --done <file>` refuses a staged run folder without its completion record. **Prose only:** "frequently".                                                                                                                                                  |
+
+## What the guard does not see
+
+- **Who** wrote a field. `approval.by` is compared with `author`; an agent that writes the author's
+  name passes. Keep the approval in its own commit, quoting the author's message.
+- **The population** behind the registry. The guard checks the registry's own consistency, not that
+  the frame was the whole population.
+- **Key shapes it does not know.** `scan-keys` matches forms (key blocks, token prefixes,
+  `NAME=value` with a secret-looking name). A credential in an unknown shape passes; the environment
+  probe, which works on names, is the first line.
+- **Whether a run was read.** Nothing mechanical tells a read transcript from an unread one.

package/skills/run-agent-grid/references/registry-and-ledger.md

@@ -0,0 +1,142 @@
+# The registry, the attempt ledger, and a driver that obeys the guard
+
+`grid-guard` is `scripts/grid-guard.js` in this skill; every command below is run with
+`node ${CLAUDE_SKILL_DIR}/scripts/grid-guard.js`. Exit codes: 0 allowed or clean, 1 refused or a
+finding, 2 usage, 3 (`gate`) already counted.
+
+## The registry — `grid.json`, committed
+
+```json
+{
+  "author": "the-author-handle",
+  "draw": {
+    "population": "every unit of the measured kind, and where the list came from",
+    "seed": 7349,
+    "script": "draw.py"
+  },
+  "arms": ["control", "treatment"],
+  "seeds": [1, 2, 3],
+  "budget": {
+    "cap": 300,
+    "currency": "USD",
+    "stopRule": "stop at the cap; report what ran"
+  },
+  "models": [
+    {
+      "id": "model-a",
+      "smoke": {
+        "timeoutSec": 2400,
+        "costPerRun": 1.8,
+        "readBy": "who read the smoke runs",
+        "at": "ISO time"
+      }
+    }
+  ],
+  "units": [
+    {
+      "id": "unit-017",
+      "role": "counted",
+      "stratum": "s2",
+      "requirement": "the author's requirement on this unit, verbatim",
+      "approval": {
+        "by": "the-author-handle",
+        "at": "ISO time",
+        "channel": "where the author said it"
+      },
+      "proof": {
+        "toolVisible": true,
+        "firesOnPinned": true,
+        "acceptsReference": true,
+        "at": "ISO time"
+      }
+    },
+    { "id": "unit-smoke", "role": "smoke" },
+    { "id": "unit-031", "role": "excluded", "reason": "why, in one line" }
+  ]
+}
+```
+
+What `gate` requires, field by field:
+
+| field                                        | for          | refused when                                                       |
+| -------------------------------------------- | ------------ | ------------------------------------------------------------------ |
+| the file itself                              | every run    | not in git, not committed, or the working copy differs from `HEAD` |
+| `units[].id`, `arms`, `models[].id`, `seeds` | every run    | the cell's unit, arm, model or seed is not declared                |
+| `units[].role`                               | every run    | anything but `counted` or `smoke`                                  |
+| `units[].reason`                             | every run    | an `excluded` unit has none                                        |
+| `budget.cap`                                 | every run    | missing, or ledger spend + next cost would pass it                 |
+| `units[].approval`                           | counted runs | `by` ≠ `author`, or `at` / `channel` empty                         |
+| `units[].proof`                              | counted runs | any of the three facts is not `true`                               |
+| `draw.seed`                                  | counted runs | missing                                                            |
+| `models[].smoke`                             | counted runs | `timeoutSec` or `costPerRun` is not a number                       |
+
+On success `gate` prints one JSON line: `{"allowed":true,"cell":{…},"attempt":"…","counted":…,"timeoutSec":…,"spent":…}`,
+and has already appended a `reserved` record for that attempt to the ledger (read, check and append
+under `<ledger>.lock`, so concurrent drivers cannot both take one cell or both fit under the cap).
+The driver takes the run's timeout from `timeoutSec`.
+
+## The attempt ledger — `attempts.jsonl`, append-only
+
+One JSON object per line. `gate` writes the first line of an attempt (a reservation); the driver
+writes a line when the attempt **starts** and another when it **ends**, all with the same `attempt`
+id; later lines add to earlier ones. A crashed attempt therefore stays on record with a start and
+no end, and its reservation lapses at `expires`.
+
+```json
+{"attempt":"a-0042","unit":"unit-017","arm":"control","model":"model-a","seed":"1","status":"reserved","cost":1.8,"start":"2026-01-01T09:59:58Z","expires":"2026-01-01T15:59:58Z"}
+{"attempt":"a-0042","agent":"cli-a","model":"model-a","unit":"unit-017","arm":"control","seed":1,"root":"/scratch/w1","start":"2026-01-01T10:00:00Z"}
+{"attempt":"a-0042","end":"2026-01-01T10:31:00Z","status":"complete","cost":1.62}
+```
+
+| field    | meaning                                                                                                                                                          |
+| -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `status` | `reserved` — held by `gate` until the end record or `expires`; `complete` — counted, whatever the outcome; `harness-error` — kept on record, not counted, re-run |
+| `root`   | the working-copy root the attempt used: what `overlap` compares                                                                                                  |
+| `agent`  | the agent CLI that ran; with `model` and `<unit>/<arm>/<seed>` it names the cell a label refers to                                                               |
+| `cost`   | what the attempt cost, counted toward the cap even for a harness error                                                                                           |
+
+The outcome itself lives in the run folder (`runs/<attempt>/`), whose completion record
+(`done.json` by default) is written **last**.
+
+## A driver loop
+
+```bash
+#!/usr/bin/env bash
+set -uo pipefail
+G="node ${CLAUDE_SKILL_DIR}/scripts/grid-guard.js"
+ROOT=/scratch/w1            # this driver's own working-copy root
+$G lock "$ROOT" --owner "driver-w1" --pid $$ || exit 1
+trap '$G release "$ROOT" --owner driver-w1' EXIT
+
+while read -r unit arm model seed; do
+  id="a-$(date +%s%N)-$$-$unit-$arm-$model-$seed"
+  verdict=$($G gate --registry grid.json --ledger attempts.jsonl \
+            --unit "$unit" --arm "$arm" --model "$model" --seed "$seed" --attempt "$id")
+  rc=$?
+  [ $rc -eq 3 ] && continue           # counted, or reserved by another driver: skip it
+  [ $rc -ne 0 ] && exit $rc           # refused: stop the grid, read why
+  timeout_sec=$(node -e 'console.log(JSON.parse(process.argv[1]).timeoutSec ?? 3600)' "$verdict")
+  # 1. start record   2. fresh working copy of the unit in $ROOT   3. the agent CLI under
+  #    `timeout $timeout_sec`   4. the scorer   5. runs/$id/done.json LAST   6. end record
+  #    (status complete or harness-error, real cost) — it releases the reservation
+done < cells.txt
+```
+
+Start it detached so the agent session's limits cannot kill it, and let something else wake the
+session:
+
+```bash
+setsid nohup ./driver.sh > driver.log 2>&1 < /dev/null &
+```
+
+## The pre-commit hook
+
+`.git/hooks/pre-commit` (or the hook manager the repository already uses):
+
+```bash
+#!/usr/bin/env bash
+exec node "<path to this skill>/scripts/grid-guard.js" precommit --runs runs --done done.json
+```
+
+A hook in `.git/hooks/` is not versioned; record in the study's README that it is installed, and
+run the same command in CI if the repository has CI.

package/skills/run-agent-grid/run-agent-grid.eval.ts

@@ -0,0 +1,59 @@
+/**
+ * run-agent-grid — the PAID tier: does this skill's description fire on its own requests, and
+ * stay quiet on a sibling's?
+ *
+ * Declared here, whole: the prompts it must fire on, the prompts it must not, and the
+ * measurement — no shared table, no kit. `vigiles eval` discovers this file by its name and runs
+ * it; nothing runs at import.
+ *
+ * The should-not-fire prompts are the neighbours' territory, written inline: `build-benchmark`
+ * (designing the study), `build-label-page` (a page a person labels on), `label-agent-runs`
+ * (labelling the outcomes) and `draft-paper`. They are inline rather than imported from
+ * `label-agent-runs.eval.ts` because that file imports this one's prompts, and two evals that
+ * import each other's constants deadlock at module load.
+ *
+ * The skills it competes with are every skill this package ships (the `skills/` directory this
+ * file sits in), the same set in a checkout and in `node_modules`.
+ *
+ * ⚠️ NEVER RUN. Written with the skill; its rates are UNKNOWN, not assumed good.
+ *
+ * Costs money; not CI.
+ *   npx vigiles eval skills/run-agent-grid/run-agent-grid.eval.ts
+ */
+import { dirname } from "node:path";
+import { fileURLToPath } from "node:url";
+import { defineEval, skillResolved } from "vigiles";
+
+/** The namespace vigiles installs a loose skills directory under. */
+const LOOSE_NS = "vigiles-loose-skills";
+
+/** Requests this skill owns. Exported: `label-agent-runs.eval.ts` uses them as its should-not-fire set. */
+export const TRIGGER_PROMPTS: readonly string[] = [
+  "запускаю платную сетку прогонов агента — что проверить до первого запуска",
+  "the grid driver died halfway, how do I resume without re-running the counted cells",
+  "два драйвера писали в одну рабочую папку, какие прогоны теперь испорчены",
+  "gate every agent run on the approved sample and a hard budget cap before we spend money",
+];
+
+export default defineEval({
+  measureTriggerRate: {
+    name: "run-agent-grid-trigger",
+    skillsDir: dirname(dirname(fileURLToPath(import.meta.url))),
+    prompts: TRIGGER_PROMPTS,
+    irrelevantPrompts: [
+      "надо спроектировать сам замер, а не описывать его — как считать честно",
+      "build a gold-labelling page for these 40 items so I can answer on my phone",
+      "which of the finished runs does a person have to read, and which can the classifier label",
+      "числа есть, пора писать текст",
+    ],
+    fired: (trace) => skillResolved(trace, `${LOOSE_NS}:run-agent-grid`),
+    // Four named phrasings per side, deliberately, as for every skill of this package.
+    minPrompts: 4,
+    // A grid in progress, so "the driver died" and "the registry" have something to refer to.
+    fixture: {
+      "grid.json": '{ "arms": ["control", "treatment"], "seeds": [1, 2] }\n',
+      "attempts.jsonl":
+        '{"attempt":"a1","unit":"u1","arm":"control","seed":1,"start":"2026-01-01T00:00:00Z"}\n',
+    },
+  },
+});