@sleepy_panda_srl/portulan 0.1.3 → 0.3.0

package/cli/skill-goldens.mjs

@@ -1,111 +1,23 @@
 #!/usr/bin/env node
-// The core-skill golden corpus — every mandate a core skill states, bound to the live artifacts it
-// governs, and graded.
-//
-// Milestone 8, the row's **original first subject**: *Golden tasks per core skill.* Untouched by either
-// amendment — the 2026-07-28 expansion says it "stands exactly as written", and clause (a) *widened its
-// subject* to reach the gates "as well as the skills" without narrowing the subject it already had
-// (`../docs/milestones/m08.md` line 18). The gates half landed at (a) as `./goldens.mjs`; this is the
-// half that sentence always named and nobody had built.
-//
-// ## What a golden task IS here, and it was RULED rather than derived
-//
-// The criterion reads two ways — a deterministic corpus, or task specifications run against a live
-// model — and `../.portulan/gate-map.md`'s *"Session-open runs `clarify` against the milestone row
-// itself"* exists because milestone 4 guessed at exactly this shape of ambiguity and cost a
-// session-blocking question. So it was put to the maintainer instead of argued. **His ruling: a
-// deterministic corpus graded against LIVE ARTIFACTS in the tree.** The model-run sense belongs to the
-// A/B clause, which is separately named in the same row.
-//
-// ## The object: a mandate, its live artifacts, and the carrier that enforces it
-//
-// A case binds one **numbered step** of one skill's `## The pass` to the artifact set it governs, and
-// records which existing rail enforces it — or `null`, which is the interesting value.
-//
-// Four properties, and every one of them exists because a checkpoint found the version without it:
-//
-//  1. **The denominator is DERIVED, not chosen.** The runner enumerates the numbered steps under each
-//     skill's `## The pass` heading and requires every one to be either `bound` or `unbindable`. A
-//     fifteenth step is red. The first draft let the author pick which mandates to write cases for,
-//     and a census over a set its own author drew reports "5 of 5" and means nothing — the difference
-//     `./goldens.mjs` already draws by deriving its denominator from the yielded policy.
-//  2. **The heading is PREFIX-matched, and an empty step list is could-not-run.** Three spellings exist
-//     — `## The pass`, `## The pass (bounded)`, `## The pass, in order` — so an exact match finds one
-//     skill of three and the other two get an empty denominator that satisfies "every step is bound"
-//     vacuously. That is `../evals/README.md`'s own sentence about clause (d): *"a check whose
-//     enumeration went empty … reports green and has stopped being a rail."* The match is also anchored
-//     at `The pass`, because a loose `/pass/` grabs `consolidate`'s `## The one move this pass may not
-//     make`.
-//  3. **The quote must anchor EXACTLY ONCE** in the skill's own file, or the case is could-not-run —
-//     `./mutants.mjs`'s discipline, adopted rather than re-derived. A skill whose wording is reworded,
-//     softened or deleted reddens instead of drifting, and nothing in this tree did that before.
-//  4. **The predicates use the carriers' own exported functions rather than re-spelling them.** This
-//     module imports `RETIRE_WHEN` and `parseProvenance` from `./doctor.mjs`, so the retire-when check
-//     is `doctor`'s regex and not a second one.
-//
-//     **What that does NOT do, stated because three carriers of this file's prose once said it did:**
-//     it does not verify the `carrier` field. Nothing links `c.carrier.symbol` to an import — the
-//     pre-commit checkpoint rewrote every carrier in a corpus file to a module that does not exist and
-//     the corpus stayed green, and moved `.portulan/verify/index.sh` out of the tree with the same
-//     result. `carrier` is a **declared, reviewed field**: it records which rail a mandate's enforcement
-//     lives in, for the census, and a reviewer is what checks it. And a carrier that deleted an export
-//     would fail this module's own import with a stack trace and exit **1**, not the 2 the earlier
-//     sentence promised.
-//
-// ## What the carrier field does NOT establish, said here because a census implies more than it means
-//
-// An import proves the carrier **contains** the check. It does not prove the carrier **runs** it: a
-// rail could keep `RETIRE_WHEN` exported, stop calling it, and this corpus would not notice, because
-// `expect` is computed here over the artifacts rather than by asking the rail. Closing that needs a
-// per-mandate drill — perturb a record, require the named rail to fire with its own tell — which is
-// `./drills.mjs`'s shape at a finer grain and a second clause's work.
-//
-// ## `unbindable` is adjudicated, never asserted
-//
-// The dodge is obvious: call a mandate unbindable and it needs no case. So the reason takes a **closed
-// vocabulary** rather than free prose, on `./mutants.mjs`'s rule that a `survives` record is admissible
-// only as a proof and never as a standing note that a gap exists:
-//
-//   * `judgement-only`  — the mandate is about a human or agent decision and no artifact can witness it
-//                         ("Get the answers from the human"). **Such a step may name no artifact path**:
-//                         if you can name the artifacts, it is not judgement, it is unbuilt.
-//   * `no-artifact`     — the mandate governs something that leaves no trace in this tree.
-//   * `cross-language`  — a carrier exists but in another language, so binding it here would put two
-//                         spellings of one rule across a language boundary.
-//   * `already-carried` — a carrier exists IN THIS LANGUAGE and already holds the mandate, so a case
-//                         here would be a third carrier of a rail that is already drilled. Added at the
-//                         pre-commit checkpoint, which found `consolidate` step 5 classed `bound` with
-//                         a carrier the runner never reads: deleting `.portulan/verify/index.sh` left
-//                         the corpus green. The row was inflating the headline, and the vocabulary had
-//                         no honest term for what it actually was.
-//
-// **And the ratio is a finding, not bookkeeping.** How much of a core skill is artifact discipline and
-// how much is agent judgement is a measurement about the engine, which is what this row exists to
-// produce. It is printed as such.
+// The core-skill golden corpus — every mandate a core skill states, bound to the live artifacts it governs, and graded.
 
 import fs from "node:fs";
 import path from "node:path";
 import { pathToFileURL } from "node:url";
 
-// The carriers, imported rather than grepped — property 4 above. A carrier that stops exporting these
-// fails this module at load, which is the strongest form of the check available for free.
+// The carriers' own exports, never a second spelling of their checks.
 import { RETIRE_WHEN, parseProvenance } from "./doctor.mjs";
 
 export const CASE_KINDS = ["load-bearing", "census"];
 export const UNBINDABLE_REASONS = ["judgement-only", "no-artifact", "cross-language", "already-carried"];
 
-// Anchored at `The pass` and prefix-matched, for the two reasons in property 2.
+// Prefix-matched, as three spellings exist; anchored, as a bare `pass` matches another `consolidate` heading.
 const PASS_HEADING = /^##\s+The pass\b/i;
-// **Enumerated on the number, and the bold is read separately.** This was
-// `/^(\d+)\.\s+\*\*(.+?)\*\*/`, which made an UNBOLDED step invisible: the pre-commit checkpoint
-// added `5. A brand new fifth mandate, not bolded` to a pass and the corpus stayed green at 5 of 15.
-// The empty-enumeration guard catches the all-or-nothing case; this catches the incremental one, which
-// is the realistic one.
+// On the number alone, the bold read separately, so an unbolded step still counts.
 const STEP = /^(\d+)\.\s+(.*)$/;
 
 export class CouldNotRun extends Error {}
 
-/** Every core skill in the engine, derived from the tree. A skill with no corpus file is a finding. */
 export function skillSet(repoRoot) {
     const dir = path.join(repoRoot, "core/skills");
     let entries;
@@ -122,7 +34,6 @@ export function skillSet(repoRoot) {
     return skills;
 }
 
-/** The numbered steps under a skill's `## The pass`. Zero is could-not-run, never a vacuous green. */
 export function passSteps(repoRoot, skill) {
     const file = path.join(repoRoot, "core/skills", skill, "SKILL.md");
     let text;
@@ -154,7 +65,6 @@ export function passSteps(repoRoot, skill) {
     return steps;
 }
 
-/** A quote must place exactly once in the skill's own file, or the case cannot be graded. */
 export function anchorQuote(repoRoot, skill, quote) {
     const text = fs.readFileSync(path.join(repoRoot, "core/skills", skill, "SKILL.md"), "utf8");
     let count = 0;
@@ -171,11 +81,6 @@ export function anchorQuote(repoRoot, skill, quote) {
     }
 }
 
-/**
- * The artifact set a case grades, resolved through the workspace's DECLARED slots rather than a
- * hard-coded path. A slot rename must not silently empty a denominator, and an empty match is
- * could-not-run for the same reason a zero-step list is.
- */
 export function slotFiles(workspaceDir, manifest, slot) {
     const rel = manifest?.slots?.[slot];
     if (typeof rel !== "string" || rel.length === 0) {
@@ -196,11 +101,9 @@ export function slotFiles(workspaceDir, manifest, slot) {
 }
 
 // ---------------------------------------------------------------------------------------------
-// The predicates. Each is named in its case, because a figure computed before its rule was decided is
-// how both of this session's opening passes produced a different number for the same question.
+// The predicates
 // ---------------------------------------------------------------------------------------------
 
-/** A bolded field's block: the `**Field.**` line, then lines until the next bolded field or heading. */
 export function fieldBlock(text, field) {
     const lines = text.split("\n");
     const start = lines.findIndex((l) => new RegExp(`^\\s*\\*\\*${field}\\.?\\*\\*`, "i").test(l));
@@ -213,7 +116,6 @@ export function fieldBlock(text, field) {
     return out.join("\n");
 }
 
-/** Bullets with wrapped continuation lines rejoined — a wrapped criterion is one criterion. */
 export function bulletsOf(block) {
     const out = [];
     for (const line of block.split("\n")) {
@@ -223,17 +125,7 @@ export function bulletsOf(block) {
     return out.filter(Boolean);
 }
 
-// EARS as `core/templates/task.md` states it. The shape is NOT restated in this module's prose: the
-// template is the rule and this is a reduction of it, cited rather than re-spelled.
-//
-// **Three carriers state the shape** — that template, `core/operating/operating`-adjacent
-// `core/operating/verification.md`, and `core/skills/clarify/SKILL.md` — all `core/` at tier `propose`.
-// Reducing them to one is deferred on **budget** and on a real design question (making a CLI module the
-// carrier that core doctrine cites inverts the cascade), filed as
-// https://github.com/sleepy-panda-srl/portulan/issues/359. _(An earlier version of this comment said
-// there were four, one of them `docs/vision.md` at tier `prohibited`, and concluded the reduction was
-// impossible to an agent. `docs/vision.md` names EARS in a comparison-table row and nowhere states the
-// shape; the count and the conclusion were both wrong, and the pre-commit checkpoint measured it.)_
+// A reduction of EARS as `core/templates/task.md` states it; the template is the rule.
 export const isEars = (b) => /\bwhen\b/i.test(b) && /\bshall\b/i.test(b);
 
 export const PREDICATES = {
@@ -247,25 +139,15 @@ export const PREDICATES = {
             ? { ok: true, total: bs.length }
             : { ok: false, why: `${bad.length} of ${bs.length} criteria are not EARS-shaped`, total: bs.length };
     },
-    // `codify` step 3 — *"Attach how it earns its place"*. core/templates/proposal.md mandates an
-    // **Enforcement.** field, and nothing checks it. The pre-commit checkpoint found this step classed
-    // judgement-only on the argument that presence-checking would be "the presence floor with none of
-    // the value" — an argument this module refutes in its own printed output, and which equally refutes
-    // the provenance binding it did ship. Same spelling tolerance, same reason.
     "enforcement-present": (text) => {
         const has = /^\s*\*\*Enforcement\.?\*\*/im.test(text) || /^##+\s+Enforcement/im.test(text);
         return has ? { ok: true } : { ok: false, why: "no enforcement field in any spelling" };
     },
-    // Any spelling of a provenance field. Deliberately shape-tolerant: the tree carries `**Provenance.**`
-    // and `## Provenance` and a literal predicate would report the renamed ones as absent — the false-red
-    // this session's own opening pass produced once against the task corpus.
     "provenance-present": (text) => {
         const has = /^\s*\*\*Provenance\.?\*\*/im.test(text) || /^##+\s+Provenance/im.test(text);
         return has ? { ok: true } : { ok: false, why: "no provenance field in any spelling" };
     },
-    // The two defined forms, answered by `doctor`'s own exported parser. Scoped to `type: rule`, which
-    // is the scope the mandate has and the scope `doctor` enforces at — a `decision` carrying prose
-    // provenance is reported by `doctor` and is not a violation of this mandate.
+    // Scoped to `type: rule`, the scope `doctor` enforces the two forms at.
     "provenance-two-form": (text) => {
         const type = (/^\s*\*\*type:\*\*\s*(\w+)/im.exec(text) || [])[1];
         if (type !== "rule") return { ok: true, skipped: "not a rule" };
@@ -276,11 +158,6 @@ export const PREDICATES = {
         RETIRE_WHEN.test(text) ? { ok: true } : { ok: false, why: "no **Retire when:** line" },
 };
 
-/**
- * The containment case: every budget id the Workspace Definition declares must be named in the
- * routing sentence `consolidate` states. Derived from the schema on both sides — a fifth budget, or a
- * rename, reddens rather than drifting.
- */
 export function budgetIds(repoRoot) {
     const schema = JSON.parse(fs.readFileSync(path.join(repoRoot, "spec/workspace.schema.json"), "utf8"));
     const mem = schema.properties?.memory?.properties ?? {};
@@ -294,8 +171,7 @@ export function budgetIds(repoRoot) {
 }
 
 // ---------------------------------------------------------------------------------------------
-// Grading. Every case's `expect` is compared against what the predicate answers over the LIVE
-// artifacts; nothing here is graded against a fixture the corpus invented.
+// Grading
 // ---------------------------------------------------------------------------------------------
 
 function gradeBound(skill, c, ctx) {
@@ -322,13 +198,6 @@ function gradeBound(skill, c, ctx) {
     return { total: considered, failing, detail: `${considered - failing.length}/${considered} compliant` };
 }
 
-/**
- * Grade one skill's corpus file against the tree.
- *
- * Every numbered step must be accounted for. `unbindable` is adjudicated: the reason comes from a
- * closed vocabulary, and a `judgement-only` step may name no artifact — if you can name the artifacts
- * the mandate governs, it is not judgement, it is unbuilt.
- */
 export function gradeSkill(skill, corpus, ctx) {
     const findings = [];
     const steps = passSteps(ctx.repoRoot, skill);
@@ -345,9 +214,7 @@ export function gradeSkill(skill, corpus, ctx) {
             );
         }
     }
-    // Step 0 is reserved for a mandate outside `## The pass` — `consolidate`'s routing sentence lives
-    // under `## When to use it`. Numbered 0 so the pass accounting stays a clean 1..N and the row cannot
-    // be mistaken for a step. Any OTHER number the pass does not have is a finding.
+    // Step 0 is a mandate outside `## The pass`, such as `consolidate`'s routing sentence.
     for (const n of declared.keys()) {
         if (n !== 0 && !steps.some((s) => s.n === n)) {
             findings.push(`${skill}: case names step ${n}, which the pass does not have`);
@@ -363,11 +230,6 @@ export function gradeSkill(skill, corpus, ctx) {
             if (!UNBINDABLE_REASONS.includes(c.reason)) {
                 findings.push(`${skill}: step ${step.n} is unbindable for ${JSON.stringify(c.reason)}, which is not one of ${UNBINDABLE_REASONS.join(" | ")}`);
             }
-            // **A reason without an argument is a label, not an adjudication.** `./mutants.mjs` refuses
-            // an operator carrying no `why` — *"a mutation nobody can read is not reviewable"* — and
-            // `./goldens.mjs` refuses a `documented-hole` naming no record. This carried neither until
-            // the pre-commit checkpoint stripped every `why` from all ten unbindable cases and watched
-            // the corpus stay green.
             if (typeof c.why !== "string" || c.why.trim().length < 40) {
                 findings.push(
                     `${skill}: step ${step.n} is unbindable and carries no argument. ` +
@@ -393,11 +255,6 @@ export function gradeSkill(skill, corpus, ctx) {
             if (e instanceof CouldNotRun) throw e;
             throw new CouldNotRun(`${skill}: step ${step.n} could not be graded — ${e.message}`);
         }
-        // **`expect.accepted` is the larger dodge, and it was entirely unadjudicated.** Any finding
-        // could be silenced by appending a bare filename — the pre-commit checkpoint created a
-        // non-compliant task file, listed it, and the corpus went green over eighteen silenced
-        // findings. So an entry is an object carrying its own `why`, on the same rule as `unbindable`
-        // above: the both-directions check stops a STALE entry, and this stops an UNJUSTIFIED one.
         const acceptedEntries = c.expect?.accepted ?? [];
         for (const a of acceptedEntries) {
             if (typeof a !== "object" || typeof a.file !== "string") {
@@ -413,8 +270,6 @@ export function gradeSkill(skill, corpus, ctx) {
         const unexpected = result.failing.filter((f) => !accepted.has(f.split(" — ")[0]));
         const repaired = [...accepted].filter((a) => !result.failing.some((f) => f.split(" — ")[0] === a));
         for (const u of unexpected) findings.push(`${skill}: step ${step.n} — ${u}`);
-        // Both directions, as `documented-hole` has on the gate side: an accepted drift that is
-        // REPAIRED is a finding too, or an accepted-drift list outlives the drift it records.
         for (const r of repaired) {
             findings.push(
                 `${skill}: step ${step.n} — ${r} now complies, and is still listed as accepted drift. ` +
@@ -438,10 +293,6 @@ export function run(argv = process.argv.slice(2), io = console) {
     let repoRoot = ".";
     let workspaceDir = null;
     for (let i = 0; i < argv.length; i += 1) {
-        // **A missing option value must not swallow the next flag.** `argv[++i]` on a trailing
-        // `--repo-root` yields `undefined` and on `--repo-root --workspace x` yields `"--workspace"`,
-        // so the tool would grade a directory nobody named and say nothing. `./rule-carriers.mjs`
-        // treats a missing value as exit 2 and this now matches it. Copilot round 1 on #360.
         const value = (flag) => {
             const v = argv[i + 1];
             if (v === undefined || v.startsWith("--")) throw new CouldNotRun(`${flag} needs a value`);
@@ -478,8 +329,6 @@ export function run(argv = process.argv.slice(2), io = console) {
     for (const skill of skills) {
         const file = path.join(corpusDir, `${skill}.json`);
         if (!fs.existsSync(file)) {
-            // The derived skill set is what makes this a coverage rail: a new core skill cannot ship
-            // with no mandates accounted for.
             findings.push(`${skill}: core/skills/${skill}/ exists and evals/goldens/skills/${skill}.json does not`);
             continue;
         }
@@ -491,8 +340,6 @@ export function run(argv = process.argv.slice(2), io = console) {
             return 2;
         }
         if (corpus.skill !== skill) {
-            // The filename is checked against the file's own field, the repair `../evals/README.md`
-            // records for the gate corpus after a misfiled fixture graded cleanly.
             io.error(`skill-goldens: ${skill}.json declares skill ${JSON.stringify(corpus.skill)}`);
             return 2;
         }
@@ -530,22 +377,12 @@ export function run(argv = process.argv.slice(2), io = console) {
     io.log(`  ${bound} of ${bound + unbindable} mandate(s) bound to live artifacts — ${loadBearing} load-bearing, ${census} census`);
     const reasons = {};
     for (const g of all) for (const r of g.rows) if (r.state === "unbindable") reasons[r.reason] = (reasons[r.reason] ?? 0) + 1;
-    // **The split is printed, because the aggregate hides which unbindable means what.** Three prose
-    // carriers of this file's own output once said "two thirds of these mandates are judgement" when
-    // two thirds were merely UNBINDABLE and `cross-language` and `already-carried` mean the opposite —
-    // a carrier exists. Caught at the pre-commit checkpoint.
     io.log(`  ${unbindable} adjudicated unbindable — ${Object.entries(reasons).map(([k, v]) => `${v} ${k}`).join(", ")}`);
     io.log("");
     io.log("  THE RATIO IS A FINDING, not bookkeeping: it measures how much of a core skill is artifact");
     io.log("  discipline and how much is agent judgement. Read the SPLIT, not the total — the");
     io.log("  judgement-only rows are the A/B clause's FIRST subject; cross-language and already-carried");
     io.log("  mean a carrier exists and is somewhere else.");
-    // **Widened 2026-08-28 on the maintainer's ruling**; this printed "only the judgement-only rows are
-    // the A/B clause's subject" until then. That reading could not stand, and the reason is measured:
-    // `vendor --host` carries `core/skills/` not at all, so all five judgement rows reach
-    // nothing in a vendored arm. The A/B clause's subject is the mandates core SHIPS that an arm
-    // actually receives. Four carriers held the narrower reading and all four moved together —
-    // ../evals/ab/corpus.md, and the claim is registered in ../.portulan/rule-carriers.json.
     io.log("  A `census` row re-indexes a figure an existing recipe already prints; it is not a new check.");
     io.log("  `unbindable` is the dodge, so its reason comes from a closed vocabulary and is adjudicated.");
     io.log("  `carrier` is a DECLARED, reviewed field. Nothing links it to a check: a corpus naming a");
@@ -572,10 +409,7 @@ export function run(argv = process.argv.slice(2), io = console) {
   }
 }
 
-// The entry guard, in the ONE form `./rule-carriers.mjs` designates: `file://${argv[1]}` percent-encodes
-// differently from `import.meta.url` and this working copy lives under a path with spaces, so the naive
-// spelling exits 0 having run nothing — met four times in this repository, once in the session before
-// this one.
+// Not `file://${argv[1]}`: `import.meta.url` percent-encodes, so a path with a space would never match.
 function isMain() {
     const invoked = process.argv[1];
     if (!invoked) return false;