shapeup-sdlc 3.13.0 → 3.14.0

package/.claude-plugin/plugin.json

@@ -1,7 +1,7 @@
 {
   "name": "shapeup-sdlc-plugin",
   "displayName": "ShapeUp SDLC Plugin",
-  "version": "3.13.0",
+  "version": "3.14.0",
   "description": "Shape Up SDLC harness for Claude Code: shaping, intake, orient, scope-mapping, building (T0-verified, sandboxed, scope-contracted), evaluation and QA skills orchestrated by a tech-lead.",
   "author": {
     "name": "Liberty Nguyen",

package/AGENTS.md

@@ -37,7 +37,7 @@ Betting Table: PO decides; rejected pitches loop back to raw idea.
 | Map Scopes | ⏸ **L1b** — Board Review (+ substrate disjointness lint) | `/scope-architect` (scope contracts ✦ — sole writer); traceability oracle advisory ✚. A registered requirement that no acceptance criterion grades and no scope claims is **red** here, and L1b prints the `REQ → AC` table: the two ways out are an AC carrying `(covers: REQ-…)` or the PO marking the clause `CUT (PO-approved)`. Red only where the plan is still cheap to change — after L1b nobody re-reads the pitch |
 | Build Vertically | ⏸ **L2** — Board 100% ✅ + T0-green ✦ | per dispatch: compile order → `/task-executor` (--order) → ingest result; T0-verified per attempt (fixtures + DB probe ✦), substrate-sandboxed ✦. Scopes build **concurrently** ✦ — `--parallel-scopes N` caps it (default 4), a scope is released the moment its own dependencies are green, and a scope green in this round is skipped rather than rebuilt. Then the **round build gate** ⚙: the ledger's run command, then the profile's `build_probe` and `launch_probe`, run once per round before EVAL — a red gate ends the round with no verdict and its failing step is compiled into the next round's orders as bugs; a `mobile` profile with no `launch_probe` is warned about every round, so the install/launch risk has an owner |
 | EVAL (once per round) | ⏸ **L3** — Verdict | `/spec-evaluator` (--order), only over a round whose build gate ⚙ is not red: spec- + test-surface-conformance ★, T0 citation ✦; refuted boxes/verdict applied by ingest. The order names the round's build gate artifact and the profile's `launch_probe`, so a `[ui]` row is graded on the app the gate launched rather than recorded as no evidence — a project with no `launch_probe` still gets no on-device grade |
-| FAIL → round r+1 | — | regression rule ★: bugs + full Test Surface of touched UC |
+| FAIL → round r+1 | — | regression rule ★: bugs + full Test Surface of touched UC. Every criterion the verdict graded FAIL reaches the round, whether or not the judge filed a bug for it, addressed to the scope that owns its use case — a round with a FAIL verdict and nothing to fix is how a run stalls. A row whose check the scope rewrote between a failing trial and a passing one is listed for the judge, who reads it against the row before the pass counts |
 
 ✦ = requires scope contracts (`shapeup/<slug>/scopes/*.md`); ✚ = requires the spine artifacts (`requirements.md`, `wiring-map.md`, `project-profile.md`). Traceability stays advisory until `covers:` is populated. Absent artifact ⇒ arm skipped (non-regression).
 

package/kernel/compile.mjs

@@ -37,7 +37,7 @@ import { readRunId, dispatchReceipts, legLedger, readReceipt, receipt } from "./
 // --spec-overridden directory, and the import is the convention-derived default.
 import {
   tasksDir, specDir as defaultSpecDir, roundLedger, trials, verdictsDir, ordersDir,
-  relShared, relLocal, globLocal, globShared, relKnowledgeBase, resultsDir, scopesDir,
+  relShared, relLocal, globLocal, globShared, relKnowledgeBase, resultsDir, scopesDir, localRoot,
 } from "./lib/paths.mjs";
 import { readContract, readAllContracts, tasksForScope, SCOPE_CONTRACT, reqId } from "./lib/contract.mjs";
 import { writeActiveOrder } from "./probe/resume.mjs";
@@ -527,6 +527,57 @@ export function verdictBugs(cwd, slug, round) {
   return v.bugs.filter((b) => !refuted.has(String(b?.id)) && !refuted.has(String(b?.criterion)));
 }
 
+/**
+ * The previous round's FAILED CRITERIA that the judge filed no bug for, as bug entries.
+ *
+ * A verdict carries two lists: the criteria it graded and the bugs it filed. Only the second
+ * reached the next round, so a criterion graded FAIL — "no evidence: no check names this row", a
+ * device row the judge could not observe — with no matching bug handed the fix round nothing to do.
+ * Measured: three fix rounds in a row compiled with no bugs over a FAIL verdict, and the run ended
+ * where it started. Every failed criterion is a thing the round must address, so each one without a
+ * bug becomes one, addressed to the scope whose contract lists the use case the criterion names.
+ *
+ * @param {string} cwd - Project root.
+ * @param {string} slug - Feature slug.
+ * @param {number} [round] - The round being compiled.
+ * @returns {Array<object>} `{criterion, severity, expected, actual, source:"criterion", scope_id?}`
+ *   per uncovered FAIL criterion; [] for round 1, a PASS, or an unreadable result.
+ */
+export function criteriaBugs(cwd, slug, round) {
+  if (!round || round < 2) return [];
+  const p = join(resultsDir(cwd, slug), `evaluate-r${round - 1}.json`);
+  if (!existsSync(p)) return [];
+  let v;
+  try { v = JSON.parse(readFileSync(p, "utf8"))?.verdict; } catch { return []; }
+  if (v?.overall !== "FAIL" || !Array.isArray(v.criteria)) return [];
+  const filed = (Array.isArray(v.bugs) ? v.bugs : []).map((b) => String(b?.criterion ?? ""));
+  const refuted = new Set((Array.isArray(v.refuted) ? v.refuted : [])
+    .flatMap((r) => [r?.id, r?.ac_id, r?.criterion, typeof r === "string" ? r : null]).filter(Boolean).map(String));
+  const owners = new Map();
+  for (const { contract, id } of readAllContracts(scopesDir(cwd, slug))) {
+    const sid = contract?.scope_id || id;
+    for (const uc of Array.isArray(contract?.use_cases) ? contract.use_cases : []) if (!owners.has(uc)) owners.set(uc, sid);
+  }
+  const out = [];
+  for (const c of v.criteria) {
+    if (c?.verdict !== "FAIL" || typeof c.criterion !== "string") continue;
+    const name = c.criterion;
+    if (refuted.has(name)) continue;
+    if (filed.some((f) => f && (f === name || f.includes(name) || name.includes(f)))) continue;
+    const uc = (name.match(/\bUC-[A-Za-z0-9_-]+/) || [])[0];
+    const owner = uc ? owners.get(uc) : undefined;
+    out.push({
+      criterion: name,
+      severity: "major",
+      expected: "the criterion is met, with evidence the judge can cite",
+      actual: String(c.evidence ?? "graded FAIL").slice(0, 400),
+      source: "criterion",
+      ...(owner ? { scope_id: owner } : {}),
+    });
+  }
+  return out;
+}
+
 /**
  * The previous round's RED BUILD GATE, as bug entries the fix round can act on.
  *
@@ -748,6 +799,38 @@ export function t0ArtifactsFor(cwd, slug, round) {
   return { artifacts, missing };
 }
 
+// --- checks a scope rewrote between a failing trial and a passing one --------------------------
+
+/**
+ * Every revised check any trial of the round recorded, with the scope it belongs to.
+ *
+ * Read across all of the round's trials rather than off the cited verdict alone: a check rewritten
+ * on attempt 2 and re-run unchanged on attempt 3 leaves no mark on attempt 3's verdict, which is the
+ * one the judge cites.
+ *
+ * @param {string} cwd - Project root.
+ * @param {string} slug - Feature slug.
+ * @param {number} [round] - The round being evaluated; omitted, every round.
+ * @returns {Array<{scope_id:string, id:string, file:string}>} Deduplicated by scope, id and file.
+ */
+export function revisedChecksFor(cwd, slug, round) {
+  const seen = new Set();
+  const out = [];
+  for (const tr of readTrials(trials(cwd, slug))) {
+    if (round != null && tr.round !== round) continue;
+    if (!tr.artifact) continue;
+    let v;
+    try { v = JSON.parse(readFileSync(join(localRoot(cwd, slug), tr.artifact), "utf8")); } catch { continue; }
+    for (const r of Array.isArray(v?.revised_checks) ? v.revised_checks : []) {
+      const key = `${tr.scope_id}\u0000${r.id}\u0000${r.file}`;
+      if (seen.has(key)) continue;
+      seen.add(key);
+      out.push({ scope_id: tr.scope_id, id: r.id, file: r.file });
+    }
+  }
+  return out;
+}
+
 // --- the launch evidence the judge grades `[ui]` rows against ---------------------------------
 //
 // WHY THE KERNEL DERIVES IT. A `[ui]` criterion is graded on the RUNNING app, and the evaluator's
@@ -1016,7 +1099,7 @@ export async function cli(rawArgv) {
   // this line, and none of them can pass a payload to a build order (see the banner above).
   // Two sources, one channel: the judge's cited defects and the build gate's failing steps.
   const bugs = scope
-    ? bugsForScope([...verdictBugs(cwd, slug, round), ...buildBugs(cwd, slug, round)], scope.scope_id, scopeSubstrates(cwd, slug))
+    ? bugsForScope([...verdictBugs(cwd, slug, round), ...criteriaBugs(cwd, slug, round), ...buildBugs(cwd, slug, round)], scope.scope_id, scopeSubstrates(cwd, slug))
     : [];
 
   // A ROUND CARRYING CITED DEFECTS IS A `fix`, AND THE ORDER HAS TO SAY SO.
@@ -1138,6 +1221,10 @@ export async function cli(rawArgv) {
   }
   // The launch evidence, for every lane (see launchEvidenceFor). Same rule as above: an explicit
   // `--payload` value outranks the derivation.
+  if (operation === "evaluate" && payloadExtra.revised_checks === undefined) {
+    const revised = revisedChecksFor(cwd, slug, round);
+    if (revised.length) payloadExtra.revised_checks = revised;
+  }
   if (operation === "evaluate") {
     const ev = launchEvidenceFor(cwd, slug, round);
     if (ev.build_gate !== undefined && payloadExtra.build_gate === undefined) payloadExtra.build_gate = ev.build_gate;

package/kernel/probe/digest.mjs

@@ -33,6 +33,11 @@ const PATTERNS = [
   // ("ERROR in the build pipeline", "ERROR in test suite failed to run") is left unmatched
   // instead of handing back a fabricated file.
   { re: /^(?:ERROR|WARNING)\s+in\s+(\.{1,2}\/[^\s:]*|[^\s:]+\.[A-Za-z0-9]{1,10})\b/i, kind: "compiler-diagnostic" },
+  // hvigor / ArkTS compiler: the message and the location arrive on ONE line,
+  //   "Error Message: Expected 5 arguments, but got 3. At File: /abs/path/Foo.test.ets:75:33"
+  // — the commonest failure on that toolchain, and one no other pattern here anchored, so every red
+  // compile handed the next attempt an empty error list.
+  { re: /^\s*Error Message:\s*(.+?)\s+At File:\s*(.+?):(\d+):\d+\s*$/, kind: "arkts-compiler" },
   // A test that FAILED BY NAME, with no file:line: "FAIL TS-05-05 step 4: no text 'Bread' on screen"
   // or jest's "FAIL src/cart.test.js". Runners that drive an app from outside it (a device flow, an
   // end-to-end script) report a case this way and nothing else, and the line is the whole signal —
@@ -77,6 +82,11 @@ export function digest(rawText) {
         pendingMessage = coreMessage(m[1]);
         continue; // wait for the stack frame that follows to get a file:line
       }
+      if (kind === "arkts-compiler") {
+        triples.push({ file: m[2].trim(), line: Number(m[3]), core_message: coreMessage(m[1]), kind });
+        pendingMessage = null;
+        break;
+      }
       const named = kind === "named-test-failure";
       const file = named ? (/[\\/]|\.[A-Za-z0-9]{1,10}$/.test(m[1]) ? m[1] : null) : m[1]?.trim();
       const lineNo = !named && m[2] ? Number(m[2]) : null;

package/kernel/schemas/domain.schema.json

@@ -362,6 +362,7 @@
       "run_cmd",
       "launch_cmd",
       "build_gate",
+      "revised_checks",
       "t0_artifacts",
       "browser",
       "tasks"
@@ -2429,6 +2430,19 @@
           "type": "string",
           "description": "spec-evaluator: the project profile's launch probe — installs the built artifact, starts it and asserts the first screen. Derived by `harness compile` from project-profile.md, absent when the profile declares none. Where `run_cmd` is only a build, this is how the app is brought up; a non-zero exit is a finding, not a reason to guess another way."
         },
+        "revised_checks": {
+          "type": "array",
+          "items": {
+            "type": "object",
+            "properties": {
+              "scope_id": { "type": "string" },
+              "id": { "type": "string" },
+              "file": { "type": "string" }
+            },
+            "required": ["id", "file"]
+          },
+          "description": "spec-evaluator: rows that FAILed in one of the round's T0 trials and PASS in a later one whose own check file (named by the row id) changed in between. Derived by harness compile. Each must be read against its row before its PASS counts; absent when none."
+        },
         "build_gate": {
           "type": "string",
           "description": "spec-evaluator: this run's newest round build gate artifact (build/r<N>-t<T>.json) — each step's exit code and output tail, the record that the build ran and the app launched. Derived by `harness compile`, absent when the gate never ran. Read it before grading a [ui] row NO EVIDENCE."

package/kernel/verify/t0.mjs

@@ -34,8 +34,8 @@
 //
 // Exit code: 0 = overall green, 1 = overall red (mirrors the oracle convention), 2 = bad argv.
 
-import { readFileSync, writeFileSync, appendFileSync, mkdirSync, existsSync, readdirSync } from "node:fs";
-import { join, dirname } from "node:path";
+import { readFileSync, writeFileSync, appendFileSync, mkdirSync, existsSync, readdirSync, statSync } from "node:fs";
+import { join, dirname, basename, extname, resolve as resolvePath, relative } from "node:path";
 import { spawnSync } from "node:child_process";
 import { createHash } from "node:crypto";
 import { digest } from "../probe/digest.mjs";
@@ -389,6 +389,97 @@ export function nextTrialNo(dir, round, attempt) {
   return max + 1;
 }
 
+// --- a check rewritten to pass ----------------------------------------------------------------
+//
+// A fixture that names Test Surface rows (`PASS TS-02-04`, `FAIL TS-02-04 step 10: …`) reads its
+// checks from files the scope itself writes. Measured on a live run: a row FAILed on attempt 1 —
+// the check renamed a list and found the old name still on screen — and went green on attempt 2
+// because the check had been rewritten to stop renaming, while the code was unchanged. Nothing
+// recorded that the evidence had moved. So each verdict records the checks it read, by digest, and
+// the named results they printed; a row that failed in this scope's previous trial and passes now,
+// whose own check file changed in between, is a REVISED check — a pass the judge must read against
+// its row before it can count. Recorded, never refused: rewriting a check that overreached is
+// legitimate, and only a person or the judge can tell which it was.
+
+/**
+ * The files a fixture's arguments name, with their digests — a named directory contributes its
+ * direct children. Paths resolve against the project root and are recorded relative to it.
+ *
+ * @param {string[]} commands - The fixture command lines.
+ * @param {string} cwd - Project root.
+ * @returns {Object<string,string>} Relative path → sha256 of its bytes. Empty when no argument names
+ *   an existing path.
+ */
+export function checkFiles(commands, cwd) {
+  const out = {};
+  /**
+   * Record one file's digest under its project-relative path.
+   * @param {string} abs - Absolute path of the file.
+   * @returns {void} Nothing; an unreadable file is skipped, since it is not evidence.
+   */
+  const add = (abs) => {
+    try { out[relative(cwd, abs)] = createHash("sha256").update(readFileSync(abs)).digest("hex"); } catch { /* unreadable — not evidence */ }
+  };
+  for (const cmd of commands || []) {
+    const tokens = String(cmd).match(/"[^"]*"|'[^']*'|\S+/g) || [];
+    for (const raw of tokens.slice(1)) {
+      const tok = raw.replace(/^["']|["']$/g, "");
+      if (!tok || tok.startsWith("-")) continue;
+      const abs = resolvePath(cwd, tok);
+      let st; try { st = statSync(abs); } catch { continue; }
+      if (st.isFile()) add(abs);
+      else if (st.isDirectory()) {
+        for (const f of readdirSync(abs).sort()) {
+          const child = join(abs, f);
+          try { if (statSync(child).isFile()) add(child); } catch { /* skip */ }
+        }
+      }
+    }
+  }
+  return out;
+}
+
+/**
+ * The rows a fixture's output names, and how each came out.
+ *
+ * @param {Array<{stdout?:string}>} results - Fixture results with their full stdout.
+ * @returns {Object<string,("PASS"|"FAIL")>} Row id → its result; a FAIL anywhere wins over a PASS.
+ */
+export function namedResults(results) {
+  const out = {};
+  for (const r of results || []) {
+    for (const line of String(r?.stdout || "").split(/\r?\n/)) {
+      const m = line.match(/^(PASS|FAIL)\s+(\S+)/);
+      if (!m) continue;
+      if (m[1] === "FAIL" || out[m[2]] !== "FAIL") out[m[2]] = m[1];
+    }
+  }
+  return out;
+}
+
+/**
+ * Rows that failed in the previous trial and pass now while their own check file changed.
+ *
+ * A row's check is the file whose name, without extension, is the row id — the convention a
+ * per-row check follows. A row with no such file is not reported: nothing can be said about it.
+ *
+ * @param {({check_files?:Object<string,string>, named_results?:Object<string,string>}|null)} prev
+ *   The scope's previous verdict in this round, or null.
+ * @param {{check_files:Object<string,string>, named_results:Object<string,string>}} curr - This one.
+ * @returns {Array<{id:string, file:string}>} One entry per revised check.
+ */
+export function revisedChecks(prev, curr) {
+  if (!prev?.named_results || !prev?.check_files) return [];
+  const out = [];
+  for (const [id, now] of Object.entries(curr.named_results || {})) {
+    if (now !== "PASS" || prev.named_results[id] !== "FAIL") continue;
+    const file = Object.keys(curr.check_files || {}).find((f) => basename(f, extname(f)) === id);
+    if (!file) continue;
+    if (prev.check_files[file] !== curr.check_files[file]) out.push({ id, file });
+  }
+  return out;
+}
+
 /**
  * Distill every failing command's output into AEGIS {file,line,core_message} triples.
  * @param {{fixtures:{results:Array<{pass:boolean,stdout:string,stderr:string}>},
@@ -514,6 +605,10 @@ export async function cli(rawArgv) {
   const dbProbe = runDbProbe(contract.db_probe, cwd);
   const verdict = computeVerdict({ fixtures, dbProbe });
   const discovered = verdict.overall === "red" ? digestFailures({ fixtures, dbProbe }) : [];
+  const checks = {
+    check_files: checkFiles(fixtures.results.map((r) => r.cmd), cwd),
+    named_results: namedResults(fixtures.results),
+  };
 
   // ---- the ratchet ---------------------------------------------------------------------
   // `current` is the incumbent: the score of the most recent trial whose TREE is the one on disk
@@ -526,6 +621,11 @@ export async function cli(rawArgv) {
   const verdictBetter = better(s, baseline ? baseline.score : null);
   const crashed = fixtures.results.some((r) => r.error) || !!dbProbe?.error;
   const { status, action } = decideStatus(verdictBetter, crashed);
+  // The scope's previous trial in this round, read back for the checks it recorded.
+  const prevTrial = [...priorTrials].reverse().find((tr) => tr.round === round);
+  let prevVerdict = null;
+  if (prevTrial?.artifact) { try { prevVerdict = JSON.parse(readFileSync(join(outDir, prevTrial.artifact), "utf8")); } catch { /* none */ } }
+  const revised = revisedChecks(prevVerdict, checks);
 
   // The run key, read from the receipt that lives in the run root this script was pointed at.
   // `--out` IS that root, so identity comes from the receipt rather than from parsing a slug back
@@ -550,6 +650,9 @@ export async function cli(rawArgv) {
     ...verdict,
     score: s,
     discovered_tasks: discovered,
+    ...(Object.keys(checks.check_files).length ? { check_files: checks.check_files } : {}),
+    ...(Object.keys(checks.named_results).length ? { named_results: checks.named_results } : {}),
+    ...(revised.length ? { revised_checks: revised } : {}),
   });
 
   // The tree operation. `--no-ratchet` leaves the working tree exactly as the attempt left it —

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "shapeup-sdlc",
-  "version": "3.13.0",
+  "version": "3.14.0",
   "description": "Shape Up for coding agents \u2014 with gates the agent can't talk its way past. Harness for Claude Code.",
   "bin": {
     "shapeup-sdlc": "bin/init.mjs"

package/skills/spec-evaluator/SKILL.md

@@ -41,6 +41,7 @@ Invoked as `--order <path>`. Fields you may rely on (absent = unknown, never inf
 | `payload.run_cmd` | How to start the running app. On a stack where building and launching are different acts it is only the build — prefer `payload.launch_cmd` when the order carries one. Absent standalone → ask; absent orchestrated with no `launch_cmd` either → ESCALATE, do not guess |
 | `payload.launch_cmd` | The project profile's launch probe: installs the built artifact, starts it and asserts the first screen. This is how the app is brought up for `[ui]` probing. A non-zero exit is a finding to cite, not a reason to try another way. Absent → the profile declares none; fall back to `run_cmd` |
 | `payload.build_gate` | This run's newest round build gate artifact — each step's exit code and output tail. It records that the build ran and whether the app launched, and the launch step's output names what it captured. Read it before grading any `[ui]` row NO EVIDENCE: a launch that succeeded is evidence the app can be probed, and the row is graded on the app, not on its absence. Absent → the gate never ran |
+| `payload.revised_checks[]` | Rows that FAILed in one trial of this round and PASS in a later one whose own check file changed in between — a pass obtained by editing the check. Read each listed file against its row's Expect before you let its PASS count; a check that no longer asserts what the row asks makes that row a FAIL, and the bug names the check. Absent → no check was rewritten |
 | `payload.t0_artifacts[]` | Per-scope T0 verdict paths for this round (scoped specs), compiled from each scope's green verdict. An artifact listed but missing/red on disk, or a scoped spec with none listed → the round is NOT gradeable: return `status: failed` with the reason, naming the scope, as your FIRST deviation — a structural precondition, not a criterion |
 | `payload.browser` | `cli` (default, ~4x cheaper) \| `mcp` \| `none` |
 | `payload.tasks[]` | Traceability only (which UCs a task claims): NEVER a grading source — the committed UC text is the criterion, a paraphrase mismatch is a finding |