@mmerterden/multi-agent-pipeline 16.2.2 → 16.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -34,11 +34,22 @@ node $HOME/.claude/scripts/evidence-gate.mjs --claim test --status passed --evi
34
34
 
35
35
  This prevents a false "it built" claim with no log behind it. On exit 1, treat the gate as failed (do NOT proceed to AI review) and surface the gate's `reason`.
36
36
 
37
+ **Inherited failures (when `state.baseline.tests` exists).** Phase 0 Step 7.6 recorded whether the suite was already red, so Gate 3 blocks on what this work broke, not what it walked into:
38
+
39
+ | baseline status | Gate 3 |
40
+ |---|---|
41
+ | `green` | unchanged; every failure is this run's |
42
+ | `red` + `failing[]` | subtract those ids. Nothing left -> pass, logged `test:pass (inherited {N})`. A NEW failure still blocks. |
43
+ | `red`, empty `failing[]` | do NOT pass and do NOT silently block: report `test:inherited-red (not attributable, <logPath>)` and ask. Inventing a set here masks regressions. |
44
+ | `unknown` / absent | unchanged from today |
45
+
46
+ The subtraction never widens: match on identifier only, and when identifiers cannot be compared fall to the `not attributable` row.
47
+
37
48
  **Gate results:**
38
49
 
39
50
  - All pass (including the evidence gate) -> proceed to AI review
40
51
  - Any fail -> fix immediately, re-run gates (no AI review until clean)
41
- Log: "Phase 4: Gates - build:{pass/fail} lint:{pass/fail} test:{pass/fail} secrets:{clean/found} evidence:{ok/unverified}"
52
+ Log: "Phase 4: Gates - build:{pass/fail} lint:{pass/fail} test:{pass/fail/inherited-red} secrets:{clean/found} evidence:{ok/unverified}"
42
53
 
43
54
  ##### Gate 5 - Fortify SSC findings (runs when `state.contextLinks[]` contains a `fortify` entry, or when `prefs.global.fortify.alwaysCheck === true`)
44
55
 
@@ -374,7 +385,7 @@ Exit 0 = valid. Exit 2 = contradiction (approved=true with blocking findings) -
374
385
  1. Compute disagreement: reviewers agree iff all return `approved=true` with no `blocking` findings, OR all return `approved=false` with overlapping `blocking` findings. Anything else is disagreement.
375
386
  2. Agreement → skip the rebuttal round, go straight to Step 3 triage.
376
387
  3. Disagreement → one rebuttal round:
377
- - For each reviewer, re-prompt with: (a) their original output, (b) the OTHER reviewers' blocker findings verbatim, (c) instruction: *"Given the opposing arguments, keep / withdraw / modify each of your findings. You may also newly agree with a finding you previously missed. Return the SAME JSON schema - this is a revision, not a new review."*
388
+ - For each reviewer, re-prompt with: (a) their original output, (b) the OTHER reviewers' blocker findings **anonymized** through `node $HOME/.claude/scripts/anonymize-findings.mjs` (labels `Source A/B/C`, no model name, order deterministic per `taskId:iteration`), (c) instruction: *"Given the opposing arguments, keep / withdraw / modify each of your findings. You may also newly agree with a finding you previously missed. Return the SAME JSON schema - this is a revision, not a new review."*
378
389
  - Launch all reviewers in parallel (same CLI-aware set as Step 2).
379
390
  - Max one round. Results replace the original outputs.
380
391
  4. Proceed to Step 3 triage with the round-2 outputs.
@@ -393,15 +404,27 @@ Optional: when `ai-analyst-toolkit` is enabled and a finding blames a third-part
393
404
 
394
405
  Opt-in empirical layer: when `prefs.global.verifyByTest.enabled` is `true`, accepted blocking findings additionally go through Step 3.7 (verify-by-test), which tries to reproduce each one with a minimal failing test before the Phase 3 rework loop fires. Full wiring: `$HOME/.claude/multi-agent-refs/features/verify-by-test.md`.
395
406
 
396
- ##### 3.0 Merge the deterministic findings in
407
+ ##### 3.0 Anonymize the reviewer findings, then merge the deterministic ones
408
+
409
+ **Anonymize first (required).** On both CLIs the triage model is also a reviewer (Fable on Claude Code, Opus on Copilot), and a judge that can see which findings are its own is marking its own homework:
397
410
 
398
- Append the Step 1.76 test-integrity findings to the reviewer findings before counting, so they are adjudicated like any reviewer finding rather than resolved by triage never seeing them:
411
+ ```bash
412
+ ANON=$(jq -n --argjson r "$REVIEWERS_JSON" --arg t "$TASK_ID" --argjson i "$ITERATION" \
413
+ '{taskId: $t, iteration: $i, reviewers: $r}' \
414
+ | node "$HOME/.claude/scripts/anonymize-findings.mjs" --map "/tmp/review-$TASK_ID-$ITERATION-map.json")
415
+ ```
416
+
417
+ `$REVIEWERS_JSON` is `state.reviewIterations[i].reviewers`. Findings come back with `foundBy: "Source A|B|C"` and every identity key removed. Persist the map to `state.reviewIterations[i].anonymizationMap` for Phase 7 per-reviewer telemetry, and **never put the map in a prompt**.
418
+
419
+ Then append the Step 1.76 test-integrity findings, so they are adjudicated rather than never seen:
399
420
 
400
421
  ```bash
401
422
  MERGED=$(jq -s '.[0] + (.[1].findings // [])' \
402
- <(printf '%s' "$REVIEWER_FINDINGS_JSON") <(printf '%s' "${TEST_INTEGRITY_JSON:-{\}}"))
423
+ <(printf '%s' "$ANON") <(printf '%s' "${TEST_INTEGRITY_JSON:-{\}}"))
403
424
  ```
404
425
 
426
+ Deterministic findings keep `tag: test_integrity` and carry no `foundBy`: a reviewer finding may be a hallucination, a gate finding is a fact.
427
+
405
428
  ##### 3.1 Short-circuit: no findings
406
429
 
407
430
  If **merged** findings `length === 0`, **skip triage**: write empty result `{"accepted": [], "deferred": [], "rejected": [], "approved": true}`, log, proceed to Phase 5. Note this is the merged count from 3.0: a run with zero reviewer findings but a non-empty test-integrity set must NOT short-circuit.
@@ -410,7 +433,7 @@ If **merged** findings `length === 0`, **skip triage**: write empty result `{"ac
410
433
 
411
434
  Launch **1 Agent** (subagent_type: `general-purpose`, model: `fable` on Claude Code / `opus` on Copilot CLI) with:
412
435
 
413
- - Raw findings from Reviewer 1 + Reviewer 2 (merged JSON)
436
+ - The anonymized merged findings from 3.0 (`Source A/B/C` labels; no model name anywhere in the prompt)
414
437
  - Task scope (Phase 1 analysis summary + Phase 2 plan)
415
438
  - Full diff being reviewed
416
439
  - **Prior-art context (advisory)** - per raw finding, `triage-memory.mjs query --top <prefs.global.priorArtEnrichment.topN>` (default 3). Pass `--top`: without it the script falls back to `memoryRecall.maxResults`, a different concern, and `topN` silently does nothing. Off when `priorArtEnrichment.enabled = false`.
@@ -27,6 +27,24 @@ Phase 6 consumes the latest Phase 4 triage output object conforming to `$HOME/.c
27
27
 
28
28
  The "Build passes" checklist item below is evidence-gated, not self-asserted: Phase 6 trusts the `buildStatus.ok` that Phase 3 / Phase 4 recorded through `$HOME/.claude/scripts/evidence-gate.mjs` (a pass without a substantiating build/test log is treated as unverified and blocks the commit). The secret scanner (`pre-commit-check.sh`) runs on the staged diff as the final pre-commit gate.
29
29
 
30
+ #### Step 0a - Plan coverage gate (BLOCKING)
31
+
32
+ Phase 4 answers whether the diff is correct, not whether everything the plan promised landed: the criteria manifest counts rule IDs, Step 1.45 counts planned tests, and the per-step rollup is rendered in Phase 7, after this commit.
33
+
34
+ ```bash
35
+ # One --analysis per doc: state.analysis.docPath[] is an array (one per platform),
36
+ # and reading only the first reports a one-platform verdict as the whole run's.
37
+ ANALYSIS_ARGS=()
38
+ while IFS= read -r d; do [ -n "$d" ] && ANALYSIS_ARGS+=(--analysis "$d"); done \
39
+ < <(jq -r '.analysis.docPath[]? // empty' "$STATE_FILE")
40
+ node "$HOME/.claude/scripts/plan-coverage-gate.mjs" --state "$STATE_FILE" \
41
+ ${ANALYSIS_ARGS[@]+"${ANALYSIS_ARGS[@]}"} --root "$WORKTREE" || COVERAGE_GAP=1
42
+ ```
43
+
44
+ A step is accounted for when it is `completed`, `skipped` + `skipReason`, or `failed` + `failureReason`; a bare `pending` / `in_progress` is what this catches. Every Section 14 row tagged `Add new` must name a file that exists in the tree, and an empty `docPath[]` leaves that half reporting `skipped`, never `passed`.
45
+
46
+ On exit 1: show the gate's list verbatim, then either return to Phase 3 or have the user mark each step deliberately (`plan-todos.sh` writes the reason). Never proceed without that decision. Exit 2 means the plan was missing or empty: modes without Phase 2 skip this step explicitly.
47
+
30
48
  #### Step 0 - Multi-Repo Integration Build
31
49
 
32
50
  **Fires only when** `state.projects.length >= 2` (multi-repo task). Single-repo tasks skip this step silently - no prompt, no overhead.
@@ -58,7 +58,7 @@ default: option 1
58
58
  Comment body: a short "Multi-agent readiness review" heading, the score + verdict, then the gap list grouped Blockers / Warnings / Gaps, each with a one-line fix suggestion. Tone rules (same as `channels/issue-comment.md`): no AI/Claude/Copilot attribution, no "generated by", no em-dash/section-sign, plain ASCII; if referencing another item use `Ref: #N` never `Closes/Fixes`. READY items get a short "ready to pick up" confirmation instead of a gap list.
59
59
 
60
60
  Provider dispatch:
61
- - **Jira** (`review-jira`): post via `$HOME/.claude/multi-agent-refs/channels/jira.md` comment path - `POST /rest/api/2/issue/{id}/comment`, Bearer token from `keychainMapping.jira` via `credential-store.sh`, UTF-8 verbatim (`jq --rawfile` + `curl --data-binary @payload.json`), markdown->wiki conversion.
61
+ - **Jira** (`review-jira`): post via `$HOME/.claude/multi-agent-refs/channels/jira.md` comment path - `POST /rest/api/2/issue/{id}/comment`, Bearer token from `keychainMapping.jira` via `credential-store.sh`, UTF-8 verbatim (`jq --rawfile` + `curl --data-binary @payload.json`), markdown->wiki conversion, then `node "$HOME/.claude/scripts/jira-wiki-escape.mjs"` on the converted body before POST.
62
62
  - **GitHub** (`review-issue`): `gh issue comment "$N" --repo "$org/$repo" --body-file <file>` per `channels/issue-comment.md` (auth via the Phase 0 `gh` account).
63
63
 
64
64
  On post failure, surface the failing endpoint on stderr; the chat verdict remains the source of truth.
@@ -522,6 +522,42 @@
522
522
  }
523
523
  }
524
524
  },
525
+ "baseline": {
526
+ "type": "object",
527
+ "additionalProperties": true,
528
+ "description": "Pre-work state of the repo, captured in Phase 0. Exists so Phase 4 can tell an inherited failure from one this run caused.",
529
+ "properties": {
530
+ "tests": {
531
+ "type": "object",
532
+ "additionalProperties": true,
533
+ "description": "Outcome of the Phase 0 baseline test run (gated by prefs.global.testBaseline.enabled). status is three-valued on purpose: collapsing unknown into green would let a skipped baseline read as a clean tree, which is the failure this whole record exists to prevent.",
534
+ "properties": {
535
+ "status": {
536
+ "type": "string",
537
+ "enum": ["green", "red", "unknown"],
538
+ "description": "green = the suite passed before any change. red = it did not. unknown = no test command, the time cap was hit, or the baseline was disabled."
539
+ },
540
+ "failing": {
541
+ "type": "array",
542
+ "items": { "type": "string" },
543
+ "description": "Identifiers of tests already failing before this run. Empty on a red status means the output could not be parsed into names: the evidence is then the logPath alone, and Phase 4 reports 'inherited red, not attributable' rather than inventing a set."
544
+ },
545
+ "logPath": {
546
+ "type": "string",
547
+ "description": "Path to the tee'd baseline log. The evidence behind the status; cited when failing[] could not be parsed."
548
+ },
549
+ "capturedAt": {
550
+ "type": "string",
551
+ "format": "date-time"
552
+ },
553
+ "command": {
554
+ "type": "string",
555
+ "description": "The exact test command that produced this baseline. Phase 4 compares against its own Gate 3 command and treats a mismatch as unknown."
556
+ }
557
+ }
558
+ }
559
+ }
560
+ },
525
561
  "reviewIterations": {
526
562
  "type": "array",
527
563
  "items": {
@@ -550,7 +586,43 @@
550
586
  "enum": ["fix", "accept", "escalate"]
551
587
  },
552
588
  "reviewers": {
553
- "type": "array"
589
+ "type": "array",
590
+ "description": "One entry per reviewer dispatch that RETURNED. Typed because two consumers depend on the shape: anonymize-findings.mjs needs model+findings to build the label map, and run-metrics.mjs reports acceptedRatio per reviewer. Extra keys are allowed; nothing is required, so a run written before this shape existed still validates and surfaces as model \"unknown\" rather than failing.",
591
+ "items": {
592
+ "type": "object",
593
+ "additionalProperties": true,
594
+ "properties": {
595
+ "model": {
596
+ "type": "string",
597
+ "description": "The model that produced this review (fable | sonnet | opus | gpt-*). Absent is not the same as unknown-by-name: an entry with no model is reported as \"unknown\" in the per-reviewer metric instead of being folded into another reviewer's count."
598
+ },
599
+ "findings": {
600
+ "type": "array",
601
+ "description": "Raw findings from this reviewer, before triage."
602
+ },
603
+ "roundCount": {
604
+ "type": "integer",
605
+ "minimum": 1,
606
+ "description": "1 normally, 2 when the Step 2.5 rebuttal round replaced this reviewer's output."
607
+ }
608
+ }
609
+ }
610
+ },
611
+ "anonymizationMap": {
612
+ "type": "object",
613
+ "additionalProperties": true,
614
+ "description": "Written by Phase 4 Step 3.0 from anonymize-findings.mjs --map, read by run-metrics.mjs to attribute accepted findings back to a reviewer. Declared because the phase doc names it and a consumer reads it; absent on runs from before anonymization existed, which run-metrics reports as perReviewerAttribution \"unavailable\" rather than as a zero. Never goes into a prompt: it is the mapping the anonymization exists to withhold.",
615
+ "properties": {
616
+ "seed": {
617
+ "type": "string",
618
+ "description": "taskId:iteration - the seed that produced the finding order, so a resume reproduces it."
619
+ },
620
+ "labelToModel": {
621
+ "type": "object",
622
+ "additionalProperties": { "type": "string" },
623
+ "description": "Source A|B|C -> model name. \"unknown\" for a reviewer entry that declared no model."
624
+ }
625
+ }
554
626
  },
555
627
  "triage": {
556
628
  "type": "object"
@@ -797,6 +797,25 @@
797
797
  }
798
798
  }
799
799
  },
800
+ "testBaseline": {
801
+ "type": "object",
802
+ "additionalProperties": false,
803
+ "description": "Phase 0 test baseline. When enabled, Phase 0 runs the SAME test command Phase 4 Gate 3 uses and records which tests were already failing before this run touched anything, so Phase 4 stops attributing an inherited red suite to the current work. Three outcomes are stored, never two: green, red (with the failing set when it can be parsed, otherwise the log path alone), or unknown when the command is absent or the time cap was hit. Off by default because on iOS the run costs a full xcodebuild test before any work starts. Pattern source: obra/superpowers using-git-worktrees Step 3 'Verify Clean Baseline', extended from ask-the-user to a stored set Phase 4 can subtract.",
804
+ "properties": {
805
+ "enabled": {
806
+ "type": "boolean",
807
+ "default": false,
808
+ "description": "Master switch. Off by default - a baseline run costs one full test suite before Phase 3 starts."
809
+ },
810
+ "timeoutSeconds": {
811
+ "type": "integer",
812
+ "minimum": 30,
813
+ "maximum": 3600,
814
+ "default": 600,
815
+ "description": "Hard cap on the baseline run. Exceeding it records status unknown rather than a partial failing set: a truncated suite would look like passing tests that never ran."
816
+ }
817
+ }
818
+ },
800
819
  "reviewDisagreementRound": {
801
820
  "type": "boolean",
802
821
  "default": false,
@@ -36,6 +36,6 @@
36
36
  "warn_tokens": 5600
37
37
  }
38
38
  },
39
- "total_max_tokens": 56600,
39
+ "total_max_tokens": 57600,
40
40
  "note": "Token estimate = ceil(chars / 4). Per-phase budget rule: warn = current+10% (rounded to nearest 50), max = current+25%. Gives ~6 edit cycles of headroom before warn trips - intentionally quiet under normal maintenance, loud when a phase grows unusually. Only the active phase is loaded (lazy). Recalibrated at v10.0.0 after the validator/consistency/simplifier/lesson gate contracts landed in phases 1-4. Recalibrated again at v10.9.0 after the verify-by-test (Phase 4 Step 3.7), update-check (Phase 0 Step 0.6), immutable-test (Phase 3 GREEN) and redTests re-entry contracts landed - Step 3.7 prose was compressed to a pointer into refs/features/verify-by-test.md before the recalibration. Total bumped 50000 -> 51000 at v12.5.0 after the worktree residue/traversal-prune contract (Phase 0 + Phase 5 heal) and the Reflexion causal-diagnosis contract (Phase 4 lesson memory) landed; the prose was compressed first (161 tokens reclaimed) and every per-phase max still passes - only the aggregate needed room. Recalibrated again at v13.6.0 after the install-relative path correction: an instruction that names `pipeline/scripts/x` resolves only from a repo checkout, and a run happens in the user's worktree, so 157 references across these docs moved to `$HOME/.claude/...` at +5 bytes each - 196 tokens of pure correctness cost. Same discipline as before: prose was compressed FIRST (149 tokens reclaimed, by pointing Phase 1's Figma tier table at the Phase 0 probe that already resolved it and Phase 4's Codex constraints at the always-loaded AGENTS.md block), and only then were the budgets moved. Five warn lines had been permanently amber, which makes the amber tier useless as a signal, so every warn was reset to the documented current+10% and the four maxes that the new warn would have collided with were reset to current+25%. Aggregate 51000 -> 51500. Total bumped 51500 -> 52200 at v14.0.0 after Phase 4 Review entered the four --dev mode phase sets and the criteria-resolution contract (Step 1.78) landed. Same discipline as every prior bump: prose was compressed FIRST, 820 tokens reclaimed, before the number moved. Two of those compressions are structural rather than cosmetic - the hardcoded SwiftUI interaction list in Step 1.5 and the SwiftUI convention paragraph in Step 2.8 were transcriptions of rules that now live in a scoped registry, so keeping them here would have re-created the drift this release exists to remove, and the third moved the Step 1.78 full contract into refs/features/skill-conformance.md leaving a pointer. What remains is contract text that cannot be inferred: the manifest's four consumer-visible parts, the conformance checklist the reviewers must return, and the fail-closed semantics. Every per-phase max still passes (phase-4 12405/14750); only the aggregate needed room. Total bumped 52200 -> 52700 at v14.1.0 after two more contracts landed: stack skill routing (Phase 3 pre-flight step 9) and worktree finalize (Phase 6 step 9). Compression came first, as always, and twice: 224 tokens out of Phase 3 by pointing its criteria-ledger and routing steps at their feature files instead of restating them, and 190 out of Phase 6 by moving the finalize contract into refs/features/worktree-finalize.md and leaving the invocation plus the exit-3 semantics. Both new contracts follow the pattern the earlier ones set: the phase doc carries the call and the decision, the feature file carries the reasoning, and the feature files are outside this budget because it loops only the eight phase-N-* keys. Every per-phase max still passes (phase-3 7677/8950, phase-6 5223/6150 and both under warn); only the aggregate needed room. Total bumped 52700 -> 52750 for the Phase 0 Step 3 branch-persistence correction: the step wrote the legacy `projects[].branches` while the TTL filter two sections below read `global.recentBranches`, and both spots named a `{name, lastUsed}` shape the schema rejects (`branch` required, `additionalProperties: false`), so the recent-branch picker option could never populate and a literal implementation would have failed prefs validation. Naming the right target, the right key and the legacy field to avoid costs 41 tokens over the one line it replaces. Compression came first and was applied three times to the replacement text itself, from 120 tokens down to 66, by moving the rationale out of the phase doc entirely: the reasoning now lives where it is enforced, in the migrate-prefs carry-forward comment and the smoke-pref-migration f7 block, leaving the phase doc with only the instruction. 50 was the smallest step that clears it; phase-0-init sits at 10893/12400, far under its own max, so this is purely an aggregate ceiling. v15.0.0: total 52750 -> 53100, the stack-skill tables in phase-1/2/4 now carry plugin-namespaced names (ai-<stack>-toolkit:<skill>) - functional prefixes, ~170 tokens. v15.10.0: total 53350 -> 53950 for the memory-recall + context-offload contracts (Phase 1 two-block durable-knowledge injection and its telemetry, Phase 3 build-log offload pipe, Phase 4 ranked prior art, offload pipe and recall telemetry). Compression came first and twice, taking the new prose from 1168 tokens to 580: the reasoning behind the two blocks lives in multi-agent-refs/prompt-assembly.md and the reasoning behind the offload filter lives in the offload-ref.sh header, both outside this budget, so the phase docs carry only the call, the pref that gates it and the one fact an agent cannot infer - that the evidence gate still reads the whole build log, so offloading changes what is read, never what counts as a verified pass. Every per-phase max still passes (phase-3 7985/8950, phase-4 12997/14750); phase-3 and phase-4 crossed their warn lines and are left amber on purpose, because that is the signal that those two docs are the next ones needing structural compression rather than another bump. v15.13.0: total 53950 -> 54050 for the prefs-to-flag bridges. Five settings had shipped declared-but-inert: contextOffload.minLines and .tailLines (fixed in 15.11.0), learningsLedger.maxBriefEntries, and testGap.scanTree and .promoteSeverity - the last two declared in the schema AND implemented as flags in the scanner, with nothing in between reading the pref and passing the flag. Wiring three of them costs the phase docs 94 tokens, which is the wiring itself and not prose: two `--max` substitutions and a three-line GAP_FLAGS block. Compression came first and twice, as always: the rationale that would have sat in phase-5 now lives in the header of smoke-prefs-consumed.sh, the gate that makes this class fail a build instead of shipping, and a `--severity-promote` table row was dropped because the invocation above it now shows the flag and names the pref that triggers it, which the row did not. 100 was the smallest step that clears it. Every per-phase max still passes; phase-3 and phase-4 remain amber on purpose. v15.14.0: total 54050 -> 54400 for the supported-version gate. Phase 0 Step 0.6 stopped being purely advisory: a release can now publish an npm dist-tag `required` that names the oldest runnable version, and below it the run halts instead of nagging. What the phase doc has to carry is the part an agent cannot infer - the third stdout field, that the halt is identical in autopilot, and that the run must NOT continue on the freshly updated install because its docs were already loaded from the old version. Compression came first, as always, and took the new prose from 469 tokens to 337: the rationale for the floor, the exemption list, the fail-open rules and the `npm dist-tag add` recipe all moved to multi-agent-refs/rules.md \"Supported Version Gate\" (loaded by 25 commands, outside this budget) and to the header of require-supported-version.sh, leaving the phase doc with the call, the decision table and the halt. 350 was the smallest step that clears it. Every per-phase max still passes (phase-0-init 11230/12400); phase-3 and phase-4 remain amber on purpose. v15.17.0: total 54400 -> 54900 for the Phase 1 analysis-document step. Phase 2 and Phase 3 pre-flights had BLOCKED on `analysis/<feature>-<platform>.md` since v9.0.0 while nothing produced it, so a full run either aborted at Phase 2 or the model ignored its own BLOCKING contract; Step 4 is the producer. What the phase doc carries is only what cannot be inferred: the when-table (taskType x Figma reference), the four refs in load order, the two artefacts, and that the doc validator fails closed. Compression came first and took the step from 745 tokens to 497: the history of why the gap existed moved to the CHANGELOG, the per-ref one-line descriptions moved into the refs' own headers, and the autopilot carve-out collapsed to one clause. The 17.4k-token analysis engine itself is NOT in this budget - it moved out of commands/ into multi-agent-refs/analysis/{locked,evidence,synthesis,render}.md, loaded on demand, which also took analysis/SKILL.md from 18081 to 5974 tokens and retired its lint grace entry. 500 was the smallest step that clears it; phase-1-analysis sits at 4338/4600 and is amber on purpose, like phase-3 and phase-4. v15.18.0: total 54900 -> 55250 for analysis mode. Three phase docs gained a mode branch that cannot be inferred: Phase 4 reviews a document instead of a diff (validator, the one question reviewers answer, the open-question walk), and Phase 6 publishes instead of committing. Compression came first and was applied twice to the new prose and once to old: the Phase 4 branch went from 320 tokens to 180 and the Phase 6 branch from 190 to 120 by pointing at multi-agent-refs/analysis/{resolve,render}.md, which now hold the walks themselves, and the front-matter parse contract stopped being spelled out in both pre-flights. The analysis engine keeps leaving this budget rather than entering it: intake joined locked/evidence/synthesis/render/resolve in multi-agent-refs/analysis/, which is what let analysis/SKILL.md drop under the 6000 hard cap after its grace entry was retired. 350 was the smallest step that clears it; phase-4 and phase-6 are amber on purpose, as phase-1 and phase-3 already were. v15.20.0: total 55250 -> 55500 for the TDD bridge. Phase 3 pre-flight read the analysis doc's concept table and even said test method names come from it, while nothing read Section 15 - so the RED step invented tests and the analysis test matrix never reached development. Phase 3 step 5b now loads it into state.dev.testPlan[] and Phase 4 step 1.45 cross-checks every planned row against a real test, which is what turns \"analysis quality is output quality\" from a slogan into a finding. Compression came first on both blocks, 300 tokens down to 175, by dropping the enumerated failure modes to one line each and the rationale to one clause; the reasoning lives in the CHANGELOG. 250 was the smallest step that clears it. v15.21.0: total 55500 -> 55800 for the post-analysis confirmation. Phase 2 gained Step 0.9, the last human checkpoint before Phase 3: derived values are shown for confirmation and only Section 20 rows are asked, through the resolve engine that already exists in refs. It belongs here rather than Phase 4 because Phase 4 runs after development, where an answer arrives too late to change anything. Compression came first and twice, 430 tokens down to 250, by collapsing the derived-vs-asked explanation to one sentence each and moving the walk itself to multi-agent-refs/analysis/resolve.md, which Phase 4 and analysis-resolve already mount. 300 was the smallest step that clears it. v15.22.0: total 55800 -> 55900 for the analyst-toolkit hooks. Phase 1 Step 4 now names the two prefs that decide whether a document is produced at all and how deep it goes (forceFull, mode) - the first of those had shipped declared-but-inert and smoke-prefs-consumed caught it - and Phase 4 triage gained one clause: a finding that blames a third-party library asks evidence-github whether it is already open upstream, which turns it into a deferred item with a citation instead of Phase 3 rework on code that is not ours. Compression came first and three times, taking the new prose from 220 tokens to 110, and the Phase 1d evidence contract itself never entered this budget - it lives in multi-agent-refs/analysis/evidence.md beside the phases it belongs to. 100 was the smallest step that clears it, leaving 34 tokens of headroom. phase-4 stays amber and the debt named at v15.10.0 stands: it is the doc that needs structural compression rather than another bump, and the two candidates are the inline triage JSON shape and the 3.4 telemetry block, both of which restate something already authoritative elsewhere. v16.0.0: total 55900 -> 56350 for the depth picker. `--dev` and the four dev-* commands are gone; depth is Phase 0 Step 7.5, which costs phase-0-init a step it did not have. Compression came first and three times, taking the step from 530 tokens to 300: the question wording, the per-taskType recommendation and the mode tables all live in phases/modes.md (outside this budget), so the phase doc carries only what an agent cannot infer - that the step runs after Step 7 and why, who is exempt, that ASK_CHOICE_DEFAULT must be passed explicitly because ask-choice.sh takes the FIRST option on a non-TTY, and that Short flips the Phase 1/2 tiles late rather than pre-marking them. The phase-4 telemetry block named as compression debt at v15.22.0 was collapsed to an emit() helper (-27) and the four dev-* mode files left the tree entirely, but neither offsets a genuinely new phase step. 450 was the smallest step that clears it, leaving 119 tokens of headroom. phase-4 remains amber and its other named candidate, the inline triage JSON shape, was left alone on purpose: it is the prompt the triage agent is handed, not a restatement for readers. v16.2.0: total 56350 -> 56600 for the spec-freshness and reuse-tag contracts. Phase 3 step 3 had compared `state.run.lastAnalysisDigest` since it was written, against a key nothing ever set and that the state schema did not declare, so the staleness branch was unreachable and every run reported fresh by default. Phase 1 now persists the digest and a `base_commit` anchor, and step 3 gained the repo-drift half the digest cannot see: a reused document keeps a matching digest precisely because its evidence inputs did not change, while the code underneath it moved. The second contract is the Section 14 tag reaching development: Phase 2 carries it onto the todo as `sourceTag` and Phase 3 treats it as an instruction, which is what stops a Reuse row from being re-implemented. Compression came first and took the four additions from 380 tokens to 214, by moving every rationale clause out of the phase docs: why the commit anchor exists rather than a digest recomputation lives in this note and the CHANGELOG, and the schema descriptions carry the field semantics. The baseline had 9 tokens of headroom, so no addition of any size could have fit without a bump. 250 was the smallest step that clears it, leaving 45 tokens. phase-3 and phase-4 remain amber."
41
41
  }
@@ -1,10 +1,10 @@
1
1
  # Pipeline Scripts - Category Index
2
2
 
3
- The `pipeline/scripts/` directory holds 148 shell scripts + 45 Node.js `.mjs` scripts in a flat layout. v6.0.0 evaluated moving them into `smoke/`, `hooks/`, `runtime/` subdirs and decided against it: each path change would need to propagate through `package.json` globs, `install.js` deploy logic, npm scripts, sibling script calls, and CI workflow references - a 40+ file touch radius with high regression risk for minor polish value.
3
+ The `pipeline/scripts/` directory holds 186 shell scripts + 58 Node.js `.mjs` scripts in a flat layout. v6.0.0 evaluated moving them into `smoke/`, `hooks/`, `runtime/` subdirs and decided against it: each path change would need to propagate through `package.json` globs, `install.js` deploy logic, npm scripts, sibling script calls, and CI workflow references - a 40+ file touch radius with high regression risk for minor polish value.
4
4
 
5
5
  Instead, this README is a curated category index of the key scripts, not an exhaustive listing. Use it to navigate.
6
6
 
7
- ## Smoke scripts (118 files - `smoke-*.sh`)
7
+ ## Smoke scripts (151 files - `smoke-*.sh`)
8
8
 
9
9
  Validate contracts. Each emits `══ <name> smoke: N passed, M failed ══` on completion (the `smoke-personal-data.sh` + `smoke-token-budget.sh` suites use `PASS:` / `warnings:` variants). Run via `npm test` (chained) or individually.
10
10
 
@@ -0,0 +1,199 @@
1
+ #!/usr/bin/env node
2
+ // anonymize-findings.mjs - strip reviewer identity before a model judges findings.
3
+ //
4
+ // Phase 4 dispatches 2 reviewers on Claude Code (Fable + Sonnet) and 3 on
5
+ // Copilot CLI (GPT + Opus + Sonnet), then hands the merged findings to triage -
6
+ // Fable on Claude Code, Opus on Copilot. On both sides the triage model is also
7
+ // one of the reviewers, and Step 3.2 used to label the input "Reviewer 1 +
8
+ // Reviewer 2", so the judge could tell which findings were its own. Same leak in
9
+ // the Step 2.5 rebuttal round, which showed each reviewer "the OTHER reviewers'"
10
+ // findings by attribution.
11
+ //
12
+ // Pattern source: karpathy/llm-council backend/council.py stage 2, which labels
13
+ // peer answers "Response A/B/C" and keeps label_to_model out of the prompt.
14
+ //
15
+ // A residual worth naming: with only two reviewers on Claude Code the label set
16
+ // is {Source A, Source B} and the triage model is one of them, so it retains a
17
+ // 50% prior on which findings are its own. Anonymization removes the signal, not
18
+ // the guess - llm-council's four-model council has a stronger version of the
19
+ // same property. The value here is that nothing TELLS the judge.
20
+ //
21
+ // What this does NOT do: scrub identity out of free text. A reviewer that writes
22
+ // "as Sonnet would" in `issue`, or whose prose style is recognisable, is not
23
+ // anonymized by removing keys. Scrubbing prose was rejected deliberately - it
24
+ // would mangle a legitimate finding about `ClaudeService.swift` - so the
25
+ // guarantee here is structural (no identity FIELD survives), not semantic.
26
+ //
27
+ // Determinism matters more here than it looks: /multi-agent:resume re-enters
28
+ // Phase 4, and a different shuffle would produce a different triage input for
29
+ // the same review. Findings are therefore sorted by a stable content key first
30
+ // (so reviewer completion order cannot leak through position) and then shuffled
31
+ // with a PRNG seeded from taskId + iteration.
32
+ //
33
+ // Usage:
34
+ // anonymize-findings.mjs [--file <reviewers.json>] [--map <out.json>] [--seed <text>]
35
+ //
36
+ // Exit codes: 0 ok, 64 usage, 65 input is not JSON.
37
+
38
+ import { readFileSync, writeFileSync } from "node:fs";
39
+
40
+ const IDENTITY_KEYS = [
41
+ "reviewer",
42
+ "reviewerName",
43
+ "reviewerId",
44
+ "model",
45
+ "agent",
46
+ "source",
47
+ "author",
48
+ ];
49
+
50
+ function labelFor(index) {
51
+ let n = index;
52
+ let out = "";
53
+ do {
54
+ out = String.fromCharCode(65 + (n % 26)) + out;
55
+ n = Math.floor(n / 26) - 1;
56
+ } while (n >= 0);
57
+ return `Source ${out}`;
58
+ }
59
+
60
+ function seedFrom(text) {
61
+ let h = 2166136261;
62
+ for (let i = 0; i < text.length; i++) {
63
+ h ^= text.charCodeAt(i);
64
+ h = Math.imul(h, 16777619);
65
+ }
66
+ return h >>> 0;
67
+ }
68
+
69
+ function rng(seed) {
70
+ let a = seed || 1;
71
+ return () => {
72
+ a = (a + 0x6d2b79f5) >>> 0;
73
+ let t = Math.imul(a ^ (a >>> 15), 1 | a);
74
+ t = (t + Math.imul(t ^ (t >>> 7), 61 | t)) ^ t;
75
+ return ((t ^ (t >>> 14)) >>> 0) / 4294967296;
76
+ };
77
+ }
78
+
79
+ function stableKey(finding) {
80
+ return [
81
+ finding?.file ?? "",
82
+ String(finding?.line ?? ""),
83
+ finding?.severity ?? "",
84
+ finding?.ruleId ?? "",
85
+ finding?.issue ?? "",
86
+ ].join(" ");
87
+ }
88
+
89
+ function stripIdentity(finding) {
90
+ const out = {};
91
+ for (const [k, v] of Object.entries(finding || {})) {
92
+ if (!IDENTITY_KEYS.includes(k)) out[k] = v;
93
+ }
94
+ return out;
95
+ }
96
+
97
+ // An entry with no `model` becomes "unknown" rather than being folded into a
98
+ // neighbour: absent and named are two different states, and a per-reviewer
99
+ // metric that quietly merges them reads as coverage it does not have.
100
+ export function anonymize(input, { seed } = {}) {
101
+ const reviewers = Array.isArray(input?.reviewers) ? input.reviewers : [];
102
+ const seedText = seed ?? `${input?.taskId ?? ""}:${input?.iteration ?? ""}`;
103
+ const labelToModel = {};
104
+ const tagged = [];
105
+ const malformed = [];
106
+
107
+ reviewers.forEach((reviewer, i) => {
108
+ const label = labelFor(i);
109
+ labelToModel[label] =
110
+ typeof reviewer?.model === "string" && reviewer.model ? reviewer.model : "unknown";
111
+ // A reviewer whose `findings` is not an array used to contribute nothing and
112
+ // say nothing, so triage adjudicated fewer findings than the panel produced
113
+ // with no trace of the loss. It is recorded instead: the caller can see that
114
+ // a dispatch came back malformed.
115
+ if (reviewer?.findings !== undefined && !Array.isArray(reviewer.findings)) {
116
+ malformed.push({ label, kind: typeof reviewer.findings });
117
+ }
118
+ const findings = Array.isArray(reviewer?.findings) ? reviewer.findings : [];
119
+ for (const finding of findings) {
120
+ tagged.push({ ...stripIdentity(finding), foundBy: label });
121
+ }
122
+ });
123
+
124
+ tagged.sort((a, b) => {
125
+ const ka = stableKey(a);
126
+ const kb = stableKey(b);
127
+ return ka < kb ? -1 : ka > kb ? 1 : 0;
128
+ });
129
+
130
+ const next = rng(seedFrom(seedText));
131
+ for (let i = tagged.length - 1; i > 0; i--) {
132
+ const j = Math.floor(next() * (i + 1));
133
+ [tagged[i], tagged[j]] = [tagged[j], tagged[i]];
134
+ }
135
+
136
+ return { findings: tagged, map: { seed: seedText, labelToModel, malformed } };
137
+ }
138
+
139
+ const isMain = process.argv[1] && import.meta.url === `file://${process.argv[1]}`;
140
+ if (isMain) {
141
+ const args = process.argv.slice(2);
142
+ const flag = (name) => {
143
+ const i = args.indexOf(name);
144
+ return i === -1 ? undefined : args[i + 1];
145
+ };
146
+ const file = flag("--file");
147
+ const mapOut = flag("--map");
148
+ const seed = flag("--seed");
149
+
150
+ if (args.includes("--help")) {
151
+ process.stdout.write(
152
+ [
153
+ "usage: anonymize-findings.mjs [--file <reviewers.json>] [--map <out.json>] [--seed <text>]",
154
+ " input: {taskId, iteration, reviewers:[{model, findings:[]}]} on stdin or --file",
155
+ " stdout: anonymized findings, identity stripped, order deterministic per seed",
156
+ " --map: writes {seed, labelToModel} for telemetry. Never put this file in a prompt.",
157
+ "",
158
+ ].join("\n"),
159
+ );
160
+ process.exit(0);
161
+ }
162
+
163
+ // Without this, a bare invocation blocks on a TTY stdin forever and the
164
+ // documented exit 64 is unreachable. It was in the first draft and was lost
165
+ // when the file was rewritten.
166
+ if (!file && process.stdin.isTTY) {
167
+ process.stderr.write(
168
+ "usage: anonymize-findings.mjs [--file <reviewers.json>] [--map <out.json>] [--seed <text>]\n" +
169
+ " (reads the reviewers JSON from stdin when --file is omitted; --help for detail)\n",
170
+ );
171
+ process.exit(64);
172
+ }
173
+
174
+ let raw;
175
+ if (file) {
176
+ raw = readFileSync(file, "utf8");
177
+ } else {
178
+ const chunks = [];
179
+ for await (const chunk of process.stdin) chunks.push(chunk);
180
+ raw = Buffer.concat(chunks).toString("utf8");
181
+ }
182
+
183
+ let parsed;
184
+ try {
185
+ parsed = JSON.parse(raw);
186
+ } catch (e) {
187
+ process.stderr.write(`input is not JSON: ${e.message}\n`);
188
+ process.exit(65);
189
+ }
190
+
191
+ const { findings, map } = anonymize(parsed, { seed });
192
+ for (const m of map.malformed) {
193
+ process.stderr.write(
194
+ `warning: ${m.label} returned findings as ${m.kind}, not an array - its findings do not reach triage\n`,
195
+ );
196
+ }
197
+ if (mapOut) writeFileSync(mapOut, `${JSON.stringify(map, null, 2)}\n`);
198
+ process.stdout.write(`${JSON.stringify(findings, null, 2)}\n`);
199
+ }
@@ -0,0 +1,92 @@
1
+ #!/usr/bin/env node
2
+ import { readFileSync } from "node:fs";
3
+
4
+ const SEQUENCES = [
5
+ ":)",
6
+ ":(",
7
+ ":P",
8
+ ":D",
9
+ ";)",
10
+ "(y)",
11
+ "(n)",
12
+ "(i)",
13
+ "(?)",
14
+ "(!)",
15
+ "(/)",
16
+ "(x)",
17
+ "(+)",
18
+ "(-)",
19
+ "(on)",
20
+ "(off)",
21
+ "(*)",
22
+ "(*r)",
23
+ "(*g)",
24
+ "(*b)",
25
+ "(*y)",
26
+ ];
27
+
28
+ const ALTERNATION = [...SEQUENCES]
29
+ .sort((a, b) => b.length - a.length)
30
+ .map((s) => s.replace(/[.*+?^${}()|[\]\\]/g, "\\$&"))
31
+ .join("|");
32
+
33
+ const EMOTICON = new RegExp(`(?<!\\\\)(${ALTERNATION})`, "g");
34
+
35
+ function segments(text) {
36
+ const out = [];
37
+ const opener = /\{code(?::[^}\n]*)?\}|\{noformat\}/g;
38
+ let idx = 0;
39
+ let match;
40
+ while ((match = opener.exec(text)) !== null) {
41
+ out.push({ literal: false, text: text.slice(idx, match.index) });
42
+ const closer = match[0].startsWith("{noformat") ? "{noformat}" : "{code}";
43
+ const end = text.indexOf(closer, match.index + match[0].length);
44
+ const stop = end === -1 ? text.length : end + closer.length;
45
+ out.push({ literal: true, text: text.slice(match.index, stop) });
46
+ idx = stop;
47
+ opener.lastIndex = stop;
48
+ }
49
+ out.push({ literal: false, text: text.slice(idx) });
50
+ return out;
51
+ }
52
+
53
+ export function escapeJiraEmoticons(text) {
54
+ return segments(text)
55
+ .map((s) => (s.literal ? s.text : s.text.replace(EMOTICON, "\\$1")))
56
+ .join("");
57
+ }
58
+
59
+ export function findUnescaped(text) {
60
+ const hits = [];
61
+ let offset = 0;
62
+ for (const s of segments(text)) {
63
+ if (!s.literal) {
64
+ for (const m of s.text.matchAll(EMOTICON)) {
65
+ const at = offset + m.index;
66
+ hits.push({ line: text.slice(0, at).split("\n").length, sequence: m[1] });
67
+ }
68
+ }
69
+ offset += s.text.length;
70
+ }
71
+ return hits;
72
+ }
73
+
74
+ async function readStdin() {
75
+ const chunks = [];
76
+ for await (const chunk of process.stdin) chunks.push(chunk);
77
+ return Buffer.concat(chunks).toString("utf8");
78
+ }
79
+
80
+ const isMain = process.argv[1] && import.meta.url === `file://${process.argv[1]}`;
81
+ if (isMain) {
82
+ const args = process.argv.slice(2);
83
+ const check = args.includes("--check");
84
+ const file = args.find((a) => !a.startsWith("--"));
85
+ const input = file ? readFileSync(file, "utf8") : await readStdin();
86
+ if (check) {
87
+ const hits = findUnescaped(input);
88
+ for (const h of hits) process.stderr.write(`${file || "-"}:${h.line}: ${h.sequence}\n`);
89
+ process.exit(hits.length === 0 ? 0 : 1);
90
+ }
91
+ process.stdout.write(escapeJiraEmoticons(input));
92
+ }