@mmerterden/multi-agent-pipeline 16.19.0 → 16.20.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (47) hide show
  1. package/CHANGELOG.md +23 -0
  2. package/install/_codex-agents.mjs +2 -2
  3. package/install/_dev-only-files.mjs +1 -0
  4. package/package.json +4 -3
  5. package/pipeline/agents/android-architect.md +1 -0
  6. package/pipeline/agents/backend-architect.md +1 -0
  7. package/pipeline/agents/code-reviewer.md +1 -0
  8. package/pipeline/agents/dev-critic.md +2 -1
  9. package/pipeline/agents/explorer.md +1 -0
  10. package/pipeline/agents/ios-architect.md +1 -0
  11. package/pipeline/agents/security-auditor.md +1 -0
  12. package/pipeline/agents/task-clarifier.md +1 -0
  13. package/pipeline/commands/multi-agent/manual-test/SKILL.md +5 -1
  14. package/pipeline/commands/multi-agent/resume/SKILL.md +1 -0
  15. package/pipeline/commands/multi-agent/store-ready/SKILL.md +22 -0
  16. package/pipeline/commands/sim-test.md +10 -5
  17. package/pipeline/multi-agent-refs/channels/pr.md +22 -4
  18. package/pipeline/multi-agent-refs/features/autopilot-circuit-breaker.md +15 -2
  19. package/pipeline/multi-agent-refs/features/review-delta.md +89 -0
  20. package/pipeline/multi-agent-refs/features/scope-check.md +41 -0
  21. package/pipeline/multi-agent-refs/features/verify-by-test.md +6 -5
  22. package/pipeline/multi-agent-refs/features/worktree-finalize.md +1 -1
  23. package/pipeline/multi-agent-refs/payload-contracts.md +1 -1
  24. package/pipeline/multi-agent-refs/phases/phase-3-dev.md +17 -2
  25. package/pipeline/multi-agent-refs/phases/phase-4-review.md +46 -8
  26. package/pipeline/multi-agent-refs/phases/phase-5-test.md +10 -0
  27. package/pipeline/multi-agent-refs/phases/phase-6-commit.md +2 -0
  28. package/pipeline/multi-agent-refs/phases/phase-7-report.md +4 -2
  29. package/pipeline/multi-agent-refs/rules.md +2 -2
  30. package/pipeline/schemas/agent-state.schema.json +129 -0
  31. package/pipeline/schemas/dev-critic-output.schema.json +5 -0
  32. package/pipeline/schemas/prefs.schema.json +47 -0
  33. package/pipeline/schemas/reviewer-output.schema.json +7 -2
  34. package/pipeline/schemas/scope-check.schema.json +55 -0
  35. package/pipeline/schemas/token-budget.json +3 -3
  36. package/pipeline/schemas/triage-output.schema.json +12 -2
  37. package/pipeline/scripts/README.md +3 -2
  38. package/pipeline/scripts/_fingerprint.mjs +173 -0
  39. package/pipeline/scripts/evidence-gate.mjs +73 -5
  40. package/pipeline/scripts/finding-fingerprint.mjs +101 -0
  41. package/pipeline/scripts/review-delta.mjs +217 -0
  42. package/pipeline/scripts/run-metrics.mjs +20 -0
  43. package/pipeline/scripts/scope-check-gate.mjs +90 -0
  44. package/pipeline/scripts/smoke-cross-cli-behavior.sh +11 -4
  45. package/pipeline/scripts/validate-reviewer.mjs +6 -0
  46. package/pipeline/scripts/validate-triage.mjs +20 -0
  47. package/pipeline/skills/shared/core/multi-agent-store-ready/SKILL.md +5 -0
package/CHANGELOG.md CHANGED
@@ -16,6 +16,29 @@ Internal file-layout changes that don't affect the slash-command surface are sti
16
16
 
17
17
  ## [Unreleased]
18
18
 
19
+ ## [16.20.0] - 2026-09-07
20
+
21
+ A comparison against `msitarzewski/agency-agents`, a 150k-star persona catalog, found nothing to copy wholesale and three things to fix: the personas were never registered, the circuit-breaker was prose, and a review finding had no identity from one round to the next. This release gives findings that identity and builds the loop controls, the handoff records and the evidence rules on top of it.
22
+
23
+ ### Added
24
+
25
+ - **A finding keeps its identity across review rounds.** `finding-fingerprint.mjs` stamps every reviewer, triage and dev-critic finding with `F:xxxxxxxx`, computed from the file and either the cited rule ID or the normalised issue text; the line, the severity and the fix text take no part, so a finding whose line moved after the fix is still the same finding. `review-delta.mjs` compares one round's triage with the previous one and reports `stillPresent`, `resolved`, `downgraded` and `new`, plus how many consecutive rework cycles each survivor has lasted. Phase 4 keeps one `.pipeline/triage-round-<N>.json` per round, injects the previous round's accepted findings into the shared reviewer prefix so reviewers echo instead of rediscover, and stores the delta in `state.reviewIterations[i].delta`. The Phase 3 reflection prompt puts the survivors first, marked `STILL PRESENT`.
26
+ - **The circuit-breaker has two triggers in code.** Trigger 2 (a blocking or important finding that survives `identicalFindingCycles` consecutive rework cycles, default 2) fires from `review-delta.mjs` exit 3 at Phase 4 Step 3.8; trigger 3 (the rework cap) is recorded by the Phase 3 re-entry. Both write `state.circuitBreaker` and halt visibly in autopilot; interactive modes show the survivors and ask. `prefs.global.autopilotCircuitBreaker` and `state.circuitBreaker` now exist in the schemas the feature doc had been naming, `resume` clears the trip, and `smoke-autopilot-circuit-breaker` asserts the schema fields, the scripts and the phase wiring instead of grepping the prose.
27
+ - **Dev states its scope before review reads it.** Phase 3 Step 3.7 writes `.pipeline/scope-check.json`: a reason per touched file, the changes deliberately not made, and the code-simplifier rationales that used to be discarded. `scope-check-gate.mjs` compares it with the real diff; Phase 4 renders it as `<scope-self-check>` so reviewers judge against the stated scope and do not re-propose what was rejected; Phase 6 builds the PR `Changes` bullets and a "Follow-ups not done in this PR" list from it.
28
+ - **PR bodies carry a risk section when the diff earns one.** `channels/pr.md` gains the conditional `## Risk and Security` section (auth flow touched, secret handling changed, data migration, rollback), required whenever `state.diffRisk.signals` carries a high-stakes signal. Phase 4 now persists the diff-risk totals to `state.diffRisk`, and `run-metrics.mjs` reports `diff.filesTouched`, `locAdded`, `locRemoved` and the cross-round delta.
29
+ - **Manual test results are evidence, not a chat reply.** Phase 5 writes `.pipeline/manual-test.json` with one row per acceptance criterion (the quoted spec, the observation, the verdict, the screenshot) and `evidence-gate.mjs --claim manual` refuses an "ok" whose criteria are missing, failed or untested without a reason. The simulator UI report pairs every bug with its spec quote and its before/after captures and ends with a production-readiness verdict that defaults to FAILED.
30
+ - **One green run is not proof.** Verify-by-test re-runs a passing repro test `verifyByTest.repeatCount` times (default 3) before it may downgrade a blocking finding; a run that disagrees makes the verdict `inconclusive` with a `flaky` note. Phase 3 runs every new or changed test `testStability.repeatCount` times (default 3) and records `test.flake_signal` when outcomes disagree: a test that passes only on retry is a flake signal, not a pass.
31
+ - **Persona files round-trip and store readiness asks about symbols and rollout.** `lint-personas.mjs` checks every persona's frontmatter, its `## Output Format` section, and that the Codex TOML rendered from it decodes back to the same description and body byte for byte; the renderer no longer appends a newline the source did not have. The smokes that hardcoded six persona names now derive the list from the directory. `store-ready` flags an archive without dSYMs (or a minified bundle without `mapping.txt`) as blocking, checks that a symbol upload step exists, and ends the report with the phased-rollout thresholds and the rollback owner a release needs written down.
32
+
33
+ ### Changed
34
+
35
+ - **Token budgets: phase-4-review max 14750 -> 15150, total 58250 -> 60250.** The full contracts for the review delta and the scope self-check live in `multi-agent-refs/features/review-delta.md` and `features/scope-check.md`, outside the budget; the phase docs carry the call, the pref and the exit table, and were compressed twice before the numbers moved. `rules.md` was tightened by 150 bytes to keep the fixed per-run load under its 60000-byte ceiling. Reasoning in `schemas/token-budget.json`.
36
+
37
+ ### Fixed
38
+
39
+ - **Phase 4 and Phase 7 disagreed on the triage file name.** Phase 4 wrote `.pipeline/triage.json`; Phase 7, `worktree-finalize.sh`, `render-work-summary.sh` and `diff-explain.mjs` read `triage-output.json`. The salvage step bridged it only when the worktree was removed, and everywhere else the triage corpus and the learnings ledger silently stopped being fed. Phase 4 now copies the latest round to `triage-output.json` itself.
40
+ - **The eight personas were never registered as subagents.** `pipeline/agents/*.md` carried `description`, `model`, `preferredModel` and `modelRationale` but no `name:`, and Claude Code drops a subagent file without one silently: no error, no agent. Every `subagent_type: "code-reviewer"`, `"ios-architect"`, `"security-auditor"`, `"dev-critic"` and `"task-clarifier"` dispatch in the phase docs named an agent the host did not have. Each persona now declares `name:` equal to its filename stem, and `smoke-agent-model-routing` fails when a persona lacks it, when the name differs from the stem, or when it is not lowercase kebab-case. Found while comparing the persona format against `msitarzewski/agency-agents`, whose linter treats `name` as a required field.
41
+
19
42
  ## [16.19.0] - 2026-09-02
20
43
 
21
44
  A review of the three repositories together (pipeline, toolkit MCP, plugin marketplace) with every test suite green found the defects the suites could not see. This release fixes what was found in the pipeline; `@mmerterden/multi-agent-toolkit-mcp` 3.8.0 and the marketplace plugins carry their own halves.
@@ -94,10 +94,10 @@ export function tomlString(s) {
94
94
  return JSON.stringify(s);
95
95
  }
96
96
  if (!s.includes("'''")) {
97
- return `'''\n${s}\n'''`;
97
+ return `'''\n${s}'''`;
98
98
  }
99
99
  const escaped = s.replace(/\\/g, "\\\\").replace(/"""/g, '\\"\\"\\"');
100
- return `"""\n${escaped}\n"""`;
100
+ return `"""\n${escaped}"""`;
101
101
  }
102
102
 
103
103
  /**
@@ -59,6 +59,7 @@ const DEV_ONLY_TOOLING = Object.freeze([
59
59
  "run-smokes.mjs",
60
60
  "scorecard.mjs",
61
61
  "lint-skills.mjs",
62
+ "lint-personas.mjs", // imports install/_codex-agents.mjs, which never ships to scripts/
62
63
  "lint-mcp-refs.mjs",
63
64
  "check-md-links.mjs",
64
65
  "validate-schemas.mjs", // validates the repo's own schema files, needs ajv
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@mmerterden/multi-agent-pipeline",
3
- "version": "16.19.0",
3
+ "version": "16.20.0",
4
4
  "description": "8-phase AI development pipeline with full orchestration on Claude Code, Copilot CLI and Codex CLI. Analysis, planning, TDD, CLI-aware parallel review with consensus surfacing + Fable triage, default-FAIL evidence gates, secret + intent guards, per-phase cost ledger, persistent learnings memory, wiki generation, commit automation. Token-preserving uninstall.",
5
5
  "type": "module",
6
6
  "main": "index.js",
@@ -14,11 +14,11 @@
14
14
  },
15
15
  "scripts": {
16
16
  "start": "node index.js",
17
- "test": "npm run format:check && node --test test/*.test.mjs && node pipeline/scripts/run-smokes.mjs && node pipeline/scripts/lint-skills.mjs && node pipeline/scripts/lint-mcp-refs.mjs && node pipeline/scripts/eval-triage.mjs && node pipeline/scripts/eval-golden-tasks.mjs && node pipeline/scripts/eval-intent.mjs && node pipeline/scripts/eval-recall.mjs && node pipeline/scripts/validate-schemas.mjs && node pipeline/scripts/validate-prefs.mjs && node pipeline/scripts/scorecard.mjs",
17
+ "test": "npm run format:check && node --test test/*.test.mjs && node pipeline/scripts/run-smokes.mjs && node pipeline/scripts/lint-skills.mjs && node pipeline/scripts/lint-personas.mjs && node pipeline/scripts/lint-mcp-refs.mjs && node pipeline/scripts/eval-triage.mjs && node pipeline/scripts/eval-golden-tasks.mjs && node pipeline/scripts/eval-intent.mjs && node pipeline/scripts/eval-recall.mjs && node pipeline/scripts/validate-schemas.mjs && node pipeline/scripts/validate-prefs.mjs && node pipeline/scripts/scorecard.mjs",
18
18
  "test:unit": "node --test test/*.test.mjs",
19
19
  "test:smoke": "node pipeline/scripts/run-smokes.mjs",
20
20
  "lint:skills": "node pipeline/scripts/lint-skills.mjs",
21
- "test:quick": "node --test test/*.test.mjs && node pipeline/scripts/lint-skills.mjs && node pipeline/scripts/lint-mcp-refs.mjs",
21
+ "test:quick": "node --test test/*.test.mjs && node pipeline/scripts/lint-skills.mjs && node pipeline/scripts/lint-personas.mjs && node pipeline/scripts/lint-mcp-refs.mjs",
22
22
  "test:coverage": "c8 --clean --reporter=none node --test test/*.test.mjs && c8 --clean=false --check-coverage --reporter=text --reporter=lcov node pipeline/scripts/run-smokes.mjs",
23
23
  "lint": "eslint .",
24
24
  "lint:fix": "eslint . --fix",
@@ -85,6 +85,7 @@
85
85
  "!pipeline/scripts/eval-*.mjs",
86
86
  "!pipeline/scripts/scorecard.mjs",
87
87
  "!pipeline/scripts/lint-skills.mjs",
88
+ "!pipeline/scripts/lint-personas.mjs",
88
89
  "!pipeline/scripts/lint-mcp-refs.mjs",
89
90
  "!pipeline/scripts/check-md-links.mjs",
90
91
  "!pipeline/scripts/validate-schemas.mjs",
@@ -1,4 +1,5 @@
1
1
  ---
2
+ name: android-architect
2
3
  description: Android architect - evaluates design decisions, patterns, and module structure
3
4
  model: fable
4
5
  preferredModel: fable
@@ -1,4 +1,5 @@
1
1
  ---
2
+ name: backend-architect
2
3
  description: Backend architect - evaluates API design, data modeling, and system architecture
3
4
  model: fable
4
5
  preferredModel: fable
@@ -1,4 +1,5 @@
1
1
  ---
2
+ name: code-reviewer
2
3
  description: "Code reviewer for multi-agent Phase 4 - security, architecture, quality, performance. Default model is fable (opus is the first fallback); Phase 4 orchestrator overrides to sonnet for Reviewer 3."
3
4
  model: fable
4
5
  preferredModel: fable
@@ -1,4 +1,5 @@
1
1
  ---
2
+ name: dev-critic
2
3
  description: "Phase 3.5 evaluator - runs after Dev's last edit, before Phase 4. Verifies build/test/checklist gates; returns pass | fix-list. Sonnet by default; Phase 3 orchestrator may override."
3
4
  model: sonnet
4
5
  preferredModel: sonnet
@@ -79,7 +80,7 @@ Apply the platform checklist verbatim. **Cite the rule file + line/section** so
79
80
  - **Max 2 critic iterations.** Generator gets feedback, edits, you re-evaluate. If round 2 still fails gates, **escalate**: return `escalate: true` and let the orchestrator decide (pause user / abort / continue with known failures).
80
81
  - Round 1 budget: full critic pass with all gates + checklist.
81
82
  - Round 2 budget: ONLY re-check the items that failed round 1. Don't re-flag what the generator fixed.
82
- - Never add new findings in round 2 that weren't in round 1's output - that's scope creep, not iteration.
83
+ - Never add new findings in round 2 that weren't in round 1's output - that's scope creep, not iteration. Round-2 findings carry the round-1 `fingerprint` (`finding-fingerprint.mjs annotate --kind dev-critic`); a round-2 finding with no round-1 fingerprint is the scope creep this rule forbids.
83
84
 
84
85
  ## Output Format
85
86
 
@@ -1,4 +1,5 @@
1
1
  ---
2
+ name: explorer
2
3
  description: "Codebase explorer agent for multi-agent Phase 1 - scans project structure, patterns, and impact areas"
3
4
  model: sonnet
4
5
  preferredModel: sonnet
@@ -1,4 +1,5 @@
1
1
  ---
2
+ name: ios-architect
2
3
  description: iOS architect - evaluates design decisions, patterns, and module structure
3
4
  model: fable
4
5
  preferredModel: fable
@@ -1,4 +1,5 @@
1
1
  ---
2
+ name: security-auditor
2
3
  description: Security specialist - analyzes code for vulnerabilities and compliance issues
3
4
  model: opus
4
5
  preferredModel: opus
@@ -1,4 +1,5 @@
1
1
  ---
2
+ name: task-clarifier
2
3
  description: "Phase 0 Step 9 task ambiguity scorer - reads task description, emits clarity score (0-10) and up to N clarifying questions. Haiku by default; cost stays under 1¢ per call."
3
4
  model: haiku
4
5
  preferredModel: haiku
@@ -47,5 +47,9 @@ Lets you switch to the task branch for manual testing in Xcode before the PR is
47
47
  5. **Wait for the user's reply**
48
48
 
49
49
  6. **Branch on the answer**:
50
- - **OK** → `phase-tracker.sh update 5 completed` + `phase-tracker.sh meta 5 Result "local test passed (user)"`, recreate the worktree, continue to Phase 6
50
+ - **OK** → first write `$WORKTREE/.pipeline/manual-test.json` (one entry per acceptance criterion from the analysis doc test plan, the plan tasks, or the user's own words):
51
+ ```json
52
+ {"criteria":[{"spec":"<quote>","source":"analysis 15.2 | plan task 3 | user","observed":"<what was seen>","verdict":"pass|fail|not-tested","reason":"<required when not-tested>","screenshot":"<path or null>"}],"verdict":"passed|failed"}
53
+ ```
54
+ then run `node $HOME/.claude/scripts/evidence-gate.mjs --claim manual --status passed --evidence "$WORKTREE/.pipeline/manual-test.json"`. Exit 1 means the "ok" is not accepted: name the criterion that is missing evidence and wait for the next reply. Exit 0 → `phase-tracker.sh update 5 completed` + `phase-tracker.sh meta 5 Result "local test passed (user)"`, recreate the worktree, continue to Phase 6. Full contract: `$HOME/.claude/multi-agent-refs/phases/phase-5-test.md` step 5.
51
55
  - **Fix needed** → `phase-tracker.sh now 5 "applying fix: <summary>"`, recreate the worktree, apply the fix
@@ -23,6 +23,7 @@ Resume a paused or failed task from the last successful phase.
23
23
  - `currentPhase` - last completed phase
24
24
  - `status` - `paused` | `failed` | `in_progress`
25
25
  - `haltReason` - if set, show it so the user knows why the run stopped; clear it on successful re-entry
26
+ - `circuitBreaker` - if `tripped`, show `trigger` + `detail`, then set `tripped: false` and keep `counters`; if the same trigger fires again at the next checkpoint the breaker re-trips (no silent bypass)
26
27
  - `autopilot` - preserve the mode
27
28
 
28
29
  3. **Load context** - rebuild working context from durable artifacts, never from conversation memory:
@@ -218,10 +218,22 @@ is a guess the user must be able to correct.
218
218
  Use the fallback only when the tool is genuinely absent, and say in the report
219
219
  which path ran - a rule set that silently differed between two invocations is
220
220
  worse than a missing gate.
221
+
222
+ Then, whichever path ran, count `<archive>/dSYMs/*.dSYM`. Zero is a blocking
223
+ finding: `[SYMBOLS] archive carries no dSYM; crash reports will not symbolicate`,
224
+ with the hint `DEBUG_INFORMATION_FORMAT = dwarf-with-dsym` for the Release
225
+ configuration. This check runs from the package alone and does not need the
226
+ MCP tool.
221
227
  - **Android**: `android_apk_audit` on the artifact, plus the
222
228
  `google-play-compliance` skill's 21 rules - `bundletool validate` and manifest
223
229
  dump, `aapt2 dump badging`, `apksigner verify`, ABI / native scan.
224
230
 
231
+ Then, when the module has `minifyEnabled true` and the bundle build produced no
232
+ `mapping.txt`, raise the same class of blocking finding:
233
+ `[SYMBOLS] minified bundle carries no mapping.txt; crash reports will not
234
+ deobfuscate`. This check reads the module config and the build output alone
235
+ and does not need the MCP tool.
236
+
225
237
  `error` findings are blocking; `warning` is advisory. Group by severity and keep
226
238
  each finding's ITMS / Play policy code - Gate 2 may return the same code on iOS,
227
239
  and seeing it in both places tells the user it is real rather than a heuristic.
@@ -259,6 +271,7 @@ that catches what a human reviewer rejects, so it reads source, not the binary.
259
271
  | Privacy policy | reachable in-app and in the metadata |
260
272
  | IAP | anything unlocking features goes through StoreKit, with no external purchase path |
261
273
  | Sign in with Apple | present when a third-party social login is offered |
274
+ | Crash symbolication | a dSYM upload step exists: an Xcode run-script calling `upload-symbols`, or a Crashlytics / Sentry / Datadog upload in CI |
262
275
 
263
276
  **Android** - `ai-android-toolkit:play-store-review`:
264
277
 
@@ -272,6 +285,7 @@ that catches what a human reviewer rejects, so it reads source, not the binary.
272
285
  | Account deletion | if the app creates accounts, an in-app deletion path exists, plus the web deletion URL Play requires |
273
286
  | Content rating | the questionnaire answers match the app's actual content |
274
287
  | Signing | Play App Signing configured, upload key distinct from the app signing key |
288
+ | Crash symbolication | a `mapping.txt` upload exists: the Firebase Crashlytics Gradle plugin, or the Play App Bundle deobfuscation file |
275
289
 
276
290
  For each: `pass` / `fail` / `not-applicable` with the evidence path that justifies
277
291
  it. `not-applicable` needs a reason - an unexamined area is not a pass.
@@ -303,6 +317,12 @@ Advisory
303
317
 
304
318
  Not run
305
319
  Gate 2: no local Play validator - authoritative check is server-side only
320
+
321
+ Before rollout (human inputs, not verified)
322
+ Phased rollout: <1% -> 10% -> 50% -> 100% | full>
323
+ Halt thresholds: crash-free < 99.5% (iOS, Android) or ANR > 0.47% (Android) => pause the rollout
324
+ Rollback / forward-fix owner: <name>
325
+ Forward-fix plan: <one line>
306
326
  ```
307
327
 
308
328
  Rules for the report:
@@ -311,6 +331,8 @@ Rules for the report:
311
331
  skip. An Android run therefore reads `2 of 3 gates cleared, 1 skipped` at best.
312
332
  - Every blocking finding carries a file path or a store code. A finding the user
313
333
  cannot act on is noise.
334
+ - The `Before rollout` block is filled by a human, never inferred. When it is
335
+ left unfilled the verdict line gains the suffix `, rollout plan missing`.
314
336
  - Humanize via `--lang en` by default (`promptLanguage` is locked to `"en"`); pass
315
337
  `--lang=tr` explicitly to opt into Turkish.
316
338
  - No AI or assistant attribution anywhere, per
@@ -253,16 +253,18 @@ Then output full report:
253
253
  - **Steps**: 1. Open app -> 2. Tap {X} -> 3. Observe {issue}
254
254
  - **Expected**: {correct behavior}
255
255
  - **Actual**: {what's wrong}
256
+ - **Spec**: "<quoted acceptance criterion from the analysis doc Section 15 / 20, or: no spec, model expectation>" (<source>)
257
+ - **Evidence**: before=<png path> after=<png path>
256
258
 
257
259
  ### BUG-2: ...
258
260
 
259
261
  ## Screens Visited ({N})
260
262
 
261
- | # | Screen | Light | Dark | Large Text | Bugs |
262
- | --- | -------- | ----- | ----- | ---------- | ---- |
263
- | 1 | Home | ok | BUG-1 | ok | 1 |
264
- | 2 | Login | ok | ok | BUG-2 | 1 |
265
- | 3 | Settings | ok | ok | ok | 0 |
263
+ | # | Screen | Light | Dark | Large Text | Bugs | Evidence |
264
+ | --- | -------- | ----- | ----- | ---------- | ---- | -------- |
265
+ | 1 | Home | ok | BUG-1 | ok | 1 | 3 |
266
+ | 2 | Login | ok | ok | BUG-2 | 1 | 4 |
267
+ | 3 | Settings | ok | ok | ok | 0 | 2 |
266
268
 
267
269
  ## Summary
268
270
 
@@ -272,8 +274,11 @@ Then output full report:
272
274
  - Critical: {N} | Major: {N} | Minor: {N}
273
275
  - Accessibility audit (accessibility scenario): {N} audited, {N} not audited (reason per screen), deep pass: {ran / skipped: reason}
274
276
  - Crashes (full scenario): {N} during this run
277
+ - Production readiness: FAILED | NEEDS WORK | READY
275
278
  ```
276
279
 
280
+ `Evidence` in a bug block is the captures already written to files in Step 4: a tap-driven finding needs both `before` and `after`, a static finding (layout, contrast, dark mode, large text) needs `after` only. The `Evidence` column in Screens Visited is the count of screenshot files written for that screen. Production readiness defaults to FAILED; it is NEEDS WORK when only Minor bugs remain, and READY only when there are zero Critical / Major bugs and every planned screen was visited.
281
+
277
282
  Save to: `$HOME/.claude/logs/sim-test/{bundle_id}/{timestamp}.md`
278
283
 
279
284
  ### Step 7 - Fix Offer
@@ -14,14 +14,15 @@ The PR description targets code reviewers - it stays technical. Every adapter
14
14
  | 2 | `changes` | `## Değişiklikler` | `## Changes` | always |
15
15
  | 3 | `architecture` | `## Mimari Kararlar` | `## Architecture Decisions` | when a non-trivial design choice was made |
16
16
  | 4 | `verification` | `## Doğrulama` | `## Verification` | always |
17
- | 5 | `dependencies` | `## Bağımlılıklar` | `## Dependencies` | when deps added/removed/bumped |
18
- | 6 | `related` | `## İlgili` | `## Related` | always (Jira/issue ref; never `Closes/Fixes`) |
17
+ | 5 | `risk` | `## Risk ve Güvenlik` | `## Risk and Security` | when `state.diffRisk.signals` carries a high-stakes signal (`security_path`, `migration`, `public_api`, `no_test_change`, `test_lines_removed`) |
18
+ | 6 | `dependencies` | `## Bağımlılıklar` | `## Dependencies` | when deps added/removed/bumped |
19
+ | 7 | `related` | `## İlgili` | `## Related` | always (Jira/issue ref; never `Closes/Fixes`) |
19
20
 
20
21
  ### Section content rules
21
22
 
22
23
  **`summary`** - 1-3 sentences in `outputLanguage`. The "why" of the change. Past tense, no marketing voice. Code identifiers stay verbatim.
23
24
 
24
- **`changes`** - bullet list, one item per logically distinct change. Each bullet starts with the touched component and ends with a one-line "what". Use the stack's native file extensions / module paths - the example below shows the **shape**, not a stack lock-in:
25
+ **`changes`** - bullet list, one item per logically distinct change. Each bullet starts with the touched component and ends with a one-line "what". The source is `$WORKTREE/.pipeline/scope-check.json` `files[].reason` (Phase 3 Step 3.7): a file the dev could not justify there is a file this list cannot describe either, so the bullet quotes the gate output instead of inventing a reason. Use the stack's native file extensions / module paths - the example below shows the **shape**, not a stack lock-in:
25
26
 
26
27
  ```markdown
27
28
  ## Changes
@@ -50,6 +51,17 @@ Skeleton (the adapter fills the body with the actual stack-appropriate lines at
50
51
 
51
52
  Multi-repo PRs (one PR per repo) emit verification commands for that repo's stack only - never mix iOS + Android commands into a single PR body.
52
53
 
54
+ **`risk`** - only when `state.diffRisk.signals` (Phase 4 Step 1.75) contains a high-stakes signal. Four fixed lines, each answered, never left as a placeholder; the source is Phase 1 `touchedAreas` plus the signals themselves, and when a signal is present the absence of this section is a Phase 6 Step 3 blocker:
55
+
56
+ ```markdown
57
+ ## Risk and Security
58
+
59
+ - Auth flow touched: yes | no
60
+ - Secret handling changed: yes | no
61
+ - Data migration: yes | no
62
+ - Rollback: feature flag <name> | git revert <sha> | none, and why
63
+ ```
64
+
53
65
  **`dependencies`** - only when `Package.swift` / `Podfile` / `build.gradle` / `package.json` changed. Each entry: `package@old → new - reason`.
54
66
 
55
67
  **`related`** - flat list, plain text. Examples:
@@ -61,15 +73,21 @@ Multi-repo PRs (one PR per repo) emit verification commands for that repo's stac
61
73
  - Issue: #123
62
74
  - Confluence: <page-url> (if work referenced a spec)
63
75
  - Figma: <design-url> (if work referenced a design)
76
+
77
+ Follow-ups not done in this PR:
78
+ - <scope-check.json notDone[].what> - <why>
79
+ - <deferred triage finding> - <triage reason>
64
80
  ```
65
81
 
82
+ The follow-up list is present only when `scope-check.json` `notDone[]` or the final triage `deferred[]` is non-empty; the two sources merge into one list.
83
+
66
84
  Never use `Closes #N`, `Fixes #N`, `Resolves PROJ-X`. Issues require 4-approval close, the auto-close keywords break that contract.
67
85
 
68
86
  ### Assembly order (per run)
69
87
 
70
88
  ```
71
89
  1. Read agent-state.json (taskId, contextLinks, identity, language).
72
- 2. Build section bodies in markdown - summary first, then in the table order, skipping conditional sections that don't apply.
90
+ 2. Build section bodies in markdown - summary first, then in the table order, skipping conditional sections that don't apply. Section order is fixed: `summary` → `changes` → `architecture` (cond.) → `verification` → `risk` (cond.) → `dependencies` (cond.) → `related`.
73
91
  3. Run the assembled body through the `humanizer` skill.
74
92
  4. Apply Multi-repo cross-links (## Related PRs prepend when projects.length > 1).
75
93
  5. Dispatch per the Behaviour-by-remote table.
@@ -2,7 +2,7 @@
2
2
 
3
3
  **Pattern**: autopilot runs with zero interaction, which is exactly when a silent failure loop is most expensive - an agent can burn a budget re-attempting the same broken fix, or thrash between two phases, with nobody watching. A circuit-breaker converts "keep going no matter what" into "keep going until a defined unsafe condition, then halt and hand back to the user." This is the sanctioned autopilot pause (same class as the Phase 7 channels pause): the run stops, records why, and waits for an explicit `resume`.
4
4
 
5
- **Gated by `prefs.global.autopilotCircuitBreaker`** (default: enabled; thresholds tunable). Halting is always safe, so the breaker itself defaults on. Disable per-run only with an explicit override. Complements, does not replace, the existing autopilot safety rules (build-fail max 3 retries, Phase 4 blocking-finding rework, destructive-op confirmations).
5
+ **Gated by `prefs.global.autopilotCircuitBreaker`** (`enabled` default true, `identicalFindingCycles` default 2, `maxReworkCycles` default 3; `schemas/prefs.schema.json`). Halting is always safe, so the breaker itself defaults on. Disable per-run only with an explicit override. Complements, does not replace, the existing autopilot safety rules (build-fail max 3 retries, Phase 4 blocking-finding rework, destructive-op confirmations).
6
6
 
7
7
  ## Trip conditions
8
8
 
@@ -18,12 +18,25 @@ Any one trips the breaker. All are evaluated from `agent-state.json` + telemetry
18
18
 
19
19
  Trigger 2 is the key addition over the plain build-retry cap: a build can "fail differently" three times (legitimate iteration) or "fail identically" twice (stuck). Only the identical-failure case is a stall; the retry cap catches the rest.
20
20
 
21
+ ## Wiring status
22
+
23
+ | Trigger | Evaluated by | Status |
24
+ |---|---|---|
25
+ | 2, finding half | `review-delta.mjs` exit 3 at Phase 4 Step 3.8: a blocking/important finding whose `fingerprint` (finding-fingerprint.mjs) stays in the accepted set for `identicalFindingCycles` consecutive rounds | **code** (v16.20.0) |
26
+ | 3 | Phase 3 re-entry item 6: the `retryCount === 3` hard-kill records the trip | **code** (v16.20.0) |
27
+ | 2, build-error half | needs a build-log signature normaliser | documented behaviour, no script yet |
28
+ | 1 | needs checkpoint-to-checkpoint artifact diffing | documented behaviour, no script yet |
29
+ | 4 | belongs to `cost-budget-check.mjs` | documented behaviour, no script yet |
30
+ | 5 | Phase 6 push | documented behaviour, no script yet |
31
+
32
+ State shape: `state.circuitBreaker = {tripped, trigger, detail, checkpoint: {phase, step, iteration}, trippedAt, counters: {identicalFindingCycles, reworkCycles}}` (`schemas/agent-state.schema.json`). The per-round classification the finding half reads lives in `state.reviewIterations[i].delta` (`new`, `stillPresent`, `resolved`, `downgraded`, `recurrence`, `plateau`). `smoke-autopilot-circuit-breaker.sh` asserts the schema fields, the scripts and the phase wiring, not only this prose.
33
+
21
34
  ## Action on trip
22
35
 
23
36
  1. Set `agent-state.json.circuitBreaker = {tripped: true, trigger: <#>, detail, checkpoint}` and flip `autopilot` handling to paused (the run does not continue unattended).
24
37
  2. Emit one actionable line per the progress contract: what tripped, the evidence (error signature / cycle count / spend vs ceiling), and the single next action (`resume #N` after a fix, or `kill #N`).
25
38
  3. Never auto-resolve the underlying cause - no force-anything, no conflict auto-merge, no budget self-raise. The breaker hands control back; it does not paper over the problem.
26
- 4. `resume #N` clears the tripped flag and continues from the recorded checkpoint. If the same trigger fires again immediately, the breaker re-trips (no silent bypass).
39
+ 4. `resume #N` clears `circuitBreaker.tripped` (keeping `counters`) and continues from the recorded checkpoint. If the same trigger fires again immediately, the breaker re-trips (no silent bypass).
27
40
 
28
41
  ## Why this is the right autopilot exception
29
42
 
@@ -0,0 +1,89 @@
1
+ # Feature: Cross-round review delta (Phase 4 Steps 2.1, 2.2, 3.8)
2
+
3
+ **Pattern**: reviewers re-read the whole diff every round with no memory of the round before, so they rediscover last round's findings in new words and nothing can tell "still broken" from "new". A finding therefore needs an identity that survives the fix: `finding-fingerprint.mjs` computes `F:xxxxxxxx` from the file and either the cited `ruleId` or the normalised issue text (lowercase, quotes stripped, paths reduced to basenames, digit runs collapsed). The line, the severity, the fix text and the reviewer take no part, because all of them change between rounds without the finding changing. `review-delta.mjs` then compares round N with round N-1 and reports `stillPresent`, `resolved`, `downgraded` (re-reported but no longer accepted by triage) and `new`, plus `recurrence`: how many consecutive rework cycles each survivor has lasted. That count is the autopilot circuit-breaker's trigger 2.
4
+
5
+ Gated by `prefs.global.autopilotCircuitBreaker` (`enabled` default true, `identicalFindingCycles` default 2). Computed by scripts, never by the model: an older triage JSON gets its fingerprints on the fly, and a reviewer that echoes one is respected but not relied on.
6
+
7
+ ## Files per round
8
+
9
+ Phase 4 Step 3.2.1 writes `$WORKTREE/.pipeline/triage-round-<N>.json` (N = `state.reviewIterations | length`), validates it, annotates it and copies it to `$WORKTREE/triage-output.json`, the name Phase 7, `worktree-finalize.sh`, `render-work-summary.sh` and `diff-explain.mjs` read. A copy rather than a symlink because the salvage is `cp -R`, and `.pipeline/` is on the salvage list, so every round survives into `artifactsPath`. Step 3.7 rewrites the round file; the copy is repeated after it.
10
+
11
+ ## Step 2.1 block: previous-round findings (iteration >= 2)
12
+
13
+ ```bash
14
+ ITERATION=$(jq '.reviewIterations | length' "$STATE_FILE")
15
+ PREV_ROUND="$WORKTREE/.pipeline/triage-round-$((ITERATION-1)).json"
16
+ PREV_BLOCK=""
17
+ [ "$ITERATION" -ge 2 ] && [ -f "$PREV_ROUND" ] && PREV_BLOCK=$(node $HOME/.claude/scripts/finding-fingerprint.mjs annotate "$PREV_ROUND" 2>/dev/null \
18
+ | jq -r '[.accepted[] | select(.severity=="blocking" or .severity=="important")] | sort_by(.severity != "blocking") | .[:40][] | "- \(.fingerprint) [\(.severity)] \(.file): \(.issue)"')
19
+ ```
20
+
21
+ Rendered at the end of the shared prefix (Step 1.9), identical for every reviewer and for the triage call of that iteration:
22
+
23
+ ```
24
+ <previous-round-findings>
25
+ Each entry below was accepted last round and sent for rework.
26
+ - If the issue is still present, report it again with the SAME fingerprint value and the current line.
27
+ - If it is fixed, omit it. Omission is how you report resolution; never emit a "resolved" finding.
28
+ - Any finding not listed here is new: leave fingerprint unset.
29
+ {PREV_BLOCK}
30
+ </previous-round-findings>
31
+ ```
32
+
33
+ The cap of 40 entries drops `important` before `blocking` so the prefix stays bounded. The block never enters the repo-stable prefix `prompt-assembly.md` describes: the diff it follows is already per-run. After each reviewer's validator gate, `finding-fingerprint.mjs annotate --in-place` fills in any fingerprint the reviewer left unset; `anonymize-findings.mjs` strips only identity keys, so the fingerprint reaches triage, and the triage prompt tells the model to preserve it verbatim.
34
+
35
+ ## Step 2.2 block: scope self-check (every iteration)
36
+
37
+ Phase 3 Step 3.7 wrote `$WORKTREE/.pipeline/scope-check.json` (contract: `features/scope-check.md`). Render it so reviewers judge the diff against the dev's stated scope and do not re-propose what was rejected:
38
+
39
+ ```bash
40
+ SCOPE_JSON="$WORKTREE/.pipeline/scope-check.json"
41
+ SCOPE_GATE=$(git -C "$WORKTREE" diff --name-only "origin/$BASE_BRANCH"...HEAD \
42
+ | node $HOME/.claude/scripts/scope-check-gate.mjs --check "$SCOPE_JSON" --diff-files - --advisory 2>/dev/null)
43
+ ```
44
+
45
+ ```
46
+ <scope-self-check>
47
+ Files and the reason the dev gave for touching each:
48
+ {jq -r '.files[] | "- \(.path): \(.reason)"' "$SCOPE_JSON"}
49
+ Files in the diff with no stated reason (flag as scope drift if the change is not obviously required):
50
+ {jq -r '.unjustified[]' <<< "$SCOPE_GATE"}
51
+ Deliberately not done (do not raise these as findings; they are known):
52
+ {jq -r '.notDone[] | "- \(.what) (\(.why))"' "$SCOPE_JSON"}
53
+ </scope-self-check>
54
+ ```
55
+
56
+ A missing record renders the block with `no scope-check.json written` and a `review.scope_check=missing` metric; the review proceeds, and the absence is itself information for the reviewer.
57
+
58
+ ## Step 3.8: delta + trigger 2
59
+
60
+ ```bash
61
+ TRIP=$(jq -r '.global.autopilotCircuitBreaker.identicalFindingCycles // 2' "$PREFS_FILE")
62
+ DELTA_JSON=$(node $HOME/.claude/scripts/review-delta.mjs --rounds-dir "$WORKTREE/.pipeline" --iteration "$ITERATION" --trip-cycles "$TRIP"); DELTA_RC=$?
63
+ jq -c --argjson d "$DELTA_JSON" --argjson i "$((ITERATION-1))" \
64
+ '{reviewIterations: (.reviewIterations | .[$i] += {delta: ($d + {computedAt: (now | todate)})})}' "$STATE_FILE" \
65
+ | node $HOME/.claude/scripts/write-state.mjs "$STATE_FILE"
66
+ $HOME/.claude/scripts/log-metric.sh "$TASK_ID" 4 review.delta iteration=$ITERATION \
67
+ new=$(jq '.counts.new // 0' <<< "$DELTA_JSON") still_present=$(jq '.counts.stillPresent // 0' <<< "$DELTA_JSON") \
68
+ resolved=$(jq '.counts.resolved // 0' <<< "$DELTA_JSON") plateau=$(jq '.plateau // false' <<< "$DELTA_JSON") tripped=$([ "$DELTA_RC" -eq 3 ] && echo true || echo false)
69
+ ```
70
+
71
+ | Exit | Meaning | Action |
72
+ |---|---|---|
73
+ | 0 | Progress, or nothing to compare (iteration 1, missing previous round) | Continue to Step 4. |
74
+ | 3 | A blocking/important finding survived `identicalFindingCycles` consecutive rework cycles | **Autopilot** with `enabled`: trip the breaker. `state.circuitBreaker = {tripped: true, trigger: 2, detail: "finding <fingerprint> (<file>: <issue>) survived <n> consecutive rework cycles", checkpoint: {phase: 4, step: "3.8", iteration: N}, trippedAt, counters: {identicalFindingCycles: <max recurrence>, reworkCycles: N-1}}`, then the halt-visibility protocol from `phases/operations.md` (`status=paused`, `haltReason="4:circuit-breaker:identical-finding"`, tracker meta, the `>&2 HALT` line, the usage report). **Interactive modes**: do not trip; show `delta.stillPresent` and ask (picker-contract) `Continue rework` / `Escalate to me` / `Accept as deferred`, the last moving those findings to `deferred[]` with reason `circuit-breaker: accepted by user after <n> cycles`. |
75
+ | 1 | Unreadable round file | Log `review.delta_skipped reason=invalid` and continue; the delta is advisory and never blocks on its own failure. |
76
+
77
+ `plateau` (the still-present set unchanged from the previous delta) is logged, not acted on: at the default threshold it coincides with trigger 2 and with the Phase 3 `retryCount` hard-kill. The rework-storm cap itself is trigger 3, recorded by the Phase 3 re-entry. The Phase 3 reflection prompt names each accepted finding by `fingerprint` and quotes `delta.stillPresent` first, marked `STILL PRESENT after round N-1's fix`.
78
+
79
+ ## Why fingerprints are computed, not stored
80
+
81
+ Same argument `shortId()` in `_retrieval.mjs` makes for corpus rows: derived from the finding's own content, so existing triage files get ids without a migration and a re-annotated file keeps the ids it had. The known trade-off is over-merging: two findings in one file that differ only by a number share a fingerprint. The delta output prints `file: issue` beside every id and the trip needs two consecutive recurrences, so a merge is visible and cannot halt a run by itself. Under-merging (a reviewer that rewrites rather than echoes) only ever suppresses a trip; it never causes one.
82
+
83
+ ## Telemetry
84
+
85
+ `review.delta` once per iteration >= 2: `iteration`, `new`, `still_present`, `resolved`, `plateau`, `tripped`. `run-metrics.mjs` reports `reviewDelta.stillPresentFinal`, `resolvedTotal` and `tripped`; Phase 7 renders the three counts per round.
86
+
87
+ ## Reference
88
+
89
+ Scripts: `$HOME/.claude/scripts/_fingerprint.mjs`, `finding-fingerprint.mjs`, `review-delta.mjs`. Schemas: `reviewer-output` 1.2.0, `triage-output` 3.4.0, `dev-critic-output` (optional `fingerprint`), `agent-state` (`reviewIterations[].delta`, `circuitBreaker`). Breaker: `features/autopilot-circuit-breaker.md`. Smokes: `smoke-review-delta.sh`, `smoke-autopilot-circuit-breaker.sh`.
@@ -0,0 +1,41 @@
1
+ # Feature: Scope self-check (Phase 3 Step 3.7)
2
+
3
+ **Pattern**: Phase 4 reconstructs everything from the diff. The one thing it cannot reconstruct is why each file was touched and what was left out on purpose, so Dev states both before the handoff, and a deterministic gate checks the statement against the real diff. The record also carries the code-simplifier rationales (Step 3.6) that used to be discarded, and it feeds two later consumers: the `<scope-self-check>` block in the Phase 4 reviewer prefix and the PR body in Phase 6.
4
+
5
+ ## The record
6
+
7
+ `$WORKTREE/.pipeline/scope-check.json`, schema `schemas/scope-check.schema.json`:
8
+
9
+ ```json
10
+ {
11
+ "version": "1.0.0",
12
+ "taskId": "<taskId>",
13
+ "files": [{ "path": "<repo-relative path>", "reason": "<why this task needs this exact file>" }],
14
+ "notDone": [{ "what": "<change considered and not made>", "why": "out-of-scope | follow-up | rejected-abstraction" }],
15
+ "simplifier": { "applied": 0, "skipped": 0, "rationales": ["<Step 3.6 rationale strings>"] }
16
+ }
17
+ ```
18
+
19
+ Rules:
20
+
21
+ - One entry per file in the diff, no entry for a file outside it. A reason names the task requirement, never "while I was here".
22
+ - `notDone` carries every hypothetical the dev chose not to defend against and every abstraction it considered and rejected, so review does not re-propose them.
23
+ - Component tasks (`taskType === "component"`) write the record too; the plugin skill's file table is the `files[]` source.
24
+
25
+ ## The gate
26
+
27
+ ```bash
28
+ git -C "$WORKTREE" diff --name-only "origin/$BASE_BRANCH"...HEAD \
29
+ | node $HOME/.claude/scripts/scope-check-gate.mjs --check "$WORKTREE/.pipeline/scope-check.json" --diff-files -
30
+ ```
31
+
32
+ Output `{ok, justified, unjustified[], unlisted[], notDone}`. Exit 1 lists `unjustified[]` (in the diff, no reason or an empty one); `unlisted[]` (in the record, not in the diff) is reported and never fatal. Complete the record once and re-run. A second exit 1 does not block: log `dev.scope_check=incomplete unjustified=<n>` and hand off; Phase 4 receives the gate output inside `<scope-self-check>`, so reviewers see which files arrived without a stated reason. `--advisory` (used by Phase 4) always exits 0. Progress line: ` → checking scope self-check ({n} files, {m} not done)`.
33
+
34
+ ## Consumers
35
+
36
+ - Phase 4 Step 2.2 renders file reasons, unjustified files and `notDone[]` into the shared reviewer prefix (`features/review-delta.md`).
37
+ - Phase 6 Step 3 builds the PR `## Changes` bullets from `files[].reason` and lists `notDone[]` under `## Related` as "Follow-ups not done in this PR", merged with the final triage `deferred[]` (`channels/pr.md`).
38
+
39
+ ## Reference
40
+
41
+ Script: `$HOME/.claude/scripts/scope-check-gate.mjs`. Schema: `$HOME/.claude/schemas/scope-check.schema.json`. Smoke: `smoke-scope-check.sh`. Unit: `test/scope-check-gate.test.mjs`.
@@ -5,7 +5,7 @@
5
5
  **Gated by `prefs.global.verifyByTest.enabled`** (default: `false`). When enabled, after triage 3.6 and before Step 4, IF the validated triage output contains at least one `accepted` blocking finding:
6
6
 
7
7
  1. Dispatch ONE verifier sub-agent for the iteration (model: `verifyByTest.model`, default `sonnet`) - never one dispatch per finding. Input: up to `verifyByTest.maxFindings` (default 3) accepted blocking findings, the diff hunks for their files, and the Phase 1 test conventions.
8
- 2. Per finding, the verifier writes ONE minimal repro test asserting the correct behavior the finding claims is broken, then runs ONLY that test via the Phase 3 single-test invocation (`xcodebuild test -only-testing:`, `pytest {file}::{name}`, `npm test -- --testPathPattern=`, `./gradlew test --tests`) under `acquire_build_lock`/`release_build_lock`, log tee'd to `$WORKTREE/.pipeline/verify-<i>.test.log`.
8
+ 2. Per finding, the verifier writes ONE minimal repro test asserting the correct behavior the finding claims is broken, then runs ONLY that test via the Phase 3 single-test invocation (`xcodebuild test -only-testing:`, `pytest {file}::{name}`, `npm test -- --testPathPattern=`, `./gradlew test --tests`) under `acquire_build_lock`/`release_build_lock`. One green run is not trusted: the same single-test invocation runs `verifyByTest.repeatCount` times (default 3; value 1 disables the repeat) as a shell loop, each run's log tee'd to `$WORKTREE/.pipeline/verify-<i>-<k>.test.log` for `k` in `1..repeatCount`. ALL runs must pass, and each log must clear `evidence-gate.mjs --claim test --status passed`, before the outcome counts as "passes". A first run that fails as predicted ends the loop early (`confirmed` needs no repeat).
9
9
  3. Stamp each processed finding with a `verification` object (triage-output schema v3.2.0) and re-run `validate-triage.mjs` on the mutated triage file under the standard 3.2.1 gate protocol.
10
10
  4. Findings beyond `maxFindings` keep their judgment-only verdict (log `verify_by_test=cap-exceeded`).
11
11
  5. The whole step is bounded by `verifyByTest.stepTimeoutSec` (default 600); on breach or verifier crash, remaining findings keep judgment-only verdicts and the pipeline proceeds. Never blocks.
@@ -15,7 +15,8 @@
15
15
  | Repro test outcome | `verification.result` | Action |
16
16
  |---|---|---|
17
17
  | Fails as the finding predicts | `confirmed` | Finding stays accepted blocking. Repro test KEPT in the worktree, recorded in `redTests[]`. |
18
- | Passes (not reproducible) | `not-reproduced` | Downgrade gated by `evidence-gate.mjs --claim test --status passed` on the test log (exit 0 required). Finding moves `accepted[]` -> `deferred[]` with reason `verify-by-test: not reproduced - repro test <testRef> passed`; repro test file deleted. Evidence-gate failure -> treat as `inconclusive`. |
18
+ | Passes (not reproducible) | `not-reproduced` | Only when ALL `verifyByTest.repeatCount` runs pass and every `verify-<i>-<k>.test.log` clears `evidence-gate.mjs --claim test --status passed` (exit 0 required on each). Finding moves `accepted[]` -> `deferred[]` with reason `verify-by-test: not reproduced - repro test <testRef> passed`; repro test file deleted. Evidence-gate failure on any run -> treat as `inconclusive`. |
19
+ | Passes on some runs, fails on others | `inconclusive` | Flake, not proof either way. Finding stays accepted blocking, `verification.note = "flaky: passed k/N"`, repro test file deleted, telemetry line gains `flaky=<count>`. |
19
20
  | Compile error / timeout / not unit-testable | `inconclusive` | Finding stays accepted blocking (judgment stands). Partial test deleted, cause in `verification.note`. |
20
21
 
21
22
  Downgrades go to `deferred`, never `rejected`: triage judged the issue real, and deferred items surface in the Phase 7 report for a human eye.
@@ -30,12 +31,12 @@ After Step 3.7, the only uncommitted verifier artifacts are the confirmed repro
30
31
 
31
32
  ## Telemetry
32
33
 
33
- One `review.verify_by_test` metric per iteration: `attempted`, `confirmed`, `downgraded`, `inconclusive`, `duration_ms` (plus `tokens_in/out` when available), forwarded to the tracker like all Phase 4 metrics. Timeout emits `triage=verify-by-test-timeout`.
34
+ One `review.verify_by_test` metric per iteration: `attempted`, `confirmed`, `downgraded`, `inconclusive`, `flaky` (findings whose repeat runs disagreed), `duration_ms` (plus `tokens_in/out` when available), forwarded to the tracker like all Phase 4 metrics. Timeout emits `triage=verify-by-test-timeout`.
34
35
 
35
36
  ## Off by default reason
36
37
 
37
- Adds one Sonnet call plus up to `maxFindings` single-test runs (and build-lock contention on Xcode projects) per review iteration that has accepted blockers. On clean runs it never fires, but on noisy-reviewer repos it can add minutes per iteration. Flip on for security-critical work, release branches, or repos where reviewer false-positive rate is high.
38
+ Adds one Sonnet call plus up to `maxFindings` single-test runs (and build-lock contention on Xcode projects) per review iteration that has accepted blockers. Each passing repro test is re-run `verifyByTest.repeatCount` times (default 3), so a downgrade costs up to `repeatCount` single-test runs rather than one. On clean runs it never fires, but on noisy-reviewer repos it can add minutes per iteration. Flip on for security-critical work, release branches, or repos where reviewer false-positive rate is high.
38
39
 
39
40
  ## Reference
40
41
 
41
- Wiring: `$HOME/.claude/multi-agent-refs/phases/phase-4-review.md` Step 3.7. Schema: `$HOME/.claude/schemas/triage-output.schema.json` v3.2.0 (`$defs.verification`). Evidence gate: `$HOME/.claude/scripts/evidence-gate.mjs`. Prefs: `prefs.global.verifyByTest` in `$HOME/.claude/schemas/prefs.schema.json`.
42
+ Wiring: `$HOME/.claude/multi-agent-refs/phases/phase-4-review.md` Step 3.7. Schema: `$HOME/.claude/schemas/triage-output.schema.json` v3.2.0 (`$defs.verification`). Evidence gate: `$HOME/.claude/scripts/evidence-gate.mjs`. Prefs: `prefs.global.verifyByTest` (including `repeatCount`) in `$HOME/.claude/schemas/prefs.schema.json`.
@@ -49,7 +49,7 @@ This is why the removal is safe:
49
49
 
50
50
  | Consumer | Reads | Without salvage |
51
51
  |---|---|---|
52
- | Phase 7 triage-memory ingest | `triage-output.json` | `[ -f ]`-guarded, so it degrades **silently**: the triage corpus and learnings ledger stop being fed and no error appears |
52
+ | Phase 7 triage-memory ingest | `triage-output.json` | `[ -f ]`-guarded, so it degrades **silently**: the triage corpus and learnings ledger stop being fed and no error appears. Since v16.20.0 Phase 4 writes `triage-output.json` itself (Step 3.2.1, the latest copy of `.pipeline/triage-round-<N>.json`), so the salvage is a second copy, not the only bridge |
53
53
  | Phase 7 learnings-ledger distill | same file | same silent degradation |
54
54
  | `render-work-summary.sh` | `agent-state.json`, `phase-tracker.json` (falls back to `logs/multi-agent/<task>/tracker-state.json`, which survives removal) | loses the salvaged copies but keeps the tracker via the logs fallback |
55
55
  | `:resume` | `agent-state.json` | cannot continue a Phase 7 pause |
@@ -10,7 +10,7 @@ description: "Canonical required-reading list for outward-facing payloads (PR bo
10
10
 
11
11
  | Read | Before | Governs |
12
12
  |---|---|---|
13
- | [`channels/pr.md`]($HOME/.claude/multi-agent-refs/channels/pr.md) | assembling the PR body | fixed section set (`summary` → `changes` → `architecture` cond. → `verification` → `dependencies` cond. → `related`), Markdown-only rule, reviewer-preserving Bitbucket PUT payload |
13
+ | [`channels/pr.md`]($HOME/.claude/multi-agent-refs/channels/pr.md) | assembling the PR body | fixed section set (`summary` → `changes` → `architecture` cond. → `verification` → `risk` cond. → `dependencies` cond. → `related`), Markdown-only rule, reviewer-preserving Bitbucket PUT payload |
14
14
  | [`phases/phase-6-commit.md`]($HOME/.claude/multi-agent-refs/phases/phase-6-commit.md) | committing | commit convention, default-reviewer fetch, draft/ready prompt, push-must-succeed loop |
15
15
  | [`rules.md`]($HOME/.claude/multi-agent-refs/rules.md) "External System Outputs" | any REST payload | real newlines, no HTML entities, no hand-rolled JSON, markup dialect per surface |
16
16