@mmerterden/multi-agent-pipeline 20.37.0 → 20.37.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +25 -0
- package/docs/facts.json +2 -2
- package/docs/features.md +1 -1
- package/docs/recovery-guide.md +1 -1
- package/install/catalog-history.json +1 -1
- package/manifest.json +40 -35
- package/package.json +1 -1
- package/pipeline/commands/multi-agent/manual-test/SKILL.md +12 -7
- package/pipeline/multi-agent-refs/audit-guide.md +20 -0
- package/pipeline/multi-agent-refs/features/commit-multi-repo.md +63 -0
- package/pipeline/multi-agent-refs/features/dev-multi-repo.md +51 -0
- package/pipeline/multi-agent-refs/features/test-gap.md +40 -0
- package/pipeline/multi-agent-refs/features/worktree-finalize.md +1 -1
- package/pipeline/multi-agent-refs/orchestrator/phase-3-user-test.md +12 -9
- package/pipeline/multi-agent-refs/phases/phase-1-plan.md +4 -4
- package/pipeline/multi-agent-refs/phases/phase-2-dev.md +8 -47
- package/pipeline/multi-agent-refs/phases/phase-3-review.md +21 -65
- package/pipeline/multi-agent-refs/phases/phase-4-commit.md +9 -59
- package/pipeline/multi-agent-refs/phases/phase-5-report.md +0 -31
- package/pipeline/multi-agent-refs/rules.md +1 -1
- package/pipeline/office/index.html +6 -3
- package/pipeline/office/toolkit.json +1 -1
- package/pipeline/rules/figma-pipeline.md +6 -3
- package/pipeline/schemas/prefs.schema.json +1 -1
- package/pipeline/scripts/_memory-policy.mjs +27 -5
- package/pipeline/scripts/checkpoint-fold.sh +59 -0
- package/pipeline/scripts/gate-ledger.mjs +1 -1
- package/pipeline/scripts/office-control.mjs +41 -15
- package/pipeline/scripts/offload-ref.sh +1 -1
- package/pipeline/scripts/rework-rulings.mjs +5 -3
- package/pipeline/scripts/scope-check-gate.mjs +15 -2
- package/pipeline/scripts/user-test-handoff.sh +203 -0
- package/pipeline/scripts/validate-planning.mjs +5 -3
- package/pipeline/scripts/validate-triage.mjs +2 -1
- package/pipeline/scripts/write-state.mjs +13 -4
- package/pipeline/skills/.skill-manifest.json +2 -2
- package/pipeline/skills/shared/core/multi-agent-manual-test/SKILL.md +12 -7
|
@@ -221,7 +221,7 @@ Progress line: `→ analysis doc: <produced|reused|not-applicable> (<N> platform
|
|
|
221
221
|
|
|
222
222
|
Two artefacts, both read downstream: `state.analysis` (the object below, for Phase 1 decomposition) and the Step 4 document (Phase 1 pre-flight + Phase 2's sole design source, Locked 29).
|
|
223
223
|
|
|
224
|
-
Phase 1 produces an object conforming to `$HOME/.claude/schemas/analysis-output.schema.json` and persists it to `state.analysis`. Required fields (exact names per the schema): `stack` (detected stack identifier + primary language), `touchedAreas[]` (path + why), `risks[]` (existing-code hazards/observations the planner must respect - each `{risk, severity, mitigation}`; use an empty array when none), `summary` (one-paragraph human-readable).
|
|
224
|
+
Phase 1 produces an object conforming to `$HOME/.claude/schemas/analysis-output.schema.json` and persists it to `state.analysis`. Required fields (exact names per the schema): `stack` (detected stack identifier + primary language), `touchedAreas[]` (path + why), `risks[]` (existing-code hazards/observations the planner must respect - each `{risk, severity, mitigation}`; use an empty array when none), `summary` (one-paragraph human-readable). The planning half reads this object as its sole input (Input contract below).
|
|
225
225
|
|
|
226
226
|
**Required: validator gate (deterministic) - run on the persisted file immediately after the analysis object is produced; the validator's exit code decides, not the LLM turn:**
|
|
227
227
|
|
|
@@ -273,7 +273,7 @@ Contract and rationale: `progress-contract.md` -> Token telemetry forwarding.
|
|
|
273
273
|
|
|
274
274
|
## Mid-phase checkpoint (BLOCKING)
|
|
275
275
|
|
|
276
|
-
The planning half consumes what the analysis half just produced.
|
|
276
|
+
The planning half consumes what the analysis half just produced. **Figma** MCP / REST forbidden; the toolkit MCP is not.
|
|
277
277
|
|
|
278
278
|
1. **Analysis document presence**: read `state.analysis.docStatus`, which Step 4 above set.
|
|
279
279
|
- `produced` | `reused` -> the file is at `state.analysis.docPath[]`; continue with steps 2-4.
|
|
@@ -469,9 +469,9 @@ reasons=$(jq -r '.reasons[] | "• \(.rule) (+\(.weight)): \(.detail)"' <<< "$ve
|
|
|
469
469
|
| `false` | Skip 5a/5b as before - autopilot proceeds to Phase 2. Log `🧠 Phase 1: Safety classifier - score {N}, autopilot proceeds`. |
|
|
470
470
|
| `true` | Inject a one-time manual approval prompt even though we are in autopilot. Render the plan (5b shape) with the `reasons[]` list prepended. User sees `⚠️ Autopilot safety gate tripped (score {N})` banner and must explicitly choose Approve / Cancel (or edit via Other) in the 5b `AskUserQuestion` picker. Log `🧠 Phase 1: Safety classifier - score {N}, autopilot paused, reasons={rules}`. |
|
|
471
471
|
|
|
472
|
-
**
|
|
472
|
+
**Opt-out by design:** a pause on a high-blast-radius plan costs seconds, a silent merge of a bad one costs a revert. `prefs.global.autopilotSafetyGate = false` is for task classes the user has validated as safe.
|
|
473
473
|
|
|
474
|
-
**Rules
|
|
474
|
+
**Rules, weights and the trigger** live in the `classify-plan-safety.mjs` header, the only copy: it pauses on any single heavy signal (weight 30+), on two signals of weight 20+, or on a total score of 50.
|
|
475
475
|
|
|
476
476
|
**Telemetry:** emit `phase.plan.safety` OTel span (when `MULTI_AGENT_OTEL_SPANS=1`) with `{score, recommendPause, rules}` so post-hoc analysis can tune weights.
|
|
477
477
|
|
|
@@ -173,7 +173,7 @@ For each task (respecting dependency order), on start and on end: `bash $HOME/.c
|
|
|
173
173
|
git -C "{worktreePath}" commit -F "/tmp/wip-message-$TASK_ID.txt"
|
|
174
174
|
```
|
|
175
175
|
Write `wip({scope}): {task summary} [{jiraId}]` to `/tmp/wip-message-$TASK_ID.txt` with the file tool first. The summary comes from the issue, and inside a double-quoted `-m` the shell would run any `$(...)` in it.
|
|
176
|
-
|
|
176
|
+
Standard path item 3 of Phase 4 Step 2 folds these back before the real commit. Keep the subject shape: that step recognises a checkpoint by it. This prevents work loss if a later task fails or the session crashes.
|
|
177
177
|
7. Update task status: `completed` (apply the same verify rule - re-read the Status field and retry once on mismatch)
|
|
178
178
|
8. Log: "Phase 2: {task} - Test + Build passed"
|
|
179
179
|
|
|
@@ -279,53 +279,11 @@ Exit 1 lists `unjustified[]`: complete the record, re-run once. A second exit 1
|
|
|
279
279
|
|
|
280
280
|
#### v2.1.0+ Multi-Repo Mode
|
|
281
281
|
|
|
282
|
-
|
|
282
|
+
**Only when `state.projects[].length > 1`.** Load `$HOME/.claude/multi-agent-refs/features/dev-multi-repo.md` and follow it: the per-todo `repo` tag and worktree switch, one status update per todo, build serialization across worktrees, per-repo failure isolation with the evidence-gated `buildStatus`, and per-repo telemetry. A single-repo run skips it.
|
|
283
283
|
|
|
284
|
-
|
|
284
|
+
#### Token forwarding
|
|
285
285
|
|
|
286
|
-
|
|
287
|
-
{
|
|
288
|
-
"id": "T-3",
|
|
289
|
-
"title": "Add tokenResolver public API",
|
|
290
|
-
"repo": "common", // ← matches state.projects[i].name
|
|
291
|
-
"files": ["Sources/Common/TokenResolver.swift"],
|
|
292
|
-
"tdd": { "test": "...", "code": "..." }
|
|
293
|
-
}
|
|
294
|
-
```
|
|
295
|
-
|
|
296
|
-
If a todo has no `repo` tag in multi-repo mode → log warning + ask user, do not auto-pick.
|
|
297
|
-
|
|
298
|
-
**Per-todo worktree switch**: Before each todo's TDD cycle, resolve `WT_PATH = state.projects[ findIndex(name === todo.repo) ].worktreePath` and run **all** subsequent commands with `git -C "$WT_PATH"` and `cd "$WT_PATH"` for build/test. Identity (`user.name`/`user.email`) was already pinned per worktree by Phase 0 Step 8 - no re-config needed here.
|
|
299
|
-
|
|
300
|
-
**Status update (single ticket, multi-repo)**: The Jira/GitHub issue is shared across all repos in the group - call the status-update API exactly **once per todo** (not once per repo per todo). The post-mutation verify rule still applies.
|
|
301
|
-
|
|
302
|
-
**Build serialization**:
|
|
303
|
-
- Xcode worktrees still share `BUILD_LOCK = /tmp/claude-xcodebuild.lock` - multi-repo just means more callers contending for the same lock; the existing `mkdir`-based queue handles it correctly.
|
|
304
|
-
- Non-Xcode repos may build in parallel across worktrees - but ONLY when builds are independent. If repo B's build depends on repo A's freshly-published artifact (e.g. SPM local path dependency), serialize them in dependency order. Plan items declare dependencies via Phase 1's `dependsOn` field; multi-repo dispatch respects that order.
|
|
305
|
-
|
|
306
|
-
**Failure isolation**: A build/test failure in one repo's worktree must NOT silently leave the other repos in a half-built state. On failure:
|
|
307
|
-
1. Capture the failure into `state.projects[i].buildStatus = { ok: false, attempts: N, lastError: "..." }`
|
|
308
|
-
2. The ENTIRE Phase 2 retries within that repo's worktree (max 3, same as single-repo)
|
|
309
|
-
3. After 3 attempts, escalate; do NOT proceed to Phase 3 with any unbuilt repo
|
|
310
|
-
|
|
311
|
-
**Recording a pass (default-FAIL evidence gate):** before setting `buildStatus.ok = true`, the build output must be tee'd to a log and that log must substantiate the success - a zero exit code alone is not trusted. Run the evidence gate; on exit 1, do NOT record a pass:
|
|
312
|
-
```bash
|
|
313
|
-
<build-command> 2>&1 | tee "{worktreePath}/.build.log" \
|
|
314
|
-
| bash $HOME/.claude/scripts/offload-ref.sh --phase 3 --label build --root "$WORKTREE"
|
|
315
|
-
node $HOME/.claude/scripts/evidence-gate.mjs --claim build --status passed --evidence "$WORKTREE/.build.log"
|
|
316
|
-
```
|
|
317
|
-
Non-zero from the gate: the build pass is unverified, so treat it as a failure and keep `buildStatus.ok=false`.
|
|
318
|
-
This closes the gap where an agent records "built" without ever producing build output.
|
|
319
|
-
|
|
320
|
-
**Why the pipe (opt-in via `prefs.global.contextOffload.enabled`).** `tee` decides where the log is written, not how much of it the model reads. The filter parks the full text at `.multi-agent/refs/<node_id>.md` and prints a stub plus the tail, where a failing build's error already is; read that file before re-running a failed build. The evidence gate still reads the whole `.build.log`, so what counts as a verified pass is unchanged. Pref off = pass-through.
|
|
321
|
-
|
|
322
|
-
**Telemetry**: Per-repo metrics in addition to per-task metrics:
|
|
323
|
-
```bash
|
|
324
|
-
$HOME/.claude/scripts/log-metric.sh "$TASK_ID" 2 build.completed repo=common duration_ms=$D status=ok
|
|
325
|
-
$HOME/.claude/scripts/log-metric.sh "$TASK_ID" 2 build.completed repo=uicomponents duration_ms=$D status=ok
|
|
326
|
-
```
|
|
327
|
-
|
|
328
|
-
**Token forwarding:** every TDD round (red, green, refactor) that hits the dev model MUST forward token totals into the tracker so Phase 5's Cost Breakdown captures Phase 2:
|
|
286
|
+
Every TDD round (red, green, refactor) that hits the dev model MUST forward token totals into the tracker so Phase 5's Cost Breakdown captures Phase 2:
|
|
329
287
|
|
|
330
288
|
```bash
|
|
331
289
|
LOG_METRIC_FORWARD_TO_TRACKER=1 $HOME/.claude/scripts/log-metric.sh "$TASK_ID" 2 dev.tdd_round \
|
|
@@ -399,7 +357,8 @@ Review inherits the logs instead of regenerating them.
|
|
|
399
357
|
# Gate 0: Owned paths (repo profile), before the build
|
|
400
358
|
node $HOME/.claude/scripts/owned-path-gate.mjs --repo "$WORKTREE" --base "origin/$BASE_BRANCH" --state "$STATE_FILE"
|
|
401
359
|
# Gate 1: Build (xcodebuild/gradle assemble/tsc/py compile - stack-dependent; Xcode uses the build queue lock, see Phase 2) - tee output to a log
|
|
402
|
-
<build-command> 2>&1 | tee "{worktreePath}/.build.log"
|
|
360
|
+
<build-command> 2>&1 | tee "{worktreePath}/.build.log" \
|
|
361
|
+
| bash $HOME/.claude/scripts/offload-ref.sh --phase 2 --label build --root "$WORKTREE"
|
|
403
362
|
# Gate 2: Lint (swiftlint/ktlint/ruff/eslint - stack-dependent)
|
|
404
363
|
# Gate 3: Tests pass (xcodebuild/gradle/pytest/the resolved node command) - tee output to a log
|
|
405
364
|
<test-command> 2>&1 | tee "{worktreePath}/.test.log"
|
|
@@ -419,6 +378,8 @@ node $HOME/.claude/scripts/evidence-gate.mjs --claim test --status passed --evi
|
|
|
419
378
|
|
|
420
379
|
This prevents a false "it built" claim with no log behind it. On exit 1, treat the gate as failed (do NOT proceed to AI review) and surface the gate's `reason`.
|
|
421
380
|
|
|
381
|
+
**Why the pipe (opt-in via `prefs.global.contextOffload.enabled`).** `tee` decides where the log is written, not how much of it the model reads. The filter parks the full text at `.multi-agent/refs/<node_id>.md` and prints a stub plus the tail, where a failing build's error already is; read that file before re-running a failed build. The evidence gate still reads the whole `.build.log`, so what counts as a verified pass is unchanged. Pref off = pass-through.
|
|
382
|
+
|
|
422
383
|
**Autopilot runs** (`state.autopilot` or `MULTI_AGENT_UNATTENDED=1`) also run the stack-aware checks, each recording its verdict in `state.gates[]`: `evidence-gate.mjs --stack`, `test-summary.mjs` (zero tests executed parks the run as `verification-failed` instead of reworking), `test-strength.mjs --head worktree` and `symbol-existence-gate.mjs`. With the variable set, the Phase 4 commit hook refuses a commit whose ledger lacks them. Commands and verdicts: `$HOME/.claude/multi-agent-refs/features/unattended-gates.md` section 4.
|
|
423
384
|
|
|
424
385
|
**Scaffolded repo** (`.scaffold.json` at the worktree root): at this phase's entry and exit, `scaffold-gate.mjs --dir "$WORKTREE" --phase story --story <id>`; exit 1 or 3 halts and the next story does not start; 2 is a call error, never a pass (`$HOME/.claude/multi-agent-refs/features/scaffold.md`).
|
|
@@ -437,7 +437,7 @@ bash $HOME/.claude/scripts/log-metric.sh "$TASK_ID" 3 memory.hit rows=$CITED_COU
|
|
|
437
437
|
**Bulky payloads (opt-in via `prefs.global.contextOffload.enabled`).** Test output and whole-file diffs go through the offload filter, which leaves a `[[ref:<node_id>]]` line plus the tail in context and the full text under `.multi-agent/refs/`. Read that file when the tail is not enough; with the pref off it is a pass-through.
|
|
438
438
|
|
|
439
439
|
```bash
|
|
440
|
-
<test-command> 2>&1 | bash $HOME/.claude/scripts/offload-ref.sh --phase
|
|
440
|
+
<test-command> 2>&1 | bash $HOME/.claude/scripts/offload-ref.sh --phase 3 --label tests
|
|
441
441
|
```
|
|
442
442
|
|
|
443
443
|
**Triage prompt skeleton:**
|
|
@@ -497,7 +497,7 @@ One file per round; `triage-output.json` is the latest copy every reader expects
|
|
|
497
497
|
| ----- | ---------------------------- | ------------------------------------------------- |
|
|
498
498
|
| **0** | Valid and clean | Act on triage output as-is |
|
|
499
499
|
| **1** | Invalid structure | Gate protocol (strict, fails CLOSED): (a) emit the validator's stderr and `errors[]` JSON into the log verbatim; (b) attempt ONE self-correction rework - re-prompt triage with the errors quoted, overwrite `$TRIAGE_FILE`; (c) re-run the validator. Second failure -> HALT the phase with the recovery hint: `ERR: triage output failed validate-triage.mjs twice. Inspect $TRIAGE_FILE against $HOME/.claude/schemas/triage-output.schema.json, fix or regenerate it, then resume with /multi-agent:resume #N.` |
|
|
500
|
-
| **2** | Over-rejection guard tripped | Pause for human confirm (autopilot: log + accept) |
|
|
500
|
+
| **2** | Over-rejection guard tripped | Pause for human confirm (autopilot: log + accept); act on `result.corrected` when set |
|
|
501
501
|
| **3** | Contradiction auto-corrected | Proceed with `result.corrected` |
|
|
502
502
|
|
|
503
503
|
Capture stdout into `state.reviewIterations[-1].validatorResult` for Phase 5 audit.
|
|
@@ -649,36 +649,7 @@ echo "$GAP_JSON" | node $HOME/.claude/scripts/validate-test-gap.mjs - >/dev/null
|
|
|
649
649
|
|
|
650
650
|
Skipped when `prefs.global.testGap.enabled` is `false`. Add `--scan-tree` when `testGap.scanTree` is true and `--severity-promote` when `testGap.promoteSeverity` is. Exit 3: the analysed stack (`ios|swift`, `android|kotlin`, `python`, `node|typescript|js`) has no gap rules, so there is no report.
|
|
651
651
|
|
|
652
|
-
|
|
653
|
-
|
|
654
|
-
| Field | Meaning |
|
|
655
|
-
|---|---|
|
|
656
|
-
| `gaps[].sourcePath` | source file with the unprotected symbol |
|
|
657
|
-
| `gaps[].symbol` | symbol name |
|
|
658
|
-
| `gaps[].kind` | rule id (e.g. `public_func`, `composable_fun`, `named_export`) |
|
|
659
|
-
| `gaps[].severity` | `blocking` / `important` / `suggestion` (see severity table below) |
|
|
660
|
-
| `gaps[].expectedTestPaths` | likely test paths the user should land at, in priority order |
|
|
661
|
-
| `gaps[].hint` | stack-specific testing reminder (e.g. swiftui-qa.md 3-layer) |
|
|
662
|
-
|
|
663
|
-
**Severity defaults**:
|
|
664
|
-
|
|
665
|
-
| Symbol kind | Severity |
|
|
666
|
-
|---|---|
|
|
667
|
-
| `composable_fun`, `view_struct`, `config_struct`, `interface`, `objc_export`, `public_proto` | important |
|
|
668
|
-
| Other public API additions (`public_func`, `open_fun`, `named_export`, `default_export`, ...) | suggestion |
|
|
669
|
-
|
|
670
|
-
**Gating** (opt-in): if `prefs.testGap.blockingThreshold` is set and `gapBySeverity.important + gapBySeverity.blocking` exceeds it, Phase 3 surfaces the report as a Phase 3 rework finding and loops back. **Default off** - gaps render as advisory under "Test Gap Report" only.
|
|
671
|
-
|
|
672
|
-
**Telemetry**:
|
|
673
|
-
|
|
674
|
-
```bash
|
|
675
|
-
LOG_METRIC_FORWARD_TO_TRACKER=0 $HOME/.claude/scripts/log-metric.sh "$TASK_ID" 3 test_gap.scanned \
|
|
676
|
-
stack=$SCAN_STACK \
|
|
677
|
-
sources=$(jq '.totals.sourcesScanned' <<< "$GAP_JSON") \
|
|
678
|
-
gaps=$(jq '.totals.gapCount' <<< "$GAP_JSON")
|
|
679
|
-
```
|
|
680
|
-
|
|
681
|
-
(No tracker forwarding - the scanner has no token cost.)
|
|
652
|
+
When a report came back, load `$HOME/.claude/multi-agent-refs/features/test-gap.md` for its fields, severity defaults, the opt-in `prefs.testGap.blockingThreshold` gate (default off, advisory only) and the `test_gap.scanned` metric.
|
|
682
653
|
|
|
683
654
|
**Figma reference panel (when `state.evidence.figma[]` is non-empty).** Before the local-checkout prompt, print a single block listing each captured frame so the user has a side-by-side reference during manual test:
|
|
684
655
|
|
|
@@ -700,22 +671,19 @@ Tier 1 / Tier 2 records print `screenshotUrl` from the captured evidence (Tier 2
|
|
|
700
671
|
- `{ label: "Skip", description: "Stay in the worktree and go to Phase 4" }`
|
|
701
672
|
- **Skip** → set `state.phases["3"].status = "skipped"` (so Phase 4 can offer the local-checkout prompt) → Phase 4
|
|
702
673
|
- **Test now** → set `state.phases["3"].status = "in_progress"` → continue:
|
|
703
|
-
2. **
|
|
704
|
-
```
|
|
705
|
-
git -C {worktree-path} add -A
|
|
706
|
-
git -C {worktree-path} commit -m "WIP: {jiraId} - changes for user test"
|
|
707
|
-
|
|
708
|
-
3. Remove worktree, checkout branch in main repo:
|
|
709
|
-
```
|
|
710
|
-
git worktree remove .worktrees/{jiraId} --force
|
|
711
|
-
git checkout {branch-name}
|
|
674
|
+
2. **Hand the branch over.** Make the WIP commit yourself, so it passes the same approval and secret-scan hooks as any commit (Standard path item 3 of Phase 4 Step 2 folds it back with Phase 2's checkpoints). The script then holds the run artefacts and the main repo's current branch, removes the worktree and checks the branch out in the main repo. `{worktree-path}` and `{jiraId}` come from the state (`worktreePath`, `jiraId`):
|
|
675
|
+
```bash
|
|
676
|
+
git -C "{worktree-path}" add -A
|
|
677
|
+
git -C "{worktree-path}" diff --cached --quiet || git -C "{worktree-path}" commit -m "WIP: {jiraId} - changes for user test"
|
|
678
|
+
bash $HOME/.claude/scripts/user-test-handoff.sh checkout --task "{jiraId}" --worktree "{worktree-path}" --root "$PROJECT_ROOT" --branch "{branch-name}"
|
|
712
679
|
```
|
|
713
|
-
|
|
680
|
+
A refused commit stops here with nothing moved. Script exits: 3 the main repo has uncommitted changes; 5 the worktree or a submodule holds uncommitted, unpushed or stashed work (a submodule with no remote needs one added); both move nothing. 6 a handoff is already held, or git refused part way: run `restore` below (if it answers 7, nothing had moved). 1 the worktree is not on the branch.
|
|
681
|
+
3. On any failure report the reason and stop.
|
|
714
682
|
4. Show test instructions:
|
|
715
683
|
```
|
|
716
684
|
Switched to branch: {branch-name}
|
|
717
685
|
To test: Xcode -> build -> run -> manual test
|
|
718
|
-
"ok" -> proceeds to Phase 4 (
|
|
686
|
+
"ok" -> proceeds to Phase 4 (the checkpoint commits are folded into one proper commit)
|
|
719
687
|
"fix: ..." -> worktree is recreated, returns to Phase 2
|
|
720
688
|
```
|
|
721
689
|
5. Mark the tracker and the state as waiting (`local-test`, operations.md "Attended waits"), then wait for the user response:
|
|
@@ -725,6 +693,12 @@ Tier 1 / Tier 2 records print `screenshotUrl` from the captured evidence (Tier 2
|
|
|
725
693
|
```
|
|
726
694
|
The waiting state persists in `tracker-state.json` across the handoff; `/multi-agent:resume` and `/multi-agent:manual-test` CONTINUE this state file and never re-init it (`$HOME/.claude/multi-agent-refs/tracker-contract.md` "Continuation runs").
|
|
727
695
|
|
|
696
|
+
**Either answer starts by handing the branch back**, which puts the main repo on the branch it was on, re-adds the worktree and returns the held artefacts:
|
|
697
|
+
```bash
|
|
698
|
+
bash $HOME/.claude/scripts/user-test-handoff.sh restore --task "{jiraId}" --worktree "{worktree-path}" --root "$PROJECT_ROOT" --branch "{branch-name}"
|
|
699
|
+
```
|
|
700
|
+
Exit 3: the user edited files on the branch in the main repo. Ask them to `git stash`, run `restore` again, then `git -C "{worktree-path}" stash pop`; a commit made there instead would stop Phase 4 from folding the checkpoints. Exit 6: the main repo could not leave the branch (an untracked file in the way) or the worktree could not be re-added; the message names the cause, the hold is kept, and `restore` runs again once it is cleared. Exit 7: nothing was held, so the worktree was never removed.
|
|
701
|
+
|
|
728
702
|
**"ok" is a structured result, not a word.** Before "ok" is accepted, the run writes `$WORKTREE/.pipeline/manual-test.json`: one entry per acceptance criterion, the criteria taken from the analysis doc test plan (Section 15 / 20), the plan tasks, and the user's own words in the reply. Every criterion records what was seen; a criterion that was not tried says so with a reason.
|
|
729
703
|
```json
|
|
730
704
|
{"criteria":[{"spec":"<quote>","source":"analysis 15.2 | plan task 3 | user","observed":"<what was seen>","verdict":"pass|fail|not-tested","reason":"<required when not-tested>","screenshot":"<path or null>"}],"verdict":"passed|failed"}
|
|
@@ -735,23 +709,13 @@ Tier 1 / Tier 2 records print `screenshotUrl` from the captured evidence (Tier 2
|
|
|
735
709
|
```
|
|
736
710
|
Add `--require-screenshot` when `state.visualEvidence.required` is true: a passing criterion then has to name a file that exists, because a `screenshot` key pointing nowhere is not evidence.
|
|
737
711
|
|
|
738
|
-
Exit 1 means the "ok" is not accepted: tell the user which criterion is missing evidence (a `fail` verdict, or `not-tested` without a reason) and wait for the next reply. Exit 0 marks Phase 3 completed with `Result "local test passed (user)"`.
|
|
739
|
-
6. If fix needed:
|
|
740
|
-
- Branch already has WIP commit (from step 2) - changes are safe
|
|
741
|
-
- **Heal stale admin state first** (same contract as Phase 0 - step 3's
|
|
742
|
-
`worktree remove` or an interrupted run can leave a stale entry, so a bare
|
|
743
|
-
re-add fails with `already exists`/`already registered`):
|
|
744
|
-
```bash
|
|
745
|
-
git -C "$PROJECT_ROOT" worktree unlock "{worktree-path}" 2>/dev/null || true
|
|
746
|
-
git -C "$PROJECT_ROOT" worktree prune 2>/dev/null || true
|
|
747
|
-
```
|
|
748
|
-
Unlock first: prune skips a locked entry. Phase 0's repo residue guard is already in `.git/info/exclude` - no re-add.
|
|
749
|
-
- Recreate worktree from branch: `git -C $PROJECT_ROOT worktree add {worktree-path} {branch}`
|
|
712
|
+
Exit 1 means the "ok" is not accepted: tell the user which criterion is missing evidence (a `fail` verdict, or `not-tested` without a reason) and wait for the next reply. Exit 0 marks Phase 3 completed with `Result "local test passed (user)"`.
|
|
713
|
+
6. If fix needed (the worktree is back after the restore above):
|
|
750
714
|
- Re-set git identity: `git -C {worktree-path} config user.name/email` (from state)
|
|
751
715
|
- Go back to Phase 2
|
|
752
716
|
7. Log: "Phase 3: Review (user test) - {result}"
|
|
753
717
|
|
|
754
|
-
**CRITICAL**: Never remove
|
|
718
|
+
**CRITICAL**: Never remove the worktree by hand. `user-test-handoff.sh` is the only path: it refuses whenever work would be lost.
|
|
755
719
|
|
|
756
720
|
#### Automated Device Checks (on-demand)
|
|
757
721
|
|
|
@@ -767,18 +731,10 @@ Before or during user testing, run device-level audits via Bash if user requests
|
|
|
767
731
|
| Snapshot regression | Component / pixel-stable UI changes | ios: `mcp__multi-agent-toolkit__ios_visual_diff` · android: `mcp__multi-agent-toolkit__android_screenshot` + compare |
|
|
768
732
|
| Store screenshots | `taskType === screenshot` | ios: `ios_status_bar({preset: "clean"})` · android: `android_screenshot` |
|
|
769
733
|
|
|
770
|
-
|
|
771
|
-
|
|
772
|
-
Phase 2 Step 3.55 is the primary host and runs in every mode. Phase 3 is the richer one where it exists: the device is up and a person is watching, so the flow is one somebody confirmed. It adds, never replaces.
|
|
773
|
-
|
|
774
|
-
Run only when `visualEvidence.required` and `visualEvidence.video.file` is empty: re-check the device with `$HOME/.claude/scripts/probe-evidence-capability.sh --only device`, then `$HOME/.claude/scripts/capture-evidence.sh video start` -> drive the flow (`run-ui-tests.sh run`, or the Phase 3 scenarios by hand) -> `video stop` -> `fit`. The cap is `visualEvidence.maxVideoSeconds`, read through `capture-evidence.sh limits` so one value serves both. Update `videoTier` and `videoTierReason` with what ran.
|
|
775
|
-
|
|
776
|
-
When the intake answer was `unit` there is no recording here either: overriding it in a phase the user may not be watching makes the question decorative.
|
|
734
|
+
When `visualEvidence.required` is true and `visualEvidence.video.file` is empty, record the UI flow video now: load `$HOME/.claude/multi-agent-refs/audit-guide.md` "Phase 3 evidence flows", which also carries the snapshot regression flow.
|
|
777
735
|
|
|
778
736
|
Results included in Phase 5 report. MCP tools preferred when available - concise structured output, lower token cost.
|
|
779
737
|
|
|
780
|
-
**Snapshot regression flow (optional):** when the task changes a stable component, capture a screenshot before the change (baseline) and after (current), then call `ios_visual_diff({baseline, current, max_diff_pct: 1.0})`. Threshold can be relaxed for animated / non-deterministic regions - keep `max_diff_pct ≤ 1.0` for static layouts.
|
|
781
|
-
|
|
782
738
|
#### Telemetry - token forwarding
|
|
783
739
|
|
|
784
740
|
When the security-auditor or any other Phase 3 sub-agent runs, forward its token totals so Phase 5's Cost Breakdown captures Phase 3:
|
|
@@ -104,7 +104,11 @@ Branch **deterministically**, no implicit fallback. Read `agent-state.json` and
|
|
|
104
104
|
- Default: option 1 (by index, not by label - the label is localized)
|
|
105
105
|
- If the user picks option 2 (WIP checkout): follow Phase 3 Steps 2-6. On `"fix:"` → Phase 2. On `"ok"` → resume Phase 4.
|
|
106
106
|
2. Commit confirm prompt (skip in autopilot), in `outputLanguage`: "Commit now?" with "Commit" / "Pause and resume later", branched on the option index. No → pause (`resume` later). Yes → continue; where prompts are off, record it with `confirm-command.mjs --command '<the exact commit>'` (agent-guard asks on the run's commit).
|
|
107
|
-
3.
|
|
107
|
+
3. Fold the run's checkpoint commits back into the working tree: Phase 2's `wip(...): ... [{jiraId}]` commits and the user-test `WIP: {jiraId}` commit, counted from HEAD down and stopping at the first commit that is neither, so earlier commits on the branch are never touched. Item 4 stages and commits properly.
|
|
108
|
+
```bash
|
|
109
|
+
bash $HOME/.claude/scripts/checkpoint-fold.sh --repo "$WORKTREE" --task "{jiraId}" --base "{baseBranch}"
|
|
110
|
+
```
|
|
111
|
+
Exit 6: every commit down to the root is a checkpoint; stop and ask.
|
|
108
112
|
4. Stage changes: `git add` with specific files (NOT `git add -A` - avoid sensitive files, `.worktrees/`, agent files)
|
|
109
113
|
5. **Multi-submodule check**: If project has submodules (e.g. uicomponents + common):
|
|
110
114
|
- Detect which submodules have changes: `git -C {worktree} diff --name-only | cut -d'/' -f1-3 | sort -u`
|
|
@@ -113,7 +117,7 @@ Branch **deterministically**, no implicit fallback. Read `agent-state.json` and
|
|
|
113
117
|
6. Commit with convention: `{type}({scope}): {description} [{jiraId}]` or `[#{shortId}]`. When `node $HOME/.claude/scripts/repo-profile.mjs resolve commit.format --repo "$PROJECT_ROOT"` returns `use: true`, its `ticketPosition` places the ticket (`prefix` first, `suffix` last; `none` keeps this default unless confirmed or manual), `ticketStyle` its form and `scopeRequired` the scope. A profile hook with `amends-commit` or `writes-files` rewrites the commit: re-read the SHA after committing ([`features/repo-profile.md`]($HOME/.claude/multi-agent-refs/features/repo-profile.md)). **Local-only repos** (`projects[i].provider == "local"`) have no ticket, and that wins over any profile placement: the subject keeps the conventional prefix with no ticket, and the taskId (`LOCAL-...`) goes in the body footer: `Local-Task: {taskId}`.
|
|
114
118
|
6a. **Valven (required):** `features/valven.md`.
|
|
115
119
|
6b. **Record the commit**: operations.md "Completion evidence".
|
|
116
|
-
7. Push to remote - **skip per-repo when `projects[i].provider == "local"`** (no `git push`, no upstream config). Log `Phase 4: local-only commit {sha} (no push)` for that repo.
|
|
120
|
+
7. Push to remote through the Push-must-succeed policy below - **skip per-repo when `projects[i].provider == "local"`** (no `git push`, no upstream config). Log `Phase 4: local-only commit {sha} (no push)` for that repo.
|
|
117
121
|
8. Ask: "Want to open a Pull Request?" (attended wait, as in step 1)
|
|
118
122
|
- **Skip the prompt entirely when `state.offlineOnly == true`** (all repos local - no PR target exists). Proceed to Phase 5 with `pr.status = "skipped-offline"` for each project.
|
|
119
123
|
- In mixed mode (some local, some remote), only prompt for the remote-backed repos; local ones auto-skip.
|
|
@@ -257,25 +261,9 @@ Before creating PR, ask: "READY or DRAFT? [1/2], default `defaultPrMode` or 1"
|
|
|
257
261
|
|
|
258
262
|
#### v2.1.0+ Multi-Repo Mode
|
|
259
263
|
|
|
260
|
-
|
|
261
|
-
|
|
262
|
-
##### Per-repo commit (shared message, per-repo scope)
|
|
263
|
-
|
|
264
|
-
For each repo in `state.projects[]` (sequentially, in dependency order from Phase 1):
|
|
264
|
+
**Only when `state.projects[].length > 1`.** Load `$HOME/.claude/multi-agent-refs/features/commit-multi-repo.md` and follow it: per-repo commit (shared subject, per-repo scope), per-repo PR with sibling cross-links and the serial worktree finalize, the issue-body PR list, and the `recentGroups` LRU bump. A single-repo run skips it; the push policy below applies to every run.
|
|
265
265
|
|
|
266
|
-
|
|
267
|
-
2. Build the commit message using the **shared subject** + **per-repo scope** rule:
|
|
268
|
-
```
|
|
269
|
-
{type}({repo-scope}): {shared subject} [{tracker-id}]
|
|
270
|
-
|
|
271
|
-
Part of multi-repo change - see sibling PR(s) in repo `{other repo names}`.
|
|
272
|
-
```
|
|
273
|
-
`repo-scope` is the repo's short name (e.g. `common`, `uicomponents`). The subject and `[tracker-id]` are identical across all repos.
|
|
274
|
-
3. `git add` (specific files only - same rules as single-repo Step 5)
|
|
275
|
-
4. `git -C "$WT_PATH" commit -m "$MESSAGE"` (heredoc body) - using identity already pinned in Phase 0 Step 8.
|
|
276
|
-
5. Capture `state.projects[i].commit = {"sha": "<sha>"}`.
|
|
277
|
-
|
|
278
|
-
##### Push-must-succeed policy (gated by `prefs.global.settings.pushMustSucceed`, default `true`)
|
|
266
|
+
#### Push-must-succeed policy (gated by `prefs.global.settings.pushMustSucceed`, default `true`)
|
|
279
267
|
|
|
280
268
|
The build passed → the commit MUST reach the remote. Silent push failures are unacceptable.
|
|
281
269
|
|
|
@@ -305,45 +293,7 @@ Persist `state.projects[i].pushStatus` ∈ `"pushed" | "rebase-pushed" | "skippe
|
|
|
305
293
|
|
|
306
294
|
The legacy single-repo path uses the same loop with `state.projects[].length === 0`; semantics unchanged.
|
|
307
295
|
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
After all repos are pushed, open PRs sequentially, one per `state.projects[]` entry, each through its remote's usual path (gh / Bitbucket REST) with the per-stack body template (Standard / UI Component / etc). Append to each body:
|
|
311
|
-
|
|
312
|
-
```
|
|
313
|
-
## Sibling PRs (multi-repo task)
|
|
314
|
-
This PR is part of a multi-repo change. Sibling PRs (review together):
|
|
315
|
-
- <url of each PR opened before this one>
|
|
316
|
-
```
|
|
317
|
-
|
|
318
|
-
After the last PR opens, `update_sibling_links`: go back to the first N-1 PRs and add the sibling URLs that were not yet known when they were created.
|
|
319
|
-
|
|
320
|
-
**Then finalize each repo's worktree**, serially and only now - `update_sibling_links` updates one PR per repo and the loop `cd`s per repo, so removing repo 1's worktree mid-loop breaks 2..N. Run `worktree-finalize.sh` once per repo with that repo's `--worktree` / `--project-root` / `--project`; each skips or removes independently.
|
|
321
|
-
|
|
322
|
-
`update_sibling_links` does an UPDATE call per PR (Bitbucket: the `channels/pr.md` PUT payload, current `version`; GitHub: `gh pr edit --body-file`). Idempotent.
|
|
323
|
-
|
|
324
|
-
##### GitHub issue body - list all PRs
|
|
325
|
-
|
|
326
|
-
Step 10's "Issue body update" extends to multi-repo: instead of a single `### Pull Requests` line, write one row per repo:
|
|
327
|
-
|
|
328
|
-
```
|
|
329
|
-
### Pull Requests
|
|
330
|
-
- **common:** {pr-1-url}
|
|
331
|
-
- **uicomponents:** {pr-2-url}
|
|
332
|
-
```
|
|
333
|
-
|
|
334
|
-
Match the existing template's table/list shape - do not introduce new markers (per existing single-repo rule).
|
|
335
|
-
|
|
336
|
-
##### `recentGroups` update on success
|
|
337
|
-
|
|
338
|
-
After ALL repos in the task have `pushStatus ∈ {"pushed", "rebase-pushed"}`:
|
|
339
|
-
|
|
340
|
-
1. Find the matching `recentGroups[]` entry by **order-independent compare** of `repos` array (sort + JSON.stringify).
|
|
341
|
-
2. If found: `entry.count += 1`, `entry.lastUsed = ISO timestamp`, bump to front of array (LRU).
|
|
342
|
-
3. If not found: prepend a new entry `{ label: <auto-generated>, repos: [...], count: 1, lastUsed: ... }`. Cap the array at 10 (drop oldest).
|
|
343
|
-
|
|
344
|
-
A task ending with one or more `pushStatus === "skipped"` repos does NOT bump `recentGroups` count - only fully-successful multi-repo tasks earn LRU bumps.
|
|
345
|
-
|
|
346
|
-
##### Telemetry
|
|
296
|
+
#### Telemetry
|
|
347
297
|
|
|
348
298
|
Per `state.projects[]` entry (`{name}`), then once for the task:
|
|
349
299
|
|
|
@@ -380,37 +380,6 @@ Append to every Phase 5 report (knowledge-capture pass) a **Quality & Metrics**
|
|
|
380
380
|
|
|
381
381
|
The block is generated by `output-quality-check.sh` + `token-budget-report.mjs --task $TASK_ID --json` and pasted verbatim. If either script is missing, that subsection is omitted (best-effort).
|
|
382
382
|
|
|
383
|
-
#### metrics.jsonl schema extension
|
|
384
|
-
|
|
385
|
-
Telemetry events may carry an `outputQuality` object inside `details`, plus a `channels` sub-object with per-adapter status:
|
|
386
|
-
|
|
387
|
-
```json
|
|
388
|
-
{
|
|
389
|
-
"ts": "2026-04-21T...",
|
|
390
|
-
"task_id": "PROJ-12345",
|
|
391
|
-
"phase": "7",
|
|
392
|
-
"event": "task.completed",
|
|
393
|
-
"details": {
|
|
394
|
-
"phases": 8,
|
|
395
|
-
"review_cycles": 2,
|
|
396
|
-
"channels": {
|
|
397
|
-
"pr": "posted",
|
|
398
|
-
"jira": "posted",
|
|
399
|
-
"confluence": "skipped",
|
|
400
|
-
"wiki": "posted"
|
|
401
|
-
},
|
|
402
|
-
"outputQuality": {
|
|
403
|
-
"commitFormat": "pass",
|
|
404
|
-
"prBody": "pass",
|
|
405
|
-
"jiraComment": "pass",
|
|
406
|
-
"reviewerJsonValid": true,
|
|
407
|
-
"triageJsonValid": true,
|
|
408
|
-
"severityRatio": "1:2:4"
|
|
409
|
-
}
|
|
410
|
-
}
|
|
411
|
-
}
|
|
412
|
-
```
|
|
413
|
-
|
|
414
383
|
#### Final metrics report appended to PR body at release time
|
|
415
384
|
|
|
416
385
|
When the pipeline cuts a release, append the latest 30-day rollup to the release PR body:
|
|
@@ -189,7 +189,7 @@ Same rule applies to any future provider wrapper (e.g. `cloudflare-deploy.sh`, `
|
|
|
189
189
|
|
|
190
190
|
These rules govern when and how the pipeline asks the user, and when it must NOT ask.
|
|
191
191
|
|
|
192
|
-
- **No auto-commit.** Never run `git commit` without explicit user approval, even when build + review pass. The Phase 4 commit step pauses and asks ("commit message X, OK?"); only on `yes` does the commit run. Applies to checkpoint / WIP commits during Dev (Phase
|
|
192
|
+
- **No auto-commit.** Never run `git commit` without explicit user approval, even when build + review pass. The Phase 4 commit step pauses and asks ("commit message X, OK?"); only on `yes` does the commit run. Applies to checkpoint / WIP commits during Dev (Phase 2) and to the orchestrator's own state-save commits. Enforced on Claude Code: agent-guard asks on a `git commit` from the session driving an attended run, and pre-commit-check refuses it without the run's build, test and (from Phase 4) review evidence.
|
|
193
193
|
- **No micro-confirmations for reversible local work.** Local edits, lint fixes, test additions, scaffold files, in-worktree rewrites - just do them. Ask only for irreversible or shared-state actions: `git push`, `git push --force`, tag, third-party POST (Confluence, Jira, Slack), publish, deletes spanning many files, branch deletion. The threshold is "can the user undo this in 30 seconds via Cmd+Z or `git restore`?" - if yes, do not ask.
|
|
194
194
|
- **Empty `AskUserQuestion` answer is not consent.** When the user submits without picking any option (empty answers map), do NOT silently apply the "Recommended" default. Re-ask the question, or proceed treating each item as unselected. On Claude Code the `answer-check.sh` PostToolUse hook names each unanswered question back to the model. Future-behavior preference questions are not preemptive - only ask when the answer changes what happens NOW.
|
|
195
195
|
- **Question phrasing in Plan Mode.** Don't reference "the plan" in `AskUserQuestion` text while in plan mode - the user cannot see the plan until `ExitPlanMode`. Use `ExitPlanMode` for approval; `AskUserQuestion` only for choosing between concrete options.
|