@mmerterden/multi-agent-pipeline 20.27.2 → 20.29.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (99) hide show
  1. package/CHANGELOG.md +41 -0
  2. package/README.md +20 -6
  3. package/README.tr.md +6 -5
  4. package/docs/adr/0008-installer-modularization-and-secret-leak-defense.md +0 -1
  5. package/docs/architecture.md +4 -4
  6. package/docs/ecosystem.md +5 -5
  7. package/docs/facts.json +14 -12
  8. package/docs/features.md +2 -0
  9. package/install/_dev-only-files.mjs +1 -0
  10. package/install/catalog-history.json +1 -1
  11. package/install/claude.mjs +112 -0
  12. package/install/index.mjs +0 -2
  13. package/install/templates/claude-hooks.json +30 -1
  14. package/manifest.json +101 -67
  15. package/package.json +4 -1
  16. package/pipeline/agents/code-reviewer.md +1 -0
  17. package/pipeline/agents/test-gen-analyst.md +98 -0
  18. package/pipeline/commands/multi-agent/analysis/SKILL.md +3 -1
  19. package/pipeline/commands/multi-agent/resume/SKILL.md +2 -1
  20. package/pipeline/commands/multi-agent/sync/SKILL.md +3 -3
  21. package/pipeline/commands/multi-agent/test-gen/SKILL.md +124 -0
  22. package/pipeline/contract/types/index.d.ts +1 -1
  23. package/pipeline/lib/hook-registration.mjs +252 -0
  24. package/pipeline/lib/prefs-value.mjs +44 -0
  25. package/pipeline/lib/run-completion.mjs +148 -0
  26. package/pipeline/multi-agent-refs/analysis/render.md +4 -4
  27. package/pipeline/multi-agent-refs/cross-cli-contract.md +11 -5
  28. package/pipeline/multi-agent-refs/features/doctor.md +24 -0
  29. package/pipeline/multi-agent-refs/features/test-classification.md +76 -0
  30. package/pipeline/multi-agent-refs/features/usage-reporting.md +17 -0
  31. package/pipeline/multi-agent-refs/help/en.md +1 -0
  32. package/pipeline/multi-agent-refs/help/tr.md +1 -0
  33. package/pipeline/multi-agent-refs/phases/operations.md +19 -2
  34. package/pipeline/multi-agent-refs/phases/phase-0-init.md +6 -2
  35. package/pipeline/multi-agent-refs/phases/phase-1-plan.md +2 -2
  36. package/pipeline/multi-agent-refs/phases/phase-2-dev.md +7 -8
  37. package/pipeline/multi-agent-refs/phases/phase-3-review.md +5 -5
  38. package/pipeline/multi-agent-refs/phases/phase-4-commit.md +7 -6
  39. package/pipeline/multi-agent-refs/phases/phase-5-report.md +10 -8
  40. package/pipeline/multi-agent-refs/test-gen/stages.md +227 -0
  41. package/pipeline/preferences-template.json +10 -0
  42. package/pipeline/rules/figma-pipeline.md +2 -0
  43. package/pipeline/schemas/agent-state.schema.json +20 -5
  44. package/pipeline/schemas/phases.json +3 -2
  45. package/pipeline/schemas/phases.schema.json +5 -1
  46. package/pipeline/schemas/prefs.schema.json +51 -0
  47. package/pipeline/schemas/runs-index.schema.json +1 -1
  48. package/pipeline/schemas/test-gen-discovery.schema.json +73 -0
  49. package/pipeline/schemas/test-gen-gaps.schema.json +122 -0
  50. package/pipeline/schemas/test-gen-inventory.schema.json +59 -0
  51. package/pipeline/schemas/test-gen-results.schema.json +44 -0
  52. package/pipeline/scripts/_active-run.mjs +158 -0
  53. package/pipeline/scripts/_autopilot-outcomes.mjs +1 -0
  54. package/pipeline/scripts/_guard_rules.py +605 -0
  55. package/pipeline/scripts/_handle-patterns.mjs +166 -0
  56. package/pipeline/scripts/_http-post.mjs +141 -0
  57. package/pipeline/scripts/_stack-routing.mjs +4 -0
  58. package/pipeline/scripts/_test-gaming.mjs +227 -0
  59. package/pipeline/scripts/_unified-diff.mjs +104 -0
  60. package/pipeline/scripts/_worktree-snapshot.mjs +113 -0
  61. package/pipeline/scripts/agent-guard.py +118 -7
  62. package/pipeline/scripts/agent-guard.sh +50 -3
  63. package/pipeline/scripts/argv-secret-scan.mjs +114 -0
  64. package/pipeline/scripts/autopilot-runner.mjs +8 -2
  65. package/pipeline/scripts/confirm-command.mjs +70 -0
  66. package/pipeline/scripts/doctor.mjs +105 -0
  67. package/pipeline/scripts/evidence-gate.mjs +82 -7
  68. package/pipeline/scripts/gate-ledger.mjs +163 -7
  69. package/pipeline/scripts/handle-diff-audit.mjs +188 -0
  70. package/pipeline/scripts/plan-coverage-gate.mjs +9 -8
  71. package/pipeline/scripts/pre-commit-check.sh +12 -0
  72. package/pipeline/scripts/routine-registry.mjs +5 -1
  73. package/pipeline/scripts/run-scope-guard.mjs +188 -0
  74. package/pipeline/scripts/run-scope-guard.sh +23 -0
  75. package/pipeline/scripts/run-stop-guard.mjs +246 -0
  76. package/pipeline/scripts/run-stop-guard.sh +25 -0
  77. package/pipeline/scripts/run-ui-tests.sh +27 -3
  78. package/pipeline/scripts/test-classify.mjs +296 -0
  79. package/pipeline/scripts/test-gen-gate.mjs +409 -0
  80. package/pipeline/scripts/test-gen-state.mjs +313 -0
  81. package/pipeline/scripts/test-integrity-gate.mjs +76 -11
  82. package/pipeline/scripts/uninstall.mjs +2 -0
  83. package/pipeline/scripts/usage-register-beacon.mjs +239 -0
  84. package/pipeline/scripts/usage-register.mjs +128 -33
  85. package/pipeline/scripts/usage-report.mjs +43 -33
  86. package/pipeline/scripts/validate-state.mjs +26 -0
  87. package/pipeline/scripts/write-state.mjs +114 -0
  88. package/pipeline/skills/.skill-manifest.json +23 -3
  89. package/pipeline/skills/.skills-index.json +57 -2
  90. package/pipeline/skills/shared/README.md +10 -5
  91. package/pipeline/skills/shared/core/multi-agent-sync/SKILL.md +3 -3
  92. package/pipeline/skills/shared/core/multi-agent-test-gen/SKILL.md +120 -0
  93. package/pipeline/skills/shared/external/fix-review-verdicts/SKILL.md +133 -0
  94. package/pipeline/skills/shared/external/handle-request/SKILL.md +97 -0
  95. package/pipeline/skills/shared/external/scenario-gap-analysis/SKILL.md +129 -0
  96. package/pipeline/skills/shared/external/scenario-gap-analysis/references/scenario-schema.md +60 -0
  97. package/pipeline/skills/shared/external/test-result-triage/SKILL.md +160 -0
  98. package/pipeline/skills/skills-index.md +7 -2
  99. package/install/_telemetry.mjs +0 -61
@@ -32,6 +32,9 @@
32
32
  * 4 - the lock was taken from us mid-write; nothing was written, retry
33
33
  * 3 - I/O error (disk full, permission denied, parent dir missing)
34
34
  * 5 - --if-rev did not match; the record moved, nothing was written
35
+ * 6 - the write moves the run into `status: complete` without the evidence
36
+ * lib/run-completion.mjs requires (last phase of the mode, Phase 4
37
+ * evidence, a plan-coverage verdict); nothing was written
35
38
  *
36
39
  * Tunables (env):
37
40
  * WRITE_STATE_LOCK_TIMEOUT_MS acquire window before exit 2 (default 15000)
@@ -67,6 +70,7 @@ import { dirname, basename } from "path";
67
70
  import { Buffer } from "node:buffer";
68
71
  import { runMain } from "../lib/fatal.mjs";
69
72
  import { invokedDirectly } from "../lib/invoked-directly.mjs";
73
+ import { completionGaps } from "../lib/run-completion.mjs";
70
74
 
71
75
  /**
72
76
  * @typedef {{ replace: boolean, path: string, ifRev: number|null }} ParsedArgs
@@ -320,6 +324,104 @@ function undeclaredKeys(payload, current) {
320
324
  );
321
325
  }
322
326
 
327
+ const COMPLETE = new Set(["complete", "completed"]);
328
+ const STOP_GUARD_HALT = /^\d+:stop-guard:/;
329
+
330
+ /**
331
+ * Lift the halt run-stop-guard.mjs records after its repeat ceiling once the
332
+ * run advances a phase. That halt says only "the session kept stopping"; a
333
+ * later phase transition is the run going on, and leaving the halt in place
334
+ * would show a moving run as paused and make every later Stop let it through.
335
+ * A halt any other step recorded, or a write that sets status or haltReason
336
+ * itself, is left alone.
337
+ *
338
+ * @param {object} next
339
+ * @param {object} current
340
+ * @param {unknown} payload
341
+ */
342
+ export function clearStopGuardHalt(next, current, payload) {
343
+ if (!payload || typeof payload !== "object" || Array.isArray(payload)) return;
344
+ if (Object.hasOwn(payload, "status") || Object.hasOwn(payload, "haltReason")) return;
345
+ if (typeof current?.haltReason !== "string" || !STOP_GUARD_HALT.test(current.haltReason)) return;
346
+ const was = Number(current.currentPhase);
347
+ const now = Number(payload.currentPhase);
348
+ if (!Number.isInteger(now) || !(now > (Number.isInteger(was) ? was : -1))) return;
349
+ next.haltReason = null;
350
+ next.status = "in_progress";
351
+ }
352
+
353
+ /**
354
+ * Clear a wait the run has moved past. A write that advances `currentPhase`
355
+ * without setting `waitingFor` itself ends any wait recorded in the earlier
356
+ * phase, so a clear the step forgot cannot keep the run marked as waiting
357
+ * (and the stop guard off) for the rest of the run. `question` is left to
358
+ * answer-question.mjs, which clears it together with its pendingQuestion.
359
+ *
360
+ * @param {object} next
361
+ * @param {object} current
362
+ * @param {unknown} payload
363
+ */
364
+ export function clearPassedWait(next, current, payload) {
365
+ if (!payload || typeof payload !== "object" || Array.isArray(payload)) return;
366
+ if (Object.hasOwn(payload, "waitingFor")) return;
367
+ if (typeof current?.waitingFor !== "string" || current.waitingFor === "question") return;
368
+ const was = Number(current.currentPhase);
369
+ const now = Number(payload.currentPhase);
370
+ if (!Number.isInteger(now) || !(now > (Number.isInteger(was) ? was : -1))) return;
371
+ next.waitingFor = null;
372
+ }
373
+
374
+ /**
375
+ * The refusal message for a write that moves the run into `complete` without
376
+ * the evidence lib/run-completion.mjs asks for, or null.
377
+ *
378
+ * Only the transition is judged: a merge that sets `status: complete`, or a
379
+ * replace whose document is complete over one that was not. A later write to
380
+ * an already complete run that does not touch `status` goes through, so the
381
+ * report steps after the transition keep working.
382
+ *
383
+ * @param {object} next
384
+ * @param {unknown} payload
385
+ * @param {boolean} replace
386
+ * @param {string|null} diskStatus
387
+ * @returns {string|null}
388
+ */
389
+ export function completionRefusal(next, payload, replace, diskStatus) {
390
+ if (!COMPLETE.has(next?.status)) return null;
391
+ const entering = replace
392
+ ? !COMPLETE.has(diskStatus)
393
+ : Boolean(payload && typeof payload === "object" && COMPLETE.has(payload.status));
394
+ if (!entering) return null;
395
+ const gaps = completionGaps(next);
396
+ if (!gaps.length) return null;
397
+ return `write-state: refusing status "${next.status}": ${gaps.join("; ")}. Nothing was written. Finish the missing step, or record a halt (status "paused" with a haltReason).`;
398
+ }
399
+
400
+ /**
401
+ * Bind the run to the host session that drives it.
402
+ *
403
+ * `hostSessionId` is the Claude Code session (CLAUDE_CODE_SESSION_ID in the
404
+ * Bash environment) that made the run's last phase transition. The Stop hook
405
+ * (run-stop-guard.mjs) and the run-scoped PreToolUse guards read it to find
406
+ * the run a hook call belongs to. Stamped only when the write creates the
407
+ * document or sets `currentPhase`: those are the driver's writes, while a
408
+ * second session editing a running task (`/multi-agent:steer`) sets neither
409
+ * and so does not take the run over.
410
+ *
411
+ * @param {object} next
412
+ * @param {unknown} payload
413
+ * @param {boolean} existed
414
+ * @param {Record<string,string|undefined>} [env]
415
+ */
416
+ export function stampHostSession(next, payload, existed, env = process.env) {
417
+ const sid = env.CLAUDE_CODE_SESSION_ID;
418
+ if (typeof sid !== "string" || !sid.trim()) return;
419
+ const transition =
420
+ payload && typeof payload === "object" && Object.hasOwn(payload, "currentPhase");
421
+ if (existed && !transition) return;
422
+ next.hostSessionId = sid.trim();
423
+ }
424
+
323
425
  /**
324
426
  * @param {object} base
325
427
  * @param {object} patch
@@ -397,6 +499,8 @@ async function main() {
397
499
  }
398
500
  }
399
501
  next = deepMerge(current, payload);
502
+ clearStopGuardHalt(next, current, payload);
503
+ clearPassedWait(next, current, payload);
400
504
  }
401
505
 
402
506
  // A state document is an object. Anything else cannot carry `rev`, and
@@ -415,9 +519,11 @@ async function main() {
415
519
  // would tell the next reader that nothing changed.
416
520
  let priorRev = Number.isInteger(next.rev) ? next.rev : 0;
417
521
  let diskRev = null;
522
+ let diskStatus = null;
418
523
  if (existsSync(path)) {
419
524
  try {
420
525
  const parsed = JSON.parse(readFileSync(path, "utf-8"));
526
+ diskStatus = parsed?.status ?? null;
421
527
  if (Number.isInteger(parsed?.rev)) {
422
528
  diskRev = parsed.rev;
423
529
  priorRev = Math.max(priorRev, parsed.rev);
@@ -448,7 +554,15 @@ async function main() {
448
554
  }
449
555
  }
450
556
 
557
+ const refusal = completionRefusal(next, payload, replace, diskStatus);
558
+ if (refusal) {
559
+ console.error(refusal);
560
+ releaseLock(path);
561
+ process.exit(6);
562
+ }
563
+
451
564
  next.rev = priorRev + 1;
565
+ stampHostSession(next, payload, fileExistedBefore);
452
566
 
453
567
  const unknown = undeclaredKeys(payload, replace ? {} : current);
454
568
  if (unknown.length) {
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "schemaVersion": "1.0.0",
3
- "generatedAt": "2026-10-20T13:08:30Z",
4
- "skillCount": 229,
3
+ "generatedAt": "2026-10-24T14:34:39Z",
4
+ "skillCount": 234,
5
5
  "entries": [
6
6
  {
7
7
  "path": "shared/core/apple-archive-compliance/SKILL.md",
@@ -237,7 +237,7 @@
237
237
  },
238
238
  {
239
239
  "path": "shared/core/multi-agent-sync/SKILL.md",
240
- "sha256": "1e418fe472a93cc7b4b5bc19ff9e9b8b9d89b545e45d343a643f17a492cb0624"
240
+ "sha256": "e96afa9b89a2ab6e2fe13a01158f98272c6828e5cef03c8909240766aaefd44b"
241
241
  },
242
242
  {
243
243
  "path": "shared/core/multi-agent-test-accessibility/SKILL.md",
@@ -251,6 +251,10 @@
251
251
  "path": "shared/core/multi-agent-test-dynamic-type/SKILL.md",
252
252
  "sha256": "e1cadd5eb8d680b0edfc40a4912733dc6a432493c868f4997b2dec0db6638dd4"
253
253
  },
254
+ {
255
+ "path": "shared/core/multi-agent-test-gen/SKILL.md",
256
+ "sha256": "0da450ed0c74ab610d3c8f0cf1b0facd2a4ae179afc832440939c3367de7e762"
257
+ },
254
258
  {
255
259
  "path": "shared/core/multi-agent-test-screenshots/SKILL.md",
256
260
  "sha256": "a41507304e4650079e6359b3ad5854f90f45a90172911f487c4c923ef8839f91"
@@ -487,6 +491,10 @@
487
491
  "path": "shared/external/firebase/SKILL.md",
488
492
  "sha256": "d377217a03e7fc63bd7eba285c6e2b450f293b13c6e65680e71b022cd14872a3"
489
493
  },
494
+ {
495
+ "path": "shared/external/fix-review-verdicts/SKILL.md",
496
+ "sha256": "a728b7c87fbf175e959e23b655c626294b7a7a2949c62a37e7b3d4a172ce57d5"
497
+ },
490
498
  {
491
499
  "path": "shared/external/github-actions-templates/SKILL.md",
492
500
  "sha256": "13b6fec454c2ebf69b06fcdc00033a6dab98d6111e0346447c8d474db547f801"
@@ -495,6 +503,10 @@
495
503
  "path": "shared/external/gradle-kotlin-dsl/SKILL.md",
496
504
  "sha256": "af7b1e30e2448f43b5100b2a935bbaadde86ccd490844c901736c9270f8cc3ce"
497
505
  },
506
+ {
507
+ "path": "shared/external/handle-request/SKILL.md",
508
+ "sha256": "05ecf72d8e40eeca8dfc0913764b1188b9b1189ea098ad23b4f0c8256f2c671c"
509
+ },
498
510
  {
499
511
  "path": "shared/external/healthkit/SKILL.md",
500
512
  "sha256": "fece1986bc35925e34fed8d8cb13aa59a3577b8cedefdc02c736c0d4a5d38aff"
@@ -703,6 +715,10 @@
703
715
  "path": "shared/external/room-database/SKILL.md",
704
716
  "sha256": "45fe377469f1fb4036431194d3b525d3caaa79231808625f643468863c6e814a"
705
717
  },
718
+ {
719
+ "path": "shared/external/scenario-gap-analysis/SKILL.md",
720
+ "sha256": "fcd9f614dc18ea0d4610469f7edcf9214d3b14d65976fa3fb7c4a524330a392a"
721
+ },
706
722
  {
707
723
  "path": "shared/external/search-first/SKILL.md",
708
724
  "sha256": "c9d5ffb698bbfb76a89da6a35dfc0f1d72a31d2b4a0e9e9f3e027be7a8fddfde"
@@ -855,6 +871,10 @@
855
871
  "path": "shared/external/tailwind-css/SKILL.md",
856
872
  "sha256": "9eda0a871962f52e381781b95c948128626984a82ddbd47be53ea869cd250b7e"
857
873
  },
874
+ {
875
+ "path": "shared/external/test-result-triage/SKILL.md",
876
+ "sha256": "72b0e0049fcf549b61a189900ca8e503b779e93cd26575d7b7328ae785465e38"
877
+ },
858
878
  {
859
879
  "path": "shared/external/testing-backend/SKILL.md",
860
880
  "sha256": "d3eea1c6d1e2a40a0b66bc6460194ee313e17fc2af8f75047030b7840c2db857"
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "schemaVersion": "1.0.0",
3
- "skillCount": 229,
3
+ "skillCount": 234,
4
4
  "entries": [
5
5
  {
6
6
  "name": "accessibility-compliance-accessibility-audit",
@@ -191,7 +191,7 @@
191
191
  },
192
192
  {
193
193
  "name": "app-store-review",
194
- "description": "App Store submission readiness and rejection prevention: Review Guidelines 2.1, 2.3, 2.5.1, 3.1.1, 4.2 and 5.1.2, PrivacyInfo.xcprivacy manifests and required-reason codes, IAP and StoreKit rules, ATT consent, EU DMA distribution, entitlements and usage strings, metadata limits, screenshots, TestFli",
194
+ "description": "App Store submission readiness and rejection prevention: Review Guidelines 1.2, 2.1, 2.3, 2.3.14, 2.5.1, 2.5.19, 3.1.1, 4.2 and 5.1.2, upload floors, age rating and medical device declarations, PrivacyInfo.xcprivacy manifests and required-reason codes, IAP and StoreKit rules, ATT consent, EU DMA dis",
195
195
  "platform": null,
196
196
  "group": "external",
197
197
  "plugin": "ai-ios-toolkit",
@@ -596,6 +596,17 @@
596
596
  "triggerPaths": [],
597
597
  "relativePath": "shared/external/firebase/SKILL.md"
598
598
  },
599
+ {
600
+ "name": "fix-review-verdicts",
601
+ "description": "Independent fix review with three verdicts, test-gaming, scope and root-cause, each backed by file:line findings; the result is computed from findings, never from a claim. Use when reviewing a fix made to turn tests green. Not a full review checklist.",
602
+ "platform": null,
603
+ "group": "external",
604
+ "plugin": "ai-common-toolkit",
605
+ "invokeAs": "ai-common-toolkit:fix-review-verdicts",
606
+ "triggerKeywords": [],
607
+ "triggerPaths": [],
608
+ "relativePath": "shared/external/fix-review-verdicts/SKILL.md"
609
+ },
599
610
  {
600
611
  "name": "github-actions-templates",
601
612
  "description": "Authoring GitHub Actions workflows: triggers, matrices, caching, concurrency, least-privilege permissions, SHA pinning, OIDC, reusable workflows, composite actions, Xcode runners. Use when writing or hardening a workflow file.",
@@ -629,6 +640,17 @@
629
640
  "triggerPaths": [],
630
641
  "relativePath": "shared/external/gradle-kotlin-dsl/SKILL.md"
631
642
  },
643
+ {
644
+ "name": "handle-request",
645
+ "description": "Add the test handles a scenario needs and answer each request with ADDED, ALREADY or COULD NOT ADD plus file:line or reason. Identity values only, attribute-only diffs; iOS accessibilityIdentifier, Compose testTag, web data-testid. Use when a test needs an element handle the UI lacks.",
646
+ "platform": null,
647
+ "group": "external",
648
+ "plugin": "ai-common-toolkit",
649
+ "invokeAs": "ai-common-toolkit:handle-request",
650
+ "triggerKeywords": [],
651
+ "triggerPaths": [],
652
+ "relativePath": "shared/external/handle-request/SKILL.md"
653
+ },
632
654
  {
633
655
  "name": "healthkit",
634
656
  "description": "HealthKit on iOS, iPadOS, watchOS and visionOS: HKHealthStore authorization, sample, statistics and chart-ready collection queries, saving HKQuantitySample values, background delivery, live workouts (HKWorkoutSession, HKLiveWorkoutBuilder), HKUnit, HKQuantityTypeIdentifier. Use when reading, writing",
@@ -1685,6 +1707,17 @@
1685
1707
  "triggerPaths": [],
1686
1708
  "relativePath": "shared/core/multi-agent-test-dynamic-type/SKILL.md"
1687
1709
  },
1710
+ {
1711
+ "name": "multi-agent-test-gen",
1712
+ "description": "Write the tests an app lacks, stage by stage, each gated on its artefact: inventory, read-only gap analysis, handles, tests, runs, per-test classification. Use when screens, flows or a change need tests.",
1713
+ "platform": null,
1714
+ "group": "core",
1715
+ "plugin": null,
1716
+ "invokeAs": "multi-agent-test-gen",
1717
+ "triggerKeywords": [],
1718
+ "triggerPaths": [],
1719
+ "relativePath": "shared/core/multi-agent-test-gen/SKILL.md"
1720
+ },
1688
1721
  {
1689
1722
  "name": "multi-agent-test-screenshots",
1690
1723
  "description": "App Store screenshot set from a booted simulator / emulator in a given locale, taken as an argument and defaulting to tr. Alias pinning the screenshot scenario. Use when generating store screenshots or checking a localised build renders.",
@@ -1927,6 +1960,17 @@
1927
1960
  "triggerPaths": [],
1928
1961
  "relativePath": "shared/external/room-database/SKILL.md"
1929
1962
  },
1963
+ {
1964
+ "name": "scenario-gap-analysis",
1965
+ "description": "Set an inventory against existing tests, sort each behaviour into COVERED, WRONGLY COVERED or MISSING, and write gaps as measurable, prioritised scenarios with cross-screen flows. Use when planning tests or auditing coverage. Not for writing tests or judging a run.",
1966
+ "platform": null,
1967
+ "group": "external",
1968
+ "plugin": "ai-common-toolkit",
1969
+ "invokeAs": "ai-common-toolkit:scenario-gap-analysis",
1970
+ "triggerKeywords": [],
1971
+ "triggerPaths": [],
1972
+ "relativePath": "shared/external/scenario-gap-analysis/SKILL.md"
1973
+ },
1930
1974
  {
1931
1975
  "name": "search-first",
1932
1976
  "description": "Research-before-coding workflow. Before writing a custom implementation, check what already exists (repo utilities, installed libraries, packages, internal components) and decide adopt / extend / compose / build on evidence. Use at the start of any non-trivial feature or utility, especially when you",
@@ -2345,6 +2389,17 @@
2345
2389
  "triggerPaths": [],
2346
2390
  "relativePath": "shared/external/tailwind-css/SKILL.md"
2347
2391
  },
2392
+ {
2393
+ "name": "test-result-triage",
2394
+ "description": "Judge each test result on changed code as product-bug, stale-test, valid or ambiguous (the default), and list behaviour no test covers. stale-test needs a diff file:line, product-bug needs evidence. Use when tests ran on a new commit, PR or fix. Not for writing or editing tests.",
2395
+ "platform": null,
2396
+ "group": "external",
2397
+ "plugin": "ai-common-toolkit",
2398
+ "invokeAs": "ai-common-toolkit:test-result-triage",
2399
+ "triggerKeywords": [],
2400
+ "triggerPaths": [],
2401
+ "relativePath": "shared/external/test-result-triage/SKILL.md"
2402
+ },
2348
2403
  {
2349
2404
  "name": "testing-backend",
2350
2405
  "description": "Backend testing: pytest patterns (Python), Jest patterns (Node.js), integration testing, database fixtures/factories, API contract testing. Use when writing or reviewing backend tests: unit, integration, database and API level.",
@@ -2,24 +2,24 @@
2
2
 
3
3
  Single source of truth for skills delivered to both Claude Code (`~/.claude/skills/`) and Copilot CLI (`~/.copilot/skills/`) by the installer.
4
4
 
5
- **Total:** 229 skills (68 core + 161 external). Auto-generated by `scripts/gen-skills-index.mjs` - do not edit by hand.
5
+ **Total:** 234 skills (69 core + 165 external). Auto-generated by `scripts/gen-skills-index.mjs` - do not edit by hand.
6
6
 
7
7
  ## Directory layout
8
8
 
9
- - **`core/`** - 68 `multi-agent*` orchestration skills that are pipeline-critical. Edits here are core-code changes.
10
- - **`external/`** - 161 iOS / Android / generic skills that ship into the stack plugins. Each names its origin in `metadata.source`: `multi-agent-pipeline` when written here, or the upstream and license a `NOTICE-*.md` in this directory covers.
9
+ - **`core/`** - 69 `multi-agent*` orchestration skills that are pipeline-critical. Edits here are core-code changes.
10
+ - **`external/`** - 165 iOS / Android / generic skills that ship into the stack plugins. Each names its origin in `metadata.source`: `multi-agent-pipeline` when written here, or the upstream and license a `NOTICE-*.md` in this directory covers.
11
11
  - Install destinations (ADR-0009): Claude Code gets NO local copy of `external/` - it loads those skills from the `multi-agent-plugins` marketplace, namespaced (`ai-<stack>-toolkit:<name>`); only the two compliance catalogs from `core/` land in `~/.claude/skills/`. Copilot CLI and Codex CLI receive a flat copy filtered to the enabled stacks. `external/` remains the single authoring source that `build-stack-plugins.mjs` publishes from.
12
12
 
13
13
  Source layout is logical grouping only - skill discovery at runtime is unchanged.
14
14
 
15
15
  ## Categories
16
16
 
17
- - [Pipeline Orchestration](#pipeline-orchestration) - 68
17
+ - [Pipeline Orchestration](#pipeline-orchestration) - 69
18
18
  - [iOS / Apple Ecosystem](#ios-apple-ecosystem) - 94
19
19
  - [Android / Kotlin](#android-kotlin) - 16
20
20
  - [Web](#web) - 10
21
21
  - [Backend / API](#backend-api) - 11
22
- - [Cross-cutting](#cross-cutting) - 30
22
+ - [Cross-cutting](#cross-cutting) - 34
23
23
 
24
24
  ## Pipeline Orchestration
25
25
 
@@ -89,6 +89,7 @@ Source layout is logical grouping only - skill discovery at runtime is unchang
89
89
  | [`multi-agent-test-accessibility`](./core/multi-agent-test-accessibility/) | `core` | Accessibility audit on a booted simulator / emulator: VoiceOver labels, sub-44pt tap targets, contrast, traits. Alias pinning the accessibil |
90
90
  | [`multi-agent-test-dark-mode`](./core/multi-agent-test-dark-mode/) | `core` | Dark mode UI test on a booted simulator / emulator: walk every screen light then dark, report contrast + colour bugs. Alias pinning the dark |
91
91
  | [`multi-agent-test-dynamic-type`](./core/multi-agent-test-dynamic-type/) | `core` | Dynamic Type layout test on a booted simulator / emulator: re-walk every screen at XL through accessibility-extra-large, report truncation a |
92
+ | [`multi-agent-test-gen`](./core/multi-agent-test-gen/) | `core` | Write the tests an app lacks, stage by stage, each gated on its artefact: inventory, read-only gap analysis, handles, tests, runs, per-test |
92
93
  | [`multi-agent-test-screenshots`](./core/multi-agent-test-screenshots/) | `core` | App Store screenshot set from a booted simulator / emulator in a given locale, taken as an argument and defaulting to tr. Alias pinning the |
93
94
  | [`multi-agent-testflight-validation`](./core/multi-agent-testflight-validation/) | `core` | iOS-pinned alias of store-ready: same three gates on an iOS build, one implementation. Validates a TestFlight / App Store package, never upl |
94
95
  | [`multi-agent-uninstall`](./core/multi-agent-uninstall/) | `core` | Uninstall the pipeline from Claude Code + Copilot CLI. Keychain access tokens are always left untouched; --all-data also clears pipeline set |
@@ -264,15 +265,19 @@ Source layout is logical grouping only - skill discovery at runtime is unchang
264
265
  | [`e2e-testing`](./external/e2e-testing/) | `external` | \| |
265
266
  | [`evidence-github`](./external/evidence-github/) | `external` | Search GitHub for facts an analysis can cite - a known issue in a dependency, the PR that fixed it, what a release actually changed, prior a |
266
267
  | [`evidence-registry`](./external/evidence-registry/) | `external` | Check a package's real version, deprecation status and release notes on npm, PyPI, Maven Central or Swift Package Index. Use when a spec or |
268
+ | [`fix-review-verdicts`](./external/fix-review-verdicts/) | `external` | Independent fix review with three verdicts, test-gaming, scope and root-cause, each backed by file:line findings; the result is computed fro |
267
269
  | [`github-actions-templates`](./external/github-actions-templates/) | `external` | Authoring GitHub Actions workflows: triggers, matrices, caching, concurrency, least-privilege permissions, SHA pinning, OIDC, reusable workf |
270
+ | [`handle-request`](./external/handle-request/) | `external` | Add the test handles a scenario needs and answer each request with ADDED, ALREADY or COULD NOT ADD plus file:line or reason. Identity values |
268
271
  | [`humanizer`](./external/humanizer/) | `external` | \| |
269
272
  | [`localization-reuse-map`](./external/localization-reuse-map/) | `external` | Localization reuse map for one redesigned screen on iOS, Android and web: new keys set against each platform's legacy key, legacy values and |
270
273
  | [`pdfkit`](./external/pdfkit/) | `external` | PDFKit on iOS: PDFView display and SwiftUI wrapping, creating and editing PDFDocument and PDFPage, highlight, note, signature and form annot |
274
+ | [`scenario-gap-analysis`](./external/scenario-gap-analysis/) | `external` | Set an inventory against existing tests, sort each behaviour into COVERED, WRONGLY COVERED or MISSING, and write gaps as measurable, priorit |
271
275
  | [`search-first`](./external/search-first/) | `external` | \| |
272
276
  | [`security-review`](./external/security-review/) | `external` | Run a defensive, static security review of a diff or a repo: build a run-scoped threat model, find vulnerabilities joined to OWASP + CWE wit |
273
277
  | [`signal-community`](./external/signal-community/) | `external` | Check whether other people hit the same problem, via Stack Overflow and Hacker News (keyless) plus opt-in Reddit and X. Use when a bug repor |
274
278
  | [`skill-creator`](./external/skill-creator/) | `external` | House guide for authoring plugin skills: creating a new skill, trimming or splitting a bloated one, writing or sharpening a description, and |
275
279
  | [`spm-build-analysis`](./external/spm-build-analysis/) | `external` | Analyze Swift Package Manager dependencies, package plugins, module variants, and CI-oriented build overhead that slow Xcode builds. Use whe |
280
+ | [`test-result-triage`](./external/test-result-triage/) | `external` | Judge each test result on changed code as product-bug, stale-test, valid or ambiguous (the default), and list behaviour no test covers. stal |
276
281
  | [`valven-rules`](./external/valven-rules/) | `external` | Delivery-flow rules from metrics Valven reports: PR size, one concern per PR, tests with source, issue keys, no WIP commits, churn, PR body, |
277
282
  | [`xcode-build-benchmark`](./external/xcode-build-benchmark/) | `external` | Benchmark Xcode clean and incremental builds with repeatable inputs, timing summaries, and timestamped `.build-benchmark/` artifacts. Use wh |
278
283
  | [`xcode-build-fixer`](./external/xcode-build-fixer/) | `external` | Apply approved Xcode build optimization changes following best practices, then re-benchmark to verify improvement. Use when a developer has |
@@ -33,7 +33,7 @@ Run all steps automatically:
33
33
  ```
34
34
  Step 0: DOCTOR node $HOME/.claude/scripts/doctor.mjs - exit 2 or 4 STOPS the sync
35
35
  Step 1: DETECT Compare timestamps, find stale targets
36
- Step 2: COPILOT Claude Code -> Copilot CLI (instructions + 65 sub-command skills)
36
+ Step 2: COPILOT Claude Code -> Copilot CLI (instructions + 66 sub-command skills)
37
37
  Step 2b: CODEX Claude Code -> Codex CLI (1 router skill + sub-command specs as refs + agent TOML)
38
38
  Step 3: REPO Claude Code -> pipeline repo (genericized, personal data scrub)
39
39
  Step 3d: DEV-TOOLKIT Companion MCP server -> detect movement, ship gates, commit + publish
@@ -229,7 +229,7 @@ When invoked with the `release` argument:
229
229
  |-------------|-------------|
230
230
  | `~/.claude/commands/multi-agent/{cmd}/SKILL.md` | `~/.copilot/skills/multi-agent-{cmd}/SKILL.md` |
231
231
 
232
- **65 commands are synced** (canonical inventory - must match `cross-cli-contract.md` section 1; drift = contract violation):
232
+ **66 commands are synced** (canonical inventory - must match `cross-cli-contract.md` section 1; drift = contract violation):
233
233
 
234
234
  ```
235
235
  analysis, analysis-jira, analysis-resolve, autopilot, autopilot-off,
@@ -241,7 +241,7 @@ prune-prompts, purge, refactor, research, resume, review,
241
241
  review-analysis, review-issue, review-jira, route-off, route-on,
242
242
  route-status, routines, save, scaffold, scan, scenario-audit, search,
243
243
  security-review, serve, setup, stack, status, steer, store-ready, sync, test, test-accessibility,
244
- test-dark-mode, test-dynamic-type, test-screenshots, testflight-validation,
244
+ test-dark-mode, test-dynamic-type, test-gen, test-screenshots, testflight-validation,
245
245
  uninstall, update
246
246
  ```
247
247
 
@@ -0,0 +1,120 @@
1
+ ---
2
+ name: multi-agent-test-gen
3
+ description: "Write the tests an app lacks, stage by stage, each gated on its artefact: inventory, read-only gap analysis, handles, tests, runs, per-test classification. Use when screens, flows or a change need tests."
4
+ user-invocable: true
5
+ argument-hint: "[scope] [--platform ios|android|web] [--repo <path>] - scope: a screen, route or flow, or empty for what changed since the last test-gen run"
6
+ metadata:
7
+ language: en
8
+ not-for: test, bug-bash, scenario-audit, manual-test
9
+ ---
10
+
11
+ # multi-agent-test-gen
12
+
13
+ **Input**: $ARGUMENTS
14
+
15
+ Writes the tests an app is missing, in the repo's own frameworks, and reports
16
+ what they found. Seven stages run as subagents, each leaving one artefact that a
17
+ script gates before the next stage starts: inventory, discovery, gaps, handles,
18
+ write, run, classify. The two analysis stages run as a read-only agent type, so
19
+ nothing they do can change the repository.
20
+
21
+ Stage contract, read before Step 1:
22
+
23
+ ```
24
+ $HOME/.claude/multi-agent-refs/test-gen/stages.md
25
+ $HOME/.claude/multi-agent-refs/features/test-classification.md
26
+ ```
27
+
28
+ Scripts: `test-gen-gate.mjs` (artefact, handle and write gates),
29
+ `handle-diff-audit.mjs` (handles only), `test-classify.mjs`,
30
+ `test-gen-state.mjs` (change base, revision cap), `evidence-gate.mjs`, all under
31
+ `$HOME/.claude/scripts/`. Schemas: `test-gen-inventory`, `test-gen-gaps`,
32
+ `test-gen-results` under `$HOME/.claude/schemas/`.
33
+
34
+ ## Gotchas
35
+
36
+ - An analysis stage that wants to "just fix" something is out of its lane. Discovery and gaps go to `test-gen-analyst`, whose grant is an allowlist of read-only tools.
37
+ - The handle stage adds handles and nothing else. A wrapped view, a renamed element or a changed condition fails `handle-diff-audit.mjs`; revert the hunk, do not argue with the audit.
38
+ - `HANDLES: none-needed` is a claim that the handles exist. It needs the `file:line` of each.
39
+ - A test that passed once is not green. Unstable is a fail, and a red test with no base to compare is `not_measured`, not a regression.
40
+ - Test data is named in `dataNeeds[]`, never filled in by the run. A credential is a `secretRef`.
41
+ - A product bug is reported, never fixed here. In autopilot it parks the run.
42
+
43
+ ## Step 1 - Prepare
44
+
45
+ 1. Repo: `--repo`, else the current directory; confirm it in one line.
46
+ Platform: `--platform`, else detect as `multi-agent-test` does. Run id
47
+ `tg-<timestamp>`; `RUN="$HOME/.claude/logs/test-gen/<repo>/<timestamp>"`,
48
+ created with `mkdir -p "$RUN"`.
49
+ 2. Toolkit: `test_results_classify`, `coverage_gap` and `ui_locator_codegen`
50
+ need `@mmerterden/multi-agent-toolkit-mcp` >= v3.25.0. Check the tool list
51
+ once. Each missing tool has a fallback in `stages.md` (classify from the
52
+ repeats, the analyst matches coverage itself with every match `inferred`,
53
+ locators written by hand from the handles); say which fallbacks are in use,
54
+ once.
55
+ 3. Change base. `TG_STATE="$HOME/.claude/knowledge/<project>/test-gen-state.json"`:
56
+
57
+ ```bash
58
+ node "$HOME/.claude/scripts/test-gen-state.mjs" base-scan --state "$TG_STATE" --repo "$REPO" \
59
+ --graph "$HOME/.claude/knowledge/<project>/code-graph.json" > "$RUN/base-scan.json"
60
+ ```
61
+
62
+ Exit 0: the changed files and their dependents narrow an empty scope. Exit 1
63
+ (`measured: false`, first run or base gone): nothing is narrowed, and the
64
+ reason goes in the report.
65
+
66
+ ## Step 2 - Inventory and priority
67
+
68
+ Stage 1 of `stages.md`, then the priority pick: ask which items to cover
69
+ (one question to the user; autopilot: the top three, recorded as `autopilot-top3`). Gate:
70
+
71
+ ```bash
72
+ node "$HOME/.claude/scripts/test-gen-gate.mjs" check --stage inventory --file "$RUN/inventory.json"
73
+ node "$HOME/.claude/scripts/evidence-gate.mjs" --claim artefact --status passed \
74
+ --evidence "$RUN/inventory.json" --stage-start "$STAGE_START"
75
+ ```
76
+
77
+ ## Step 3 - Discovery and gaps (read-only)
78
+
79
+ Stages 2 and 3, both with `subagent_type: "test-gen-analyst"`. Write each answer
80
+ with the Write tool, then gate each file with `evidence-gate.mjs --claim
81
+ artefact` against its own stage start, and each with `test-gen-gate.mjs check`
82
+ (`--stage discovery`, `--stage gaps`). Show the gaps (bucket, priority,
83
+ expectation) and the `dataNeeds[]`; ask for the data or the credential names
84
+ (autopilot: write only the scenarios whose needs are met).
85
+
86
+ ## Step 4 - Handles
87
+
88
+ Stage 4. Snapshot first (`test-gen-gate.mjs baseline`), then the subagent, then
89
+ both gates against that snapshot: `test-gen-gate.mjs handles` and
90
+ `handle-diff-audit.mjs --baseline` exit 0. The gates read only the stage's own
91
+ change; the person's uncommitted work is never the stage's to revert. Skip the
92
+ stage when no gap has `handlesNeeded`, and say so.
93
+
94
+ ## Step 5 - Write, run, classify
95
+
96
+ Stages 5 to 7: a new snapshot, the tests and `written.json`, the
97
+ `test-gen-gate.mjs written --baseline` gate (test files only, inside the repo
98
+ and changed by the stage, no test gaming, a discovered runner, more than zero
99
+ tests run), the repeated runs, then
100
+ `results.json` through `test-classify.mjs` and `test-gen-gate.mjs results`.
101
+
102
+ ## Step 6 - Triage
103
+
104
+ Stage 8, one verdict per non-pass result, each through `test-gen-state.mjs
105
+ revise`. Exit 3 parks the run: stop, keep every artefact, report why, and record
106
+ no change base, because the run did not finish.
107
+
108
+ ## Step 7 - Report and base
109
+
110
+ Print, in this order: the tests written (file, scenario, expectation), the
111
+ results table from `results.json` with every `fail` and `not_measured` reason,
112
+ the triage verdicts, the handles added (`handles.md`), the scenarios blocked by
113
+ data, and `gaps.json`'s `unmeasured[]`. State which parts are mechanical (the
114
+ gates, the classification, the handle audit) and which are judgement (the
115
+ inventory, the gaps, the triage). Then record the change base
116
+ (`test-gen-state.mjs base-record`).
117
+
118
+ Offer, one question: open the product bugs through `multi-agent` with each
119
+ failing test as the acceptance check, or stop. **No commit, no push, no issue
120
+ from this command.**