@mmerterden/multi-agent-pipeline 20.27.1 → 20.28.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +38 -0
- package/README.md +6 -5
- package/README.tr.md +6 -5
- package/docs/adr/0008-installer-modularization-and-secret-leak-defense.md +0 -1
- package/docs/architecture.md +4 -4
- package/docs/ecosystem.md +5 -5
- package/docs/facts.json +14 -12
- package/docs/features.md +2 -0
- package/install/_dev-only-files.mjs +1 -0
- package/install/catalog-history.json +1 -1
- package/install/index.mjs +0 -2
- package/manifest.json +74 -51
- package/package.json +4 -1
- package/pipeline/agents/code-reviewer.md +1 -0
- package/pipeline/agents/test-gen-analyst.md +98 -0
- package/pipeline/commands/multi-agent/sync/SKILL.md +3 -3
- package/pipeline/commands/multi-agent/test-gen/SKILL.md +124 -0
- package/pipeline/multi-agent-refs/cross-cli-contract.md +11 -5
- package/pipeline/multi-agent-refs/features/doctor.md +13 -0
- package/pipeline/multi-agent-refs/features/test-classification.md +76 -0
- package/pipeline/multi-agent-refs/features/usage-reporting.md +17 -0
- package/pipeline/multi-agent-refs/help/en.md +1 -0
- package/pipeline/multi-agent-refs/help/tr.md +1 -0
- package/pipeline/multi-agent-refs/phases/operations.md +2 -2
- package/pipeline/multi-agent-refs/phases/phase-2-dev.md +6 -7
- package/pipeline/multi-agent-refs/phases/phase-3-review.md +3 -3
- package/pipeline/multi-agent-refs/phases/phase-5-report.md +6 -5
- package/pipeline/multi-agent-refs/store-ready/gate-3-policy.md +6 -2
- package/pipeline/multi-agent-refs/test-gen/stages.md +227 -0
- package/pipeline/preferences-template.json +3 -0
- package/pipeline/rules/app-store-guidelines.md +29 -41
- package/pipeline/schemas/prefs.schema.json +14 -0
- package/pipeline/schemas/test-gen-discovery.schema.json +73 -0
- package/pipeline/schemas/test-gen-gaps.schema.json +122 -0
- package/pipeline/schemas/test-gen-inventory.schema.json +59 -0
- package/pipeline/schemas/test-gen-results.schema.json +44 -0
- package/pipeline/scripts/_handle-patterns.mjs +166 -0
- package/pipeline/scripts/_http-post.mjs +141 -0
- package/pipeline/scripts/_stack-routing.mjs +4 -0
- package/pipeline/scripts/_test-gaming.mjs +227 -0
- package/pipeline/scripts/_unified-diff.mjs +104 -0
- package/pipeline/scripts/_worktree-snapshot.mjs +113 -0
- package/pipeline/scripts/doctor.mjs +51 -0
- package/pipeline/scripts/evidence-gate.mjs +74 -3
- package/pipeline/scripts/handle-diff-audit.mjs +188 -0
- package/pipeline/scripts/routine-registry.mjs +5 -1
- package/pipeline/scripts/run-ui-tests.sh +27 -3
- package/pipeline/scripts/test-classify.mjs +296 -0
- package/pipeline/scripts/test-gen-gate.mjs +409 -0
- package/pipeline/scripts/test-gen-state.mjs +313 -0
- package/pipeline/scripts/test-integrity-gate.mjs +76 -11
- package/pipeline/scripts/uninstall.mjs +1 -0
- package/pipeline/scripts/usage-register-beacon.mjs +239 -0
- package/pipeline/scripts/usage-register.mjs +128 -33
- package/pipeline/scripts/usage-report.mjs +43 -33
- package/pipeline/skills/.skill-manifest.json +25 -5
- package/pipeline/skills/.skills-index.json +57 -2
- package/pipeline/skills/shared/README.md +11 -6
- package/pipeline/skills/shared/core/apple-archive-compliance/SKILL.md +5 -4
- package/pipeline/skills/shared/core/apple-archive-compliance/references/rules.yml +8 -8
- package/pipeline/skills/shared/core/multi-agent-sync/SKILL.md +3 -3
- package/pipeline/skills/shared/core/multi-agent-test-gen/SKILL.md +120 -0
- package/pipeline/skills/shared/external/app-store-review/SKILL.md +69 -8
- package/pipeline/skills/shared/external/app-store-review/evals/evals.json +12 -0
- package/pipeline/skills/shared/external/app-store-review/references/review-checklists.md +10 -1
- package/pipeline/skills/shared/external/fix-review-verdicts/SKILL.md +133 -0
- package/pipeline/skills/shared/external/handle-request/SKILL.md +97 -0
- package/pipeline/skills/shared/external/scenario-gap-analysis/SKILL.md +129 -0
- package/pipeline/skills/shared/external/scenario-gap-analysis/references/scenario-schema.md +60 -0
- package/pipeline/skills/shared/external/test-result-triage/SKILL.md +160 -0
- package/pipeline/skills/skills-index.md +7 -2
- package/install/_telemetry.mjs +0 -61
package/CHANGELOG.md
CHANGED
|
@@ -16,6 +16,44 @@ Internal file-layout changes that don't affect the slash-command surface are sti
|
|
|
16
16
|
|
|
17
17
|
## [Unreleased]
|
|
18
18
|
|
|
19
|
+
## [20.28.0] - 2026-10-24
|
|
20
|
+
|
|
21
|
+
### Added
|
|
22
|
+
|
|
23
|
+
- A registration that fails for a reason the person did not choose reports it: `usage-register.mjs` starts `usage-register-beacon.mjs` detached, which sends `{ u, v, reason, os }` to `<register URL>/failure` once per reason and version a day, with no token and no failure text, and never for an opted-out machine or one with no login. `/multi-agent:doctor` gains the `usage-register` check, which shows the last registration outcome and whether the failure report reached the site.
|
|
24
|
+
- Four test-planning knowledge skills join the common toolkit: `scenario-gap-analysis` (covered, wrongly covered and missing scenarios, with a could-not-measure section), `test-result-triage` (product-bug, stale-test, valid or ambiguous per test, ambiguous unless evidence says otherwise), `handle-request` (added, already present or could not add, per platform) and `fix-review-verdicts` (test-gaming, scope and root-cause verdicts computed from findings). They are routed to `ai-common-toolkit` through `COMMON_SKILLS`.
|
|
25
|
+
- `pipeline/scripts/permission-baseline.mjs` freezes the tool grants the pipeline ships in `pipeline/contract/frozen/permission-baseline.json`: each command, skill and agent's `allowed-tools` / `tools` (an agent with no or an empty `tools` inherits everything and is recorded as `*`) and `disallowedTools`, the host-read frontmatter keys that change what a file may do (`permissionMode`, `mcpServers`, `hooks`, `context`, `agent`, `shell`), the unattended profile (allow and deny lists, wideAllows, sandbox lists and values, `defaultMode`, `env`), `RESEARCH_DENIED_TOOLS`, the grant every saved routine is written with (`ROUTINE_ALLOWED_TOOLS`), and the permission flags in the spawn argv of `pipeline/scripts` and `pipeline/lib` with every literal value they take (comment lines are skipped). `check` fails when an entry joins an allow list, leaves a deny list or a value changes, and when a grant cannot be read (a block scalar, a multi-line flow list, a folded continuation, a quoted or duplicated grant key, a flow-mapping or indented frontmatter, a missing code-level declaration); `accept` records an intended grant, so every widening lands as a reviewed line in the baseline diff. Narrowing passes on its own, a deleted file is reported as removed and a file is reported as moved only when its tools, deny list and frontmatter keys all match the old path. Enforced by `test/permission-baseline.test.mjs` in `npm test`; the script and its baseline are maintainer tooling and are not published.
|
|
26
|
+
- `/multi-agent:test-gen` writes the tests an app lacks, as seven subagent stages each gated on its own artefact: inventory (`inventory.json`, then a priority pick; autopilot takes the top three), discovery (`discovery.json`) and gaps (`gaps.json`, with `dataNeeds[]` that name test data and never hold it), handles (`handles.md`), the tests (`written.json`), repeated runs and per-test classification (`results.json`). Discovery and gaps run on the new read-only `test-gen-analyst` persona, whose tool grant is an allowlist of Read, Grep, Glob and read-only toolkit tools. `scripts/test-gen-gate.mjs` checks the artefacts against the new `test-gen-inventory`, `test-gen-gaps` and `test-gen-results` schemas without a dependency, snapshots the working tree (untracked files included) before each stage that may edit the repository and judges the stage by its own delta: the handle stage must add a handle line (or say `HANDLES: none-needed` with `file:line` evidence, inside the repository, of handles that exist), and the write stage may change test files only, inside the repository and changed by the stage, with no test gaming in its diff, a run command one of the discovered runners owns, and more than zero tests run. It also checks `discovery.json` (new `test-gen-discovery` schema) and writes the three-state `results.json`. Toolkit tools `test_results_classify`, `coverage_gap` and `ui_locator_codegen` need `@mmerterden/multi-agent-toolkit-mcp` ≥ v3.25.0, recorded in `cross-cli-contract.md`; each has a named fallback on an older server. `smoke-test-gen-contract.sh` holds the schemas, the stage artefact paths and the read-only agent type.
|
|
27
|
+
- `scripts/handle-diff-audit.mjs` checks that a diff only adds automation handles (iOS `accessibilityIdentifier`, Compose `testTag` / semantics, web `data-testid`). The removed lines and the added lines with their handle constructs stripped are compared as ordered code tokens, operators and punctuation included and whitespace ignored, so re-wrapping a tag passes while a flipped comparison, a dropped `!` or swapped arguments fail as lost, extra or reordered tokens; a new file passes only when it holds handle constants and nothing else. With `--baseline` (or `--since <tree>`) it reads only what changed after the stage-start snapshot, so the person's uncommitted work is never read as the stage's change; refs are verified and an option-shaped value is refused. Each file is judged by its own platform's pattern, so a Flutter `Key` or a selector string in a test file is not a handle. Exit 0 handles only, 2 violation, 3 unreadable input; `smoke-handle-diff-audit.sh` holds the codes on fixed fixtures.
|
|
28
|
+
- `evidence-gate.mjs --stage-start <time>` holds a stage to its own output: the evidence must have been written during the stage (mtime no earlier than the start less 2s) and be whole: a `.json` file must parse, any other file under 400 bytes or ending in a letter or digit with no trailing newline is partial. The new `--claim artefact` checks only that, for stage outputs that are not logs.
|
|
29
|
+
- Per-test classification at the Phase 2 exit gate (`features/test-classification.md`). The toolkit's `test_results_classify` (`@mmerterden/multi-agent-toolkit-mcp` ≥ v3.25.0) reads the Gate 3 report, the repeat runs as confirmation and the Phase 0 report as the base; `scripts/test-classify.mjs merge` applies the pipeline's rules on top: `unstable` is never green, `not_measured` stays red with its reason, the Phase 0 baseline joins the preexisting set (and a test it lists that the toolkit calls a regression becomes `unstable`), a tripped breaker is not green. On an older toolkit `test-classify.mjs fallback` classifies from the repeat runs' JUnit XML alone; a test that failed and passed inside one run (a retry, per-device reports) is `unstable`, and both attribute quote styles are read. Exit 4 marks a crash, apart from a red verdict. `run-ui-tests.sh run` now names a per-test report next to its log (`UI_TEST_RESULT`, `UI_TEST_RESULT_FORMAT`: the xcresult bundle on iOS, Playwright's JUnit report, the module's connected test results on Android).
|
|
30
|
+
- `scripts/test-gen-state.mjs` keeps the project-level state of test generation. `base-record` writes the commit a run finished at with a reason that must name the run; `base-scan` diffs base..HEAD into changed files and `graph-affected.mjs` dependents, and a scan that cannot be made still writes a `measured: false` row with the reason. `revise` counts stale-test revisions per run and per calendar day against the new `prefs.global.testGen.revisionCap` (default 20), so many small runs cannot slip under the cap; at the cap autopilot parks, and a `product-bug` verdict is never applied in autopilot. The state file is read and replaced under a lock; a state or prefs file that does not parse parks autopilot instead of counting from zero, and a cap outside 0..500 falls back to the default.
|
|
31
|
+
- Test gaming check: `test-integrity-gate.mjs --diff <unified.diff>` (or `--diff-base <ref>`) reports added lines that make a suite green without fixing the code, as `test_integrity` findings with `signal: "test_gaming"` and the line. Product code that branches on the test environment (`navigator.webdriver`, `process.env.CI`, ProcessInfo and XCTest, `isRunningTests`, a test-harness check) is blocking; a swallowed error (an empty catch, `.catch(() => {})`, `except: pass`, a statement-level Swift `try?`, a Kotlin `runCatching` nobody reads; one finding per occurrence), a timeout raised to twice the nearest removed one or a new wait of 30s or more in a test is important. Units follow the language (JavaScript and Kotlin milliseconds, Swift and Python seconds). Only source files are read and comment lines are skipped; configuration (`*.config.*`, CI workflows), YAML and documentation are out of scope. `--diff-base` reads the working tree, untracked files included, against the merge base with the base branch; a ref is verified, a missing or option-shaped value is a setup error, and an unreadable diff is reported as `gaming.scanned: false`, never as clean. Phase 3 Step 1.76 passes the run's diff, and the code reviewer persona gains a test-gaming lens that defers to the `fix-review-verdicts` skill when it is enabled.
|
|
32
|
+
|
|
33
|
+
### Fixed
|
|
34
|
+
|
|
35
|
+
- A finished, halted or parked run now reports its end: Phase 5 and the halt and park blocks send `usage-report.mjs --task-id`, and resolve the state file with the new `--print-state` instead of a `$STATE_FILE` that an earlier shell never handed over. An empty `--state` is recorded in `usage-last-send.json` as `empty-state-path` instead of returning silently. `test/usage-report-callers.test.mjs` fails any `--state "$VAR"` whose variable is not set in the same fenced block.
|
|
36
|
+
- Usage registration and reporting work behind a corporate proxy or a TLS-inspecting gateway: `_http-post.mjs` retries a connection-level failure of Node's fetch through curl, which honours the proxy variables and the system trust store, with secret headers read from a 0600 file and never placed on argv. A failure keeps both causes (`fetch SELF_SIGNED_CERT_IN_CHAIN, curl curl-exit-7`), and every registration attempt is recorded in `usage-last-register.json`.
|
|
37
|
+
- Lint and format skip `.claude/worktrees/`, where agent worktrees of other branches live.
|
|
38
|
+
|
|
39
|
+
### Removed
|
|
40
|
+
|
|
41
|
+
- The opt-in anonymous install ping (`MULTI_AGENT_TELEMETRY=1`, `install/_telemetry.mjs`). Install and update are already reported by `usage-report.mjs --lifecycle` under the machine's registered token, and the endpoint the ping used no longer accepts unauthenticated posts.
|
|
42
|
+
|
|
43
|
+
### Security
|
|
44
|
+
|
|
45
|
+
- `fast-uri` updated within its semver range to clear four moderate advisories (GHSA-w74x-v6f9-j3xg, GHSA-xhq7-xmqj-769h, GHSA-wjr9-j795-jxcr, GHSA-333x-w4r6-6crp).
|
|
46
|
+
|
|
47
|
+
## [20.27.2] - 2026-10-20
|
|
48
|
+
|
|
49
|
+
### Changed
|
|
50
|
+
|
|
51
|
+
- App Store compliance follows the App Review Guidelines of 16.10.2026 and the September 2026 upload requirements. `rules/app-store-guidelines.md` adds the upload floors (iOS 13 deployment target, iOS 26 SDK, iOS 27 SDK from April 2027), 1.2 for AI content creation apps and nonconsensual intimate imagery, 2.3.1(a) Notes for Review, 2.3.14 creative assets, 2.5.19 hidden features, the 3.1.1(a) External Purchase Link or, where permitted, Music Streaming Services entitlement, 4.3 saturated categories, 4.5.3 notification and Live Activity spam, the 5.1.1(v) citation, and the age rating and medical device declarations; 4.8 asks for an equivalent privacy-focused login rather than Sign in with Apple only.
|
|
52
|
+
- `store-ready` Gate 3 (iOS) gathers evidence for 1.2, 2.5.19, the upload floor and the App Store Connect declarations, and its IAP and login rows follow 3.1.1(a) and 4.8.
|
|
53
|
+
- `apple-archive-compliance`: `info-plist` covers `MinimumOSVersion` ≥ 13 for iOS and iPadOS builds, `sdk-floor` names the announced iOS 27 floor, `required-reason-api` cites ITMS-91053, and `info-plist` and `embedded-sdk` carry the guideline and ITMS codes their scanner emits. The new behaviour needs `@mmerterden/multi-agent-toolkit-mcp` ≥ 3.23.2.
|
|
54
|
+
- `app-store-review` covers the new guidelines, the upload floors and the App Store Connect declarations, with a new eval.
|
|
55
|
+
- `smoke-compliance-skills.sh` checks that the Apple catalog table and `references/rules.yml` agree on every rule's ITMS codes and SDK floor.
|
|
56
|
+
|
|
19
57
|
## [20.27.1] - 2026-10-19
|
|
20
58
|
|
|
21
59
|
### Fixed
|
package/README.md
CHANGED
|
@@ -158,7 +158,7 @@ The discipline behind all of this - bounded loops, evidence gates, token-budgete
|
|
|
158
158
|
|
|
159
159
|
## Commands
|
|
160
160
|
|
|
161
|
-
`/multi-agent` plus
|
|
161
|
+
`/multi-agent` plus 66 sub-commands. `/multi-agent:help` renders the same catalog in your terminal, in your `outputLanguage`.
|
|
162
162
|
|
|
163
163
|
### Pipeline entries
|
|
164
164
|
|
|
@@ -221,6 +221,7 @@ The discipline behind all of this - bounded loops, evidence gates, token-budgete
|
|
|
221
221
|
| `/multi-agent:test-dynamic-type` | Re-walk every screen at XL through accessibility-XL, report truncation |
|
|
222
222
|
| `/multi-agent:test-screenshots [locale]` | App Store screenshot set in a locale (defaults to `tr`) |
|
|
223
223
|
| `/multi-agent:bug-bash` | Parallel exploratory sessions by charter; only findings a failing repro test proves are reported |
|
|
224
|
+
| `/multi-agent:test-gen` | Writes the tests an app lacks, stage by stage: inventory, read-only gap analysis, handles, tests, repeated runs, per-test classification |
|
|
224
225
|
| `/multi-agent:manual-test` | Phase 3 standalone: check out the task branch and prepare it for Xcode |
|
|
225
226
|
|
|
226
227
|
### Design, build and store
|
|
@@ -407,17 +408,17 @@ This enables the matching plugin (+ the shared `ai-common` plugin) in the repo's
|
|
|
407
408
|
|
|
408
409
|
## Tool support
|
|
409
410
|
|
|
410
|
-
The pipeline runs natively on **Claude Code**, **Copilot CLI** and **Codex CLI** - all three install from the same `pipeline/` source and get the same
|
|
411
|
+
The pipeline runs natively on **Claude Code**, **Copilot CLI** and **Codex CLI** - all three install from the same `pipeline/` source and get the same 66 commands.
|
|
411
412
|
|
|
412
413
|
| Tool | Flag | What it installs |
|
|
413
414
|
| ----------- | -------------------- | ------------------------------------------------------------------------------------------------------ |
|
|
414
415
|
| Claude Code | `--claude` (default) | slash commands + skills + agents + three `PreToolUse` hooks (secret scan, agent-guard, read-size gate) |
|
|
415
|
-
| Copilot CLI | `--copilot` | instructions +
|
|
416
|
-
| Codex CLI | `--codex` | one router skill +
|
|
416
|
+
| Copilot CLI | `--copilot` | instructions + 66 sub-command skills + scripts |
|
|
417
|
+
| Codex CLI | `--codex` | one router skill + 66 specs as refs + 11 agent TOML + `AGENTS.md` block + `codex mcp add` |
|
|
417
418
|
|
|
418
419
|
Filter skills by stack with `--platform=ios\|android\|all`.
|
|
419
420
|
|
|
420
|
-
**Why Codex gets one skill and not
|
|
421
|
+
**Why Codex gets one skill and not 66.** Codex assembles every discovered skill's name
|
|
421
422
|
and description into a single prompt block and drops entries when it overflows, with no
|
|
422
423
|
error. Measured on 0.145: installing one plugin that declares 142 skills surfaced only
|
|
423
424
|
75 of them and evicted an unrelated user skill. So on Codex the pipeline ships a single
|
package/README.tr.md
CHANGED
|
@@ -157,7 +157,7 @@ Koşunun kendisinde ayarlanabilen tek şey autopilot; geri kalan her şey kendi
|
|
|
157
157
|
|
|
158
158
|
## Komutlar
|
|
159
159
|
|
|
160
|
-
`/multi-agent` ve
|
|
160
|
+
`/multi-agent` ve 66 alt komut. `/multi-agent:help` aynı katalogu terminalde, `outputLanguage` ayarına göre gösterir.
|
|
161
161
|
|
|
162
162
|
### Pipeline girişleri
|
|
163
163
|
|
|
@@ -220,6 +220,7 @@ Koşunun kendisinde ayarlanabilen tek şey autopilot; geri kalan her şey kendi
|
|
|
220
220
|
| `/multi-agent:test-dynamic-type` | Her ekranı XL'den accessibility-XL'e kadar yeniden gezer, kırpılmaları raporlar |
|
|
221
221
|
| `/multi-agent:test-screenshots [dil]` | Bir dilde App Store screenshot seti (varsayılan `tr`) |
|
|
222
222
|
| `/multi-agent:bug-bash` | Charter başına paralel keşif oturumları; yalnız fail eden bir repro testinin kanıtladığı bulgular raporlanır |
|
|
223
|
+
| `/multi-agent:test-gen` | Uygulamada eksik testleri aşama aşama yazar: envanter, salt-okunur eksik analizi, tutamaklar, testler, tekrarlı koşum, test başına sınıflama |
|
|
223
224
|
| `/multi-agent:manual-test` | Faz 5'in bağımsız hali: görev branch'ine geçer, Xcode için hazırlar |
|
|
224
225
|
|
|
225
226
|
### Tasarım, build ve store
|
|
@@ -406,17 +407,17 @@ Bu, ilgili plugin'i (+ ortak `ai-common` plugin'ini) repo'nun `.claude/settings.
|
|
|
406
407
|
|
|
407
408
|
## Araç desteği
|
|
408
409
|
|
|
409
|
-
Pipeline **Claude Code**, **Copilot CLI** ve **Codex CLI** üzerinde native çalışır - üçü de aynı `pipeline/` kaynağından kurulur ve aynı
|
|
410
|
+
Pipeline **Claude Code**, **Copilot CLI** ve **Codex CLI** üzerinde native çalışır - üçü de aynı `pipeline/` kaynağından kurulur ve aynı 66 komutu alır.
|
|
410
411
|
|
|
411
412
|
| Araç | Bayrak | Ne kurar |
|
|
412
413
|
| ----------- | ----------------------- | ---------------------------------------------------------------------------------------------------------------- |
|
|
413
414
|
| Claude Code | `--claude` (varsayılan) | slash komutları + skill'ler + agent'lar + üç `PreToolUse` hook'u (secret scan, agent-guard, okuma-boyutu geçidi) |
|
|
414
|
-
| Copilot CLI | `--copilot` | talimatlar +
|
|
415
|
-
| Codex CLI | `--codex` | bir router skill + ref olarak
|
|
415
|
+
| Copilot CLI | `--copilot` | talimatlar + 66 alt-komut skill'i + script'ler |
|
|
416
|
+
| Codex CLI | `--codex` | bir router skill + ref olarak 66 spec + 11 agent TOML + `AGENTS.md` bloğu + `codex mcp add` |
|
|
416
417
|
|
|
417
418
|
Skill'leri stack'e göre filtrele: `--platform=ios\|android\|all`.
|
|
418
419
|
|
|
419
|
-
**Codex neden
|
|
420
|
+
**Codex neden 66 değil de tek bir skill alıyor.** Codex, keşfettiği her skill'in adını
|
|
420
421
|
ve açıklamasını tek bir prompt bloğuna toplar ve blok taştığında girdileri hatasızca
|
|
421
422
|
düşürür. 0.145 üzerinde ölçüldü: 142 skill deklare eden bir plugin kurulduğunda sadece
|
|
422
423
|
75'i yüzeye çıktı ve alakasız bir kullanıcı skill'i tahliye edildi. Bu yüzden Codex'te
|
|
@@ -37,7 +37,6 @@ install/
|
|
|
37
37
|
_adapters.mjs - third-party adapter dispatch
|
|
38
38
|
_common.mjs - shared FS helpers (copyDir, ensureDir, ...)
|
|
39
39
|
_platform-filter.mjs - --platform=ios|android filter
|
|
40
|
-
_telemetry.mjs - opt-in install telemetry
|
|
41
40
|
_copilot-instructions.mjs - generator for ~/.copilot/copilot-instructions.md
|
|
42
41
|
_dev-only-files.mjs - single source of truth for excluded files
|
|
43
42
|
```
|
package/docs/architecture.md
CHANGED
|
@@ -113,8 +113,8 @@ graph TB
|
|
|
113
113
|
end
|
|
114
114
|
|
|
115
115
|
subgraph "Pipeline Specs"
|
|
116
|
-
CMD[commands/<br/>
|
|
117
|
-
AGT[agents/<br/>
|
|
116
|
+
CMD[commands/<br/>66 command files]
|
|
117
|
+
AGT[agents/<br/>11 agent personas]
|
|
118
118
|
RUL[rules/<br/>13 domain rules]
|
|
119
119
|
PHS[multi-agent-refs/phases/<br/>phase specs + contracts]
|
|
120
120
|
CMP[skills/shared/core/<br/>orchestration skills<br/>incl. compliance]
|
|
@@ -204,8 +204,8 @@ repo and the website, plus the two independently shipped repos
|
|
|
204
204
|
```mermaid
|
|
205
205
|
graph TD
|
|
206
206
|
CC["Claude Code<br/>(source of truth)"]
|
|
207
|
-
COP["Copilot CLI<br/>(instructions +
|
|
208
|
-
COD["Codex CLI<br/>(1 router skill +
|
|
207
|
+
COP["Copilot CLI<br/>(instructions + 66 sub-command skills)"]
|
|
208
|
+
COD["Codex CLI<br/>(1 router skill + 66 refs)"]
|
|
209
209
|
REPO["Pipeline Repo<br/>(npm package)"]
|
|
210
210
|
WEB["Website"]
|
|
211
211
|
PLUGREPO["multi-agent-plugins<br/>(6 plugins, own repo)"]
|
package/docs/ecosystem.md
CHANGED
|
@@ -5,7 +5,7 @@ separately, wired together at install time and at run time:
|
|
|
5
5
|
|
|
6
6
|
| Repo | What it owns | Ships as |
|
|
7
7
|
|---|---|---|
|
|
8
|
-
| **`multi-agent-pipeline`** (this repo) | Orchestration: the 6-phase flow, the
|
|
8
|
+
| **`multi-agent-pipeline`** (this repo) | Orchestration: the 6-phase flow, the 66 slash commands, quality gates, review/triage, cross-CLI parity | npm package (`@mmerterden/multi-agent-pipeline`), installs itself onto Claude Code / Copilot CLI / Codex CLI |
|
|
9
9
|
| **`multi-agent-plugins`** | Stack knowledge: per-platform component/lifecycle skills (iOS, Android, Frontend, Backend) + shared knowledge | Claude Code marketplace, 6 independently-versioned plugins |
|
|
10
10
|
| **`multi-agent-toolkit-mcp`** | The pipeline's hands on devices and browsers: 197 MCP tools across 21 categories (simulator/emulator control, end-to-end locators and assertions, memory, crash diagnostics, accessibility audit, store compliance, web automation and scraping, Figma-vs-mock design audit, code intelligence, wallet passes, an agent-DSL batch runner, a full-text context index with text-to-image rendering, provider-backed research, verified hand-off of simple text tasks to free models, video key frames, offline security scoring, read-only access to related repositories at their latest branch, travel document fixtures, a voice channel, App Store Connect and Play Console APIs, virtual iPhones) | npm package, registered as a standard stdio MCP server on every host |
|
|
11
11
|
|
|
@@ -18,7 +18,7 @@ Either can be swapped or removed without touching the other two's source.
|
|
|
18
18
|
graph LR
|
|
19
19
|
subgraph PIPE ["multi-agent-pipeline (orchestrator)"]
|
|
20
20
|
direction TB
|
|
21
|
-
PHASES["6 phases ·
|
|
21
|
+
PHASES["6 phases · 66 commands"]
|
|
22
22
|
GATES["deterministic gates + review triage"]
|
|
23
23
|
end
|
|
24
24
|
|
|
@@ -68,8 +68,8 @@ only those:
|
|
|
68
68
|
graph TD
|
|
69
69
|
CC["Claude Code<br/>~/.claude/commands/multi-agent/<br/>(source of truth)"]
|
|
70
70
|
|
|
71
|
-
CC -->|"Step 2: copy + reformat<br/>
|
|
72
|
-
CC -->|"Step 2b: transform<br/>(install.js --codex)"| COD["Codex CLI<br/>1 router skill +
|
|
71
|
+
CC -->|"Step 2: copy + reformat<br/>66 sub-command skills"| COP["Copilot CLI<br/>~/.copilot/skills/"]
|
|
72
|
+
CC -->|"Step 2b: transform<br/>(install.js --codex)"| COD["Codex CLI<br/>1 router skill + 66 refs<br/>+ 11 agent TOML"]
|
|
73
73
|
CC -->|"Step 3: genericize<br/>(strip personal data)"| REPO["multi-agent-pipeline repo<br/>pipeline/"]
|
|
74
74
|
CC -->|"Step 4: version + feature sync"| WEB["Website<br/>projects.ts / i18n.tsx"]
|
|
75
75
|
|
|
@@ -161,7 +161,7 @@ measurements behind this table):
|
|
|
161
161
|
|
|
162
162
|
| | Claude Code | Copilot CLI | Codex CLI |
|
|
163
163
|
|---|---|---|---|
|
|
164
|
-
| **Pipeline commands** |
|
|
164
|
+
| **Pipeline commands** | 66 slash-command skills, native | 66 sub-command skills, `multi-agent-{cmd}` naming, copied in | 1 router skill (`multi-agent`) + 66 command specs as reference files - Codex silently truncates its skills block past a few dozen entries, so sub-commands are not peer skills here |
|
|
165
165
|
| **Stack plugins** | Marketplace plugin, loaded natively, resolved by `.claude/settings.json` enabled-list | Enabled plugin's authored skills copied flat into `~/.copilot/skills/`; `knowledge/` **not** re-copied (already delivered via `shared/external`) | Copied as reference files under `~/.codex/multi-agent-refs/skills/`, plugin-prefixed on name clash (e.g. `architecture` → `ai-ios-toolkit-architecture`) |
|
|
166
166
|
| **Component dispatch (Phase 3)** | Marketplace plugin's `create-component`/`create-screen` skill via the Skill tool | No plugin loader - the enabled stack plugin's authored skills (incl. `create-component`) are copied flat into `~/.copilot/skills/` at install time (the old frozen `figma-*` copies are pruned, they were never a fallback) | Not part of the enforced parity axis; classification + state-shape must match, skill *inventory* does not |
|
|
167
167
|
| **multi-agent-toolkit-mcp** | `claude mcp add multi-agent-toolkit -- npx -y @mmerterden/multi-agent-toolkit-mcp` | `copilot mcp add multi-agent-toolkit -- npx -y @mmerterden/multi-agent-toolkit-mcp` | `codex mcp add multi-agent-toolkit -- npx -y @mmerterden/multi-agent-toolkit-mcp` (skipped with a warning if `codex` isn't on `PATH`) |
|
package/docs/facts.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"$comment": "Generated by pipeline/scripts/gen-facts.mjs. Do not hand-edit: the site reads this, and a number edited here instead of at its source is the drift this file removes.",
|
|
3
|
-
"generatedAt": "2026-10-
|
|
4
|
-
"version": "20.
|
|
3
|
+
"generatedAt": "2026-10-24",
|
|
4
|
+
"version": "20.28.0",
|
|
5
5
|
"phaseSchema": 2,
|
|
6
6
|
"phases": [
|
|
7
7
|
{
|
|
@@ -35,15 +35,15 @@
|
|
|
35
35
|
"autopilot": [0, 1, 2, 3, 4, 5],
|
|
36
36
|
"analysis": [0, 1, 3, 4, 5]
|
|
37
37
|
},
|
|
38
|
-
"commandCount":
|
|
39
|
-
"skillCount":
|
|
40
|
-
"skillCountAll":
|
|
41
|
-
"agentCount":
|
|
38
|
+
"commandCount": 66,
|
|
39
|
+
"skillCount": 165,
|
|
40
|
+
"skillCountAll": 234,
|
|
41
|
+
"agentCount": 11,
|
|
42
42
|
"hosts": ["Claude Code", "Codex CLI", "Copilot CLI"],
|
|
43
|
-
"toolCount":
|
|
43
|
+
"toolCount": 204,
|
|
44
44
|
"toolCategories": {
|
|
45
45
|
"ios": 42,
|
|
46
|
-
"web":
|
|
46
|
+
"web": 38,
|
|
47
47
|
"android": 33,
|
|
48
48
|
"store": 30,
|
|
49
49
|
"vphone": 10,
|
|
@@ -53,14 +53,16 @@
|
|
|
53
53
|
"voice": 6,
|
|
54
54
|
"context": 4,
|
|
55
55
|
"pass": 4,
|
|
56
|
+
"document": 3,
|
|
56
57
|
"agent": 2,
|
|
57
|
-
"document": 2,
|
|
58
58
|
"llm": 2,
|
|
59
59
|
"research": 2,
|
|
60
60
|
"security": 2,
|
|
61
|
+
"ui": 2,
|
|
62
|
+
"coverage": 1,
|
|
61
63
|
"media": 1,
|
|
62
|
-
"
|
|
64
|
+
"test": 1
|
|
63
65
|
},
|
|
64
|
-
"pluginSkillCount":
|
|
65
|
-
"toolkitVersion": "3.
|
|
66
|
+
"pluginSkillCount": 261,
|
|
67
|
+
"toolkitVersion": "3.25.0"
|
|
66
68
|
}
|
package/docs/features.md
CHANGED
|
@@ -392,6 +392,8 @@ A visual or semantic assertion is decided by a fresh judge subagent that receive
|
|
|
392
392
|
|
|
393
393
|
`/multi-agent:bug-bash` fans out 5-10 charters, each with a persona and its own budget, merges the findings, triages them against the source, and proves every candidate with a reproducing test. `bug-bash.mjs report` confirms a finding only when its repro failed with `ASSERTION_FAILED` on the assertion that encodes it; everything else is listed as rejected with the reason (artifact, environment, design, fixture, did-not-reproduce, unproven, unverified). The `e2e-testing` knowledge skill in ai-common-toolkit carries the toolkit side: locators, expect verdicts, traces, replay, parallel web sessions and synthetic MRZ document fixtures.
|
|
394
394
|
|
|
395
|
+
`/multi-agent:test-gen` writes the tests an app lacks as a chain of subagents, each gated on its own artefact: inventory (`inventory.json`, then a priority pick, the top three in autopilot), discovery and gaps (`discovery.json`, `gaps.json` with `dataNeeds[]`) on the read-only `test-gen-analyst` persona, whose tool grant is an allowlist of Read, Grep, Glob and read-only toolkit tools, handles (`handles.md`; judged against a stage-start snapshot of the working tree, the stage must add a handle line and `handle-diff-audit.mjs` must find nothing but handles in its delta, compared as ordered code tokens), the tests (`written.json`; test files only, no test gaming in the stage's diff, more than zero tests run), repeated runs, and per-test classification (`results.json`: pass, fail or not_measured, each non-pass with its reason). The run ends by recording its commit as the next run's change base, and autopilot parks at the stale-test revision cap or on any product-bug verdict. Detail: [`stages.md`](../pipeline/multi-agent-refs/test-gen/stages.md).
|
|
396
|
+
|
|
395
397
|
### Token Budget Enforcement
|
|
396
398
|
|
|
397
399
|
Per-phase token budgets prevent runaway sessions. If a phase exceeds its budget, the pipeline pauses and offers: continue (extend budget), skip phase, or abort. Budgets are configurable in `prefs.global.tokenBudgets`.
|
|
@@ -70,6 +70,7 @@ const DEV_ONLY_TOOLING = Object.freeze([
|
|
|
70
70
|
"gen-catalog-history.mjs", // reads the repo's git history; an install has none
|
|
71
71
|
"gen-ref-toc.mjs", // regenerates the repo's own reference tree; an install has no pipeline/
|
|
72
72
|
"validate-schemas.mjs", // validates the repo's own schema files, needs ajv
|
|
73
|
+
"permission-baseline.mjs", // reads the repo's own command, skill and agent tree; an install has none
|
|
73
74
|
"sync-parity-check.sh",
|
|
74
75
|
"benchmark-phase-0.sh",
|
|
75
76
|
// Publish-time only: `prepack` runs it from the repo root to write
|