@mmerterden/multi-agent-pipeline 20.15.1 → 20.17.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +45 -0
- package/README.md +6 -5
- package/README.tr.md +6 -5
- package/docs/architecture.md +3 -3
- package/docs/ecosystem.md +5 -5
- package/docs/facts.json +14 -12
- package/docs/features.md +9 -1
- package/install/catalog-history.json +1 -1
- package/install/claude.mjs +40 -1
- package/install/codex.mjs +4 -3
- package/manifest.json +70 -51
- package/package.json +1 -1
- package/pipeline/commands/multi-agent/bug-bash/SKILL.md +154 -0
- package/pipeline/commands/multi-agent/review/SKILL.md +46 -1
- package/pipeline/commands/multi-agent/setup/SKILL.md +8 -6
- package/pipeline/commands/multi-agent/sync/SKILL.md +4 -3
- package/pipeline/commands/multi-agent/update/SKILL.md +5 -1
- package/pipeline/commands/sim-test.md +71 -6
- package/pipeline/lib/plugin-skill-roots.mjs +80 -0
- package/pipeline/lib/usage-lifecycle.mjs +83 -0
- package/pipeline/multi-agent-refs/cross-cli-contract.md +20 -4
- package/pipeline/multi-agent-refs/features/assertion-judge.md +76 -0
- package/pipeline/multi-agent-refs/features/exploratory-testing.md +140 -0
- package/pipeline/multi-agent-refs/features/review-decision.md +2 -1
- package/pipeline/multi-agent-refs/features/skill-conformance.md +1 -1
- package/pipeline/multi-agent-refs/features/usage-reporting.md +59 -5
- package/pipeline/multi-agent-refs/features/valven.md +110 -0
- package/pipeline/multi-agent-refs/help/en.md +2 -0
- package/pipeline/multi-agent-refs/help/tr.md +2 -0
- package/pipeline/multi-agent-refs/phases/phase-0-init.md +2 -1
- package/pipeline/multi-agent-refs/phases/phase-1-plan.md +2 -0
- package/pipeline/multi-agent-refs/phases/phase-2-dev.md +2 -4
- package/pipeline/multi-agent-refs/phases/phase-3-review.md +5 -5
- package/pipeline/multi-agent-refs/phases/phase-4-commit.md +3 -2
- package/pipeline/schemas/agent-state.schema.json +16 -0
- package/pipeline/schemas/bug-bash-verification.schema.json +41 -0
- package/pipeline/schemas/explore-run.schema.json +108 -0
- package/pipeline/schemas/prefs.schema.json +21 -0
- package/pipeline/scripts/_stack-routing.mjs +2 -0
- package/pipeline/scripts/bug-bash.mjs +379 -0
- package/pipeline/scripts/explore-findings.mjs +410 -0
- package/pipeline/scripts/pr-timeline.mjs +135 -0
- package/pipeline/scripts/pre-commit-check.sh +4 -2
- package/pipeline/scripts/review-decision-gate.mjs +1 -1
- package/pipeline/scripts/skill-conformance.mjs +17 -71
- package/pipeline/scripts/smoke-cross-cli-behavior.sh +1 -1
- package/pipeline/scripts/uninstall.mjs +28 -3
- package/pipeline/scripts/usage-identity.mjs +36 -10
- package/pipeline/scripts/usage-register.mjs +95 -8
- package/pipeline/scripts/usage-report.mjs +52 -2
- package/pipeline/scripts/valven-gate.mjs +1069 -0
- package/pipeline/scripts/valven-port.mjs +169 -0
- package/pipeline/scripts/valven-split.mjs +341 -0
- package/pipeline/scripts/verify-citations.mjs +14 -1
- package/pipeline/skills/.skill-manifest.json +18 -6
- package/pipeline/skills/.skills-index.json +126 -71
- package/pipeline/skills/shared/README.md +8 -5
- package/pipeline/skills/shared/core/multi-agent-bug-bash/SKILL.md +150 -0
- package/pipeline/skills/shared/core/multi-agent-review/SKILL.md +2 -0
- package/pipeline/skills/shared/core/multi-agent-setup/SKILL.md +5 -1
- package/pipeline/skills/shared/core/multi-agent-sync/SKILL.md +4 -3
- package/pipeline/skills/shared/core/multi-agent-update/SKILL.md +4 -1
- package/pipeline/skills/shared/external/e2e-testing/SKILL.md +124 -0
- package/pipeline/skills/shared/external/e2e-testing/references/document-fixtures.md +65 -0
- package/pipeline/skills/shared/external/valven-rules/SKILL.md +93 -0
- package/pipeline/skills/shared/external/valven-rules/references/rules.yml +186 -0
- package/pipeline/skills/skills-index.md +76 -71
package/CHANGELOG.md
CHANGED
|
@@ -16,6 +16,51 @@ Internal file-layout changes that don't affect the slash-command surface are sti
|
|
|
16
16
|
|
|
17
17
|
## [Unreleased]
|
|
18
18
|
|
|
19
|
+
## [20.17.0] - 2026-10-07
|
|
20
|
+
|
|
21
|
+
### Added
|
|
22
|
+
|
|
23
|
+
- Install and update records for operational reporting (`pipeline/lib/usage-lifecycle.mjs`): the installed version is read from the installer's `.pipeline-version` stamp, its time from the stamp's modification time, and compared with `usageLog.reportedVersion`. When they differ, `usage-report.mjs --lifecycle` sends an `install` or `update` record (new version, the version it replaced, the host trees, the time) under the reporting login and moves `reportedVersion` forward once the server accepted it. Registration, setup and update with a token already onboarded, and every accepted run event run it; the installer still writes nothing but files. `usage-register.mjs` records `usageLog.registeredAt`, which tells a first install from an update on a machine that never reported a version.
|
|
24
|
+
- A registration is pinned: the machine keeps the verified login in `usageLog.registeredAs` (`registeredFallback` says whether it was a fallback account), every record is named after it without a gh call, and a fallback account never replaces it. `usage-register.mjs --json` reports `needsPersonalAccount`, and setup and update offer once to log in the account that has access, switch the work account back and re-register, so the personal account needs to be logged in to gh only at that moment.
|
|
25
|
+
- Valven Development Behaviors in `valven-rules` (registry 1.1.0), scored on Valven's own bands: `VLV-CYCLE-01` coding time (first commit to the pull request opening; 30 min / 2.5 h / 24 h), `VLV-CYCLE-02` pickup time (ready for review to the first review; 1 h / 3 h / 14 h) and `VLV-CYCLE-03` review time (first review to approval or merge; 30 min / 3 h / 18 h). A duration in the 50 or 0 band is a `suggestion`, because the time has passed and the pull request cannot change it. `VLV-PICKUP-01` is `important` for a pull request without a reviewer and a `suggestion` for a draft older than 3 hours (`draft_severity`), because an unattended run opens a draft on purpose; a draft is not asked for a reviewer. `VLV-REWORK-01` also reports the churn score, `100 - churn %`.
|
|
26
|
+
- `valven-gate.mjs --pr-meta-file`: the pull request's timeline, read from `gh pr view --json author,closedAt,createdAt,isDraft,mergedAt,reviewRequests,reviews` as it is or from a normalized Bitbucket file; reviews by the author never count. `pr-timeline.mjs` writes the GitHub file: `gh pr view` plus the ready-for-review time and each review author's type from GraphQL, so a pull request opened as a draft is timed from when it was marked ready and a bot reviewer is not counted as a person. Pending reviews, bot reviews and reviews left before ready are ignored, an unknown reviewer count stays unknown, and a merged or closed pull request stops the clocks. The report carries `scores` per rule and `results[].score`; an unreadable file is reported and only the timeline rules go `not-measured`. `/multi-agent:review` passes it for GitHub and Bitbucket pull requests and prints a `Valven scores` line.
|
|
27
|
+
- `pipeline/lib/plugin-skill-roots.mjs`: the marketplace plugin `skills/` roots, shared by `skill-conformance.mjs` and `valven-gate.mjs`.
|
|
28
|
+
- Exploratory testing contract for `/multi-agent:test` (`features/exploratory-testing.md`, `explore-findings.mjs`, `explore-run.schema.json`). A finding carries expected, observed, repro steps, severity 1-5, kind `issue` or `warning`, and evidence on disk; a finding that misses a field, or writes a secret in clear text instead of `secret_ref:<name>`, cannot fail the run. Each run has a step, minute and token budget (defaults 8 / 10 / 150000, clamped to 1-12 / 3-15 / 20000-400000), and `check` stops it at a spent budget or after three consecutive blocked steps that reported nothing. `CREDENTIAL_MISSING`, `ENVIRONMENT_UNAVAILABLE`, `SEED_DATA_MISSING` and locator or replay failures are blocked steps, reported by cause and apart from product failures; a run whose every step was blocked is `blocked`, not `passed`. `sim-test.md` writes the run file, prefers `ui_locate` / `*_tap_element` and `expect_*` steps over coordinates, and renders its report from `report`.
|
|
29
|
+
- Assertion judge (`features/assertion-judge.md`): a visual or semantic assertion is decided by a fresh subagent that receives only the assertion, the current screenshot or UI tree, and optionally the app's vocabulary. `explore-findings.mjs judge-packet` refuses any field describing the run, and `judge-verdict` fails an inconclusive answer, or one citing evidence it was not given, as `ASSERTION_INCONCLUSIVE`.
|
|
30
|
+
- `/multi-agent:bug-bash [scope] [--platform ios|android|web] [--charters <file>]`: 5-10 charters, each with a persona (`first-time-user`, `skeptic`, `fuzzer`, `state-keeper`, `error-paths`, or `custom`) and its own budget, run as parallel explorer subagents (at most 4, one device or web `session` each). `bug-bash.mjs` validates the charters, merges findings that describe one defect, and confirms a candidate only when its repro test failed with `ASSERTION_FAILED` on the assertion that encodes it; every other candidate is listed as rejected with its reason (artifact, environment, design, fixture, did-not-reproduce, unproven, unverified), and environment rejections can be kept as unverified production risks. No commit, push or issue.
|
|
31
|
+
- `e2e-testing` knowledge skill, shipped in ai-common-toolkit: element locators, expect steps and their verdict codes, `secret_ref`, failure traces, `save_as` / `replay`, parallel web sessions, and `document_generate` / `document_mrz_build` fixtures for MRZ and document-scan tests. The toolkit side of all three entries needs `@mmerterden/multi-agent-toolkit-mcp` ≥ v3.19.0 (`cross-cli-contract.md` section 1.4); on an older server the flows fall back to UI-tree reads and frame-centre taps.
|
|
32
|
+
- The description-surface ceiling in `smoke-context-budget.sh` moves from 91000 to 91500 for `/multi-agent:bug-bash` and its shared/core counterpart (211 bytes each, below the 317-byte average, after a trim from 391) and the `e2e-testing` skill: the tree grew, no description is padded.
|
|
33
|
+
|
|
34
|
+
### Changed
|
|
35
|
+
|
|
36
|
+
- `valven-rules` moved from `skills/shared/core` to `skills/shared/external` and ships in `ai-common-toolkit`, so the rules reach everyone who installs the marketplace, not only pipeline users. Claude Code reads it from the plugin and no longer installs a local copy; Copilot CLI and Codex receive it from the installer as before. `valven-gate.mjs` and `valven-split.mjs` read the registry installed with them, so the gate's evaluators and rules come from one release: on Claude Code the installer lays the skill down in `multi-agent-refs/registries/valven-rules/`, outside the skills dir, and a leftover `~/.claude/skills/valven-rules` is never read. The plugin copies (checkout, then every cached version, newest first) are the last resort. On Claude Code `skill-conformance.mjs` searches the plugin roots first and then that registries dir; on Copilot and Codex the installed skills dir stays first.
|
|
37
|
+
- The Claude Code installer removes the old local copy when it matches a shipped version: `gen-catalog-history.mjs` keys a skill by its directory name whether it sits in `core/` or `external/`. A dangling symlink a `--link` install left there is removed too. Uninstall removes the old copy and such links (`PIPELINE_RETIRED_SKILL_DIRS`).
|
|
38
|
+
- The repo sync leaves `~/.claude/multi-agent-refs/registries/` out: the installer generates it from `pipeline/skills/shared/external`, which the repo already holds.
|
|
39
|
+
- Phase 4 recommends opening the pull request READY rather than DRAFT: every gate has passed before the push, and a draft holds the reviewers back. A saved `defaultPrMode` still wins. The Step 2.95 log line carries the coding score.
|
|
40
|
+
|
|
41
|
+
### Fixed
|
|
42
|
+
|
|
43
|
+
- `pre-commit-check.sh` lets `.env.example`, `.env.sample` and `.env.template` through the env-file block, since a template documents names rather than values; their content is still scanned for secrets, and every other `.env*` file is still blocked.
|
|
44
|
+
- Operational reporting: when no logged-in gh account can read the pipeline repository, `usage-identity.mjs` reports under the active account (else the first), marked `usageLog.loginFallback` and cached for 1 day so an account with access added later takes over. A work machine often has only an enterprise-managed account logged in, which can never be a collaborator of a personal repository; the server decides whether it accepts that login. Only a machine with no gh account at all reports nothing.
|
|
45
|
+
- `usage-register.mjs --quiet` prints one stderr line, `multi-agent: usage reporting is off - <reason>`, when reporting stays off for a reason the user can fix, and the Phase 0 backstop no longer discards stderr. A refused login is told to add the account with access with `gh auth login`.
|
|
46
|
+
- The usage emitter waits 6 s for the endpoint instead of 2.5 s, so a cold start does not drop the record.
|
|
47
|
+
- `smoke-cross-cli-behavior.sh` checks for any `$USER_`-prefixed login in the command docs instead of one hard-coded account suffix.
|
|
48
|
+
|
|
49
|
+
## [20.16.0] - 2026-10-02
|
|
50
|
+
|
|
51
|
+
### Added
|
|
52
|
+
|
|
53
|
+
- `valven-rules` registry (`skills/shared/core/valven-rules`): delivery-flow rules derived from the metrics Valven reports (cycle time, rework rate, review responsiveness, PR complexity, commit risk, lead time, change failure rate, MTTR). `gate_rules` are measured by a script: PR size (250 lines important, 600 blocking), 25 files, renames mixed with content, tests with source, issue key on the branch and on every commit, no WIP or fixup commits, rework ratio (removed lines written in the last 21 days, blamed at the merge-base), PR body length and test section, 150-line hotfixes. `rules` (`VLV-SCOPE-02`, `VLV-PRDESC-02`) are judgement rules the reviewers return a verdict on. Thresholds are overridable per rule in `prefs.global.valven.thresholds`; `prefs.global.valven.enabled: false` turns the gate off and logs it.
|
|
54
|
+
- `valven-gate.mjs --mode dev|commit|review`: runs at Phase 2 exit (Gate 6, `--worktree` includes uncommitted and untracked files), Phase 4 Step 2.95 before push, Phase 4 before the PR is created (PR body), Phase 3 Step 1.762 and every `/multi-agent:review` (Claude Code and Copilot). Works from a checkout or, for a PR without one, from the patch, commit subjects, branch name and body; what it cannot measure is reported `not-measured`.
|
|
55
|
+
- Old-work rules: `VLV-AGE-01` (a commit older than 14 days, by author date), `VLV-BEHIND-01` (more than 50 commits behind the base or a merge-base older than 14 days) and `VLV-TRACE-03` (commits carrying another issue's key). `valven-port.mjs` carries the net change onto `<branch>-vN` cut from the current base tip with `git apply --3way`, one commit, rolling back on conflict. Phase 0 does not reuse a branch with old work; Phase 4 ports before it splits; `/multi-agent:review` puts the port plan in the finding.
|
|
56
|
+
- `valven-split.mjs`: builds on one branch, ships as stacked PRs. Plans slices under the size budget (renames first, tests with their source, data, then domain, then UI), and with `--apply` creates `<branch>-p1..pN` and checks the top of the stack has the branch's exact tree. Phase 4 builds each slice and opens one PR per slice, each based on the previous; `/multi-agent:review` adds the slice plan to the size finding.
|
|
57
|
+
|
|
58
|
+
### Changed
|
|
59
|
+
|
|
60
|
+
- `review-decision-gate.mjs` counts `tag: valven` findings as gate-backed, so a single-source valven blocker is not lowered to important.
|
|
61
|
+
- `verify-citations.mjs` resolves a `line: 0` finding by its file alone when `criteriaSource` is `valven-rules`; every other source keeps the line check.
|
|
62
|
+
- Claude Code installs `valven-rules` with the compliance catalogs as a pipeline-local skill.
|
|
63
|
+
|
|
19
64
|
## [20.15.1] - 2026-10-02
|
|
20
65
|
|
|
21
66
|
### Fixed
|
package/README.md
CHANGED
|
@@ -158,7 +158,7 @@ The discipline behind all of this - bounded loops, evidence gates, token-budgete
|
|
|
158
158
|
|
|
159
159
|
## Commands
|
|
160
160
|
|
|
161
|
-
`/multi-agent` plus
|
|
161
|
+
`/multi-agent` plus 64 sub-commands. `/multi-agent:help` renders the same catalog in your terminal, in your `outputLanguage`.
|
|
162
162
|
|
|
163
163
|
### Pipeline entries
|
|
164
164
|
|
|
@@ -219,6 +219,7 @@ The discipline behind all of this - bounded loops, evidence gates, token-budgete
|
|
|
219
219
|
| `/multi-agent:test-accessibility` | VoiceOver labels, sub-44pt tap targets, contrast, traits |
|
|
220
220
|
| `/multi-agent:test-dynamic-type` | Re-walk every screen at XL through accessibility-XL, report truncation |
|
|
221
221
|
| `/multi-agent:test-screenshots [locale]` | App Store screenshot set in a locale (defaults to `tr`) |
|
|
222
|
+
| `/multi-agent:bug-bash` | Parallel exploratory sessions by charter; only findings a failing repro test proves are reported |
|
|
222
223
|
| `/multi-agent:manual-test` | Phase 3 standalone: check out the task branch and prepare it for Xcode |
|
|
223
224
|
|
|
224
225
|
### Design, build and store
|
|
@@ -394,17 +395,17 @@ This enables the matching plugin (+ the shared `ai-common` plugin) in the repo's
|
|
|
394
395
|
|
|
395
396
|
## Tool support
|
|
396
397
|
|
|
397
|
-
The pipeline runs natively on **Claude Code**, **Copilot CLI** and **Codex CLI** - all three install from the same `pipeline/` source and get the same
|
|
398
|
+
The pipeline runs natively on **Claude Code**, **Copilot CLI** and **Codex CLI** - all three install from the same `pipeline/` source and get the same 64 commands.
|
|
398
399
|
|
|
399
400
|
| Tool | Flag | What it installs |
|
|
400
401
|
| ----------- | -------------------- | ------------------------------------------------------------------------------------------------------ |
|
|
401
402
|
| Claude Code | `--claude` (default) | slash commands + skills + agents + three `PreToolUse` hooks (secret scan, agent-guard, read-size gate) |
|
|
402
|
-
| Copilot CLI | `--copilot` | instructions +
|
|
403
|
-
| Codex CLI | `--codex` | one router skill +
|
|
403
|
+
| Copilot CLI | `--copilot` | instructions + 64 sub-command skills + scripts |
|
|
404
|
+
| Codex CLI | `--codex` | one router skill + 64 specs as refs + 10 agent TOML + `AGENTS.md` block + `codex mcp add` |
|
|
404
405
|
|
|
405
406
|
Filter skills by stack with `--platform=ios\|android\|all`.
|
|
406
407
|
|
|
407
|
-
**Why Codex gets one skill and not
|
|
408
|
+
**Why Codex gets one skill and not 64.** Codex assembles every discovered skill's name
|
|
408
409
|
and description into a single prompt block and drops entries when it overflows, with no
|
|
409
410
|
error. Measured on 0.145: installing one plugin that declares 142 skills surfaced only
|
|
410
411
|
75 of them and evicted an unrelated user skill. So on Codex the pipeline ships a single
|
package/README.tr.md
CHANGED
|
@@ -157,7 +157,7 @@ Koşunun kendisinde ayarlanabilen tek şey autopilot; geri kalan her şey kendi
|
|
|
157
157
|
|
|
158
158
|
## Komutlar
|
|
159
159
|
|
|
160
|
-
`/multi-agent` ve
|
|
160
|
+
`/multi-agent` ve 64 alt komut. `/multi-agent:help` aynı katalogu terminalde, `outputLanguage` ayarına göre gösterir.
|
|
161
161
|
|
|
162
162
|
### Pipeline girişleri
|
|
163
163
|
|
|
@@ -218,6 +218,7 @@ Koşunun kendisinde ayarlanabilen tek şey autopilot; geri kalan her şey kendi
|
|
|
218
218
|
| `/multi-agent:test-accessibility` | VoiceOver etiketleri, 44pt altı dokunma alanları, kontrast, trait'ler |
|
|
219
219
|
| `/multi-agent:test-dynamic-type` | Her ekranı XL'den accessibility-XL'e kadar yeniden gezer, kırpılmaları raporlar |
|
|
220
220
|
| `/multi-agent:test-screenshots [dil]` | Bir dilde App Store screenshot seti (varsayılan `tr`) |
|
|
221
|
+
| `/multi-agent:bug-bash` | Charter başına paralel keşif oturumları; yalnız fail eden bir repro testinin kanıtladığı bulgular raporlanır |
|
|
221
222
|
| `/multi-agent:manual-test` | Faz 5'in bağımsız hali: görev branch'ine geçer, Xcode için hazırlar |
|
|
222
223
|
|
|
223
224
|
### Tasarım, build ve store
|
|
@@ -393,17 +394,17 @@ Bu, ilgili plugin'i (+ ortak `ai-common` plugin'ini) repo'nun `.claude/settings.
|
|
|
393
394
|
|
|
394
395
|
## Araç desteği
|
|
395
396
|
|
|
396
|
-
Pipeline **Claude Code**, **Copilot CLI** ve **Codex CLI** üzerinde native çalışır - üçü de aynı `pipeline/` kaynağından kurulur ve aynı
|
|
397
|
+
Pipeline **Claude Code**, **Copilot CLI** ve **Codex CLI** üzerinde native çalışır - üçü de aynı `pipeline/` kaynağından kurulur ve aynı 64 komutu alır.
|
|
397
398
|
|
|
398
399
|
| Araç | Bayrak | Ne kurar |
|
|
399
400
|
| ----------- | ----------------------- | ---------------------------------------------------------------------------------------------------------------- |
|
|
400
401
|
| Claude Code | `--claude` (varsayılan) | slash komutları + skill'ler + agent'lar + üç `PreToolUse` hook'u (secret scan, agent-guard, okuma-boyutu geçidi) |
|
|
401
|
-
| Copilot CLI | `--copilot` | talimatlar +
|
|
402
|
-
| Codex CLI | `--codex` | bir router skill + ref olarak
|
|
402
|
+
| Copilot CLI | `--copilot` | talimatlar + 64 alt-komut skill'i + script'ler |
|
|
403
|
+
| Codex CLI | `--codex` | bir router skill + ref olarak 64 spec + 10 agent TOML + `AGENTS.md` bloğu + `codex mcp add` |
|
|
403
404
|
|
|
404
405
|
Skill'leri stack'e göre filtrele: `--platform=ios\|android\|all`.
|
|
405
406
|
|
|
406
|
-
**Codex neden
|
|
407
|
+
**Codex neden 64 değil de tek bir skill alıyor.** Codex, keşfettiği her skill'in adını
|
|
407
408
|
ve açıklamasını tek bir prompt bloğuna toplar ve blok taştığında girdileri hatasızca
|
|
408
409
|
düşürür. 0.145 üzerinde ölçüldü: 142 skill deklare eden bir plugin kurulduğunda sadece
|
|
409
410
|
75'i yüzeye çıktı ve alakasız bir kullanıcı skill'i tahliye edildi. Bu yüzden Codex'te
|
package/docs/architecture.md
CHANGED
|
@@ -113,7 +113,7 @@ graph TB
|
|
|
113
113
|
end
|
|
114
114
|
|
|
115
115
|
subgraph "Pipeline Specs"
|
|
116
|
-
CMD[commands/<br/>
|
|
116
|
+
CMD[commands/<br/>64 command files]
|
|
117
117
|
AGT[agents/<br/>10 agent personas]
|
|
118
118
|
RUL[rules/<br/>13 domain rules]
|
|
119
119
|
PHS[multi-agent-refs/phases/<br/>phase specs + contracts]
|
|
@@ -204,8 +204,8 @@ repo and the website, plus the two independently shipped repos
|
|
|
204
204
|
```mermaid
|
|
205
205
|
graph TD
|
|
206
206
|
CC["Claude Code<br/>(source of truth)"]
|
|
207
|
-
COP["Copilot CLI<br/>(instructions +
|
|
208
|
-
COD["Codex CLI<br/>(1 router skill +
|
|
207
|
+
COP["Copilot CLI<br/>(instructions + 64 sub-command skills)"]
|
|
208
|
+
COD["Codex CLI<br/>(1 router skill + 64 refs)"]
|
|
209
209
|
REPO["Pipeline Repo<br/>(npm package)"]
|
|
210
210
|
WEB["Website"]
|
|
211
211
|
PLUGREPO["multi-agent-plugins<br/>(6 plugins, own repo)"]
|
package/docs/ecosystem.md
CHANGED
|
@@ -5,7 +5,7 @@ separately, wired together at install time and at run time:
|
|
|
5
5
|
|
|
6
6
|
| Repo | What it owns | Ships as |
|
|
7
7
|
|---|---|---|
|
|
8
|
-
| **`multi-agent-pipeline`** (this repo) | Orchestration: the 6-phase flow, the
|
|
8
|
+
| **`multi-agent-pipeline`** (this repo) | Orchestration: the 6-phase flow, the 64 slash commands, quality gates, review/triage, cross-CLI parity | npm package (`@mmerterden/multi-agent-pipeline`), installs itself onto Claude Code / Copilot CLI / Codex CLI |
|
|
9
9
|
| **`multi-agent-plugins`** | Stack knowledge: per-platform component/lifecycle skills (iOS, Android, Frontend, Backend) + shared knowledge | Claude Code marketplace, 6 independently-versioned plugins |
|
|
10
10
|
| **`multi-agent-toolkit-mcp`** | The pipeline's hands on devices and browsers: 118 MCP tools across 14 categories (simulator/emulator control, memory, crash diagnostics, accessibility audit, store compliance, web automation, Figma-vs-mock design audit, code intelligence, wallet passes, an agent-DSL batch runner, a full-text context index, provider-backed research, video key frames, offline security scoring) | npm package, registered as a standard stdio MCP server on every host |
|
|
11
11
|
|
|
@@ -18,7 +18,7 @@ Either can be swapped or removed without touching the other two's source.
|
|
|
18
18
|
graph LR
|
|
19
19
|
subgraph PIPE ["multi-agent-pipeline (orchestrator)"]
|
|
20
20
|
direction TB
|
|
21
|
-
PHASES["6 phases ·
|
|
21
|
+
PHASES["6 phases · 64 commands"]
|
|
22
22
|
GATES["deterministic gates + review triage"]
|
|
23
23
|
end
|
|
24
24
|
|
|
@@ -68,8 +68,8 @@ only those:
|
|
|
68
68
|
graph TD
|
|
69
69
|
CC["Claude Code<br/>~/.claude/commands/multi-agent/<br/>(source of truth)"]
|
|
70
70
|
|
|
71
|
-
CC -->|"Step 2: copy + reformat<br/>
|
|
72
|
-
CC -->|"Step 2b: transform<br/>(install.js --codex)"| COD["Codex CLI<br/>1 router skill +
|
|
71
|
+
CC -->|"Step 2: copy + reformat<br/>64 sub-command skills"| COP["Copilot CLI<br/>~/.copilot/skills/"]
|
|
72
|
+
CC -->|"Step 2b: transform<br/>(install.js --codex)"| COD["Codex CLI<br/>1 router skill + 64 refs<br/>+ 10 agent TOML"]
|
|
73
73
|
CC -->|"Step 3: genericize<br/>(strip personal data)"| REPO["multi-agent-pipeline repo<br/>pipeline/"]
|
|
74
74
|
CC -->|"Step 4: version + feature sync"| WEB["Website<br/>projects.ts / i18n.tsx"]
|
|
75
75
|
|
|
@@ -161,7 +161,7 @@ measurements behind this table):
|
|
|
161
161
|
|
|
162
162
|
| | Claude Code | Copilot CLI | Codex CLI |
|
|
163
163
|
|---|---|---|---|
|
|
164
|
-
| **Pipeline commands** |
|
|
164
|
+
| **Pipeline commands** | 64 slash-command skills, native | 64 sub-command skills, `multi-agent-{cmd}` naming, copied in | 1 router skill (`multi-agent`) + 64 command specs as reference files - Codex silently truncates its skills block past a few dozen entries, so sub-commands are not peer skills here |
|
|
165
165
|
| **Stack plugins** | Marketplace plugin, loaded natively, resolved by `.claude/settings.json` enabled-list | Enabled plugin's authored skills copied flat into `~/.copilot/skills/`; `knowledge/` **not** re-copied (already delivered via `shared/external`) | Copied as reference files under `~/.codex/multi-agent-refs/skills/`, plugin-prefixed on name clash (e.g. `architecture` → `ai-ios-toolkit-architecture`) |
|
|
166
166
|
| **Component dispatch (Phase 3)** | Marketplace plugin's `create-component`/`create-screen` skill via the Skill tool | No plugin loader - the enabled stack plugin's authored skills (incl. `create-component`) are copied flat into `~/.copilot/skills/` at install time (the old frozen `figma-*` copies are pruned, they were never a fallback) | Not part of the enforced parity axis; classification + state-shape must match, skill *inventory* does not |
|
|
167
167
|
| **multi-agent-toolkit-mcp** | `claude mcp add multi-agent-toolkit -- npx -y @mmerterden/multi-agent-toolkit-mcp` | `copilot mcp add multi-agent-toolkit -- npx -y @mmerterden/multi-agent-toolkit-mcp` | `codex mcp add multi-agent-toolkit -- npx -y @mmerterden/multi-agent-toolkit-mcp` (skipped with a warning if `codex` isn't on `PATH`) |
|
package/docs/facts.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"$comment": "Generated by pipeline/scripts/gen-facts.mjs. Do not hand-edit: the site reads this, and a number edited here instead of at its source is the drift this file removes.",
|
|
3
|
-
"generatedAt": "2026-10-
|
|
4
|
-
"version": "20.
|
|
3
|
+
"generatedAt": "2026-10-07",
|
|
4
|
+
"version": "20.17.0",
|
|
5
5
|
"phaseSchema": 2,
|
|
6
6
|
"phases": [
|
|
7
7
|
{
|
|
@@ -35,17 +35,17 @@
|
|
|
35
35
|
"autopilot": [0, 1, 2, 3, 4, 5],
|
|
36
36
|
"analysis": [0, 1, 3, 4, 5]
|
|
37
37
|
},
|
|
38
|
-
"commandCount":
|
|
39
|
-
"skillCount":
|
|
40
|
-
"skillCountAll":
|
|
38
|
+
"commandCount": 64,
|
|
39
|
+
"skillCount": 156,
|
|
40
|
+
"skillCountAll": 223,
|
|
41
41
|
"agentCount": 10,
|
|
42
42
|
"hosts": ["Claude Code", "Codex CLI", "Copilot CLI"],
|
|
43
|
-
"toolCount":
|
|
43
|
+
"toolCount": 174,
|
|
44
44
|
"toolCategories": {
|
|
45
|
-
"ios":
|
|
46
|
-
"android":
|
|
45
|
+
"ios": 42,
|
|
46
|
+
"android": 33,
|
|
47
47
|
"store": 30,
|
|
48
|
-
"web":
|
|
48
|
+
"web": 21,
|
|
49
49
|
"vphone": 10,
|
|
50
50
|
"code": 8,
|
|
51
51
|
"design": 7,
|
|
@@ -53,10 +53,12 @@
|
|
|
53
53
|
"pass": 4,
|
|
54
54
|
"context": 3,
|
|
55
55
|
"agent": 2,
|
|
56
|
+
"document": 2,
|
|
56
57
|
"research": 2,
|
|
57
58
|
"security": 2,
|
|
58
|
-
"media": 1
|
|
59
|
+
"media": 1,
|
|
60
|
+
"ui": 1
|
|
59
61
|
},
|
|
60
|
-
"pluginSkillCount":
|
|
61
|
-
"toolkitVersion": "3.
|
|
62
|
+
"pluginSkillCount": 247,
|
|
63
|
+
"toolkitVersion": "3.19.0"
|
|
62
64
|
}
|
package/docs/features.md
CHANGED
|
@@ -161,7 +161,7 @@ When a specialized skill takes over a main pipeline phase, progress is reported
|
|
|
161
161
|
|
|
162
162
|
### Draft vs Ready Prompt
|
|
163
163
|
|
|
164
|
-
Phase 4 asks `
|
|
164
|
+
Phase 4 asks `READY or DRAFT?` before creating the PR, recommends READY because every gate has passed by then, and persists the choice in `prefs.projects[].defaultPrMode`. An unattended run does not ask: the autopilot runner always opens a draft, and a person marks it ready.
|
|
165
165
|
|
|
166
166
|
- Bitbucket: `draft: true` flag (DC 8.x+) with `[DRAFT]` title fallback for older servers.
|
|
167
167
|
- GitHub: `gh pr create --draft` + `gh pr ready` for promotion.
|
|
@@ -382,6 +382,14 @@ Adversarial fixtures that test triage resilience against adversarial reviewer ou
|
|
|
382
382
|
|
|
383
383
|
Detects drift between Claude Code instructions (`~/.claude/commands/multi-agent/SKILL.md`), Copilot CLI instructions (`~/.copilot/copilot-instructions.md`), and the repo's pipeline spec files. Reports discrepancies during Phase 0 Init.
|
|
384
384
|
|
|
385
|
+
### Exploratory Testing and Bug Bash
|
|
386
|
+
|
|
387
|
+
`/multi-agent:test` runs every exploratory session under one contract (`features/exploratory-testing.md`, `explore-findings.mjs`, `explore-run.schema.json`): each finding has expected, observed, repro steps, severity 1-5, kind `issue` or `warning`, and evidence on disk; each run has a step, minute and token budget and stops after three consecutive blocked steps that reported nothing; steps blocked by environment, credentials, seed data or the automation itself are reported apart from product failures. Credentials are typed by `secret_ref`, and locators plus expect steps replace coordinates when the toolkit offers them.
|
|
388
|
+
|
|
389
|
+
A visual or semantic assertion is decided by a fresh judge subagent that receives only the assertion and the current screenshot or UI tree (`features/assertion-judge.md`); `explore-findings.mjs judge-packet` refuses any field describing the run, and an inconclusive or uncited answer fails as `ASSERTION_INCONCLUSIVE`.
|
|
390
|
+
|
|
391
|
+
`/multi-agent:bug-bash` fans out 5-10 charters, each with a persona and its own budget, merges the findings, triages them against the source, and proves every candidate with a reproducing test. `bug-bash.mjs report` confirms a finding only when its repro failed with `ASSERTION_FAILED` on the assertion that encodes it; everything else is listed as rejected with the reason (artifact, environment, design, fixture, did-not-reproduce, unproven, unverified). The `e2e-testing` knowledge skill in ai-common-toolkit carries the toolkit side: locators, expect verdicts, traces, replay, parallel web sessions and synthetic MRZ document fixtures.
|
|
392
|
+
|
|
385
393
|
### Token Budget Enforcement
|
|
386
394
|
|
|
387
395
|
Per-phase token budgets prevent runaway sessions. If a phase exceeds its budget, the pipeline pauses and offers: continue (extend budget), skip phase, or abort. Budgets are configurable in `prefs.global.tokenBudgets`.
|