@mmerterden/multi-agent-pipeline 20.20.0 → 20.21.0

package/CHANGELOG.md

@@ -16,6 +16,31 @@ Internal file-layout changes that don't affect the slash-command surface are sti
 
 ## [Unreleased]
 
+## [20.21.0] - 2026-10-12
+
+### Added
+
+- **Free models for simple work: `/multi-agent:route-on --free`.** Layer 2 of model routing. The new `free` rung sends a large-file read (`bulk-read.sh`) and simple text tasks at the new `delegate` call site (log and document summaries, field extraction, labels, translation and commit-body drafts) to the multi-agent-toolkit's verified free-model chain first. Every answer is checked against its source by the toolkit, and `bulk-read.sh` also checks cited lines against the file; when nothing passes, the Anthropic default answers as before. Decisions, plans, code, reviews and triage never leave Claude. A subagent stays on the Anthropic ladder: the router refuses `free` there. Contract: `multi-agent-refs/features/free-models.md`.
+- `modelRouting.free` (`enabled`, `allowTraining`) and the `delegate` scope. By default only providers whose free tier does not train on prompts (Groq) and a local Ollama receive text; `--allow-training` widens it. `--no-free` turns only the free chain off, and `route-off` keeps it configured.
+- `/multi-agent:route-status` reports the last seven days of free-model work from the toolkit's usage log: tasks answered free and verified, cache hits, tasks handed back to Claude, and the tokens kept off Claude with an estimated saving at haiku rates.
+- `route-on --free` lists usable providers by key source and, with none, names the one step that fixes it. `/multi-agent:setup` Step 3e stores free-model keys under `llm-<id>` through the clipboard flow.
+- `doctor` check `free-models`: WARN when the free chain is on and no provider is usable under the data policy, or when the toolkit CLI cannot run; `--probe` sends each configured provider one tiny request and WARNs when every usable one fails.
+- `lib/toolkit-llm.sh` runs the toolkit's `multi-agent-toolkit-llm` CLI from `MULTI_AGENT_TOOLKIT_LLM`, `PATH` or `npx`.
+- Analysis evidence gathering summarizes long Confluence, document, Jira and `generic-doc` texts through the free chain when it is on; `binding: true` standards are never delegated.
+- The `task-fit` and `cost-ceiling` strategies change dispatch: `task-fit` tries the matched rule's cheapest rung first, `cost-ceiling` does so once `cost-budget-check.mjs` reports the run at the warn threshold of `budgetCeilingUsd`.
+- Unattended runs: agent-guard allows `llm_delegate` only while `route-on --free` is on, and checks its `path` like a write target. The sandbox network list adds the free-model hosts the data policy allows once the free chain is on, and a sandboxed `bulk-read.sh` keeps the toolkit's quota and usage files under `~/.multi-agent-unattended/toolkit-index`.
+- Groq (`gsk_`) and OpenRouter (`sk-or-v1-`) keys are blocked by both secret surfaces, the commit hook and the outbound gate.
+- The cross-CLI contract lists the multi-agent-toolkit 3.21.0 minimum for `llm_delegate`, `llm_providers` and the CLI.
+
+### Fixed
+
+- `route-state.sh set-rules` validates each rule against the schema and the cost table (keys, phase range, taskKind, known rungs) and writes nothing when one fails, instead of storing any JSON array.
+- `recordDecisions: false` is honoured by `model-dispatch.sh` and shown correctly by `route-status`; jq's `//` read `false` as missing, so every decision was logged.
+
+### Changed
+
+- The ecosystem doc counts the toolkit from its 3.21.0 `tools/list`: 191 tools in 20 categories.
+
 ## [20.20.0] - 2026-10-12
 
 ### Added

package/README.md

@@ -17,7 +17,7 @@ Runs natively on Claude Code, Copilot CLI and Codex CLI. macOS only. Zero runtim
 ### Prerequisites
 
 - **Node.js >= 20.11** - required; the pipeline's own tooling runs on it.
-- **`jq`** - required for nine paths, optional for the rest. 91 shell files call it. The nine that publish or decide - the autopilot queue, Jira comments, PR reviews, issue updates, the plan file, both Figma fetchers, log search and Jira auth - refuse with exit 3 rather than run, because a missing `jq` renders as empty DATA and the work carries on with it. Everywhere else it still degrades. The install prints a note when it is missing.
+- **`jq`** - required for nine paths, optional for the rest. 92 shell files call it. The nine that publish or decide - the autopilot queue, Jira comments, PR reviews, issue updates, the plan file, both Figma fetchers, log search and Jira auth - refuse with exit 3 rather than run, because a missing `jq` renders as empty DATA and the work carries on with it. Everywhere else it still degrades. The install prints a note when it is missing.
 - **`gh`** - for GitHub issue and PR work. Its built-in `--jq` is independent of the `jq` binary.
 
 ## Quick Start
@@ -359,23 +359,34 @@ turn it on.
 
 | Command                     | What it does                                                                                                                                        |
 | --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `/multi-agent:route-on`     | Pick a strategy, a scope and the rules that say which rung a call lands on. Validated against `route-config.schema.json` before anything is written |
+| `/multi-agent:route-on`     | Pick a strategy, a scope and the rules that say which rung a call lands on; `--free` sends simple text work to verified free models first. Validated against `route-config.schema.json` before anything is written |
 | `/multi-agent:route-off`    | Disarm it. The rules are KEPT, so turning it back on does not re-ask for the same configuration                                                     |
 | `/multi-agent:route-status` | Whether it is armed, which rule applies where, which rung the last dispatches took, and what this run has cost                                      |
 | `/multi-agent:model`        | Turn the top rung on or off, and move the cost ledger's pricing with it in the same step                                                            |
 
 **The scope cannot be the whole session.** `scope` accepts `subagent`,
-`bulk-read` and `research`; there is no `host-session` value and that is a schema
-gate, not a convention. Owning the host's base URL would send every call you make
-through a third layer, including work that has nothing to do with this pipeline.
-
-**The honest limit, which `route-status` prints rather than hides.** A rung on a
-non-Anthropic provider is refused at dispatch. No call site sends a model
-request itself: the one that routes, `bulk-read.sh`, delegates to the `claude`
-CLI, which speaks that ladder and nothing else. Sending a read to a third-party
-provider would also put the file's full text outside the account that owns it,
-so it is a decision a user makes rather than a default. Routing chooses inside
-the ladder.
+`bulk-read`, `delegate` and `research`; there is no `host-session` value and that
+is a schema gate, not a convention. Owning the host's base URL would send every
+call you make through a third layer, including work that has nothing to do with
+this pipeline.
+
+**Free models for simple work: `route-on --free`.** Large-file reads, log and
+document summaries, field extraction, labels and translation or commit-body
+drafts go to the multi-agent-toolkit's free-model chain first (Groq, Mistral,
+Gemini, OpenRouter free models, Z.AI GLM Flash, a local Ollama). Every answer is
+verified against its source before it is used - no truncation or refusal, schema
+and label checks, every identifier, number and file name present in the source -
+and when none passes, Claude does the task as before. By default only providers
+that do not train on prompts see the text; `--allow-training` widens that. One
+free Groq key is enough; `route-status` reports what was answered free, what went
+back to Claude and the tokens kept off Claude. Contract:
+[`free-models.md`](pipeline/multi-agent-refs/features/free-models.md).
+
+**The honest limit, which `route-status` prints rather than hides.** A subagent
+is dispatched by the host, so Phase 1-3 personas stay inside the Anthropic ladder
+whatever the rules say. The `free` rung answers only where the pipeline makes the
+call itself, `bulk-read.sh` and the `delegate` call site, and decisions, plans,
+code, reviews and triage never leave Claude.
 
 **The top rung is a switch.** `/multi-agent:model` turns `fable` on or off
 (`prefs.global.modelFallback.fableEnabled`, shipped off). The review commands,

package/README.tr.md

@@ -358,23 +358,34 @@ kadar kapalıdır.
 
 | Komut                       | Ne yapar                                                                                                                                                        |
 | --------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `/multi-agent:route-on`     | Strateji, kapsam ve hangi çağrının hangi basamağa düşeceğini söyleyen kuralları sorar. Hiçbir şey yazılmadan önce `route-config.schema.json`'a karşı doğrulanır |
+| `/multi-agent:route-on`     | Strateji, kapsam ve hangi çağrının hangi basamağa düşeceğini söyleyen kuralları sorar; `--free` basit metin işlerini önce doğrulanmış ücretsiz modellere gönderir. Hiçbir şey yazılmadan önce `route-config.schema.json`'a karşı doğrulanır |
 | `/multi-agent:route-off`    | Kapatır. Kurallar SAKLANIR, tekrar açıldığında aynı yapılandırma yeniden sorulmaz                                                                               |
 | `/multi-agent:route-status` | Açık mı, hangi kural nerede geçerli, son çağrılar hangi basamağa gitti, bu koşu ne tuttu                                                                        |
 | `/multi-agent:model`        | Üst basamağı açıp kapatır ve maliyet defterinin fiyatlamasını aynı adımda onunla taşır                                                                          |
 
-**Kapsam oturumun tamamı olamaz.** `scope` yalnız `subagent`, `bulk-read` ve
-`research` kabul eder; `host-session` diye bir değer yoktur ve bu bir teamül
+**Kapsam oturumun tamamı olamaz.** `scope` yalnız `subagent`, `bulk-read`,
+`delegate` ve `research` kabul eder; `host-session` diye bir değer yoktur ve bu bir teamül
 değil, şema kapısıdır. Host'un base URL'ini sahiplenmek, bu pipeline'la ilgisi
 olmayan işler dahil yaptığın her çağrıyı üçüncü bir katmandan geçirirdi.
 
-**Dürüst sınır, ve `route-status` bunu gizlemek yerine yazar.** Anthropic
-dışı bir sağlayıcıdaki basamak dağıtımda reddedilir. Hiçbir çağrı yeri model
-isteğini kendisi atmıyor: yönlendirilen tek yer olan `bulk-read.sh` işi `claude`
-CLI'ına devrediyor, o da yalnız bu merdiveni konuşuyor. Bir okumayı üçüncü bir
-sağlayıcıya göndermek ayrıca dosyanın tam metnini onu sahiplenen hesabın dışına
-çıkarırdı; bu yüzden varsayılan değil, kullanıcının verdiği bir karardır.
-Yönlendirme merdivenin içinde seçim yapar.
+**Basit işler için ücretsiz modeller: `route-on --free`.** Büyük dosya okumaları,
+log ve doküman özetleri, alan çıkarımı, etiketler ve çeviri ya da commit gövdesi
+taslakları önce multi-agent-toolkit'in ücretsiz model zincirine gider (Groq,
+Mistral, Gemini, OpenRouter ücretsiz modelleri, Z.AI GLM Flash, yerel Ollama).
+Her cevap kullanılmadan önce kaynağına karşı doğrulanır - kesik ya da ret değil,
+şema ve etiket kontrolü, her tanımlayıcı, sayı ve dosya adı kaynakta geçmeli -
+ve hiçbiri geçmezse işi Claude eskisi gibi yapar. Varsayılan olarak metni yalnız
+prompt'larla eğitim yapmayan sağlayıcılar görür; `--allow-training` bunu
+genişletir. Tek bir ücretsiz Groq anahtarı yeter; `route-status` neyin ücretsiz
+cevaplandığını, neyin Claude'a döndüğünü ve Claude'dan uzak tutulan token'ları
+raporlar. Sözleşme:
+[`free-models.md`](pipeline/multi-agent-refs/features/free-models.md).
+
+**Dürüst sınır, ve `route-status` bunu gizlemek yerine yazar.** Subagent'ı host
+dağıtır; bu yüzden Phase 1-3 personaları kurallar ne derse desin Anthropic
+merdiveninde kalır. `free` basamağı yalnız pipeline'ın çağrıyı kendisi yaptığı
+yerlerde cevap verir, `bulk-read.sh` ve `delegate` çağrı noktası; karar, plan,
+kod, review ve triyaj asla Claude'dan çıkmaz.
 
 **Üst basamak bir anahtar.** `/multi-agent:model` `fable`'ı açar ya da kapatır
 (`prefs.global.modelFallback.fableEnabled`, kapalı gelir). Review komutları,

package/docs/ecosystem.md

@@ -7,7 +7,7 @@ separately, wired together at install time and at run time:
 |---|---|---|
 | **`multi-agent-pipeline`** (this repo) | Orchestration: the 6-phase flow, the 64 slash commands, quality gates, review/triage, cross-CLI parity | npm package (`@mmerterden/multi-agent-pipeline`), installs itself onto Claude Code / Copilot CLI / Codex CLI |
 | **`multi-agent-plugins`** | Stack knowledge: per-platform component/lifecycle skills (iOS, Android, Frontend, Backend) + shared knowledge | Claude Code marketplace, 6 independently-versioned plugins |
-| **`multi-agent-toolkit-mcp`** | The pipeline's hands on devices and browsers: 189 MCP tools across 19 categories (simulator/emulator control, end-to-end locators and assertions, memory, crash diagnostics, accessibility audit, store compliance, web automation and scraping, Figma-vs-mock design audit, code intelligence, wallet passes, an agent-DSL batch runner, a full-text context index with text-to-image rendering, provider-backed research, video key frames, offline security scoring, travel document fixtures, a voice channel, App Store Connect and Play Console APIs, virtual iPhones) | npm package, registered as a standard stdio MCP server on every host |
+| **`multi-agent-toolkit-mcp`** | The pipeline's hands on devices and browsers: 191 MCP tools across 20 categories (simulator/emulator control, end-to-end locators and assertions, memory, crash diagnostics, accessibility audit, store compliance, web automation and scraping, Figma-vs-mock design audit, code intelligence, wallet passes, an agent-DSL batch runner, a full-text context index with text-to-image rendering, provider-backed research, verified hand-off of simple text tasks to free models, video key frames, offline security scoring, travel document fixtures, a voice channel, App Store Connect and Play Console APIs, virtual iPhones) | npm package, registered as a standard stdio MCP server on every host |
 
 None of the three depends on the others at the code level. They compose through two
 narrow contracts: the **Skill tool** (pipeline → plugin, at Phase 2) and the **MCP
@@ -234,14 +234,14 @@ Detail: [`unattended-security.md`](../pipeline/multi-agent-refs/features/unatten
 
 ### multi-agent-toolkit-mcp's tools, by category
 
-189 tools in 19 categories, counted from the server's own `tools/list` response
-at toolkit 3.20.0 rather than from a README. Three families are defined in
+191 tools in 20 categories, counted from the server's own `tools/list` response
+at toolkit 3.21.0 rather than from a README. Three families are defined in
 modules the main file only spreads in, so a count taken from the source reads
 low; the response is the only place the whole surface appears at once.
 
 The families can be narrowed per host: `MCP_TOOLKIT_CAPS` serves only the named
-ones, and unset serves all 189. A frontend repo that sets `web,code,context`
-sees 47 tools instead of 189, which matters because every registered server's
+ones, and unset serves all 191. A frontend repo that sets `web,code,context`
+sees 47 tools instead of 191, which matters because every registered server's
 tool list is charged against the context window on every turn.
 
 | Category | Tools | Primary pipeline consumers |
@@ -259,6 +259,7 @@ tool list is charged against the context window on every turn.
 | Wallet Passes | 4 (`pass_build`, `pass_validate`, `pass_inspect`, `pass_certificates`) | pass-kit work outside the pipeline; no pipeline phase consumes them |
 | Context Index | 4 (`context_index`, `context_search`, `context_get`, `context_render`) | any skill holding an offloaded payload too large to read; ranks passages instead of grepping lines. `context_render` pages long text into images and reports whether that is cheaper than the text |
 | Research | 2 (`research_search`, `research_ask`) | closes the parity exemption that had web search outside the MCP channel; `signal-community` and `/multi-agent:refactor` band E (modes, recency and academic filters, follow-up threads, arXiv). Denied in unattended runs and autonomous research |
+| Free Models | 2 (`llm_delegate`, `llm_providers`) | `route-on --free`: `bulk-read.sh` through the `multi-agent-toolkit-llm` CLI, the `delegate` call site in skills and analysis evidence gathering ([`free-models.md`](../pipeline/multi-agent-refs/features/free-models.md)). Allowed in unattended runs only while the free chain is on |
 | Media | 1 (`media_frames`) | Phase 2 visual evidence: turns a screen recording into frames a reviewer and a model can both look at |
 | Security | 2 (`security_cvss_score`, `security_dep_inventory`) | the Phase 3 Step 2.7 security audit (`security-auditor`) and `/multi-agent:security-review`; offline, no egress |
 | Travel Document Fixtures | 2 (`document_mrz_build`, `document_generate`) | the `e2e-testing` skill's MRZ and document-scan recipes |

package/docs/facts.json

@@ -1,7 +1,7 @@
 {
   "$comment": "Generated by pipeline/scripts/gen-facts.mjs. Do not hand-edit: the site reads this, and a number edited here instead of at its source is the drift this file removes.",
-  "generatedAt": "2026-10-11",
-  "version": "20.20.0",
+  "generatedAt": "2026-10-12",
+  "version": "20.21.0",
   "phaseSchema": 2,
   "phases": [
     {
@@ -40,7 +40,7 @@
   "skillCountAll": 228,
   "agentCount": 10,
   "hosts": ["Claude Code", "Codex CLI", "Copilot CLI"],
-  "toolCount": 189,
+  "toolCount": 191,
   "toolCategories": {
     "ios": 42,
     "web": 35,
@@ -54,11 +54,12 @@
     "pass": 4,
     "agent": 2,
     "document": 2,
+    "llm": 2,
     "research": 2,
     "security": 2,
     "media": 1,
     "ui": 1
   },
   "pluginSkillCount": 252,
-  "toolkitVersion": "3.20.0"
+  "toolkitVersion": "3.21.1"
 }

package/docs/features.md

@@ -354,6 +354,7 @@ Commands (`commands.mjs --json`) and run questions (`launch-request.mjs question
 
 - **Pre-Commit Secret Detection** (12 patterns): `PreToolUse` hook scans staged files for API keys/tokens, AWS access keys, private keys, `.env` files, service account JSON. Commit **blocked** if found.
 - **Read-Size Gate** (opt-in, `prefs.global.bulkRead.mode`): a `PreToolUse` hook inspects `Read` and the shell commands that read a file whole. In `observe` it only logs what it would have caught - the baseline you measure before routing anything. In `enforce` a file over `minLines` (default 350) is blocked and delegated to a haiku-rung worker (`bulk-read.sh`), which returns a line-numbered summary so the follow-up is a bounded `Read(offset:limit:)` instead of the whole file; the full text is parked under `.multi-agent/refs/`. The development phase and any file the run has already touched are exempt, because Claude Code's `Edit` requires its own `Read` first.
+- **Free models for simple work** (opt-in, `/multi-agent:route-on --free`): large-file reads, log and document summaries, field extraction, labels and translation or commit-body drafts go to the multi-agent-toolkit's free-model chain (`llm_delegate`) first. Every answer is verified against its source and discarded when it does not hold, Claude takes over whenever none passes, and decisions, plans, code and reviews never leave Claude. By default only providers that do not train on prompts receive text. `route-status` reports what was answered free and the tokens kept off Claude; `doctor --probe` checks each provider key. Contract: `multi-agent-refs/features/free-models.md`.
 - **Capture Hooks** (`SessionEnd`, `PreCompact`, `SessionStart`): every durable write used to live in Phase 5, the phase a run is least likely to reach. `SessionEnd` flushes a run that never got there; `PreCompact` flushes before an auto-compaction summarizes a long phase mid-flight, which is the same loss one level down; `SessionStart` prints at most two lines about an unfinished run. None calls a model, none reads a payload, and all exit 0 on every path - a hook that fails a session over bookkeeping is worse than the bookkeeping.
 - **Operational Reporting** (`prefs.global.usageLog`): coarse run metadata - task id and type, current phase, run status (in progress, completed, halted, parked), phase durations, whether a PR opened, token counts - sent at every phase transition and at completion, halt and park, and never prompts, code, diffs or absolute paths (task title and PR URL only when `includeTitle` / `includePrUrl` are set). Runs are reported under the GitHub account on the machine that can read the pipeline repository, found with no question asked through gh, the git credential helper or an SSH key (`usage-identity.mjs`, cached in `usageLog.login` / `loginMethod`, `--refresh` to re-check); with no such account nothing is sent. The per-machine token is REQUESTED from the endpoint by `usage-register.mjs` (setup, update, and the Phase 0 exit gate as a backstop), is write-only, and lives in the OS credential store; prefs hold only the entry name and the switch. `usageLog.optOut: true` blocks registration permanently and is checked before the network call. Registration proves the login with that account's gh or credential-helper token, or with an SSH signature over a server challenge. The server issues a token only to the repository owner or a collaborator, pins the login to it, and shows only those accounts' records on the panel. An unreachable endpoint leaves reporting off with one line and exit 0 - a run is never failed over bookkeeping.
 - **Build Queue**: All `xcodebuild` calls acquire a lock. Each worktree uses own `-derivedDataPath`. Stale locks auto-clean after 15 min. Non-Xcode builds don't need the lock.