@mmerterden/multi-agent-pipeline 20.19.0 → 20.21.0

package/pipeline/lib/model-dispatch.sh

@@ -9,11 +9,11 @@
 #   model-dispatch.sh <call-site> [--persona P] [--phase N] [--task-kind K]
 #                                 [--default RUNG] [--task-id ID]
 #
-# <call-site> is one of the `scope` members: subagent | bulk-read | research.
-# stdout is a single rung NAME. Rung names are the contract; the model id behind
-# one lives in cost-table.json and moves without a config edit.
+# <call-site> is one of the `scope` members: subagent | bulk-read | delegate |
+# research. stdout is a single rung NAME. Rung names are the contract; the model
+# id behind one lives in cost-table.json and moves without a config edit.
 #
-# Exit is always 0 and stdout is always a usable rung. A router that can fail
+# Exit is always 0 and stdout is always a usable answer. A router that can fail
 # turns every call site into a place the run can die, for a feature that ships
 # disabled; when anything is missing, unparseable or out of scope, the caller's
 # default comes back unchanged.
@@ -22,16 +22,18 @@
 #
 #   Layer 1  an Anthropic rung. Reachable from every call site, because it
 #            writes to a seam that already exists and opens no new network path.
-#   Layer 2  a non-Anthropic rung. NO call site can dispatch to one today, and
-#            the router refuses it rather than returning a rung its caller will
-#            hand to a CLI that does not know the name:
-#              - a subagent is dispatched by the HOST;
-#              - bulk-read shells out to `claude -p --model <rung>`;
-#              - research_ask runs in the MCP server and picks its own model.
-#            `provider` in cost-table.json is what makes this checkable, and it
-#            is the seam a future call site would use. Until one exists, the
-#            refusal is the honest answer: an allowed rung nothing can honour is
-#            a rule the user believes worked.
+#   Layer 2  the `free` rung: the multi-agent-toolkit's verified free-model
+#            chain. Reachable only where the pipeline makes the call itself -
+#            bulk-read and delegate - and only with modelRouting.free.enabled.
+#            Its caller must keep its default as the fallback, because the chain
+#            answers ESCALATE_TO_CLAUDE whenever no free answer survives
+#            verification. A subagent is dispatched by the HOST, so `free` and
+#            every other non-Anthropic rung is refused there and the rule's next
+#            rung answers instead.
+#
+# Strategies: manual keeps the rule's order; task-fit tries the cheapest rung
+# first; cost-ceiling keeps the order until cost-budget-check.mjs reports the run
+# at its warn threshold of budgetCeilingUsd, then tries the cheapest first.
 
 set -uo pipefail
 
@@ -74,7 +76,7 @@ fi
 [ "$DEFAULT_RUNG" = "fable" ] && [ "$FABLE_ON" != "true" ] && DEFAULT_RUNG="opus"
 
 case "$CALL_SITE" in
-  subagent|bulk-read|research) ;;
+  subagent|bulk-read|delegate|research) ;;
   *) emit "$DEFAULT_RUNG" ;;
 esac
 
@@ -103,11 +105,45 @@ MATCH=$(jq -r \
          and ((.when.taskKind // null) as $k | $k == null or $k == $kind)
        )
    ] | first | (.prefer // []) | join(" ")' "$PREFS" 2>/dev/null) || emit "$DEFAULT_RUNG"
-[ -n "$MATCH" ] && [ "$MATCH" != "null" ] || emit "$DEFAULT_RUNG"
+[ "$MATCH" = "null" ] && MATCH=""
+
+FREE_ON=$(jq -r '.global.modelRouting.free.enabled // false' "$PREFS" 2>/dev/null)
+CANDIDATES="$MATCH"
+case "$CALL_SITE" in
+  bulk-read|delegate) [ "$FREE_ON" = "true" ] && CANDIDATES="free $MATCH" ;;
+esac
+[ -n "${CANDIDATES// /}" ] || emit "$DEFAULT_RUNG"
 
 HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 COST_TABLE="$HERE/../scripts/cost-table.json"
 
+STRATEGY=$(jq -r '.global.modelRouting.strategy // "manual"' "$PREFS" 2>/dev/null)
+CHEAPEST_FIRST=false
+STRATEGY_NOTE=""
+case "$STRATEGY" in
+  task-fit) CHEAPEST_FIRST=true; STRATEGY_NOTE="task-fit" ;;
+  cost-ceiling)
+    CEILING=$(jq -r '.global.modelRouting.budgetCeilingUsd // empty' "$PREFS" 2>/dev/null)
+    if [ -n "$CEILING" ] && [ "$TASK_ID" != "unknown" ] && command -v node >/dev/null 2>&1 \
+       && [ -f "$HERE/../scripts/cost-budget-check.mjs" ]; then
+      node "$HERE/../scripts/cost-budget-check.mjs" --task-id "$TASK_ID" --max-usd "$CEILING" --enabled >/dev/null 2>&1
+      case $? in
+        10|11) CHEAPEST_FIRST=true; STRATEGY_NOTE="cost-ceiling reached" ;;
+      esac
+    fi
+    ;;
+esac
+
+if [ "$CHEAPEST_FIRST" = "true" ] && [ -f "$COST_TABLE" ]; then
+  CANDIDATES=$(
+    jq -r --arg c "$CANDIDATES" '
+      .prices as $p
+    | ($c | split(" ") | map(select(length > 0)))
+    | to_entries
+    | sort_by([($p[.value].inPerMtok // 1e9), .key])
+    | map(.value) | join(" ")' "$COST_TABLE" 2>/dev/null || printf '%s' "$CANDIDATES")
+fi
+
 # The fable rung answers only when the fable switch is on (FABLE_ON, read
 # above). Routing is a policy over what is available; it is not a second way to
 # turn a rung on, or `/multi-agent:model off` would stop meaning anything the
@@ -116,13 +152,13 @@ COST_TABLE="$HERE/../scripts/cost-table.json"
 record() {
   [ -x "$HERE/../scripts/log-metric.sh" ] || return 0
   local rec
-  rec=$(jq -r '.global.modelRouting.recordDecisions // true' "$PREFS" 2>/dev/null)
+  rec=$(jq -r 'if .global.modelRouting.recordDecisions == false then "false" else "true" end' "$PREFS" 2>/dev/null)
   [ "$rec" = "false" ] && return 0
   "$HERE/../scripts/log-metric.sh" "$TASK_ID" "${PHASE:-0}" model_routing.decision \
     call_site="$CALL_SITE" rung="$1" reason="$2" >/dev/null 2>&1 || true
 }
 
-for rung in $MATCH; do
+for rung in $CANDIDATES; do
   if [ ! -f "$COST_TABLE" ]; then
     record "$rung" "no cost table; rung taken on trust"
     emit "$rung"
@@ -139,15 +175,27 @@ for rung in $MATCH; do
     echo "model-dispatch: rule prefers 'fable' but modelFallback.fableEnabled is false; skipping it" >&2
     continue
   fi
+  if [ "$PROVIDER" = "free-chain" ]; then
+    if [ "$FREE_ON" != "true" ]; then
+      echo "model-dispatch: rule prefers 'free' but modelRouting.free.enabled is false (route-on --free); skipping it" >&2
+      continue
+    fi
+    case "$CALL_SITE" in
+      bulk-read|delegate)
+        record "$rung" "free chain${STRATEGY_NOTE:+, $STRATEGY_NOTE}"
+        emit "$rung" ;;
+    esac
+    echo "model-dispatch: rung 'free' is reachable only at bulk-read and delegate; a $CALL_SITE is dispatched by the host, skipping it" >&2
+    continue
+  fi
   if [ "$PROVIDER" != "anthropic" ]; then
-    # The honest limit, said out loud at the moment it bites. Every call site
-    # that asks here hands the rung to something that only speaks the Anthropic
-    # ladder, so returning an external rung would produce a failed call rather
-    # than a cheaper one.
-    echo "model-dispatch: rung '$rung' is $PROVIDER and no call site can dispatch to one yet; skipping it" >&2
+    # Every call site that asks here hands any other rung to something that
+    # speaks the Anthropic ladder or the free chain, so returning an external
+    # rung would produce a failed call rather than a cheaper one.
+    echo "model-dispatch: rung '$rung' is $PROVIDER and no call site dispatches to it; skipping it" >&2
     continue
   fi
-  record "$rung" "rule matched"
+  record "$rung" "rule matched${STRATEGY_NOTE:+, $STRATEGY_NOTE}"
   emit "$rung"
 done
 

package/pipeline/lib/outbound-gate.mjs

@@ -82,6 +82,8 @@ export const RULES = [
   { name: "stripe-key", certain: true, re: /\b[sr]k_live_[A-Za-z0-9]{20,}\b/g },
   { name: "perplexity-key", certain: true, re: /\bpplx-[A-Za-z0-9]{32,}\b/g },
   { name: "huggingface-token", certain: true, re: /\bhf_[A-Za-z0-9]{30,}\b/g },
+  { name: "groq-key", certain: true, re: /\bgsk_[A-Za-z0-9]{40,}\b/g },
+  { name: "openrouter-key", certain: true, re: /\bsk-or-v1-[A-Za-z0-9]{40,}\b/g },
   {
     name: "service-account-json",
     certain: true,

package/pipeline/lib/route-state.sh

@@ -10,10 +10,11 @@
 # for a configuration the user already gave.
 #
 # Usage:
-#   route-state.sh status                       print the current policy
-#   route-state.sh on  [--strategy=S] [--scope=a,b]
+#   route-state.sh status                       print the current policy and savings
+#   route-state.sh on  [--strategy=S] [--scope=a,b] [--free [--allow-training]] [--no-free]
 #   route-state.sh off
-#   route-state.sh set-rules <file.json>        replace rules[] from a JSON array
+#   route-state.sh set-rules <file.json>        replace rules[] from a JSON array,
+#                                               checked against the schema and cost table
 #
 # Exit codes:
 #   0  -  done
@@ -34,7 +35,7 @@ if [ -f "$_MA_RS_HERE/user-facing.sh" ]; then . "$_MA_RS_HERE/user-facing.sh"; e
 # included - through a third layer, breaks the subscription's auth model, and
 # silently changes which model answers. The schema encodes the same list so a
 # hand-edited preferences file is rejected too, rather than only this entry point.
-ALLOWED_SCOPE="subagent bulk-read research"
+ALLOWED_SCOPE="subagent bulk-read delegate research"
 
 die() { echo "route-state: $1" >&2; exit "${2:-3}"; }
 
@@ -47,6 +48,9 @@ jq -e . "$PREFS" >/dev/null 2>&1 || die "$PREFS is not valid JSON  -  refusing t
 
 ACTION="${1:-status}"; shift 2>/dev/null || true
 
+COST_TABLE="$_MA_RS_HERE/../scripts/cost-table.json"
+USAGE_LOG="${MCP_TOOLKIT_INDEX_DIR:-$HOME/.claude/logs/multi-agent-toolkit/.index}/llm-usage.jsonl"
+
 write_prefs() {
   local filter="$1"; shift
   local tmp
@@ -56,6 +60,57 @@ write_prefs() {
   mv "$tmp" "$PREFS"
 }
 
+# Last seven days of the toolkit's usage logs (attended, and an unattended run's
+# own under ~/.multi-agent-unattended): verified answers, cache hits and
+# escalations, and what those tokens would have cost on haiku, the rung bulk-read
+# and simple tasks fall back to. An estimate, and labelled as one.
+free_savings() {
+  local logs=() f
+  for f in "$USAGE_LOG" "$HOME/.multi-agent-unattended/toolkit-index/llm-usage.jsonl"; do
+    [ -f "$f" ] && logs+=("$f")
+  done
+  if [ "${#logs[@]}" -eq 0 ]; then
+    echo "savings: no free-model task recorded yet ($USAGE_LOG)"
+    return 0
+  fi
+  local since
+  since=$(( $(date +%s) - 7 * 86400 ))
+  jq -rs --argjson since "$since" --slurpfile ct "$COST_TABLE" '
+      map(select((.ts | sub("\\.[0-9]+Z$"; "Z") | fromdateiso8601? // 0) >= $since))
+    | ($ct[0].prices.haiku // {inPerMtok: 1, outPerMtok: 5}) as $h
+    | {
+        n: length,
+        ok: (map(select(.ok and (.cached | not))) | length),
+        cached: (map(select(.ok and .cached)) | length),
+        esc: (map(select(.ok | not)) | length),
+        pin: (map(.prompt_tokens // 0) | add // 0),
+        pout: (map(.completion_tokens // 0) | add // 0),
+        by: (map(select(.ok and .provider)) | group_by(.provider) | map("\(.[0].provider) \(length)") | join(", "))
+      }
+    | . + {usd: ((.pin * $h.inPerMtok + .pout * $h.outPerMtok) / 1000000)}
+    | "savings (7 days): \(.n) task(s), \(.ok) answered free and verified, \(.cached) from cache, \(.esc) handed back to Claude\n"
+      + "  tokens kept off Claude: \(.pin) in + \(.pout) out  (~$\(.usd * 100 | round / 100) at haiku rates, an estimate)"
+      + (if .by != "" then "\n  by provider: \(.by)" else "" end)
+  ' "${logs[@]}" 2>/dev/null || echo "savings: the usage log could not be read"
+}
+
+# What the toolkit can reach, without a single network call: configured
+# providers and where their keys come from. Names only, never values.
+free_providers() {
+  local tl="$_MA_RS_HERE/toolkit-llm.sh"
+  [ -f "$tl" ] || tl="$HOME/.claude/lib/toolkit-llm.sh"
+  [ -f "$tl" ] || { echo "free chain: toolkit-llm.sh is not installed"; return 1; }
+  # shellcheck source=/dev/null
+  . "$tl"
+  local out
+  out=$(ma_toolkit_llm providers ${1:+--allow-training} 2>/dev/null) || { echo "free chain: the multi-agent-toolkit CLI could not be run (node and npx are needed)"; return 1; }
+  jq -r '
+      "free chain: \(.configured) provider(s) usable under the \(.data_policy) data policy",
+      (.providers[] | "  \(if .configured and .allowed_by_policy then "ready  " elif .configured then "policy " else "missing" end) \(.id)\(if .key_source then " (" + .key_source + ")" else "" end)\(if (.configured | not) then "  -  " + .setup else "" end)")
+    ' <<< "$out"
+  [ "$(jq -r '.configured' <<< "$out")" -gt 0 ]
+}
+
 case "$ACTION" in
   status)
     ENABLED=$(jq -r '.global.modelRouting.enabled // false' "$PREFS")
@@ -63,12 +118,16 @@ case "$ACTION" in
     SCOPE=$(jq -r '(.global.modelRouting.scope // ["subagent"]) | join(", ")' "$PREFS")
     NRULES=$(jq -r '(.global.modelRouting.rules // []) | length' "$PREFS")
     CEIL=$(jq -r '.global.modelRouting.budgetCeilingUsd // "none"' "$PREFS")
-    REC=$(jq -r '.global.modelRouting.recordDecisions // true' "$PREFS")
+    REC=$(jq -r 'if .global.modelRouting.recordDecisions == false then "false" else "true" end' "$PREFS")
+
+    FREE=$(jq -r '.global.modelRouting.free.enabled // false' "$PREFS")
+    TRAIN=$(jq -r '.global.modelRouting.free.allowTraining // false' "$PREFS")
 
     echo "routing:   $ENABLED"
     echo "strategy:  $STRATEGY"
     echo "scope:     $SCOPE"
     echo "rules:     $NRULES"
+    echo "free:      $FREE$([ "$FREE" = "true" ] && echo " (data policy: $([ "$TRAIN" = "true" ] && echo any || echo no-training))")"
     echo "ceiling:   $CEIL"
     echo "decisions: $REC"
     echo ""
@@ -79,34 +138,46 @@ case "$ACTION" in
       exit 0
     fi
 
-    if [ "$NRULES" -eq 0 ]; then
+    if [ "$NRULES" -eq 0 ] && [ "$FREE" != "true" ]; then
       echo "Armed with no rules. Nothing matches, so nothing is routed  -  this is a"
       echo "configuration state, not an error."
-    else
+    elif [ "$NRULES" -gt 0 ]; then
       jq -r '(.global.modelRouting.rules // [])[] |
         "  when " + ([.when | to_entries[] | "\(.key)=\(.value)"] | join(" ")) +
         "  ->  " + (.prefer | join(" > "))' "$PREFS"
     fi
 
+    if [ "$FREE" = "true" ]; then
+      echo "  free chain first at: bulk-read, delegate (Anthropic default when it escalates)"
+      echo ""
+      free_savings
+    fi
+
     echo ""
     # The limit is printed every time rather than documented once, because the
     # question it answers ("routing is on, why is the reviewer still on Opus")
     # otherwise arrives days later as a bug report.
     echo "Honest limit: on Claude Code a subagent cannot be sent to a non-Anthropic"
     echo "model  -  subagent dispatch belongs to the host. Phase 1/2/3 personas stay"
-    echo "inside the Anthropic ladder whatever the rules say. External providers apply"
-    echo "only where this pipeline makes the call itself (bulk-read, research)."
+    echo "inside the Anthropic ladder whatever the rules say. Free models answer only"
+    echo "where this pipeline makes the call itself (bulk-read, delegate), only after"
+    echo "the toolkit verified the answer against its source, and Claude takes over"
+    echo "whenever none passes."
     ;;
 
   on)
-    STRATEGY="manual"; SCOPE_ARG=""
+    STRATEGY="manual"; SCOPE_ARG=""; FREE_ARG=""; TRAIN_ARG=false
     for arg in "$@"; do
       case "$arg" in
         --strategy=*) STRATEGY="${arg#*=}" ;;
         --scope=*)    SCOPE_ARG="${arg#*=}" ;;
+        --free)       FREE_ARG=on ;;
+        --no-free)    FREE_ARG=off ;;
+        --allow-training) TRAIN_ARG=true ;;
         *) die "unknown option: $arg" 2 ;;
       esac
     done
+    [ "$TRAIN_ARG" = "true" ] && [ "$FREE_ARG" != "on" ] && die "--allow-training only goes with --free" 2
     case "$STRATEGY" in
       manual|task-fit|cost-ceiling) ;;
       *) die "strategy must be manual, task-fit or cost-ceiling (got '$STRATEGY')" 2 ;;
@@ -138,6 +209,26 @@ case "$ACTION" in
         | .global.modelRouting.scope //= ["subagent"]
       ' --arg st "$STRATEGY"
     fi
+    if [ "$FREE_ARG" = "on" ]; then
+      write_prefs '
+          .global.modelRouting.free = {enabled: true, allowTraining: $tr}
+        | .global.modelRouting.scope = (((.global.modelRouting.scope // ["subagent"]) + ["bulk-read", "delegate"]) | unique)
+      ' --argjson tr "$TRAIN_ARG"
+      echo "free chain enabled at bulk-read and delegate (data policy: $([ "$TRAIN_ARG" = "true" ] && echo any || echo no-training))"
+      TRAIN_FLAG=""
+      [ "$TRAIN_ARG" = "true" ] && TRAIN_FLAG=1
+      if ! free_providers "$TRAIN_FLAG"; then
+        echo ""
+        echo "No free provider is usable yet, so every task still runs on Claude. Add one key"
+        echo "(free at https://console.groq.com/keys) with $(ma_command setup), or from a shell:"
+        echo "  bash \"\$HOME/.claude/lib/credential-store.sh\" set llm-groq -    (paste the key, then Ctrl-D)"
+        echo "or export GROQ_API_KEY for the server, then run $(ma_command route-status)."
+      fi
+      echo ""
+    elif [ "$FREE_ARG" = "off" ]; then
+      write_prefs '.global.modelRouting.free.enabled = false'
+      echo "free chain disabled; Anthropic rungs only"
+    fi
     echo "routing enabled (strategy: $STRATEGY)"
     exec "$0" status
     ;;
@@ -154,6 +245,23 @@ case "$ACTION" in
     [ -n "$FILE" ] || die "set-rules needs a JSON file containing an array of rules" 2
     [ -f "$FILE" ] || die "no such file: $FILE" 2
     jq -e 'type == "array"' "$FILE" >/dev/null 2>&1 || die "$FILE must contain a JSON ARRAY of rules" 2
+    # The schema's rule shape, enforced here so a hand-written file cannot store
+    # a rule that dispatch silently skips: when has 1-3 known keys with valid
+    # values, prefer is a non-empty list of rungs the cost table prices.
+    RUNGS=$(jq -c '.prices | keys' "$COST_TABLE" 2>/dev/null || echo '[]')
+    BAD=$(jq -r --argjson rungs "$RUNGS" '
+        to_entries[] | .key as $i | .value as $r
+      | if ($r | type) != "object" then "rule \($i): not an object"
+        elif (($r | keys) - ["when", "prefer"]) != [] then "rule \($i): unknown key(s) \((($r | keys) - ["when", "prefer"]) | join(", "))"
+        elif ($r.when | type) != "object" or ($r.when | length) == 0 then "rule \($i): when needs persona, phase or taskKind"
+        elif (($r.when | keys) - ["persona", "phase", "taskKind"]) != [] then "rule \($i): when has unknown key(s) \((($r.when | keys) - ["persona", "phase", "taskKind"]) | join(", "))"
+        elif ($r.when.persona != null and ($r.when.persona | type) != "string") then "rule \($i): persona must be a string"
+        elif ($r.when.phase != null and (($r.when.phase | type) != "number" or $r.when.phase < 0 or $r.when.phase > 5 or ($r.when.phase | floor) != $r.when.phase)) then "rule \($i): phase must be an integer 0-5"
+        elif ($r.when.taskKind != null and ([$r.when.taskKind] | inside(["bugfix", "feature", "refactor", "chore", "component"]) | not)) then "rule \($i): taskKind must be bugfix, feature, refactor, chore or component"
+        elif ($r.prefer | type) != "array" or ($r.prefer | length) == 0 then "rule \($i): prefer needs at least one rung"
+        elif ([$r.prefer[] | select((type != "string") or (. as $x | $rungs | index($x) | not))] | length) > 0 then "rule \($i): unknown rung(s) \([$r.prefer[] | select((type != "string") or (. as $x | $rungs | index($x) | not))] | map(tostring) | join(", ")) (known: \($rungs | join(", ")))"
+        else empty end' "$FILE" 2>/dev/null)
+    [ -z "$BAD" ] || die "$(printf 'rules rejected, nothing written:\n%s' "$BAD")" 2
     write_prefs '.global.modelRouting //= {} | .global.modelRouting.rules = $r' --slurpfile _ignore /dev/null --argjson r "$(cat "$FILE")"
     echo "rules replaced: $(jq -r 'length' "$FILE")"
     ;;

package/pipeline/lib/toolkit-llm.sh

@@ -0,0 +1,30 @@
+#!/usr/bin/env bash
+# toolkit-llm.sh  -  run the multi-agent-toolkit's free-model CLI.
+#
+# Sourced by bulk-read.sh and route-state.sh. `ma_toolkit_llm <args>` runs
+# `multi-agent-toolkit-llm <args>` from the first place it exists:
+#
+#   1. MULTI_AGENT_TOOLKIT_LLM            a path to the CLI script
+#   2. multi-agent-toolkit-llm on PATH    a global install
+#   3. npx -p @mmerterden/multi-agent-toolkit-mcp@latest, the same package
+#      the MCP registration runs
+#
+# A maintainer's working copy (prefs.global.devToolkit) is deliberately not a
+# source: it can sit on an unreleased branch, and a run should use what ships.
+#
+# The CLI prints one JSON object and exits 0 on a verified answer, 3 on
+# ESCALATE_TO_CLAUDE, 2 on a usage error. Any other status (no node, no network
+# for npx) is the caller's cue to keep its own default.
+
+ma_toolkit_llm() {
+  if [ -n "${MULTI_AGENT_TOOLKIT_LLM:-}" ] && [ -f "$MULTI_AGENT_TOOLKIT_LLM" ]; then
+    node "$MULTI_AGENT_TOOLKIT_LLM" "$@"
+    return $?
+  fi
+  if command -v multi-agent-toolkit-llm >/dev/null 2>&1; then
+    multi-agent-toolkit-llm "$@"
+    return $?
+  fi
+  command -v npx >/dev/null 2>&1 || return 127
+  npx -y --@mmerterden:registry=https://registry.npmjs.org -p @mmerterden/multi-agent-toolkit-mcp@latest multi-agent-toolkit-llm "$@"
+}

package/pipeline/lib/unattended-profile-check.mjs

@@ -22,9 +22,22 @@ import { join } from "node:path";
 
 const PREFS_HOST_KEYS = ["jira", "confluence", "bitbucket"];
 
+// The free-model hosts a sandboxed bulk-read may reach once route-on --free is
+// on: Groq alone under the default no-training policy, all five with
+// allowTraining. A local Ollama needs no host; allowLocalBinding covers it.
+const FREE_HOSTS_NO_TRAINING = ["api.groq.com"];
+const FREE_HOSTS_ANY = [
+  "api.groq.com",
+  "api.mistral.ai",
+  "generativelanguage.googleapis.com",
+  "openrouter.ai",
+  "api.z.ai",
+];
+
 /**
  * Hosts from prefs.global.hosts, so the sandbox's network list covers this
- * machine's own Jira / Confluence / Bitbucket.
+ * machine's own Jira / Confluence / Bitbucket, plus the free-model providers
+ * modelRouting.free allows.
  *
  * @param {string} home
  * @returns {string[]}
@@ -35,7 +48,7 @@ export function prefsHosts(home) {
       readFileSync(join(home, ".claude", "multi-agent-preferences.json"), "utf-8"),
     );
     const hosts = prefs?.global?.hosts || {};
-    return PREFS_HOST_KEYS.map((k) => hosts[k])
+    const own = PREFS_HOST_KEYS.map((k) => hosts[k])
       .filter((v) => typeof v === "string" && v.trim())
       .map(
         (v) =>
@@ -44,6 +57,14 @@ export function prefsHosts(home) {
             .replace(/^[a-z]+:\/\//i, "")
             .split("/")[0],
       );
+    const routing = prefs?.global?.modelRouting;
+    const free =
+      routing?.enabled === true && routing?.free?.enabled === true
+        ? routing.free.allowTraining === true
+          ? FREE_HOSTS_ANY
+          : FREE_HOSTS_NO_TRAINING
+        : [];
+    return [...own, ...free];
   } catch {
     return [];
   }

package/pipeline/multi-agent-refs/analysis/evidence.md

@@ -27,6 +27,8 @@ For each entry in `state.analysisSpec.contextLinks[]`, fan out by `type`. Entrie
 2. Surface a one-line warning to the user during Phase 1 progress output: `WARN: Confluence requires browser auth for <url>. Falling back to repo evidence; add the page text manually to /tmp/analysis-paste.md and re-run if needed.`
 3. Continue Phase 1 with remaining evidence sources rather than aborting.
 
+**Long sources:** with the free chain on, summarize long confluence/document/jira/generic-doc texts via `llm_delegate` first (`features/free-models.md`).
+
 For each repo in `state.analysisSpec.repos[]`, run parallel greps (lightweight surface scan; the deeper 13-bucket extraction happens in Phase 1b):
 
 ```bash
@@ -204,7 +206,7 @@ Phase 2 Section 20 Risks reads this list and emits one open question per entry.
 
 **Caching (Locked 26)**: compute `evidence_digest = sha256(featureName || sorted(platforms) || options.redesign || hash(evidence.repoEvidence) || hash(evidence.conventions))`. `options.redesign` is a digest input: without it a redesign within a day of a normal run on the same feature reuses that cache, skips Phase 1b, and renders an empty current-behaviour table every redesign check then passes over. Cache key on disk: `/tmp/multi-agent-analysis-cache/<digest>.json` with mtime <= 24h. Cache hit skips Phase 1b and Phase 1c. `--no-cache` flag forces re-run.
 
-Phase 1d is deliberately absent from the digest inputs. Community signal changes by the hour, so folding it in would produce a new digest on every run, invalidate the cache every time, and re-run the two expensive repo phases the cache exists to skip. The consequence is worth stating plainly: a cache hit reuses yesterday's signal rows. That is the correct trade for an advisory tier, and `--no-cache` is the way to refresh them.
+Phase 1d is not a digest input: community signal changes hourly, so it would invalidate the cache every run and re-run the two costly repo phases the cache skips. A cache hit reuses yesterday's signal rows, the right trade for an advisory tier; `--no-cache` refreshes them.
 
 ### Phase 1d - Outside facts (optional, analyst toolkit)
 

package/pipeline/multi-agent-refs/cross-cli-contract.md

@@ -111,6 +111,11 @@ it does not. Contracts: `features/exploratory-testing.md`,
 | Secrets by reference | `secret_ref` on the type tools | ≥ v3.19.0 |
 | Parallel web sessions, replay | `session` on the web tools; `save_as` / `replay` | ≥ v3.19.0 |
 | Travel document fixtures | `document_generate`, `document_mrz_build` | ≥ v3.19.0 |
+| ARIA assertions | `expect_aria` step, `web_assert_aria` | ≥ v3.20.0 |
+| Web states and locators | `web_route`, `web_network_state`, `web_clock`, `web_dialog`, `web_upload`, `web_drag`, `web_hover`, `web_fill_form`, `web_find`, `web_generate_locator`, `web_trace`; `request_id` on `web_network`; `locator` on `web_click` / `web_type` | ≥ v3.20.0 |
+| Document fetch and crawl | `web_scrape` (formats, cache, `change_tracking`), `url` / `sitemap` / `include_paths` / `llms_txt` on `web_crawl` and `web_map` | ≥ v3.20.0 |
+| Research modes and threads | `mode`, `recency`, `domains`, `sources`, `thread_id` on `research_ask`; `arxiv` and `fetch_content` on `research_search` | ≥ v3.20.0 |
+| Free-model delegation | `llm_delegate`, `llm_providers`; `ESCALATE_TO_CLAUDE`; the `multi-agent-toolkit-llm` CLI (`delegate`, `providers`, `probe`, `usage`) | ≥ v3.21.0 |
 
 ---
 
@@ -266,9 +271,13 @@ happen.
 The routing trio (`route-on`, `route-off`, `route-status`) has no such split -
 it writes `prefs.global.modelRouting` identically everywhere. Its honest limit is
 different and applies on every host: a subagent cannot be dispatched to a
-non-Anthropic model, because subagent dispatch belongs to the host. External
-providers are reachable only where the pipeline makes the HTTP call itself
-(`bulk-read.sh`, `research_ask`). `route-status` prints that on every run.
+non-Anthropic model, because subagent dispatch belongs to the host. The `free`
+rung (`route-on --free`) is reachable only where the pipeline makes the call
+itself: `bulk-read.sh` and the `delegate` call site, where a skill asks
+`model-dispatch.sh delegate --default host` and calls `llm_delegate` on `free`.
+The toolkit and its CLI are the same on all three hosts, so the free chain
+behaves identically everywhere. `route-status` prints the limit on every run.
+Contract: `features/free-models.md`.
 
 ## 3. Frontmatter Transform Rules (Claude ↔ Copilot)
 

package/pipeline/multi-agent-refs/features/doctor.md

@@ -303,6 +303,17 @@ other outcome (`redirected`, `rejected`, `not-accepted`, `failed`, `skipped`),
 with the status code or reason and the step that fixes it. The emitter runs
 detached from phase-tracker with its output discarded, so this record is the
 only surface on which a reporting path that stopped working shows up.
+### free-models
+
+Can `route-on --free` actually leave Claude. SKIP while routing or its free
+chain is off. Otherwise it runs the toolkit's `multi-agent-toolkit-llm
+providers` through `lib/toolkit-llm.sh`: WARN when the CLI cannot run, or when
+no provider is both configured and allowed by the data policy
+(`modelRouting.free.allowTraining`), because every free task then escalates to
+Claude while the user believes it does not. `--probe` also sends each
+configured provider one tiny request and WARNs when every usable one fails it
+(a revoked key, an exhausted quota). `--explain` lists each provider with its
+key source, never a value.
 ### xcode-skills
 
 Is Apple's `xcode-integration` plugin, which Xcode 27.1 and later materialize

package/pipeline/multi-agent-refs/features/exploratory-testing.md

@@ -123,6 +123,13 @@ taps as before and say so in the report:
   value; the value never enters the transcript, the run file or a finding.
 - **Traces.** A failed `agent_run_steps` writes a `trace.md`; put its path in
   the step's `trace` and in the finding's `evidence` as `kind: "trace"`.
+- **States the happy path skips (web, ≥ v3.20.0).** Reach error, empty, slow
+  and offline states on purpose: `web_route` fulfills an endpoint with an error
+  status or an empty body, aborts it or delays it; `web_network_state` goes
+  offline; `web_clock` moves time for expiry and countdowns; `web_dialog`
+  answers a confirm before the click that opens it. Clear the routes when the
+  state is done. `expect_aria` checks a screen's structure against a template
+  without a judge.
 - **Replay.** `save_as` stores a step sequence that reached a finding, and
   `replay` runs it again. `REPLAY_DIVERGED` means the app no longer follows the
   recorded path: an automation block, and a hint that the screen changed.

package/pipeline/multi-agent-refs/features/external-context-injection.md

@@ -35,7 +35,7 @@ One fetcher per type. Each fetcher emits a normalized JSON view that the analysi
 | `swagger` | `~/.claude/lib/fetch-swagger.sh <url>` → endpoints[], request/response examples | `state.fetchedContext.swagger[]` |
 | `confluence` | `~/.claude/lib/fetch-confluence.sh <url>` → page body, code blocks, extracted API contracts | `state.fetchedContext.confluence[]` |
 | `figma` | no standalone fetcher - resolve via the Figma 3-tier chain (Tier 1 MCP tools, Tier 2 `~/.claude/lib/figma-screenshot.sh` + REST) when the task is a component; otherwise advisory only | `state.fetchedContext.figma[]` |
-| `generic-doc` | no automatic fetcher  -  surfaced as advisory; the agent uses WebFetch on demand when the URL turns out to be load-bearing | n/a |
+| `generic-doc` | no automatic fetcher  -  surfaced as advisory. When the URL turns out to be load-bearing, the agent reads it with the toolkit's `web_scrape` (`formats: ["markdown"]`; ≥ v3.20.0) when the server is registered, else with WebFetch. `web_scrape` returns main content as Markdown fenced as untrusted with prompt-injection flags, and serves a repeat from its cache | n/a (read into the prompt, not stored) |
 
 ## Exit code handling