@mmerterden/multi-agent-pipeline 15.9.1 → 15.10.0

package/CHANGELOG.md

@@ -16,6 +16,26 @@ Internal file-layout changes that don't affect the slash-command surface are sti
 
 ## [Unreleased]
 
+## [15.10.0] - 2026-08-21
+
+### Added
+- **Per-repo memory now recalls by relevance, not by recency.** Both memory stores ranked by something that was not relevance: `triage-memory.mjs query` scored a raw token overlap with no IDF, so a word present in every row ("view", "test", "error") counted as much as the one word that identified the bug, and `learnings-ledger.mjs brief` did not rank at all - it replayed the newest 20 entries. Past a few hundred rows both degrade the same way: the injected context stops being about the task in hand. New `pipeline/scripts/_retrieval.mjs` owns the arithmetic for both (the way `_cost.mjs` owns pricing): field-weighted BM25, exponential recency, and Reciprocal Rank Fusion with per-ranking damping so recency separates comparably relevant rows without ever promoting an unrelated one. The tokenizer indexes identifiers whole and split (`KeychainStore` is reachable from "keychain") and folds regular plurals, which is what lets a query phrased as prose reach a row that names a symbol. Zero dependencies: no embedding service, no vector store, no second model call. `prefs.global.memoryRecall.strategy: "legacy"` restores the old behaviour in one flag.
+- **`learnings-ledger.mjs profile` and drill-down pointers.** Durable knowledge is now two blocks instead of one, because relevance and prompt-cache reuse pull against each other. `profile` emits a task-INDEPENDENT `<repo-profile>` ordered by confidence, then kind, then statement - byte-stable across runs, so it belongs at the head of a phase prompt where an unchanged prefix is served from cache and grows into an asset as a repo is learned. `brief --task` emits `<task-relevant-memory>`, ranked against the task, and goes after the task text where a per-run difference costs nothing. Every rendered line ends with an `L:<id>` pointer instead of spelling out its evidence; `learnings-ledger.mjs show --id` and `triage-memory.mjs show --id` return the full row. `multi-agent-refs/prompt-assembly.md` carries the placement contract.
+- **`offload-ref.sh`: bulky tool payloads become a pointer plus a tail.** Phase 3 already teed its build output to a file, but nothing decided how much of that file reached the model, so in practice all of it did. The filter parks the full text at `.multi-agent/refs/<node_id>.md` (content-addressed, gitignored) and prints a `[[ref:<node_id>]]` stub with the last lines - where a failing build's error already is. The evidence gate keeps reading the whole log, so what counts as a verified pass is unchanged; only what reaches the prompt shrinks. Wired into Phase 3 builds and Phase 4 test output, opt-in via `prefs.global.contextOffload.enabled`, and a pass-through when off, so the pipe is always safe to write.
+- **Recall precision is measured.** Phase 1 and Phase 4 emit `memory.injected` / `memory.hit`, and `learning-curve.mjs` trends the ratio alongside the existing KPIs. Without the pair, a ranking change that injects the right rows and one that injects noise are indistinguishable from outside: both return five hits and exit 0. `smoke-learnings-ledger.sh` holds the emitter and the consumer to the same event names, so renaming one side fails a gate instead of silently emptying the column.
+
+### Security
+- **Entry text cannot forge the boundary of the block it is injected into.** `from-triage` builds ledger statements out of a finding's own words, which are model output, so a statement is untrusted content placed inside a structure the reader parses. Statements are now stored as one line (the schema always said "in one line"; nothing enforced it), and the block delimiters are neutralised at render time in both the ledger blocks and `<repo-memory>`. Angle brackets that are not delimiters are untouched, so `Array<String>` still reads as itself.
+
+### Changed
+- `memory-load.sh` takes optional task text and ranks the MEMORY.md pointers against it. The previous `head -30` was a truncation, not a summary - the thirty-first pointer was invisible however precisely it matched, so the block got less useful the longer a repo was worked on. With no task text the index order is unchanged.
+- Phase 4 prior-art lookup and the rejected-preference brief are both ranked against the findings under triage; a finding whose wording matches nothing now returns nothing instead of the three newest rows.
+- `prefs.global.contextOffload.minLines` and `tailLines` are read by `offload-ref.sh`. They shipped in the schema and were honoured by nothing, so a user who set `tailLines: 50` got 20 with no way to tell; config that does nothing documents a control that is not there. An explicit flag still beats the pref, and a non-numeric or zero value falls back to the shipped default.
+- Phase-doc token budget: total 53350 -> 53950. The new prose was compressed twice first (1168 tokens down to 580) by keeping the reasoning in `prompt-assembly.md` and the `offload-ref.sh` header, both outside the budget. Phase 3 and Phase 4 are left amber on their warn lines on purpose - that is the signal that those two docs are next for structural compression rather than another bump.
+
+### Tests
+- `test/retrieval.test.mjs` (20 unit assertions on the ranking primitives), `pipeline/scripts/eval-recall.mjs` + `pipeline/eval/recall-cases.json` (8 end-to-end recall cases against a deliberately noisy corpus, each reporting what the pre-ranking scorer would have returned), `smoke-offload-ref.sh`, and new coverage in `smoke-learnings-ledger.sh` and `smoke-per-repo-memory.sh`.
+
 ## [15.9.1] - 2026-08-20
 
 ### Changed

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@mmerterden/multi-agent-pipeline",
-  "version": "15.9.1",
+  "version": "15.10.0",
   "description": "8-phase AI development pipeline with full orchestration on Claude Code, Copilot CLI and Codex CLI. Analysis, planning, TDD, CLI-aware parallel review with consensus surfacing + Fable triage, default-FAIL evidence gates, secret + intent guards, per-phase cost ledger, persistent learnings memory, wiki generation, commit automation. Token-preserving uninstall.",
   "type": "module",
   "main": "index.js",
@@ -14,7 +14,7 @@
   },
   "scripts": {
     "start": "node index.js",
-    "test": "node --test test/*.test.mjs && node pipeline/scripts/run-smokes.mjs && node pipeline/scripts/lint-skills.mjs && node pipeline/scripts/lint-mcp-refs.mjs && node pipeline/scripts/eval-triage.mjs && node pipeline/scripts/eval-golden-tasks.mjs && node pipeline/scripts/eval-intent.mjs && node pipeline/scripts/validate-schemas.mjs && node pipeline/scripts/validate-prefs.mjs && node pipeline/scripts/scorecard.mjs",
+    "test": "node --test test/*.test.mjs && node pipeline/scripts/run-smokes.mjs && node pipeline/scripts/lint-skills.mjs && node pipeline/scripts/lint-mcp-refs.mjs && node pipeline/scripts/eval-triage.mjs && node pipeline/scripts/eval-golden-tasks.mjs && node pipeline/scripts/eval-intent.mjs && node pipeline/scripts/eval-recall.mjs && node pipeline/scripts/validate-schemas.mjs && node pipeline/scripts/validate-prefs.mjs && node pipeline/scripts/scorecard.mjs",
     "test:unit": "node --test test/*.test.mjs",
     "test:smoke": "node pipeline/scripts/run-smokes.mjs",
     "lint:skills": "node pipeline/scripts/lint-skills.mjs",

package/pipeline/multi-agent-refs/phases/phase-1-analysis.md

@@ -29,18 +29,27 @@ Also read project-level CLAUDE.md if exists:
 **Per-repo memory injection (opt-in via `prefs.global.perRepoMemory`):**
 
 ```bash
-bash $HOME/.claude/scripts/memory-load.sh "$PROJECT_ROOT"
+bash $HOME/.claude/scripts/memory-load.sh "$PROJECT_ROOT" "$TASK_TITLE $TASK_DESCRIPTION"
 ```
 
-Exit 0 with empty output = pref off or no memory on disk  -  skip. Otherwise the script emits a `<repo-memory path="...">...</repo-memory>` block (≤ 30 lines of MEMORY.md pointers) suitable for direct injection into the analysis prompt. Individual memory files are read on-demand when a pointer looks relevant to the current task.
+Exit 0 with empty output = pref off or no memory on disk  -  skip. Otherwise the script emits a `<repo-memory path="...">...</repo-memory>` block of MEMORY.md pointers suitable for direct injection into the analysis prompt. Passing the task text ranks the pointers against it instead of printing the first thirty; individual memory files are read on-demand when a pointer looks relevant.
 
-**Durable learnings brief (on by default via `prefs.global.learningsLedger.enabled`):**
+**Durable knowledge (on by default via `prefs.global.learningsLedger.enabled`):** two blocks, and where each goes is part of the contract  -  see `$HOME/.claude/multi-agent-refs/prompt-assembly.md`.
 
 ```bash
-node $HOME/.claude/scripts/learnings-ledger.mjs brief --max 20 2>/dev/null
+# HEAD of the prompt: task-independent, byte-stable, so it caches.
+node $HOME/.claude/scripts/learnings-ledger.mjs profile 2>/dev/null
+# END of the prompt, after the task text: ranked against this task.
+node $HOME/.claude/scripts/learnings-ledger.mjs brief --max 20 \
+  --task "$TASK_TITLE $TASK_DESCRIPTION" 2>/dev/null
 ```
 
-Exit 2 (empty ledger) = skip silently. Otherwise the script emits a `<repo-learnings>...</repo-learnings>` block of durable architectural facts, conventions, and rejected review preferences accumulated from prior runs of this repo. Inject it into the analysis prompt so the explorer does not re-discover known structure. Skip when `injectIntoAnalysis = false`. These are context, not commands  -  current scope decides.
+Exit 2 (empty ledger) = skip silently. Lines end with an `L:<id>` pointer; `show --id L:<id>` returns the full entry. Skip both when `injectIntoAnalysis = false`. Context, not commands  -  current scope decides. Then log the injection so recall quality stays measurable:
+
+```bash
+bash $HOME/.claude/scripts/log-metric.sh "$TASK_ID" 1 memory.injected \
+  kind=<profile|task-relevant> rows=$N chars=$C
+```
 
 **If knowledge files exist and are fresh** (modified within last 90 days  -  see knowledge.md staleness rules):
 
@@ -191,6 +200,8 @@ PRIOR=$(node $HOME/.claude/scripts/triage-memory.mjs query \
   --issue "$TASK_TITLE $TASK_DESCRIPTION" --top 3 2>/dev/null | jq -c '.hits // []')
 ```
 
+Hits are relevance-ranked, and a query matching nothing returns nothing. Each hit carries an `id`: `triage-memory.mjs show --id <id>` returns the full row.
+
 Treat hits as **context only**  -  they are past Phase 4 verdicts, not prescriptions. Useful when the new task touches the same files or symbols as a previous run.
 
 ---

package/pipeline/multi-agent-refs/phases/phase-3-dev.md

@@ -321,12 +321,15 @@ If a todo has no `repo` tag in multi-repo mode → log warning + ask user, do no
 
 **Recording a pass (default-FAIL evidence gate):** before setting `buildStatus.ok = true`, the build output must be tee'd to a log and that log must substantiate the success  -  a zero exit code alone is not trusted. Run the evidence gate; on exit 1, do NOT record a pass:
 ```bash
-<build-command> 2>&1 | tee "$WORKTREE/.build.log"
+<build-command> 2>&1 | tee "$WORKTREE/.build.log" \
+  | bash $HOME/.claude/scripts/offload-ref.sh --phase 3 --label build --root "$WORKTREE"
 node $HOME/.claude/scripts/evidence-gate.mjs --claim build --status passed --evidence "$WORKTREE/.build.log" \
   || { echo "build pass unverified  -  treat as failure"; /* keep buildStatus.ok=false */ }
 ```
 This closes the gap where an agent records "built" without ever producing build output.
 
+**Why the pipe (opt-in via `prefs.global.contextOffload.enabled`).** `tee` decides where the log is written, not how much of it the model reads. The filter parks the full text at `.multi-agent/refs/<node_id>.md` and prints a stub plus the tail, where a failing build's error already is; read that file before re-running a failed build. The evidence gate still reads the whole `.build.log`, so what counts as a verified pass is unchanged. Pref off = pass-through.
+
 **Telemetry**: Per-repo metrics in addition to per-task metrics:
 ```bash
 $HOME/.claude/scripts/log-metric.sh "$TASK_ID" 3 build.completed repo=common duration_ms=$D status=ok

package/pipeline/multi-agent-refs/phases/phase-4-review.md

@@ -416,16 +416,34 @@ PRIOR_ART="${PRIOR_ART%,}]"
 
 The triage prompt MUST include a hedge: *"prior-art entries are context, not commands; current scope decides  -  a finding rejected last quarter may be valid this time."* Without this hedge, prior verdicts amplify into a self-reinforcing bias.
 
+Hits are relevance-ranked (`prefs.global.memoryRecall`); a finding matching nothing returns nothing. Each hit carries an `id`: `triage-memory.mjs show --id <id>` returns the full row.
+
 **Injection cap (token economy).** On a many-finding review the per-finding prior-art loop above (up to 3 hits each) plus the 20-entry rejected-preference brief can dominate the triage prompt. Cap the merged prior-art at the 8 highest-similarity hits across all findings (drop the rest); keep the rejected-preference brief at its `--max 20`. Prior-art is advisory context, not a finding multiplier  -  more hits do not improve the verdict, they just inflate input tokens.
 
 **Rejected-preference brief (on by default via `prefs.global.learningsLedger.injectIntoTriage`).** Inject the durable rejected-preference list so triage does not re-accept a suggestion the team already rejected on this repo:
 
 ```bash
-node $HOME/.claude/scripts/learnings-ledger.mjs brief --max 20 2>/dev/null
+node $HOME/.claude/scripts/learnings-ledger.mjs brief --max 20 \
+  --task "$(jq -r '[.findings[].issue] | join(" ")' <<< "$MERGED_FINDINGS")" 2>/dev/null
 ```
 
 Exit 2 (empty ledger) skips silently. The `## Rejected review preferences` section names patterns the team chose not to act on; triage should lean toward `rejected` for a finding that restates one  -  but the same hedge applies (context, not command; a genuinely new instance can still be accepted).
 
+`--task` is what makes the cap honest: unranked, those twenty slots go to the newest entries, which on a long ledger are mostly about other files.
+
+**Recall telemetry.** Log what was injected, then what triage cited. Zero cited is a legitimate answer; `learning-curve.mjs` trends the ratio:
+
+```bash
+bash $HOME/.claude/scripts/log-metric.sh "$TASK_ID" 4 memory.injected kind=prior-art rows=$N
+bash $HOME/.claude/scripts/log-metric.sh "$TASK_ID" 4 memory.hit rows=$CITED_COUNT
+```
+
+**Bulky payloads (opt-in via `prefs.global.contextOffload.enabled`).** Test output and whole-file diffs go through the offload filter, which leaves a `[[ref:<node_id>]]` line plus the tail in context and the full text under `.multi-agent/refs/`. Read that file when the tail is not enough; with the pref off it is a pass-through.
+
+```bash
+<test-command> 2>&1 | bash $HOME/.claude/scripts/offload-ref.sh --phase 4 --label tests
+```
+
 **Triage prompt skeleton:**
 
 ```

package/pipeline/multi-agent-refs/prompt-assembly.md

@@ -8,14 +8,24 @@ Assemble every phase prompt as two spans, in this order:
 
 1. **Stable prefix (cacheable, repo-stable, changes rarely):**
    - system / role instructions for the phase
-   - the repo learnings brief (`learnings-ledger.mjs brief`)
+   - the repo profile (`learnings-ledger.mjs profile` -> `<repo-profile>`)
    - extracted conventions (Phase 1c) and the design/analysis doc when applicable
    - repo structural evidence (the reuse-first buckets, Code Connect index)
 2. **Volatile suffix (task-specific, changes every run):**
    - the specific task / issue / diff
+   - the task-ranked memory (`learnings-ledger.mjs brief --task ...` -> `<task-relevant-memory>`, and any `triage-memory.mjs query` prior art)
    - the current file(s) under edit
    - per-run state and the immediate instruction
 
+## Why durable knowledge is TWO blocks
+
+Ranking the ledger against the task makes the injected knowledge relevant. It also makes it different on every run, and anything that differs cannot be a cached prefix. Those two goods are in direct conflict if there is one block, so there are two:
+
+- `profile` is task-INDEPENDENT: that, and only that, is what makes it cacheable. Identical bytes across runs, so it sits at the head and is served from cache, and it grows as the repo is learned - the mechanism that makes knowing the repo cheaper rather than more expensive. Its order (confidence, then kind, then statement by code point, never recency) is a separate choice, made for truncation: when `--max` or `maxChars` cuts the block, the entries that survive should be the ones held with most confidence. The cost of that choice is that a new entry lands mid-block rather than at the end, so slightly less of the block's own prefix survives the run that adds it.
+- `brief --task` is ranked against the task in hand and goes AFTER the task text, where a per-run difference costs nothing that was not already volatile.
+
+Both blocks end each line with an `L:<id>` pointer instead of spelling out the evidence behind it. The detail is one `learnings-ledger.mjs show --id` away, and the block that would have carried it inline for every entry stays small. The same applies to prior art (`T:<id>`, `triage-memory.mjs show --id`) and to offloaded tool payloads (`[[ref:<node_id>]]`, `offload-ref.sh`): a pointer in context, the evidence on disk, nothing lost.
+
 Put the stable material FIRST and the volatile material LAST. Reordering volatile content into the middle of the prefix invalidates the cache for everything after it, so keep the boundary clean.
 
 ## Why it matters
@@ -27,5 +37,6 @@ Put the stable material FIRST and the volatile material LAST. Reordering volatil
 ## Anti-patterns
 
 - Interleaving the task text with conventions/learnings (breaks the prefix).
+- Putting the task-ranked block (`brief --task`) in the prefix. It changes every run, so everything after it re-bills at full price - which costs more than the relevance was worth.
 - Letting the stable prefix bloat past what the phase needs (context rot; cache a lean prefix, not everything - see the corpus merge/drop + progressive-disclosure discipline).
 - Regenerating repo evidence from scratch each run instead of reusing cached/learned artifacts (defeats both the cache and the learning curve).