@bcmyguest/openmem 0.6.3 → 0.9.0

package/docs/evidence-plan.md

@@ -0,0 +1,216 @@
+# Evidence plan: smart compaction, run E
+
+Original experiment plan for the claim in [context-management.md](context-management.md#benchmarks): *OpenMem smart compaction keeps ContextBench scores at or above keep-everything while costing less than native compaction and keep-everything.* The budget below is the original estimate, **$58 at list price** (phases 1–5), with a $75 hard stop. Results and deviations are recorded here as trials finish.
+
+## Current run status (2026-10-31)
+
+| Work | Evidence now available | State |
+| --- | --- | --- |
+| Phase 1, run E | `bench/results-E/summary.json` contains 51 rows across e-none (13), e-native (12), e-native-153k (12), and e-smart (14), including sensitivity and rerun rows. | Seed-paired H1–H3 and H5 comparisons below; H6 remains unmeasured. |
+| Phase 2, scratchpad | `scratchpad-phase2` has commits `8631472` (pins table) and `0809dc3` (MCP write tool). Separate trial worktrees contain step2c read-path commits, including `2e55743`. | Read path is not integrated into `main`; the brief/test conflict below needs resolution before treating it as complete. |
+| Phase 4, Sonnet | `bench/results-S/summary.json` contains two seed-matched pairs on log L6400. Mean lenient reward: e-none 0.479, e-smart 0.542. Total list-price cost: $19.85 versus $7.86 for the two trials per arm. | H4 comparison below. The first pair exceeded the original $10 abort threshold; the user approved the second pair. |
+| Coding-task replay | Eight trials completed: two seeds each for `decision-fixes` and `step2c` under e-native and e-smart. `bench/replay/results/report.md` holds raw metrics. | Scoring review pending; see below. |
+| Phase 3, decision consistency | Four completed sudoku t150 rows (seeds 45 and 46, e-none and e-smart), all with lenient reward 1.0. One decision-ledger D40 trial has a `result.json`; two more trial directories are partial. | Incomplete. No active runner was visible at this check; remaining trials need a restart and a fresh summary. |
+| Phases 5 and 6 | Phase 5 threshold ablation was optional; phase 6 Compact-Bench was deferred in the original plan. | No completed results found in this check. |
+
+The three result sets above total $101.72 at list price ($65.54 run E, $27.71 Sonnet, $8.47 replay), excluding earlier runs. This exceeds the original $75 stop by $26.72. These are artifact totals, not a reconciled billing statement. Further API trials are paused pending a new budget decision.
+
+The replay exposed two harness fixes. `bench/ctx_metrics.py` now skips non-object message entries; commit `09c8ae1` contains that fix. The replay script uses a short `/tmp/rp-XXXXXX` `TMPDIR` and links it into each trial directory to keep sandbox socket paths short. The previous agent reported that a three-turn smoke run showed no sandbox error but made no Bash call; sandboxed Bash remains unverified. The replay tree was previously ignored by `.git/info/exclude`; its source and packaged task inputs are now tracked. The previous agent reported removing eight invalid pre-fix trials; the results directory now contains eight completed trials.
+
+The replay report's `tests_pass` column is unsuitable for `step2c`: the brief requires changing the standing instruction and pointer text while forbidding changes to existing assertions, yet the existing exact-string tests pin both values. The four completed `step2c` trials all have `tests_rc=1`. The observed failures differ across trials, and one trial also has Python test failures, so a passing score cannot be inferred from `tests_rc` alone. Score the brief's requested behavior, Python tests, the exact failing checks, and the agent's report separately before drawing an arm comparison. The original two-trial inspection found that agents claimed passing npm tests without running the full suite; inspect all four final reports before treating that finding as a full-run result.
+
+The replay uses Claude Haiku 4.5 and host Claude Code with bubblewrap; run E uses the ContextBench harness. Its raw means (e-native versus e-smart: 74.0 versus 132.5 tool calls, 295 versus 463 seconds, $0.85 versus $1.27 per trial) describe these eight trials only. Only two of four e-smart replay trials called `smart_compact_search`, once each; the Sonnet ContextBench smart trials called it 11 and 12 times. These results do not establish an effect of trimming or a model-specific cause for stalled coding agents because task success and the `step2c` scorer remain unresolved.
+
+### Completed seed-paired comparisons
+
+Source: `bench/results-E/summary.json` and `bench/results-S/summary.json`. Each trial row is one observation. Within each task size, pair arms by seed; resample the paired seeds with replacement 20,000 times for two-sided 95% percentile-bootstrap intervals. The planned e-native-153k sensitivity arm and phase 3 sudoku rows are outside the primary comparison. No primary row has a recorded exception or rate-limit error. With only two or three seeds per cell, these intervals describe the observed seeds and give weak coverage for new tasks or seeds.
+
+| Run E task | Pairs smart/none | Reward difference smart − none, 95% CI | Reward difference smart − native, 95% CI | Smart/none cost ratio, 95% CI |
+| --- | ---: | ---: | ---: | ---: |
+| kv R4000 | 3 | 0.000 [0.000, 0.000] | 0.000 [0.000, 0.000] | 0.130 [0.128, 0.132] |
+| log L1600 | 3 | +0.431 [0.250, 0.542] | +0.417 [0.333, 0.500] | 0.811 [0.102, 1.528] |
+| log L6400 | 3 | +0.389 [0.125, 0.542] | +0.417 [0.167, 0.583] | 0.109 [0.107, 0.111] |
+| log L12800 | 2 | +0.646 [0.625, 0.667] | +0.653 [0.583, 0.708] (3 pairs) | 0.062 [0.062, 0.063] |
+
+Pooling the available log pairs gives smart − none reward +0.469 [0.339, 0.578] across 8 pairs and smart − native +0.495 [0.384, 0.593] across 9 pairs. This pooled interval resamples individual task/seed pairs; it describes the observed mix of log sizes. R4000 is a ceiling case: all three arms scored 1.0, so it supports non-inferiority on these seeds but not strict superiority over native. H1 was not disproved. H2 is supported on log tasks and fails its strict-superiority condition on kv R4000. Smart cost was lower on every R4000, L6400, and L12800 pair. At L1600, the cost-ratio CI crosses 1 and one smart trial cost more than its baseline, so H3's pre-registered all-family cost claim is not established.
+
+For H5, the paired smart/none agent-wall-time ratios were 0.996 [0.794, 1.140] at R4000, 1.532 [0.640, 2.021] at L1600, 1.004 [0.979, 1.052] at L6400, and 0.876 [0.582, 1.169] at L12800. The wide L1600 interval includes the 1.2 slowdown threshold; these trials neither establish nor rule out a slowdown there.
+
+On Sonnet log L6400 (seeds 45–46), smart − none reward was +0.0625 [0.0417, 0.0833]. The mean paired smart/none cost ratio was 0.401 [0.361, 0.441]; the two-arm totals were $7.86 and $19.85. Total reported input, cache, and output tokens were 22.73M smart versus 69.26M none (ratio 0.328). Both observed pairs meet H4's score and cost rule, with only two seeds and one task size.
+
+H6 cannot be scored from these artifacts. In `usage.jsonl`, `trimmed=true` marks both a newly cut request and later requests carrying the same cut; the flag is not a new-trim indicator. The LiteLLM `latency_s` logger receives model-call start/end timestamps, and the saved trim events do not record hook duration. Comparing all `trimmed=true` requests with untrimmed requests would answer a different question.
+
+## 0. What the published numbers rest on
+
+| Run | What | n | Where |
+| --- | --- | --- | --- |
+| Strict | 4 arms, kv R4000 + log L6400, seeds 42–44 | 24 | context-research.md §Strict run |
+| B | s1-none, s2-native, s3a-smart by size | 1–2 seeds/cell | context-research.md §Run B |
+| D | d-sqlite + d-hybrid at 120k trim, 6 sizes × seeds 42–44 | 36 | `bench/results-D/`, chart source |
+| Dp | search-tools pointer, log L1600 seed 44 only | 5 | `bench/results-Dp/` |
+| Df | cache-stable trims, log L1600 seed 44 only | 5 | `bench/results-Df/` |
+
+Weaknesses the plan must remove or confirm:
+
+1. Baselines (run B) and smart (run D) come from different runs, LiteLLM builds and days; `docs/context-management.md:93` says so. The latency claim (trim request 4.0 s vs 1.7 s) is cross-run.
+2. Df (the config quoted as current, `docs/context-management.md:125-139`) has n = 5 on **one task instance**, seed 44, the seed it was tuned on.
+3. The README chart (`README.md:30-36`) and `site/src/data/contextbench.json` are run D (`site/scripts/bench/compute-contextbench.py:27-29` reads `results-B` s1/s2 and `results-D` d-sqlite); the text under it quotes Df. Chart and text disagree on config (120k vs 95k trim, pointer, cache fix).
+4. The 5× cost ratio depends on Haiku 5.5's >100k price tier (`bench/summarize.py:31`, `PRICE[True]`); `context-management.md:139` already concedes total tokens differ <1.5×.
+5. Native arm fired at a forced window (`bench/run_strict.sh:31` `NATIVE_WINDOW=153000`), not Claude Code's default.
+6. No task scores the agent's own earlier decisions, the thing whole-turn eviction drops.
+
+## 1. Hypotheses and decision rules (pre-registered)
+
+Primary metric: `reward_lenient` per trial (kv tag-insensitive; `bench/summarize.py:120-135` `answer_check`). Secondary: list-price USD per trial (`summarize.py:37-43`), agent wall time (result.json via `bench/ctx_metrics.py`), request latency (`usage.jsonl` `latency_s`), peak prompt tokens, trims, native compactions.
+
+| Id | Hypothesis | Disproved if |
+| --- | --- | --- |
+| H1 quality | Smart ≥ keep-everything on held-out seeds at R4000/L6400/L12800 | paired (same seed) mean difference smart − none < −0.05 with 95% bootstrap CI excluding 0 |
+| H2 quality vs native | Smart > native (default) | paired difference ≤ 0 or CI includes 0 at n = 9 pairs per task family |
+| H3 cost | Smart < native and < none per trial on Haiku | smart/none cost ratio CI includes 1.0, or smart > native on any family |
+| H4 price-tier | On Sonnet 5, at Sonnet's per-model trim threshold (125k), smart passes if it costs no more per trial than e-none (no compaction) and scores the same or better on `reward_lenient` | smart costs more than e-none per trial, or scores worse on `reward_lenient` (then the cost claim is published as Haiku-only) |
+| H5 speed | Within one run, smart's agent wall time per trial is not slower than none by more than 20% | wall-time ratio smart/none > 1.2 with CI excluding 1.2 |
+| H6 trim latency | Requests carrying a trim cost ≤ 1 s extra over the same arm's untrimmed requests | median difference > 1 s (then document the hook cost, do not hide it) |
+| H7 decisions | Whole-turn eviction drops the agent's own decisions: smart without scratchpad scores < none on the decision task | smart ≥ none (then risk (c) is not observable here; accept as-is) |
+| H8 scratchpad | Scratchpad restores H7: smart+scratchpad ≥ none and > smart | smart+scratchpad − smart ≤ 0 (then drop the feature) |
+
+Statistics for every table: mean, n, 95% percentile-bootstrap CI (10,000 resamples), paired by seed where arms share seeds; report per family and pooled; list every trial (no exclusions beyond the pre-registered harness failures below). A trial is excluded only when `summarize.py` classes it rate-limited (`RATE_LIMIT_RE`, line 29), harness-failed (`cc_stream.py` no result event), or the proxy fail-opened (`events.kind = 'fail_open'`, `core.py:355`); excluded trials are listed and rerun once (`bench/rerun_strict.sh` pattern; `ctx/queue-E.txt` accepts appended lines, `ctx_sched.py:5`).
+
+Null outcome is publishable: if H1–H3 fail, `context-management.md` Benchmarks is rewritten to say so and the README chart is removed.
+
+## 2. Phases
+
+Trial-cost and time anchors (list price; `results-D/d-sqlite/usage.jsonl`, `docs/context-research.md:152,184-195`, `context-management.md:83-88`):
+
+| Cell | $ per trial | min per trial |
+| --- | --- | --- |
+| smart log L1600 (Df, 95k) | 0.11 | 7.3 |
+| smart kv R4000 / log L6400 (run D, 120k) | 0.27 / 0.30 | 10–12 |
+| smart log L12800 (run D) | 0.58 | 24 |
+| none L1600 / R4000 / L6400 / L12800 (run B) | 0.57 / ~1.5 / ~1.5 / ~3 | 8–15 |
+| native (forced 153k) L1600 / R4000 / L6400 | 0.31 / ~0.5 / ~0.5 | 8 |
+
+Four trials run concurrently (`ctx_sched.py` MAX = 4). Wall time ≈ trials × 11 min / 4 + 20% for reruns.
+
+### Phase 0: harness prep (no API spend, ~2 h)
+
+Files:
+
+- `bench/run_ctx.sh`: add run `E` with arms `e-none`, `e-native`, `e-smart`; ports in the table at lines 51-55 (next free: 4257–4259); `ARMS` case at 40-41; `TRIM_AT=` empty for E (line 45) so `MODEL_TRIM_AT_TOKENS` gives Haiku 95k (`proxy/openmem_context/settings.py:14`); `e-smart` at the current `main` SHA exported like `DF_SHA`/`R_DF` (lines 126-127; `./run_ctx.sh prep E`). `e-none`: pass-through proxy + `DISABLE_AUTO_COMPACT=1` (as `run_strict.sh:113`). `e-native`: pass-through proxy, **no** `CLAUDE_CODE_AUTO_COMPACT_WINDOW`, **no** `DISABLE_AUTO_COMPACT` — Claude Code's default. Justification: the comparison a user sees is default Claude Code; the forced 153k window was a pilot artefact. If Claude Code's default never compacts under 1M on R4000 (then e-native ≡ e-none up to noise) the result still answers "what does the default do"; add a `e-native-95k` sensitivity arm (`CLAUDE_CODE_AUTO_COMPACT_WINDOW` sized to fire near 95k, confirmed in the first trial's `compactions` field, `cc_stream.py:171`) on log L6400 only, 3 seeds, +$1.5.
+- Record every arm's exact env in the trial `config.json` (already Harbor's) and the proxy image tag (`D_IMAGE`, line 130) in `docs/context-research.md` run E header.
+- `bench/ci.py` (new, ~60 lines): reads `summarize.py` trial rows, prints per-cell mean / n / bootstrap CI and paired differences by seed. `bench/stats.py` is a token-count script, not statistics; do not extend it.
+- Seeds: 45, 46, 47 (held out; tasks already packaged: `bench/cb/tasks/*-seed45..49` exist for all sizes). Seed 44 is never used for scoring.
+- Queue `bench/ctx/queue-E.txt`; run with `python bench/ctx_sched.py 4 ctx/queue-E.txt`; summarise with `python bench/summarize.py results-E state-E` and `python bench/ci.py results-E`.
+- Smoke: one `e-smart` log L1600 seed 45 trial ($0.11) to confirm the 95k trim fires (`state-E/context/sessions/<sid>.log.jsonl`, `summarize.py:96-100`).
+
+### Phase 1: three arms, same day, held-out seeds (H1–H3, H5, H6)
+
+Cells: {e-none, e-native, e-smart} × {kv R4000, log L1600, log L6400, log L12800} × seeds {45, 46, 47}, except e-none at L12800 limited to 2 seeds (765k prompts, ~$3 each) and kv R16000 excluded (e-none hits the 979k step limit, not a recall score; `context-research.md` Limitations).
+
+| Arm | Trials | $ |
+| --- | --- | --- |
+| e-smart | 12 | 3 × (0.11 + 0.27 + 0.30 + 0.58) = 3.8 |
+| e-native (default) | 12 | 3 × (0.31 + 0.5 + 0.5 + 1.0) = 6.9 (if it never compacts, cost rises toward e-none: +5) |
+| e-none | 11 | 3 × (0.57 + 1.5 + 1.5) + 2 × 3 = 16.7 |
+| e-native-95k (optional) | 3 | 1.5 |
+| reruns (20%) | ~7 | 5 |
+
+Phase 1 total ≈ **$34**, 38 trials, ≈ 2.5–3 h wall. All arms interleaved in one queue so API load is shared.
+
+Analysis: H1–H3 paired by seed; H5 from result.json agent wall time, same run; H6 **within e-smart**: `usage.jsonl` rows with `trimmed = 1` vs untrimmed rows of the same session, median and bootstrap CI of the difference, plus `events` table trim rows (`core.py:438`) for the hook's own time if recorded — check `record_usage` in `proxy/openmem_context/tokens.py`; if latency_s is measured outside the hook, say so. Also report requests per trial and searches per trial (sc_search, sc_sql columns, `summarize.py:232`).
+
+Stop rule: if after the first 12 trials (one seed per cell) e-smart mean `reward_lenient` < e-none − 0.15, pause and write it up before spending more.
+
+### Phase 2: scratchpad feature (no API spend, ~1 day)
+
+Design (minimal; cache-safe):
+
+- Storage: new table `pins (id INTEGER PRIMARY KEY, at TEXT, text TEXT)` as schema version 3 in `proxy/openmem_context/sessiondb.py` (migrations at lines 21-40; JS twin in `cli/mcp/context-store.mjs` / `cli/coding-agent/context-trim.mjs:370-381`).
+- Write path: MCP tool `smart_compact_pin(session_id, text, replace=false)` in `cli/mcp/context-store.mjs` next to `smart_compact_search` (line 60); ≤ 2,000 chars total, newest-wins on `replace`; returns the current pinned block so it also sits in recent context.
+- Read path: `_build_note` (`proxy/openmem_context/core.py:270-290`) appends `[openmem: pinned notes]\n<pins>` after `pointer_text`. The note is frozen per trim (`threads[root]["note"]`, line 416), so the pinned block changes only at trim time and the cached prefix stays byte-identical between trims. Pins written after a trim appear at the next trim; the tool result carries them meanwhile.
+- Prompting: one sentence in `STANDING_INSTRUCTION` (`core.py:40`) and the pointer: "Decisions you must keep: smart_compact_pin(...)". Add the same to `cli/coding-agent/context-trim.mjs` for parity.
+- Settings: `CONTEXT_PIN_MAX_CHARS` (default 2000) in `settings.py` dataclass (line 70+), 0 disables.
+
+Tests:
+
+- `proxy/tests/test_context_proxy.py` (87 tests; run `cd proxy && python3 -m pytest -q`, pytest from `bench/.venv/bin/pytest` if the host lacks it): trim note contains pins; pins survive a second trim; `replace` semantics; byte-identical note between two requests after one trim with a pin added in between (cache invariant).
+- `proxy/tests/test_session_db.py`: schema v3 migration from v2 db.
+- `scripts/tests/test_context_store.mjs` + new `scripts/tests/test_context_store_pin.mjs` (`npm test` runs `scripts/tests/test_*.mjs`, `package.json:88`): tool schema, cap, replace, read-only tools untouched.
+- `scripts/tests/test_context_parity.mjs`: JS port renders the same note.
+- Docs: `docs/context-management.md` tool table, `proxy/README.md` if it lists env vars, CLAUDE.md context section.
+
+### Phase 3: decision-consistency task (H7, H8)
+
+Research summary (what exists, why it does or does not fit):
+
+| Benchmark | What it scores | Fit for (c) | Harness cost |
+| --- | --- | --- | --- |
+| ContextBench `sudoku_sketchpad` (vendored, `bench/cb/contextbench/tasks/sudoku_sketchpad`, tasks t4–t150 seeds 42–49) | agent tracks board state across turns; the grader folds actions | **Good**: the agent's own state, not user facts; already packaged for strict mode | zero |
+| ContextBench `needle_retention` (vendored) | gold lines verbatim in the *final context window* | **Unusable**: scores retained context, so any trimmer scores 0 by construction | — |
+| Compact-Bench Task Preservation ([arXiv 2610.31710](https://arxiv.org/abs/2610.31710)); adapter `bench/compact-bench-openmem/` exists, clone `bench/compact-bench` absent (needs commit 9dd9a3f) | continuation of SWE repairs after compaction of the agent's own progress | **Best external match** for (c) | high: repo venvs, 33 tasks, 2 at a time, several hours; defer to phase 6 |
+| [LongMemEval](https://arxiv.org/abs/2410.10813), [LoCoMo](https://arxiv.org/abs/2402.17753), [MemoryAgentBench](https://arxiv.org/abs/2507.05257) | QA over user-supplied history (MemoryAgentBench adds conflict resolution) | tests retrieval of *given* facts, which kv/log already cover; none scores the agent's own decisions | medium: new stream adapter |
+
+Decision: run **sudoku-sketchpad** (existing) and add one small family, `decision_ledger`, to `bench/cb/contextbench/tasks/` (register in `FAMILIES`, `bench/cb/contextbench/tasks/scaffold.py:25`; same `generate(seed, ...)` shape as `log_triage/generate.py:74`):
+
+- Stream: N rounds. Round k gives the agent two or three valid options for item k ("pick the codename for service k from {A,B,C}") and asks it to reply `DECIDE k=<choice>`. Filler batches (log lines reused from `log_triage`) push the context past 95k. Then Q questions: "which codename did you pick for k?" → `ANSWER k=<choice>`, plus M "apply your choice" questions whose correct answer is a deterministic function of the earlier choice (e.g. "the port for k is 8000 + index of your choice").
+- Grader (`bench/cb/contextbench/graders/decision.py`, mirrors `graders/log.py:64`): parse the agent's `DECIDE` lines from early turns (not from gold), compare each `ANSWER` to the agent's own decision; reward = fraction consistent. Deterministic; no LLM judge.
+- Sizes: `D40-F2600` (40 decisions, 2,600 filler lines, ~118k streamed tokens, one trim) and `D80-F6400` (3–4 trims). Seeds 45–47.
+
+Cells: {e-none, e-smart, e-smart-pin} × {sudoku t150, ledger D40-F2600, ledger D80-F6400} × 3 seeds = 27 trials; e-native omitted (H7/H8 are about eviction, not summaries). Cost ≈ 27 × 0.35 = **$9.5**, ≈ 1.5 h.
+
+Decision rules: H7, H8 as in §1. If `e-smart` scores ≥ `e-none` on both families, record "eviction loss not observed at these sizes" and ship the scratchpad only if H8's lift is ≥ 0.05 with CI excluding 0.
+
+### Phase 4: Sonnet, one task, end of programme (H4)
+
+Model `claude-sonnet-5` (confirm id against the LiteLLM route `bench/litellm/bench-config.yaml:7-12`; flat 1M pricing, so `PRICE` in `summarize.py:31` needs a per-model table — add `PRICES[model]`, and `site/scripts/bench/compute-contextbench.py` inherits it). Trim threshold: leave `CONTEXT_TRIM_AT_TOKENS` unset so Sonnet gets its per-model default of 125k (`settings.py:11`, `DEFAULT_TRIM_AT_TOKENS`); Haiku's 95k override (`MODEL_TRIM_AT_TOKENS`, `settings.py:14`) does not apply to Sonnet.
+
+Cells: {e-none, e-smart} × log L6400 × seeds {45, 46} = 4 trials. Cost per trial at Sonnet rates (~20× Haiku base, no cliff): none ≈ 10M tokens mostly cache reads ≈ $4–6; smart ≈ 7M ≈ $2–3. Total ≈ **$16**; abort after the first pair if it exceeds $10. The first pair cost $12.71, over the abort threshold; the user waived the abort rule and approved a second pair, and Sonnet stops there. Report cost ratio with CI and the token ratio; H4 decides how the cost claim is worded.
+
+### Phase 5: threshold ablation (optional, cheap)
+
+Only if phase 1 shows trims drive cost (cost correlates with trim count across trials, Spearman ρ > 0.5). e-smart only, log L6400 seed 45, grid `CONTEXT_TRIM_AT_TOKENS ∈ {60k, 95k, 120k}` × `CONTEXT_TRIM_TO_FRACTION ∈ {0.3, 0.6}` (`settings.py:16,80,111`), 1 trial each = 6 trials ≈ **$2**, 40 min. Report score, cost, trims, cache hit rate; pick nothing from it without a confirmation seed.
+
+### Phase 6 (deferred, not budgeted): Compact-Bench Task Preservation
+
+Re-clone `cai-jianfeng/compact-bench` at 9dd9a3f into `bench/compact-bench`, run `bench/compact-bench-openmem/run.sh --instance pylint-dev__pylint-6903 --methods claude,openmem --rounds 1` as smoke, then `drive.py`. Needs its own budget line (33 tasks × 4 methods on Haiku; estimate after the smoke).
+
+### Cheap aside for risk (a)
+
+In phase 1 e-smart trials Claude Code's auto-compaction is disabled. One extra e-smart trial on log L6400 with `DISABLE_AUTO_COMPACT` unset ($0.30) shows whether the two managers ever collide at 95k; record `compactions` and trims. Not prioritised.
+
+## 3. Budget and schedule
+
+| Phase | Trials | $ | Wall |
+| --- | --- | --- | --- |
+| 0 prep + smoke | 1 | 0.1 | 2 h build |
+| 1 three arms | 38 | 34 | 3 h |
+| 2 scratchpad | 0 | 0 | 1 day build |
+| 3 decision task | 27 | 9.5 | 1.5 h + 0.5 day build |
+| 4 Sonnet | 4 | 16 | 1 h |
+| 5 ablation | 6 | 2 | 40 min |
+| a aside | 1 | 0.3 | — |
+| **Total** | **77** | **≈ 62 (cap 75)** | 2.5 days build, ~6 h run |
+
+Phase 4 is the overrun risk; its abort rule keeps the programme under $60 if Sonnet costs come in high.
+
+## 4. Harness pitfalls (from the code)
+
+- Retries must `--resume`; a retried turn with `--session-id` collides (`bench/cc_stream.py:345`). Retry events are counted in `cc_retry_events` (`summarize.py:203`).
+- Rate limits: `RATE_LIMIT_RE` (`summarize.py:29`) classes 429/529/overloaded; run B lost trials to this. Keep MAX = 4 concurrent; stagger is 5 s (`ctx_sched.py:28`).
+- Ports are hard-coded per run:arm (`run_ctx.sh:51-55`); a new run needs new entries or proxies collide.
+- `TRIM_AT` semantics: unset → 120000; set-empty → proxy per-model threshold (`run_strict.sh:29`, `run_ctx.sh:45`).
+- kv `#tag` drops make strict `reward` 0 while values are right; compare `reward_lenient`, rerun format-invalid trials with `bench/rerun_strict.sh`.
+- `ctx_sched.py` skips tasks whose dir is missing (`ctx/skipped.txt`) and only exits when `ctx/STOP` exists.
+- Results dirs contain root-owned files from Docker; write summaries elsewhere (`bench/ctx/`), never edit `results-*`.
+- `needle_retention` grades the final context window; do not run it against any trimming arm.
+- Counts in log-triage overcount when the agent's own output echoes log lines (`context-research.md` "Echo overcount"); unchanged in run E, reported separately as `count` sub-score.
+- The proxy's `fail_open` event means an untrimmed request went out silently (`core.py:355`); check `events` per session before scoring an e-smart trial.
+- `latency_s` in `usage.jsonl` may or may not include the pre-call hook; verify in `tokens.py:record_usage` before claiming H6.
+
+## 5. Docs to update after the runs
+
+- `docs/context-research.md`: new "Run E" section (method, every trial, CIs, exclusions), "Run E-S" (Sonnet), "Run E-D" (decision task), Limitations rewritten, Next steps pruned.
+- `docs/context-management.md`: Benchmarks tables and Time/Cost sections regenerated from run E (same-run baselines; drop the cross-run caveat at line 93); the "three changes since" block (lines 125-139) folds into the main table; scratchpad tool documented.
+- `site/scripts/bench/compute-contextbench.py:27-29`: point at `results-E/state-E` for all three arms; add per-model prices; `cd site && pnpm media` regenerates `contextbench-*.png`; `site/src/data/contextbench.json` committed.
+- `README.md:30-36`: chart alt text and link; today the chart is run D (120k trim, no pointer) while the text quotes Df, fix by regenerating from run E. If H1–H3 fail, replace the chart with the null result sentence.
+- `CLAUDE.md`: bench section mentions run E layout and `bench/ci.py`.

package/docs/hindsight.md

@@ -40,12 +40,15 @@ Shipped route templates live in `config/litellm.lemond.yaml` and `config/litellm
 npx @bcmyguest/openmem install-coding-agent --yes
 ```
 
-**Optional:** also install the upstream Hindsight plugin hooks:
+**Optional:** use the upstream Hindsight plugin hooks instead of OpenMem’s. Hook sets are exclusive; the choice is saved as `[agents] integration = "provider"`, OpenMem’s hooks are removed, and the unified MCP stays:
 
 ```sh
-npx @bcmyguest/openmem install-coding-agent --yes --hindsight-plugin
+npx @bcmyguest/openmem install-coding-agent --yes --provider-plugin
+npx @bcmyguest/openmem install-coding-agent --yes --no-provider-plugin   # back to OpenMem hooks
 ```
 
+`--hindsight-plugin` was renamed to `--provider-plugin`; the old flag exits with an error.
+
 MCP without session hooks:
 
 ```sh

package/docs/mcp-setup.md

@@ -2,11 +2,14 @@
 
 OpenMem registers **stdio MCP servers** in `~/.cursor/mcp.json` (and wires session hooks via `install-coding-agent`). Day-to-day you want the **unified** `openmem` server (`memory_*` tools). Provider-native and NotebookLM servers are optional add-ons.
 
-For why this exists, see [Concepts](concepts.md). Install path: [Quickstart § Wire your coding agent](quickstart.md#3-wire-your-coding-agent-optional).
+For why this exists, see [Concepts](concepts.md). Install path: [Quickstart § Wire your coding agent](quickstart.md#3-wire-your-coding-agent-optional). Per-agent support: [Compatibility matrix](compatibility.md).
 
 ## Quick path (recommended)
 
+Run the wizard first. The MCP server only forwards `memory_*` calls to a running provider; without the stack and `wizard.conf` it has nothing to talk to.
+
 ```sh
+npx @bcmyguest/openmem            # wizard: provider, stack, smoke test
 npx @bcmyguest/openmem install-coding-agent --yes
 ```
 
@@ -31,7 +34,7 @@ Restart **Cursor Agent** (or your agent’s MCP host) after changes.
 | **Provider-native** (`mem0`, `cognee`) | `install-mcp --provider-only` | Upstream / repo-specific names | Same HTTP API, different tool surface |
 | **`notebooklm`** | When NotebookLM sync is on, or `--notebooklm` | `notebooklm_list_banks`, `notebooklm_search` | Search over synced notebooks on the **active** provider — not a memory engine |
 
-Hindsight’s optional in-editor plugin (`--hindsight-plugin`) is **not** an MCP entry; it complements unified MCP on Hindsight stacks. See [Hindsight](hindsight.md).
+Hindsight’s optional in-editor plugin (`install-coding-agent --provider-plugin`) is **not** an MCP entry; it replaces OpenMem’s session hooks on Hindsight stacks and keeps the unified MCP. See [Hindsight](hindsight.md).
 
 ## When to use which
 

package/docs/mem0.md

@@ -15,7 +15,7 @@ Upstream project: [mem0ai/mem0](https://github.com/mem0ai/mem0). Stack details f
 | Bind so the indexer container can reach the server | Wizard smoke: docs, retain, search, delete-by-id |
 | | Unified MCP + optional native mem0 MCP |
 
-Compose profile `mem0` adds env + `extra_hosts` on the shared **indexer** only — there is no mem0 container in this repo yet.
+Compose profile `mem0` adds env + `extra_hosts` on the shared **indexer** only — there is no mem0 container in this repo yet. The optional profile `litellm` adds the stack LiteLLM on loopback for mem0's LLM ([below](#models-through-the-stack-litellm-optional)).
 
 ## First-time setup
 
@@ -55,6 +55,44 @@ curl -fsS "$MEM0_API_URL/docs"   # URL from wizard / stack .env
 
 Search fails if the server’s embedder is misconfigured; smoke stores with `infer: false` and looks for that fact.
 
+## Models through the stack LiteLLM (optional)
+
+mem0 picks its own LLM and embedder. The wizard can instead route mem0's LLM through the stack's LiteLLM, the gateway Hindsight uses, so a model change (Claude included) is one LiteLLM route. In step 3 answer yes to "Route mem0's LLM through the stack LiteLLM", or run `./setup --yes` with `MEM0_LITELLM=1`.
+
+| Backend | mem0 LLM (`memory-llm`) | Embeddings (`memory-embed`) | Key in `litellm.env` |
+| --- | --- | --- | --- |
+| Ollama | `ollama_chat/<model>` (default `gemma4:e2b`) | `ollama/nomic-embed-text` by default (`none` keeps mem0's embedder) | none |
+| Custom OpenAI-compatible URL | `openai/<model>` | optional `openai/<model>` (e.g. `text-embedding-3-small`) | `LEMOND_API_KEY` (same variable as Hindsight's custom route) |
+| Anthropic API key | `anthropic/<model>` (default `claude-haiku-5-5`) | none: mem0 keeps its own embedder | `ANTHROPIC_API_KEY` |
+| Claude subscription (Meridian) | `anthropic/<model>` at Meridian ([guide](meridian-litellm.md)); use is governed by Anthropic's [Consumer Terms](https://www.anthropic.com/legal/consumer-terms) | none | `MERIDIAN_API_KEY` |
+
+What the wizard does:
+
+- writes `~/.config/personal-memory/mem0-litellm.yaml` (routes only; keys are `os.environ/…` references) and points `LITELLM_CONFIG` at it;
+- keeps `LITELLM_MASTER_KEY` and provider keys in `~/.config/hindsight/litellm.env` (mode 600; `LITELLM_ENV_FILE` overrides), creating an `sk-` master key when the file is new;
+- sets `COMPOSE_PROFILES=mem0,litellm`. `litellm` has its own compose profile; `providers/mem0/compose.yaml` publishes it on **`127.0.0.1:${MEM0_LITELLM_PORT:-4000}` only** (the wizard keeps 4000 when free, else offers the next free port) and its entrypoint exits unless `LITELLM_MASTER_KEY` starts with `sk-`;
+- prints the mem0 config blocks below. It does **not** edit your mem0 server config.
+
+```json
+{
+  "llm": {"provider": "openai", "config": {"model": "memory-llm", "openai_base_url": "http://127.0.0.1:4000/v1", "api_key": "<LITELLM_MASTER_KEY>"}},
+  "embedder": {"provider": "openai", "config": {"model": "memory-embed", "openai_base_url": "http://127.0.0.1:4000/v1", "api_key": "<LITELLM_MASTER_KEY>"}},
+  "vector_store": {"config": {"embedding_model_dims": 768}}
+}
+```
+
+The `embedder` and `vector_store` blocks appear only when the backend serves embeddings (768 is `nomic-embed-text`'s size). Apply the blocks yourself: `POST /configure` on the REST server with an admin key, or merge them into the dict you pass to `Memory.from_config`.
+
+Why these keys (mem0 `main` at [26cdefb](https://github.com/mem0ai/mem0/tree/26cdefb46600c0fa5953c4b5ee5003c125539fc4)):
+
+- The self-hosted REST server bundles only the `openai`, `anthropic` and `gemini` LLM providers and the `openai` and `gemini` embedders ([server/main.py L62-63](https://github.com/mem0ai/mem0/blob/26cdefb46600c0fa5953c4b5ee5003c125539fc4/server/main.py#L62-L63)), so the snippet uses provider `openai` against LiteLLM rather than mem0's `litellm` provider ([docs](https://docs.mem0.ai/components/llms/models/litellm)), which also passes no base URL to `litellm.completion` ([mem0/llms/litellm.py](https://github.com/mem0ai/mem0/blob/26cdefb46600c0fa5953c4b5ee5003c125539fc4/mem0/llms/litellm.py)).
+- The `openai` LLM uses `openai_base_url` and `api_key` from its config, but switches to OpenRouter when `OPENROUTER_API_KEY` is set ([mem0/llms/openai.py](https://github.com/mem0ai/mem0/blob/26cdefb46600c0fa5953c4b5ee5003c125539fc4/mem0/llms/openai.py), [config](https://docs.mem0.ai/components/llms/config)).
+- The `openai` embedder reads `openai_base_url`, `api_key` and `model` ([mem0/embeddings/openai.py](https://github.com/mem0ai/mem0/blob/26cdefb46600c0fa5953c4b5ee5003c125539fc4/mem0/embeddings/openai.py), [config](https://docs.mem0.ai/components/embedders/config)). Leaving `embedding_dims` unset keeps it from sending a `dimensions` parameter.
+- pgvector's collection size is `vector_store.config.embedding_model_dims` (default 1536, [mem0/configs/vector_stores/pgvector.py](https://github.com/mem0ai/mem0/blob/26cdefb46600c0fa5953c4b5ee5003c125539fc4/mem0/configs/vector_stores/pgvector.py)). A different embedder needs a new collection or re-added memories.
+- `POST /configure` deep-merges the body into the running config and persists it ([server/main.py L332-337](https://github.com/mem0ai/mem0/blob/26cdefb46600c0fa5953c4b5ee5003c125539fc4/server/main.py#L332-L337), `update_config` in `server/server_state.py`).
+
+Limits: `127.0.0.1` reaches LiteLLM only from a mem0 server running on this host; a mem0 server inside another container needs host networking or a different route. Claude has no embedding model, so the Anthropic and Meridian backends route the LLM only. Whether mem0's extraction prompts (JSON `response_format`) work with every routed model is not verified here; test with an `infer: true` add after switching.
+
 ## Banks / scopes
 
 mem0 scopes map to **`user_id`**. OpenMem’s MCP `bank` argument and indexer banks become that `user_id`. Default unified MCP scope is `openmem-mcp` unless tools pass `bank`. Project work should pass the resolved project bank so session noise does not land in the default. See [Concepts — Context and banks](concepts.md#context-and-banks).

package/docs/memory-providers.md

@@ -65,7 +65,7 @@ after meta steps — like asdf plugin install scripts, not reimplemented npm/mem
 
 NotebookLM sync is **orthogonal** to which provider you picked: enable the `notebooklm` profile and the sync container uses the same `MEMORY_SINK` as the indexer (Hindsight, mem0, or Cognee).
 
-Coding-agent session memory uses the **same** in-repo hooks and unified MCP for every provider; optional `--hindsight-plugin` adds the upstream Hindsight plugin on Hindsight stacks only.
+Coding-agent session memory uses the **same** in-repo hooks and unified MCP for every provider; on Hindsight stacks `--provider-plugin` swaps the in-repo hooks for the upstream Hindsight plugin (`[agents] integration = "provider"`).
 
 ## Provider manifest
 
@@ -143,7 +143,7 @@ first step); default action is full **configure and start**.
 
 ### Changing provider later
 
-Prefer **`npx @bcmyguest/openmem switch <id> --yes`** (checkout: `npx . switch <id> --yes`) over a full
+Prefer **`npx @bcmyguest/openmem switch <id> --yes`** (checkout: `node cli/openmem.mjs switch <id> --yes`) over a full
 interactive setup re-run when you only need to change backends:
 
 - Updates `MEMORY_PROVIDER` in `~/.config/hindsight/wizard.conf`

package/docs/meridian-litellm.md

@@ -0,0 +1,198 @@
+# Meridian route (Claude subscription)
+
+Route the three Hindsight aliases through [Meridian](https://github.com/rynfar/meridian), a local proxy that serves a signed-in Claude subscription over the Anthropic Messages and OpenAI Chat Completions APIs. Hindsight still calls `http://litellm:4000/v1`; only the override YAML changes.
+
+```
+hindsight ──openai──▶ litellm:4000 ──anthropic/──▶ host.docker.internal:3456 (Meridian) ──Agent SDK──▶ Claude subscription
+```
+
+Checked against Meridian 1.87.0 docs and LiteLLM docs on 2026-10-29. Nothing below was run end to end; steps marked **unverified** rest on docs alone.
+
+## Prerequisites
+
+| Need | Why |
+| --- | --- |
+| Node.js 22+ | [Meridian quick start](https://github.com/rynfar/meridian#quick-start) |
+| Claude Code CLI, signed in (`claude login`) | Meridian runs requests through the Agent SDK with those credentials. Haiku 5.5 needs Claude Code 2.1.293+ ([Haiku 5.5](https://github.com/rynfar/meridian/blob/main/docs/configuration.md#haiku-55)) |
+| Claude Pro or Max plan | Requests count against plan limits |
+| OpenMem stack set up | `node scripts/setup.mjs` has run once |
+
+## Run Meridian
+
+```sh
+npm install -g @rynfar/meridian
+claude login
+```
+
+Meridian binds `127.0.0.1:3456` by default. The LiteLLM container reaches the host through `host.docker.internal` (`extra_hosts: host-gateway` in `compose.yaml`), which resolves to the Docker bridge address, not loopback, so a loopback-only Meridian is unreachable from the container on Linux. Bind all interfaces and require a key:
+
+| Variable | Value | Source |
+| --- | --- | --- |
+| `MERIDIAN_HOST` | `0.0.0.0` (default `127.0.0.1`) | [Configuration table](https://github.com/rynfar/meridian/blob/main/docs/configuration.md#configuration) |
+| `MERIDIAN_PORT` | `3456` (default) | same |
+| `MERIDIAN_API_KEY` | long random secret. Clients send it as `x-api-key` or `Authorization: Bearer`; `/` and `/health` stay open | [API Key Authentication](https://github.com/rynfar/meridian/blob/main/docs/configuration.md#api-key-authentication) |
+
+`0.0.0.0` also exposes port 3456 on every host interface; firewall it to the Docker bridge if the machine sits on a shared network.
+
+```sh
+install -d -m 700 ~/.config/meridian
+printf 'MERIDIAN_HOST=0.0.0.0\nMERIDIAN_API_KEY=%s\n' "$(openssl rand -hex 32)" > ~/.config/meridian/meridian.env
+chmod 600 ~/.config/meridian/meridian.env
+set -a; . ~/.config/meridian/meridian.env; set +a
+meridian
+```
+
+### Keep it running
+
+Meridian documents a Nix Home Manager user service and a [systemd socket-activation example](https://github.com/rynfar/meridian/blob/main/docs/configuration.md#systemd-socket-activation). For an npm install it ships `bin/meridian-launchd.sh`, a POSIX launcher it documents as usable "as a systemd `ExecStart`" ([Running as a service without drift](https://github.com/rynfar/meridian/blob/main/docs/configuration.md#running-as-a-service-without-drift)); the launcher self-updates at start unless `MERIDIAN_NO_SELF_UPDATE=1`.
+
+The unit below is **unverified** (written from those docs, not tested). Replace `ExecStart` with the output of `echo "$(npm root -g)/@rynfar/meridian/bin/meridian-launchd.sh"`, and set `PATH` so the unit finds the same `node` and `npm` as your shell (nvm and volta installs live outside the systemd default `PATH`).
+
+```ini
+# ~/.config/systemd/user/meridian.service
+[Unit]
+Description=Meridian proxy (Claude subscription)
+
+[Service]
+EnvironmentFile=%h/.config/meridian/meridian.env
+Environment=PATH=/usr/local/bin:/usr/bin:/bin
+ExecStart=/usr/lib/node_modules/@rynfar/meridian/bin/meridian-launchd.sh
+Restart=on-failure
+
+[Install]
+WantedBy=default.target
+```
+
+```sh
+systemctl --user daemon-reload
+systemctl --user enable --now meridian
+loginctl enable-linger "$USER"   # keep it running while logged out
+```
+
+**Docker alternative (unverified).** Meridian publishes no image; build it from the repo (`docker build -t meridian .`) and mount `~/.claude` ([Docker, single account](https://github.com/rynfar/meridian/blob/main/docs/deployment.md#single-account)). The image already sets `CLAUDE_PROXY_HOST=0.0.0.0`. Attach it to the stack network with `docker network connect openmem_default meridian` and use `api_base: http://meridian:3456`; nothing is then published to the host. Set `MERIDIAN_HOST_ID` and persist the volumes listed under [Persistence and host identity](https://github.com/rynfar/meridian/blob/main/docs/deployment.md#persistence-and-host-identity).
+
+## Provider choice: `anthropic/`
+
+| | `anthropic/<model>` → `/v1/messages` | `openai/<model>` → `/v1/chat/completions` |
+| --- | --- | --- |
+| Meridian path | Primary API; Meridian's docs describe LiteLLM on this route (`passthrough` adapter, [Claude Code behind a gateway](https://github.com/rynfar/meridian/blob/main/docs/configuration.md#claude-code-behind-a-gateway)) | Supported; headerless tool loops get a derived session key ([OpenAI-compatible tools](https://github.com/rynfar/meridian/blob/main/docs/agents.md#openai-compatible-tools-open-webui-continue-etc)) |
+| Tool calls (`memory-reflect`) | LiteLLM translates OpenAI `tools` to Anthropic `tool_use` ([function calling](https://docs.litellm.ai/docs/providers/anthropic#functiontool-calling)); Meridian returns calls to the client in passthrough mode ([Passthrough Mode](https://github.com/rynfar/meridian/blob/main/docs/configuration.md#passthrough-mode-and-tool-calling)) | Meridian translates OpenAI tools itself |
+| `reasoning_effort` | LiteLLM maps it to `thinking` (budget 1024/2048/4096, or adaptive + `output_config.effort` on 4.6+ models) ([thinking](https://docs.litellm.ai/docs/providers/anthropic#usage---thinking--reasoning)). Drop it, below | Meridian's handling undocumented |
+| `response_format` | LiteLLM converts it to native `output_format` on models it knows, else to a forced tool call ([structured outputs](https://docs.litellm.ai/docs/providers/anthropic#structured-outputs)). Meridian's handling of `output_format` is **unverified** | Meridian's handling undocumented |
+
+Use `anthropic/`: it is the route Meridian documents for LiteLLM and gives the request shape LiteLLM translates most predictably. Fall back to `openai/<model>` with `api_base: http://host.docker.internal:3456/v1` if retain fails on structured output.
+
+LiteLLM appends `/v1/messages` to `api_base`, so `api_base` has no `/v1` ([Custom API Base](https://docs.litellm.ai/docs/providers/anthropic#custom-api-base)). LiteLLM sends `api_key` as `x-api-key`, which Meridian accepts.
+
+## Thinking and token limits
+
+The shipped `enable_thinking: false` is a llama.cpp chat-template flag; Claude ignores it. Remove it.
+
+| Setting | Behavior through Meridian |
+| --- | --- |
+| Thinking off | Meridian refuses explicit `thinking: disabled` on Haiku 5.5 with HTTP 400 before generation ([Haiku 5.5](https://github.com/rynfar/meridian/blob/main/docs/configuration.md#haiku-55)). Send no thinking parameter; `additional_drop_params: ["reasoning_effort"]` stops LiteLLM generating one from Hindsight's `reasoning_effort` ([Specify params to drop](https://docs.litellm.ai/docs/completion/drop_params#specify-params-to-drop)) |
+| `HINDSIGHT_API_*_MAX_COMPLETION_TOKENS`, LiteLLM `max_tokens` | Ignored by default: the Agent SDK has no output cap, so Meridian does not bound the answer ([Known limitations](https://github.com/rynfar/meridian/blob/main/docs/configuration.md#known-limitations)). Reflect tool calls are never truncated, and output is unbounded |
+| `MERIDIAN_ENFORCE_MAX_TOKENS=1` | Enforces the cap on thinking plus text. With reflect at 2048, adaptive thinking can consume the budget, and a cap that lands mid tool call fails the request. Leave unset |
+| `temperature`, `top_p` | Meridian's Claude docs do not say whether these reach the model; assume ignored (**unverified**). The README's "does not expose … sampling controls" sentence describes the Antigravity CLI backend, not Claude |
+
+## Route file
+
+`~/.config/hindsight/litellm.yaml` replaces the backend route entirely. Model ids must appear in Meridian's `GET /v1/models`; `claude-haiku-5-5` is Meridian's `haiku` default and `claude-sonnet-5-5` appears in its client examples ([Pi](https://github.com/rynfar/meridian/blob/main/docs/agents.md#pi)). Reflect gets the larger model, as in the shipped routes.
+
+```yaml
+# LiteLLM routes for Hindsight, backend: Meridian on the host (Claude subscription).
+# See docs/meridian-litellm.md. MERIDIAN_API_KEY lives in ~/.config/hindsight/litellm.env.
+model_list:
+  - model_name: memory-retain
+    litellm_params:
+      model: anthropic/claude-haiku-5-5
+      api_base: http://host.docker.internal:3456
+      api_key: os.environ/MERIDIAN_API_KEY
+      max_tokens: 4096
+      timeout: 300
+      additional_drop_params: ["reasoning_effort"]
+  - model_name: memory-reflect
+    litellm_params:
+      model: anthropic/claude-sonnet-5-5
+      api_base: http://host.docker.internal:3456
+      api_key: os.environ/MERIDIAN_API_KEY
+      max_tokens: 4096
+      timeout: 300
+      additional_drop_params: ["reasoning_effort"]
+  - model_name: memory-consolidate
+    litellm_params:
+      model: anthropic/claude-haiku-5-5
+      api_base: http://host.docker.internal:3456
+      api_key: os.environ/MERIDIAN_API_KEY
+      max_tokens: 4096
+      timeout: 300
+      additional_drop_params: ["reasoning_effort"]
+
+general_settings:
+  master_key: os.environ/LITELLM_MASTER_KEY
+```
+
+## With the wizard
+
+`./setup` step "3. Model backend" option 4, **Claude subscription (Meridian)**, automates [Route file](#route-file) and [Apply](#apply). It does not install or start Meridian; run it first as in [Run Meridian](#run-meridian).
+
+| Step | What the wizard does |
+| --- | --- |
+| Ask | Meridian URL from the host (default `http://127.0.0.1:3456`, port from `MERIDIAN_PORT` in `~/.config/meridian/meridian.env`) and an optional `MERIDIAN_API_KEY` (blank reuses the environment, `meridian.env`, or `litellm.env`) |
+| Probe | `GET /health` and `GET /v1/models` (with `x-api-key`) from the host, then `/health` on the `docker0` address. Answering on 127.0.0.1 but not on `docker0` means a loopback-only bind: it warns to set `MERIDIAN_HOST=0.0.0.0`. Without `docker0` it states the requirement |
+| Models | Retain and consolidate: `claude-haiku-5-5`, reflect: `claude-sonnet-5-5`, when listed; otherwise the first listed Haiku and Sonnet ids. Unreachable Meridian: the guide defaults |
+| Write | The route file above as `~/.config/hindsight/litellm.yaml` (an existing one is copied to `litellm.yaml.bak-<timestamp>`), `MERIDIAN_API_KEY` in `litellm.env` (mode 600; `placeholder` when Meridian runs without a key), and `[model] backend = "meridian"` in `wizard.conf` and `config.toml` |
+
+Non-interactive: `PM_BACKEND=meridian MERIDIAN_BASE=http://127.0.0.1:3456 MERIDIAN_API_KEY=... ./setup --yes`. `[model.meridian] host` in `config.toml` sets `MERIDIAN_BASE`; `PM_DEFAULT_MODEL` and `PM_REFLECT_MODEL` pick models. `--dry-run` probes and prints the plan without writing. Then run the [Smoke test](#smoke-test); the wizard's host-side probe does not prove the container reaches Meridian through a firewall.
+
+## Apply
+
+```sh
+cd ~/personal-memory   # this checkout
+[ -f ~/.config/hindsight/litellm.yaml ] && cp ~/.config/hindsight/litellm.yaml ~/.config/hindsight/litellm.yaml.bak-$(date +%Y%m%d%H%M%S)
+$EDITOR ~/.config/hindsight/litellm.yaml          # paste the route file
+grep -h '^MERIDIAN_API_KEY' ~/.config/meridian/meridian.env >> ~/.config/hindsight/litellm.env
+node scripts/setup.mjs
+docker compose up -d --force-recreate litellm
+```
+
+`--force-recreate` matters: Compose does not see edits inside the mounted YAML, so a plain `up -d` can keep the old routes. Revert by moving the override aside and rerunning the last two commands.
+
+## Smoke test
+
+LiteLLM publishes no host port; run curl on the stack network (`openmem_default`, from `name: openmem`).
+
+```sh
+set -a; . ~/.config/hindsight/litellm.env; set +a
+net="--network openmem_default --add-host host.docker.internal:host-gateway"
+
+# 1. Meridian reachable from a container, models listed
+docker run --rm $net curlimages/curl -s http://host.docker.internal:3456/health
+docker run --rm $net curlimages/curl -s -H "x-api-key: $MERIDIAN_API_KEY" http://host.docker.internal:3456/v1/models
+
+# 2. Alias answers through LiteLLM
+docker run --rm $net curlimages/curl -s http://litellm:4000/v1/chat/completions \
+  -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' \
+  -d '{"model":"memory-retain","messages":[{"role":"user","content":"Reply with OK."}]}'
+
+# 3. Reflect returns a tool call (expect finish_reason "tool_calls")
+docker run --rm $net curlimages/curl -s http://litellm:4000/v1/chat/completions \
+  -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json' \
+  -d '{"model":"memory-reflect","reasoning_effort":"low","messages":[{"role":"user","content":"What is the weather in Paris? Use the tool."}],
+       "tools":[{"type":"function","function":{"name":"get_weather","description":"Weather for a city",
+       "parameters":{"type":"object","properties":{"city":{"type":"string"}},"required":["city"]}}}]}'
+```
+
+Then run a smoke retain and reflect through Hindsight, as for any route change ([Model configuration](model-configuration.md#verify-and-migrate)).
+
+## Caveats
+
+| Topic | Detail |
+| --- | --- |
+| Terms | Subscription use through third-party proxies is governed by Anthropic's [Consumer Terms](https://www.anthropic.com/legal/consumer-terms); see also Claude Code's [Authentication and credential use](https://code.claude.com/docs/en/legal-and-compliance#authentication-and-credential-use) |
+| Third-party detection | Meridian reports Claude refusing Pi traffic as third-party ("Third-party apps now draw from your extra usage, not your plan limits") until Pi's prompt fingerprint is scrubbed ([Pi](https://github.com/rynfar/meridian/blob/main/docs/agents.md#pi)). Whether Hindsight's prompts trigger the same is **unverified**; watch Meridian's `/telemetry` |
+| Plan limits | Retain runs on every ingest; indexing a large repo spends plan quota. `GET /v1/usage/quota` shows the windows |
+| Concurrency | Meridian serializes requests that share a session identity and can return `400 This session advanced while the request was waiting` ([Concurrent requests](https://github.com/rynfar/meridian/blob/main/docs/configuration.md#concurrent-requests-to-the-same-session)). Keep `HINDSIGHT_API_LLM_MAX_CONCURRENT=1` (shipped default) |
+| System prompt | Meridian can prepend Claude Code's own system prompt per adapter (SDK feature toggles at `/settings`). An [adapter instance](https://github.com/rynfar/meridian/blob/main/docs/agents.md#adapter-instances) matching `userAgentPrefix: "litellm/"` can turn it off; effect on Hindsight output is **unverified** |
+| Exposure | `MERIDIAN_HOST=0.0.0.0` without `MERIDIAN_API_KEY` gives anyone on the network your subscription |
+| Server tools | Anthropic server tools (`web_search_*`, `web_fetch_*`) return 400 through Meridian; Hindsight does not use them |

package/docs/model-configuration.md

@@ -16,6 +16,7 @@ Edit the YAML (or a per-machine override) to change models.
 | --- | --- | --- |
 | `config/litellm.lemond.yaml` | lemond on the host | Reflect: Gemma 4 E4B GGUF (`unsloth/`); retain and consolidate: E2B |
 | `config/litellm.ollama.yaml` | Ollama on the host | Used when `MEMORY_BACKEND=ollama`; `gemma4:e4b` needs Ollama newer than 0.20.3 |
+| `~/.config/hindsight/litellm.yaml` (written by `./setup`, backend `meridian`) | Meridian on the host (Claude subscription) | Retain and consolidate: `claude-haiku-5-5`; reflect: `claude-sonnet-5-5`. See [Meridian route](meridian-litellm.md#with-the-wizard) |
 
 **Why reflect uses a larger model:** E2B often returns no usable tool call on Hindsight reflect (`finish_reason=length`). The shipped lemond route uses E4B for `memory-reflect` only. Qwen3.5-4B works as an alternative if you change the route; pick any model that passes a direct tool-call check and a smoke reflect.
 
@@ -51,6 +52,16 @@ Thinking is off on all three aliases in the route YAML (`enable_thinking: false`
 
 Retain cap must exceed each retain strategy's chunk size (coding-agent strategies use 12000) or Hindsight returns `400 Invalid retain strategy`.
 
+## Retain extraction mode
+
+OpenMem’s shipped `config/hindsight.env.example` sets:
+
+```env
+HINDSIGHT_API_RETAIN_EXTRACTION_MODE=verbatim
+```
+
+**verbatim** keeps each chunk’s text as the memory unit and still calls the retain LLM for entities, time, and location (cheaper than default `concise` fact rewriting). Alternatives: `concise` / `verbose` (LLM facts), `chunks` (no LLM on ingest; pass entities yourself if needed), or `HINDSIGHT_API_LLM_PROVIDER=none` (chunks-only; reflect disabled). Per-bank strategies can override this (the indexer already uses `chunks` for `project-index`).
+
 ## Secrets
 
 `~/.config/hindsight/litellm.env` (mode 600): `LITELLM_MASTER_KEY` (Hindsight uses this as `HINDSIGHT_API_LLM_API_KEY`), plus provider keys referenced as `os.environ/NAME` in the YAML.
@@ -66,6 +77,17 @@ Example hosted reflect model:
 
 Hindsight always sends the alias names above; you do not edit model ids in `hindsight.env` for route changes.
 
+## mem0 through the same LiteLLM
+
+The `litellm` service carries its own compose profile `litellm` next to `hindsight` and `notebooklm`, so Hindsight's stack renders and starts as before. mem0's wizard can enable that profile: LiteLLM then runs without Hindsight, published on `127.0.0.1:${MEM0_LITELLM_PORT:-4000}` only, from `~/.config/personal-memory/mem0-litellm.yaml`, with the same `litellm.env` and master key.
+
+| Alias | mem0 use |
+| --- | --- |
+| `memory-llm` | mem0's LLM (fact extraction and memory updates) |
+| `memory-embed` | mem0's embedder, only for backends with an embedding model (Ollama `nomic-embed-text`, an OpenAI-compatible server) |
+
+Claude (Anthropic API key or Meridian) has no embedding model; with those backends mem0 keeps its own embedder. The mem0 config keys and their sources are in [mem0 — Models through the stack LiteLLM](mem0.md#models-through-the-stack-litellm-optional).
+
 ## Verify and migrate
 
 Health check from the compose network: