PyPI - mnemonic-proxy - Versions diffs - 0.3.2__tar.gz → 0.3.9__tar.gz - Mend

mnemonic-proxy 0.3.2tar.gz → 0.3.9tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (73) hide show

{mnemonic_proxy-0.3.2 → mnemonic_proxy-0.3.9}/PKG-INFO RENAMED Viewed

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mnemonic-proxy
-Version: 0.3.2
+Version: 0.3.9
 Summary: Transparent proxy that lets local coding agents work for hours: verbatim archive, structured recall, segments with handoff notes, saved engine state.
 Author: Maurizio Verde
 License-Expression: MIT
@@ -64,32 +64,98 @@ OpenAI-compatible backends; saved engine state depends on what the backend suppo
 Project page: <https://lastreload.github.io/mnemonic-proxy/>
-## One session, measured
+Measured on our hardware (single runs, see [What we have not proven](#what-we-have-not-proven)): a 3 h 15 min coding
+session on one 12 GB GPU with 448K tokens of history and at most 105K in the prompt
+([One session, measured](#one-session-measured)); AgentLongBench in a 32K window, 86/100 right with the proxy vs 47
+with an LLM summary and 69 with the full history in a larger window
+([benchmark](#benchmark-agentlongbench-with-a-32k-window)); a 110K-token conversation resumed from disk in 1.4 s
+instead of ≈50 s ([Saved state, measured](#saved-state-measured)).
-A real coding session — a canvas tower-defense game built from scratch by a 35B-A3B MoE model on a single 12 GB
-RTX 4070 Ti — with the proxy in between.
+## Install
-<p align="center">
-  <img src="https://raw.githubusercontent.com/lastreload/mnemonic-proxy/main/docs/assets/replay.gif" alt="Replay: a question answered from three hours earlier via one recall call" width="100%">
-</p>
+The first time: install, watch it work with no model, check your engine, start.
-<p align="center"><sub>Replay of a real exchange (animated GIF; <a href="https://raw.githubusercontent.com/lastreload/mnemonic-proxy/main/docs/assets/replay.mp4">MP4</a>).
-Three hours and two segment switches into the session, one <code>recall</code> call — resolved inside the proxy in
-61 ms — brings back the verbatim first version of the boss block, message 20. No model summarized anything.</sub></p>
+```sh
+python3 -m venv mnemonic-venv && . mnemonic-venv/bin/activate
+pip install mnemonic-proxy
+mnemonic-proxy demo      # 1 minute, no model: archive, masking, recall -> PASS
+mnemonic-proxy check --upstream http://127.0.0.1:8095 --port 8096 --data ./data --config config.json
+mnemonic-proxy --upstream http://127.0.0.1:8095 --port 8096 --data ./data --config config.json
+```
-| | |
-|---|---|
-| **448K** | tokens of history archived and recallable |
-| **105K** | peak tokens actually in the prompt (window: 128K) |
-| **3h15** | continuous agent work · 430 requests · 0 errors |
-| **1.4 s** | to resume a 110K-token conversation from disk (≈50 s re-reading it) |
+`check` ends with `READY` (exit 0) or says, line by line, what to change; `--live` adds a real generation, a tool
+call and a save/restore. `--upstream` is the engine's address. Then point the agent at the proxy (port 8096)
+instead of the engine. The engine window must be at least 16K tokens, 32K or more recommended
+([Requirements](#requirements)). Step by step with a small model on CPU: [Quick start](#quick-start).
-<p align="center">
-  <img src="https://raw.githubusercontent.com/lastreload/mnemonic-proxy/main/docs/assets/session-chart.svg" alt="Full history vs. prompt actually read, per request" width="100%">
-</p>
+**Your engine.** Start it as below, copy the config from
+[`examples/`](https://github.com/lastreload/mnemonic-proxy/tree/main/examples) and set `slot_dir` to the engine's
+save folder:
-Magenta: the full history of the session. Cyan: what the model actually reads on each request. Dashed lines: the two
-segment switches (≈2 minutes each, written handoff notes). Data from the proxy journal, 5 Oct 2026.
+| engine | engine command (minimum) | config | saved state |
+|---|---|---|---|
+| Strata ≥ 0.1.40 | Strata's own start, plus `--slot-save-path DIR` | `config.example.json`, `slot_dir` = DIR | yes |
+| Strata older, or without `--slot-save-path` | Strata's own start | `config.official-strata.json` | no (base mode) |
+| llama-server | `llama-server -m model.gguf -c 32768 --parallel 1 --jinja --slot-save-path ./slots --port 8095` | `config.llama-server.json` | yes |
+| llama-server, router mode (several models) | `llama-server --models-preset preset.ini --models-max 1 --port 8095`; preset: `[*]` with `slot-save-path` and `parallel = 1`, a `ctx-size` (`c`) ≥ 16384 per model | `config.llama-server-router.json` + `preset.router.example.ini` | yes, per model |
+| ds4-server | `ds4-server -m MODEL.gguf --ctx 65536 --kv-disk-dir ./kv --port 8110` | `config.ds4.json` | kept by ds4 itself |
+| Ollama, LM Studio | engine context set to 32768 (see below); `--tokenizer` recommended | `config.openai-32k.json`, or no `window` (read from the engine) | no (base mode) |
+| any OpenAI-compatible (vLLM, online APIs) | — | `config.openai-32k.json` at 32K; at 131072 none is needed (the defaults are for 131072) | no (base mode) |
+The proxy reads the window from Strata, llama-server (`n_ctx`, per model in router mode), ds4-server (`--ctx`),
+Ollama and LM Studio (the context of the loaded model, 0.3.8), and scales its thresholds to it, unless `window` is
+written in the config (the two Strata examples set 131072: change it if Strata runs with another window). From any
+other engine it cannot: `window` in the config is used as it is (131072 when missing), so write the engine's real
+context there; `config.openai-32k.json` has the thresholds the proxy itself uses at 32K. Details per engine:
+[Engines](#engines).
+**Ollama and LM Studio** (tested on Linux, small model, details in [Tested combinations](#tested-combinations)). Both
+serve an OpenAI-compatible API (Ollama `http://127.0.0.1:11434/v1`, LM Studio `http://127.0.0.1:1234/v1`;
+`--upstream` is the same address without `/v1`) and the proxy runs in base mode: masking, archive, `recall`, segment
+switches, tool calls, streaming, `/v1/messages` and `/v1/responses` worked, no saved state. What to know:
+- **Give the engine a context of 32768 (at least 16384).** With their defaults on our machine (Ollama 4096 on CPU;
+  LM Studio 8192) both answer HTTP 200 to longer prompts after dropping part of them, without telling the client:
+  Ollama drops the oldest messages, LM Studio keeps the beginning and the end and drops the middle.
+  Ollama: `OLLAMA_CONTEXT_LENGTH=32768 ollama serve`; `ollama ps` shows the context in use (column `CONTEXT`).
+  LM Studio: `lms load <model> --context-length 32768`, then `lms server start`; `lms ps` shows it.
+- **The proxy reads that context** (0.3.8): Ollama `GET /api/ps` (`context_length` of the loaded model: the context
+  it runs with, not the training context of `POST /api/show`), LM Studio `GET /api/v0/models`
+  (`loaded_context_length`). It reads it again before every request (one cheap GET; a model that is not loaded is
+  loaded first with a 1-token request), so the window follows the engine and a prompt never goes over it. Below
+  16384 the proxy refuses to start; a `window` in the config bigger than the engine context also refuses the start
+  and fails `check`, with the command that fixes it. You can leave `window` out of the config.
+- **`--tokenizer` is still recommended** (the model's `tokenizer.json`, `pip install 'mnemonic-proxy[exact]'`).
+  Without it the proxy estimates chars/3.5, which was up to ~10% low on code with many digits; on generic engines
+  the estimate now gets a 15% margin by default (`window_estimate_slack` 0.15) and `check` warns. With 0.3.7 the
+  notes request at a segment switch was estimated at 31.3K tokens but was 33.4K real tokens and both engines cut it;
+  with 0.3.8 and no tokenizer it was 27.7K real tokens on Ollama and nothing was cut (numbers in Tested
+  combinations).
+- **A cut prompt is reported**: when the engine says (`usage.prompt_tokens`) it read less than 0.8× what the proxy
+  counted, the journal gets `engine_truncated` and the log one warning per conversation; `check --live` sends a
+  prompt of 0.9× the window and fails with "engine context smaller than window" if it does not come back whole.
+- Ollama ignores an integrated GPU by default and runs on the CPU (much slower); `OLLAMA_IGPU_ENABLE=1` makes it use
+  it. With it, Ollama chose a default context of 262144 from the shared memory: set `OLLAMA_CONTEXT_LENGTH` anyway.
+**Your agent.** Turn off the agent's own auto-compaction, otherwise two context managers fight over the same history:
+| agent | point it at | turn off | setup |
+|---|---|---|---|
+| pi, omp | `http://127.0.0.1:8096/v1` (`openai-completions`) | `"compaction": {"enabled": false}` | [T1 step 6](#t1--a-real-model-on-cpu), [omp](#pi-hermes-continue--chat-completions) |
+| Hermes, Continue, other Chat Completions clients | `http://127.0.0.1:8096/v1` | the client's own context compaction | [Chat Completions](#pi-hermes-continue--chat-completions) |
+| Claude Code | `ANTHROPIC_BASE_URL=http://127.0.0.1:8096` (no `/v1`) | `DISABLE_AUTO_COMPACT=1` | [Claude Code](#claude-code-anthropic-messages) |
+| Codex | `base_url = "http://127.0.0.1:8096/v1"`, `wire_api = "responses"` | no switch: `model_auto_compact_token_limit = 100000000` | [Codex](#codex-openai-responses) |
+| any MCP client (Claude Code, Codex, Hermes) | `mnemonic-mcp --db ./data/archive.sqlite` (read-only search of the archive) | — | [MCP](#mcp-recall-server-claude-code-codex-hermes) |
+**Your OS.**
+- **Linux**: the reference; the commands on this page, and all the measured sessions.
+- **Windows, native**: PowerShell commands and a one-click `examples\start-windows.bat` in [Platforms](#platforms)
+  (tested on the CI runner, not yet on a desktop). **WSL 2**: the Linux commands, with engine, proxy and `slots` all
+  inside WSL.
+- **macOS**: not tested yet; follow the Linux commands (llama.cpp from Homebrew or built with Metal), see
+  [Platforms](#platforms).
+- **Android (Termux)**: `pkg install python git`, then `pip install git+https://github.com/lastreload/mnemonic-proxy`;
+  tested in Waydroid (Android 13, x86_64) in front of an online API, not yet on an ARM phone, see
+  [Platforms](#platforms).
 ## How it works
@@ -115,71 +181,6 @@ No summaries written by a model, no vector database required.
 > "In the courier business you don't read the payload. You carry it, and you give it back exactly as it was." — the
 > design rule. Hidden content is never paraphrased; it is either in the prompt or retrievable verbatim.
-## Recall, benchmarked
-71 questions about past sessions, each with the known location of the evidence (including 8 with no answer in the
-archive). Held-out split:
-| | keyword search | structured recall |
-|---|---|---|
-| evidence found within 5 calls | 90% | 100% |
-| tokens returned per question | 5,940 | 2,737 |
-| "not in the archive" recognised | 1 / 4 | 4 / 4 |
-The benchmark guided development, so read it as a regression suite, not an independent score. Its archives are real
-private sessions and are not in this repository: `bench/` runs against your own archives (`bench/data/<name>.sqlite`)
-with questions written for them. Details: [docs/dev-notes/RECALL-RESULT.md](https://github.com/lastreload/mnemonic-proxy/blob/main/docs/dev-notes/RECALL-RESULT.md)
-(Italian).
-## Saved state, measured
-Resuming a conversation instead of re-reading it:
-| case | saved | re-read |
-|---|---|---|
-| Strata (session-files branch), 110K tokens, RTX 4070 Ti | 1.4 s | ≈50 s |
-| Strata (session-files branch), 69K tokens, after restart | 2.9 s | 33 s |
-| llama-server, 8.2K tokens, CPU, Qwen3-0.6B | 1.9 s | 28.6 s |
-| llama-server.exe, native Windows (CI runner), 3.2K tokens, CPU, Qwen3-0.6B | 2.3 s | 35.7 s |
-Times are for restoring the conversation's context; loading the model is separate.
-### Why saved state matters
-A long agent conversation is expensive to rebuild: every token of it has to be processed again (prefill) before the
-model can answer. Without saved state that cost is paid again each time the engine loses the conversation — after a
-restart, when another client or conversation used the engine in between, and when the proxy returns to a sealed
-segment. On a single 12 GB GPU that is close to a minute for 100K tokens; restoring the saved state takes a second or
-two, and the state lives on disk, not in VRAM.
-This is not specific to the proxy: any client that keeps long conversations on a local engine benefits.
-llama-server already offers it through `--slot-save-path`. Strata has it since
-[v0.1.40](https://github.com/Niko1221/Strata/releases/tag/v0.1.40) ([PR #668](https://github.com/Niko1221/Strata/pull/668)),
-opt-in with `--slot-save-path`. The Strata numbers on this page were measured on our session-files build
-(0.1.38 + PR #668, `8d8f664`), before the release; the proxy has not yet been re-measured on 0.1.40.
-## Claude Code and Codex, live on Strata
-0.2.0 in front of Strata (Qwen3.8 Flash Next IQ3_XXS, RTX 4070 Ti 12 GB, 128K window), with the real clients:
-| | Claude Code 2.1.284 (Anthropic Messages) | Codex 0.160 (Responses) |
-|---|---|---|
-| session | 1 h 41 min, 9 steps, 339 requests, **0 proxy errors** | 3 steps, 20 requests |
-| prompt tokens served from the engine cache | **92.6%** of 11.36M | **93.9%** of 351K |
-| context | 328K virtual, at most 46K in the prompt; 14 segment switches | — |
-| `recall` | 22 calls | exact: 36 tests and 8/8 error messages, word for word |
-Claude Code ran on a test config with low thresholds (window 64K, masking at 36K) so that segments would switch
-within the session. Found live and fixed in 0.3.0: in 3 writes out of 140 the model copied the shortened form of an
-old tool argument into a file (see [What we have not proven](#what-we-have-not-proven)). Report:
-[docs/dev-notes/LIVE020-RESULT.md](https://github.com/lastreload/mnemonic-proxy/blob/main/docs/dev-notes/LIVE020-RESULT.md) (Italian).
-### Long context on Strata, without the proxy
-16 `name=number` facts scattered in a generated Italian story, question at the end, temperature 0, two seeds, from
-16K to 122K tokens of prompt: **151/160 exact, 0 wrong** — every miss is an omitted line, all at 64K and above.
-Prefill stays at ~2.5K tok/s up to 122K, VRAM flat at 11.6 GB. Same report, section 4.
 ## Requirements
 - **Engine window: at least 16K tokens (16384), 32K or more (32768) recommended.** Below 16K the proxy refuses to
@@ -383,7 +384,8 @@ mnemonic-proxy --upstream http://127.0.0.1:8095 --port 8096 --data ./data --conf
 with `config.json` = `examples/config.llama-server.json` and `slot_dir` = the same `/path/to/slots` (see
 [Configuration](#configuration) for the recommended fields). With an engine that has no saved state (vLLM, online
-APIs) the proxy runs in base mode. The measurements at the top of this page (35B-A3B MoE, 128K window) come from
+APIs) the proxy runs in base mode (window: see [Install](#install)). The measurements in
+[One session, measured](#one-session-measured) (35B-A3B MoE, 128K window) come from
 Strata (our session-files build, 0.1.38 + PR #668) on a 12 GB RTX 4070 Ti — see [Engines](#engines); they say nothing about
 other hardware.
@@ -394,6 +396,18 @@ Optional: `--tokenizer` (a Hugging Face `tokenizer.json`, or a Strata pack `toke
 instead of a chars/3.5 estimate; with llama-server, `"engine_tokenize": true` counts through its `/tokenize`. A
 systemd user unit is in [examples/mnemonic-proxy.service](https://github.com/lastreload/mnemonic-proxy/blob/main/examples/mnemonic-proxy.service).
+**Window guard.** No request the proxy sends to the engine is larger than the window: before every call (answer,
+recall rounds, handoff notes, retries) the proxy checks `prompt + max_tokens + window_margin ≤ window` and, if it
+does not fit, lowers `max_tokens`, removes what the recall loop added (by value per token: results become one-line
+receipts, old reasoning becomes its conclusion line, recallable by id) or, as a last resort, returns HTTP 400 to
+the client instead of sending. With `--tokenizer` the count is the engine's own, so the guarantee is exact
+(`window_guard` events in the journal carry `guarantee=exact`). Without it the count is the calibrated chars/3.5
+estimate and the guarantee is only as good as that estimate (`guarantee=estimated`): on text with many digits or
+symbols the real count can be well above it, so use `--tokenizer` (or raise `window_estimate_slack`, e.g. `0.3`)
+when the window is small. On generic engines (Ollama, LM Studio, any OpenAI-compatible) without `--tokenizer` the
+slack is 0.15 by default (0.3.8); Strata, llama.cpp and ds4 keep 0. `"window_guard": false` restores the 0.3.2
+behaviour.
 Disable the agent's own auto-compaction in every client below, otherwise two context managers fight over the same
 history.
@@ -403,10 +417,45 @@ Point the client's OpenAI base URL at `http://127.0.0.1:8096/v1`. For pi, the tw
 `contextWindow`/`maxTokens`). For other clients: their OpenAI-compatible provider setting, plus their own way to turn
 off context compaction.
+**omp ([oh-my-pi](https://github.com/can1357/oh-my-pi))** — tested with 18.10.1 on Strata. omp reads YAML from
+`~/.omp/agent/` (or `$PI_CODING_AGENT_DIR`). Add the proxy as a provider in `models.yml`:
+```yaml
+providers:
+  mnemonic:
+    baseUrl: http://127.0.0.1:8096/v1
+    api: openai-completions
+    apiKey: none
+    auth: apiKey
+    compat:
+      supportsDeveloperRole: false
+      supportsReasoningEffort: false
+      maxTokensField: max_tokens
+      thinkingFormat: qwen-chat-template   # for Qwen models; drop it for others
+    models:
+      - id: qwen3.8-flash-next-iq3_xxs     # the id your engine reports at /v1/models
+        name: Flash Next via mnemonic-proxy
+        reasoning: true
+        input: [text]
+        contextWindow: 131072              # the engine's real window
+        maxTokens: 16384                   # leave room for reasoning (see the ds4 note below)
+        cost: {input: 0, output: 0, cacheRead: 0, cacheWrite: 0}
+```
+and turn omp's own compaction off in `config.yml`, so the proxy manages the window:
+```yaml
+compaction:
+  enabled: false
+```
+Then `omp --model mnemonic/qwen3.8-flash-next-iq3_xxs`. If you use extensions that also shorten tool output (for
+example SoL-Pi's ObservationPack), we have not tested them together with the proxy.
 ### Claude Code (Anthropic Messages)
 Checked against Claude Code 2.1.287; a 1 h 41 min session with 2.1.284 on Strata is measured
-[above](#claude-code-and-codex-live-on-strata). Claude Code may print `unrecognized_model local-model`: it works
+[below](#claude-code-and-codex-live-on-strata). Claude Code may print `unrecognized_model local-model`: it works
 anyway.
 ```sh
@@ -502,6 +551,40 @@ engine. The proxy's own:
 ## Tested combinations
+**0.3.8, Ollama and LM Studio: window read from the engine** (2026-10-19): same machine, engines and model as the
+0.3.7 run below (Ollama with the integrated GPU, `OLLAMA_IGPU_ENABLE=1`, context set with `OLLAMA_CONTEXT_LENGTH`;
+LM Studio `lms load … [--context-length 32768]`), no `--tokenizer`. Report: `docs/dev-notes/ENGINE-WINDOW-038-RESULT.md`.
+| test | Ollama | LM Studio |
+|---|---|---|
+| context read by the proxy (`/api/ps`, `/api/v0/models`) | 4096, 32768 | 8192, 32768 |
+| small context, `check` without config / with `window` 32768; proxy start | NOT READY: 4096 < 16384 / also "config window=32768 > Ollama context 4096"; start refused | same with 8192 |
+| small context, proxy allowed below the minimum (development switch), prompts of 5.4K and 10.8K estimated | window 4096: segment switches, 0.97K / 0.93K sent, codeword right, nothing cut | window 8192: 6.0K and 1.1K sent, notes 6.5K, codeword right, nothing cut |
+| forced cut: engine hidden behind a plain `/v1` forwarder, config `window` 32768 | `engine_truncated` ×2 (11.2K counted → 3.7K read; 22.0K → 3.2K), 1 warning; `check --live` fails ("request (29473 tokens) exceeds … 4096") | ×2 (11.2K → 3.8K; 22.0K → 3.2K), 1 warning; `check --live` fails (… 8192) |
+| context 32768: `check --live` full-window probe | 29,473 tokens sent, 29,473 read | 29,473 / 29,473 |
+| context 32768: the notes request that 0.3.7 cut (prompt ≈ 35.7K real) | 21 of 28 messages kept, notes request 27.7K real tokens, not cut; codeword right | 27.7K, not cut; codeword right |
+| context 32768, no `window` in the config: tool conversation (12 files) | — | 21 requests all 200, 19/19 valid tool calls, 1 mask (21.8K → 7.3K), 1 `recall` served, 0 `engine_truncated` |
+**0.3.7, Ollama and LM Studio** (2026-10-19): Linux, Ryzen AI Max+ 395 (Radeon 8060S integrated GPU, Vulkan), both
+engines installed in user space (Ollama 0.40.6 official tarball; LM Studio headless `llmster` 0.0.25 with its
+llama.cpp Vulkan runtime 2.41.0), same model in both: Qwen3-4B-Instruct-2507 Q4_K_M. Proxy from `main` with
+`config.openai-32k.json`, one request at a time from a small Python client (no agent).
+| test | Ollama | LM Studio |
+|---|---|---|
+| `check` | `READY` (exit 0); without a config `READY with warnings`: window unknown, 131072 | `READY` (exit 0) |
+| engine context by default | 4096 (CPU; with `OLLAMA_IGPU_ENABLE=1`: 262144) | 8192 |
+| longer prompts at the default context (Ollama ≈ 6–36K real tokens, LM Studio ≈ 8.6–21K), straight to the engine and through the proxy | HTTP 200, prompt cut to 2.7–3.7K tokens, a codeword from the first message lost | HTTP 200, prompt cut to 2.7–3.8K tokens, beginning and end kept |
+| engine context 32768: tool conversation over 12 files (`list_files`, `read_file`, `calc`) | 23 requests, all 200; 21 tool calls, all valid; 1 mask (prompt 19.4K → 7.4K); largest prompt 19.4K; nothing cut | 23 requests, all 200; 21 tool calls, all valid; 1 mask (19.3K → 7.3K); largest prompt 21.3K; nothing cut |
+| engine context 32768: one prompt of ≈ 35.7K real tokens | straight: cut to 29.8K, codeword lost; proxy: segment switch, 6.3K sent, codeword right | straight: cut to 14.8K; proxy: segment switch, 6.2K sent, codeword right |
+| the proxy's own notes request at that switch (no `--tokenizer`) | estimated 31.3K, cut by the engine to 30.5K | estimated 31.3K, real 33.4K, cut to 15.1K; with `--tokenizer`: 30.8K, not cut |
+| streaming, `/v1/messages`, `/v1/responses` | streamed (11 chunks), 200, 200 | streamed (11 chunks), 200, 200 |
+The model stopped before reading every file (10 of 12 on Ollama, 11 of 12 on LM Studio; the sums it made were right
+for the files it read) and, asked about a file that was still in view, called `recall` for the wrong piece and said
+it could not quote it: the proxy served the piece the model asked for. These are limits of the 4B model, not of the
+engines.
 **0.3.0, engine `-c 32768`** (2026-10-06): T1 run as written above, on the CPU of a Ryzen 7 5700G (16 threads, no
 GPU, llama-server at `nice 19`), in a fresh folder with its own `HOME`: llama.cpp `b11430` CPU build, pi 1.0.3,
 Node 24.14, Python 3.14, `config.llama-server-32k.json`. Requests sent with `pi -p` / `pi -c -p`, one at a time.
@@ -587,6 +670,34 @@ Start with `mnemonic-proxy check` (add `--live` to exercise the engine): every f
   `Get-FileHash -Algorithm SHA256`.
 - **Windows, WSL 2**: follow the Linux commands inside WSL. Keep the engine, the proxy and the `slots` folder all
   inside WSL, so that `slot_dir` and `--slot-save-path` are the same path. Not yet tested.
+- **Android (Termux)** (0.3.8; tested in Waydroid 1.6.2 on Linux — Android 13, x86_64 — with Termux 0.118.3 and its
+  Python 3.14.8; not yet on an ARM phone). The proxy is plain Python, nothing is compiled:
+  ```sh
+  pkg update -y && pkg upgrade -y
+  pkg install -y python git
+  pip install git+https://github.com/lastreload/mnemonic-proxy    # 0.3.8 (about 25 s)
+  mnemonic-proxy demo                                              # PASS
+  ```
+  Use case: an agent on the phone in front of an online API (no model on the phone), base mode:
+  ```sh
+  curl -LO https://raw.githubusercontent.com/lastreload/mnemonic-proxy/main/examples/config.openai-32k.json
+  mnemonic-proxy --upstream https://openrouter.ai/api --api-key "$OPENROUTER_API_KEY" \
+    --port 8096 --data ./data --config config.openai-32k.json
+  ```
+  then point the agent at `http://127.0.0.1:8096/v1`. Measured run (OpenRouter, `meta/muse-spark-1.3-contributor`, a
+  small stdlib client with 3 tools reading ten ~5K-token logs, 15 turns, history 55K tokens at the end): the proxy
+  never sent more than 24K tokens; prompt tokens 353K with the proxy (all rounds, the 9 `recall` rounds included) vs
+  447K for the same 15 requests sent directly (−21%; −56% counting only the 15 client turns); 12 valid tool calls,
+  0 errors; the final question about the first log (masked by then) was answered right through `recall`. Prefix
+  caching on the API side made the direct requests cheap, so in dollars the two were close (about 0.03–0.04 USD each).
+  Limits: no saved state (online API); `--api-key` is visible in the process list of the Termux user; `check --live`
+  tries the first model the API lists, so do not use `--live` against a catalogue like OpenRouter; online APIs can
+  answer HTTP 429 (rate limit), which the proxy passes to the client unchanged (the client must retry). A proxy on a
+  computer can also be reached from Termux with `adb reverse tcp:8096 tcp:8096` (tested with a fake engine).
 ## Uninstall
@@ -601,11 +712,194 @@ resuming without re-reading is lost); deleting `./data` loses the archive and th
 unreachable. Undo the client changes as well (pi `models.json`/`settings.json`, the Claude Code variables, the Codex
 provider).
+## One session, measured
+A real coding session — a canvas tower-defense game built from scratch by a 35B-A3B MoE model on a single 12 GB
+RTX 4070 Ti — with the proxy in between.
+<p align="center">
+  <img src="https://raw.githubusercontent.com/lastreload/mnemonic-proxy/main/docs/assets/replay.gif" alt="Replay: a question answered from three hours earlier via one recall call" width="100%">
+</p>
+<p align="center"><sub>Replay of a real exchange (animated GIF; <a href="https://raw.githubusercontent.com/lastreload/mnemonic-proxy/main/docs/assets/replay.mp4">MP4</a>).
+Three hours and two segment switches into the session, one <code>recall</code> call — resolved inside the proxy in
+61 ms — brings back the verbatim first version of the boss block, message 20. No model summarized anything.</sub></p>
+| | |
+|---|---|
+| **448K** | tokens of history archived and recallable |
+| **105K** | peak tokens actually in the prompt (window: 128K) |
+| **3h15** | continuous agent work · 430 requests · 0 errors |
+| **1.4 s** | to resume a 110K-token conversation from disk (≈50 s re-reading it) |
+<p align="center">
+  <img src="https://raw.githubusercontent.com/lastreload/mnemonic-proxy/main/docs/assets/session-chart.svg" alt="Full history vs. prompt actually read, per request" width="100%">
+</p>
+Magenta: the full history of the session. Cyan: what the model actually reads on each request. Dashed lines: the two
+segment switches (≈2 minutes each, written handoff notes). Data from the proxy journal, 5 Oct 2026.
+## Benchmark: AgentLongBench with a 32K window
+[AgentLongBench](https://github.com/euReKa025/AgentLongBench) (kf-c, tool_response, Count Frequency and Find
+Duplicates), 25 questions per type at 64k (~74K-token history) and 128k (~150K): 100 questions, official scorer.
+Qwen3.8 Flash Next IQ3_XXS on Strata 0.1.40, one RTX 4070 Ti 12 GB, seed 1. Four methods: the full history in an
+81,920 / 163,840-token window, and three ways to fit it into 32,768 tokens — this proxy (0.3.5, commit
+[`b8d4920`](https://github.com/lastreload/mnemonic-proxy/commit/b8d4920)), an LLM summary of at most 2,048 tokens
+written by the same model, and truncation. Preregistered test: proxy vs summary, exact McNemar. With the question
+clarified (one sentence saying which tool output "round N" means; answers and scorer unchanged):
+| correct | full window | proxy | summary | truncation |
+|---|---|---|---|---|
+| 64k (/50) | 38 | **41** | 27 | 23 |
+| 128k (/50) | 31 | **45** | 20 | 26 |
+| both (/100) | 69 | **86** | 47 | 49 |
+Proxy vs summary: 44 questions right only with the proxy, 5 only with the summary (p = 7.6e-9). With the original,
+ambiguous question every method was near chance (51 / 52 / 56 / 44; answering "0 / no" every time scores 50).
+Limits: one model, one engine, one seed, 100 questions; the clarified run exists because of the first one; end to
+end the proxy took a median 136 s / 286 s per question (64k / 128k) vs 70 s / 102 s for the full window, mostly
+replaying the whole history through the proxy before the question.
+Per-question results, logs, scripts and analysis: [mverde73/mnemonic-proxy-agentlongbench](https://huggingface.co/datasets/mverde73/mnemonic-proxy-agentlongbench).
+## Recall, benchmarked
+71 questions about past sessions, each with the known location of the evidence (including 8 with no answer in the
+archive). Held-out split:
+| | keyword search | structured recall |
+|---|---|---|
+| evidence found within 5 calls | 90% | 100% |
+| tokens returned per question | 5,940 | 2,737 |
+| "not in the archive" recognised | 1 / 4 | 4 / 4 |
+The benchmark guided development, so read it as a regression suite, not an independent score. Its archives are real
+private sessions and are not in this repository: `bench/` runs against your own archives (`bench/data/<name>.sqlite`)
+with questions written for them. Details: [docs/dev-notes/RECALL-RESULT.md](https://github.com/lastreload/mnemonic-proxy/blob/main/docs/dev-notes/RECALL-RESULT.md)
+(Italian).
+## Saved state, measured
+Resuming a conversation instead of re-reading it:
+| case | saved | re-read |
+|---|---|---|
+| Strata (session-files branch), 110K tokens, RTX 4070 Ti | 1.4 s | ≈50 s |
+| Strata (session-files branch), 69K tokens, after restart | 2.9 s | 33 s |
+| llama-server, 8.2K tokens, CPU, Qwen3-0.6B | 1.9 s | 28.6 s |
+| llama-server.exe, native Windows (CI runner), 3.2K tokens, CPU, Qwen3-0.6B | 2.3 s | 35.7 s |
+Times are for restoring the conversation's context; loading the model is separate.
+### Why saved state matters
+A long agent conversation is expensive to rebuild: every token of it has to be processed again (prefill) before the
+model can answer. Without saved state that cost is paid again each time the engine loses the conversation — after a
+restart, when another client or conversation used the engine in between, and when the proxy returns to a sealed
+segment. On a single 12 GB GPU that is close to a minute for 100K tokens; restoring the saved state takes a second or
+two, and the state lives on disk, not in VRAM.
+This is not specific to the proxy: any client that keeps long conversations on a local engine benefits.
+llama-server already offers it through `--slot-save-path`. Strata has it since
+[v0.1.40](https://github.com/Niko1221/Strata/releases/tag/v0.1.40) ([PR #668](https://github.com/Niko1221/Strata/pull/668)),
+opt-in with `--slot-save-path`. The Strata numbers on this page were measured on our session-files build
+(0.1.38 + PR #668, `8d8f664`), before the release. The proxy has since run on Strata 0.1.40 in the
+[AgentLongBench benchmark](#benchmark-agentlongbench-with-a-32k-window) and the
+[recall tests](#does-the-model-recall-on-its-own), but the restore times in the table above have not been
+re-measured on 0.1.40; a contributor measured restore on Strata 0.1.41
+([Tools](#tools-audit-the-archive-and-measure-restore)).
+## Claude Code and Codex, live on Strata
+0.2.0 in front of Strata (Qwen3.8 Flash Next IQ3_XXS, RTX 4070 Ti 12 GB, 128K window), with the real clients:
+| | Claude Code 2.1.284 (Anthropic Messages) | Codex 0.160 (Responses) |
+|---|---|---|
+| session | 1 h 41 min, 9 steps, 339 requests, **0 proxy errors** | 3 steps, 20 requests |
+| prompt tokens served from the engine cache | **92.6%** of 11.36M | **93.9%** of 351K |
+| context | 328K virtual, at most 46K in the prompt; 14 segment switches | — |
+| `recall` | 22 calls | exact: 36 tests and 8/8 error messages, word for word |
+Claude Code ran on a test config with low thresholds (window 64K, masking at 36K) so that segments would switch
+within the session. Found live and fixed in 0.3.0: in 3 writes out of 140 the model copied the shortened form of an
+old tool argument into a file (see [What we have not proven](#what-we-have-not-proven)). Report:
+[docs/dev-notes/LIVE020-RESULT.md](https://github.com/lastreload/mnemonic-proxy/blob/main/docs/dev-notes/LIVE020-RESULT.md) (Italian).
+### Does the model recall on its own?
+Recall only helps if the model uses it without being told. We built short scripted tasks where a value the model
+needs at the end exists **only** in an old, masked tool output — not on disk, not reproducible — and the final request
+never says "remember" or "earlier":
+- **A** random output of a setup script (build id, p95, skipped migrations); re-running it gives different values;
+- **B** a file read once, then deleted;
+- **C** the old version of an overwritten config, no git, no backup;
+- **D** an error from a test service that is then shut down;
+- **L** a long session: the value appears in turn 0, followed by 12 heavy turns and several segment switches.
+Qwen3.8 Flash Next IQ3_XXS on Strata 0.1.40, RTX 4070 Ti 12 GB, proxy 0.3.1 as released (no extra hints):
+| client | episodes | right values | via `recall` the model chose to call |
+|---|---|---|---|
+| Claude Code 2.1.284 | A, B, C, D × 2 | 8 / 8 | 8 / 8 |
+| Claude Code 2.1.284 | L: 127K virtual, 3 segment switches | 1 / 1 | yes |
+| pi 0.87.1 | A, B, D | 3 / 3 | 3 / 3 |
+| pi 0.87.1 | L: 231K virtual, 6 segment switches | 1 / 1 | yes (5 keyword searches) |
+| omp 18.10.1 (oh-my-pi) | A, B, C, D | 4 / 4 | 4 / 4 ¹ |
+| omp 18.10.1 (oh-my-pi) | L: 82K virtual, 2 segment switches | 1 / 1 | yes (6 calls) |
+¹ With omp the model repeated the values in its own one-line answer in B (all three) and D (two of three), so they
+stayed visible: there recall was strictly needed only for D's third value. A and C are clean.
+A control with the full history in the prompt (no masking) was right in every short episode. In the long one it
+re-ran the setup script and wrote the new, wrong values — the proxy run did better. Each episode was checked for the
+true values in the final prompt (none were there, except the omp cases in ¹); one file per configuration was also
+checked by hand.
+Across 18 real sessions (pi, Claude Code, Codex) the need is rare: 5 spontaneous recalls against 108 after a question
+about the past; 30 re-reads of a masked, unchanged file (≈37K tokens); 4 edits out of 505 failed because the model
+misremembered a masked file. When a file can be re-read, the model re-reads it — which is right when it has changed.
+Limits: one model, one run of each long session, scaled-down thresholds (32K window) for the long sessions; Cline
+not measured (it cannot resume a session headless without a Cline account). Report:
+[docs/dev-notes/RECALL-NEED-RESULT.md](https://github.com/lastreload/mnemonic-proxy/blob/main/docs/dev-notes/RECALL-NEED-RESULT.md)
+(Italian).
+### Long context on Strata, without the proxy
+16 `name=number` facts scattered in a generated Italian story, question at the end, temperature 0, two seeds, from
+16K to 122K tokens of prompt: **151/160 exact, 0 wrong** — every miss is an omitted line, all at 64K and above.
+Prefill stays at ~2.5K tok/s up to 122K, VRAM flat at 11.6 GB. Same report, section 4.
+## User reports
+Numbers measured by users on their own hardware and shared with permission. Not reproduced by us.
+| | [u/Swimming-Farmer7815](https://www.reddit.com/r/Qwen_AI/comments/1x26aj9/comment/pfb7uu7/) |
+|---|---|
+| hardware | RTX 5060 Ti 16 GB, Xeon E5-2699 v3, 64 GB DDR4 ECC |
+| engine / model | Strata, Swift-Qwen3.8-Flash-Next IQ3_XXS, 128K window |
+| client | Cline, with its automatic context compaction disabled |
+| session | 13 h; ~280K tokens of total history, ~77K in the last request |
+| saved state | ~58K tokens restored in under 4 s (about 1 min 15 s to re-read at the measured prefill rate) |
+| `recall` | 8 calls, all 8 results used by the model |
+The user reports no compaction pauses during the session. All 8 recalls happened in the last two minutes of the
+session, when the user asked for a summary of the day: during normal work the model never called `recall` on its
+own. In our tests with Claude Code, pi and omp the model did call `recall` by itself when the value could not be re-read
+from disk ([Does the model recall on its own?](#does-the-model-recall-on-its-own)); with Cline we could not test it.
+**Share your numbers.** If you run the proxy on your own hardware, a report helps — especially with clients and models
+we have not tested (Cline, OpenCode, other models). Useful: GPU/RAM, engine and model, client, session length (total
+history and tokens in the last request), restore time, how many `recall` calls and whether the model made them on its
+own. `data/journal.jsonl` holds most of it; check it before sharing, it contains your recall queries. Open an issue
+with the [user report](https://github.com/lastreload/mnemonic-proxy/issues/new?template=user-report.md) template.
 ## Engines
 | client \ engine | Strata | llama-server | ds4-server | any OpenAI-compatible |
 |---|---|---|---|---|
-| Chat Completions (pi, Hermes, Continue…) | tested · full | tested on CPU · full | tested with pi (GLM 5.3, Qwen3.8) · engine keeps its own state | base (no saved state) |
+| Chat Completions (pi, omp, Hermes, Continue…) | tested · full | tested on CPU · full | tested with pi (GLM 5.3, Qwen3.8) · engine keeps its own state | base (no saved state) |
 | Claude Code (Anthropic Messages) | **tested** (1 h 41 min live session) | experimental | experimental | experimental |
 | Codex (Responses) | **tested** (live, exact recall) | experimental | experimental | experimental |
 | MCP recall server (Claude Code, Codex, Hermes) | tested read-only on a real archive | | | |
@@ -617,11 +911,35 @@ against a mock engine and with short real client runs, not yet on long real sess
   before a switch, autosave/autorestore, `kv_archive`). The window comes from the server's `n_ctx` unless `window` is
   set; with `--parallel N` the proxy pins one slot (`slot_id`, default 0). Without `--slot-save-path` the proxy runs
   in base mode.
+- **llama.cpp in router mode** (0.3.7, llama-server started without `-m`, models from `--models-preset` or
+  `--models-dir`, loaded on demand) — detected automatically. The client's `model` picks the model; the proxy reads
+  that model's n_ctx (its slot, `GET /slots?model=…`, loading the model if needed) **before** building the prompt, so
+  the window always fits the model that will run it (thresholds scale with it). If the n_ctx cannot be read the
+  request is refused with a clear error; it is never sent with the config window. Saved files are bound to the model
+  that wrote them: a file is never restored into another model. Start every model with `--slot-save-path` (put it
+  in the `[*]` section of the preset) and `parallel = 1`, and give each model a `ctx-size` of at least 16K.
+  **Loading takes time** (on AI395 a 35B-A3B Q4 model took ~73 s from a USB disk, 12 s from the page cache): with
+  `stream: true` the proxy sends an SSE comment (`: loading model <id>`; Anthropic API: a `ping` event) every
+  `router_keepalive_s` (10 s) until the model is loaded and its n_ctx read, so a client with a read timeout does not
+  give up; if the client has gone away the request is dropped before anything is generated (journal
+  `router_client_gone`). **Non-stream requests get nothing until the answer: the client's timeout must be longer
+  than the model's load time** (plus the answer). `router_load_timeout_s` (600) bounds the wait on the proxy side.
+  **One history per conversation**: when a smaller model needs masking to fit, the masked history is what the bigger
+  model gets afterwards too (it can read the masked results again with its tools; its earlier saved file no longer
+  matches). The disk check before an autosave uses the bytes/token measured on the last save of the same model
+  (×1.25; 20000 until the model has saved once). Tested on CPU with tiny models and on AI395 (Radeon 8060S,
+  Vulkan) with Qwen3.6-35B-A3B and Gemma 4 E2B (`docs/dev-notes/ROUTER-MODE-RESULT.md`). Ready to copy:
+  `examples/preset.router.example.ini` (start with `llama-server --models-preset preset.router.example.ini
+  --models-max 1 --port 8095`) and `examples/config.llama-server-router.json`.
 - **Strata** — works in base mode on any version. **Saved state: Strata ≥ 0.1.40**
   ([PR #668](https://github.com/Niko1221/Strata/pull/668), released in
   [v0.1.40](https://github.com/Niko1221/Strata/releases/tag/v0.1.40)), started with `--slot-save-path DIR`. Session files
   must be trusted: the hashes detect corruption, not tampering. Our measurements are from the session-files build
-  (0.1.38 + PR #668, `8d8f664`); 0.1.40 has not been re-measured with the proxy yet.
+  (0.1.38 + PR #668, `8d8f664`); on 0.1.40 the proxy ran the AgentLongBench benchmark and the recall tests, but
+  restore times were not re-measured there (see [Saved state, measured](#saved-state-measured)).
+  `setup.sh` and `run-strata-proxy.sh` in the repository root (a venv, then the proxy in front of a running
+  Strata, `slot_dir` read from the Strata config) were contributed and are covered by unit tests; they are not yet
+  tested by the maintainer on a real Strata install.
   `examples/config.example.json` is the configuration used with session files;
   `examples/config.official-strata.json` the one for Strata without `--slot-save-path` (or older than 0.1.40).
@@ -654,9 +972,9 @@ mnemonic-proxy --upstream http://127.0.0.1:8110 --port 8096 --data ./data --conf
 - `checkpoint_align_tokens` (off by default): when set to ds4's checkpoint interval (by default 10240 tokens,
   `--kv-cache-continued-interval-tokens` rounded to `--kv-cache-boundary-align-tokens`), a masking block starts just
   after a multiple of it, so fewer tokens are re-read after the last disk checkpoint. Needs an exact `--tokenizer`
-  to be meaningful. Not measured yet.
-- Not proven: long real sessions; Claude Code and Codex in front of ds4; parallel sessions (`--batched-session N`)
-  — the proxy still serialises requests.
+  to be meaningful. In one long session at 32K (below) it changed no masking decision.
+- Not proven: Claude Code and Codex in front of ds4; parallel sessions (`--batched-session N`) — the proxy still
+  serialises requests.
 mnemonic-proxy is a separate tool, compatible with ds4-server and tested with it; it is not part of the ds4 project.
@@ -688,17 +1006,80 @@ Reports (Italian): [DS4-GLM-RESULT.md](https://github.com/lastreload/mnemonic-pr
 [RECALL-8K-030-RESULT.md](https://github.com/lastreload/mnemonic-proxy/blob/main/docs/dev-notes/RECALL-8K-030-RESULT.md) (the 0.3.0 run),
 [DS4-RESULT.md](https://github.com/lastreload/mnemonic-proxy/blob/main/docs/dev-notes/DS4-RESULT.md) (the adapter).
+#### Long sessions on ds4: proxy vs pi's own compaction
+Same hardware, ds4 `fc80bd6` (+ the local ROCm port for Qwen3.8), proxy 0.3.2, pi 1.0.3, window 32K, one run per
+side. Without the proxy pi compacts by itself (`reserveTokens 8192`, `keepRecentTokens 8000`: our choice for 32K).
+Every answer below is graded by tests and regexes written before the runs.
+| task, model | | with proxy | pi alone |
+|---|---|---|---|
+| 16 steps (`farmastock`), Qwen3.8 Flash Next Q2 | steps done (2 h limit) | **16/16** in 85 min | 14/16 |
+| | re-reads from token 0 | **0** | 23 |
+| | tokens read by the engine | **167K** | 267K (with fewer steps) |
+| | early facts recalled at the end | **5/5** | 4/5, one "I don't know" (asked after the limit) |
+| 29 steps (`salepren`), same model | steps done (4 h limit) | 29/29 | 10/29 |
+| | re-reads from token 0 | **1** | 38 |
+| | tokens read / prefill time | **362K / 4,447 s** | 511K / 6,368 s |
+| | memory probes at steps 8, 14, 20 | 17/18 | 5/6 (only step 8 reached) |
+| | hidden tests on the final code | 87/87 ⚠ | 59/87 |
+| 16 steps, DeepSeek V4 Flash Q2 (official ds4, no patches) | steps / re-reads / facts | 16/16 in 61 min / 0 / 7 of 8, one "I don't know" | not run |
+Prompt reading (~82 tok/s) and generation (~19 tok/s) were the same with and without the proxy.
+**⚠ The 29-step run is not a clean quality result.** Both sides ran with pi's `maxTokens 4096`, and that limit
+includes the model's reasoning. Qwen3.8 often thought for more than 4,096 tokens before acting, so the answer was
+cut before the tool call (`finish_reason: length`): 26 times with the proxy, 8 without. pi stores a cut answer
+without an action and sends the next step, so unanswered requests pile up. With the proxy they reached 13 in a
+row; the model tried to do all of them in one turn, ran out of tokens again, and kept looping. It wrote no code
+from step 2 to step 25, then rewrote the whole package from memory at step 26 (which passes all 87 tests). Without
+the proxy, pi's summaries happened to clear the backlog. The first cut came before the proxy had changed anything,
+and it also happened without the proxy: the cause is the limit, but the proxy keeps the history faithfully and so
+does not break the loop.
+What this means in practice:
+- **give reasoning models room**: set the client's max output tokens well above their usual reasoning length
+  (8192 or more for Qwen3.8 with reasoning on);
+- the 16-step runs did not show this loop (with the proxy all 16 steps produced code; 57 of the agent's own tests
+  passed at the end, 40 without the proxy) and are the cleaner comparison;
+- a rerun of the 29-step task with 8192, and a proxy note when the last answer was cut, are planned.
+Other findings: once (16-step run with `checkpoint_align_tokens`) a `recall` found the right chunk but the result
+was cut for lack of room in the window, and the model answered "never discussed" — a weak point seen once, not yet
+reproduced. With DeepSeek V4 Flash, ds4's default q8→f16 cache left only ~9 GB of free RAM on a 128 GB machine;
+the run used `DS4_CUDA_NO_Q8_F16_CACHE=1`.
+Reports (Italian): [DS4-LONG-RESULT.md](https://github.com/lastreload/mnemonic-proxy/blob/main/docs/dev-notes/DS4-LONG-RESULT.md) (16 steps, Qwen3.8),
+[DS4-LONG2-RESULT.md](https://github.com/lastreload/mnemonic-proxy/blob/main/docs/dev-notes/DS4-LONG2-RESULT.md) (29 steps, Qwen3.8),
+[DS4-FLASH-RESULT.md](https://github.com/lastreload/mnemonic-proxy/blob/main/docs/dev-notes/DS4-FLASH-RESULT.md) (16 steps, DeepSeek V4 Flash).
 `GET /v1/strata/engine` shows what was detected. Measurements: [docs/dev-notes/ENGINES-RESULT.md](https://github.com/lastreload/mnemonic-proxy/blob/main/docs/dev-notes/ENGINES-RESULT.md) (Italian).
 ## Configuration
+Output limits in managed mode: `max_completion_tokens` takes precedence over `max_tokens` when both are supplied;
+without either field, `default_response` is used (16384 by default). These values are ceilings, not a reason to
+switch segments. After masking and any necessary switch, the proxy caps the response to
+`window - estimated_prompt - reserve - 8` and sends the reduced limit to the engine (both fields are synchronized
+when present). The final window guard may reduce it further for its margin and estimate slack; it refuses a
+prompt with no space for even one generated token. Exact guarantees require `--tokenizer`; otherwise counts are
+estimates, not proof of the engine's actual token usage.
+A genuinely large prompt still triggers a segment switch when there is not enough room for a useful response:
+`max(response_floor, min(requested_output, default_response))`, bounded to at least one token and at most a quarter
+of the window. A floor never raises the actual output ceiling requested by the client. Every reduction produces
+`response_clamped` in `journal.jsonl` (requested and sent limits, prompt estimate, window and reserve), plus one
+warning per conversation in the process log, remembered across restarts. No new option is required; for example,
+a client ceiling of 120000 in a 131072 window no longer causes a switch on every short turn.
 `--config` takes a JSON object with any field of `ctxproxy.core.Config`. Defaults below are the real ones in the
 code. Most features are off by default so that 0.1 behaviour is unchanged; the last column is what we recommend
 with an engine that has saved state (`examples/config.example.json` plus the options the benchmarks above support).
 | field | default | meaning | recommended |
 |---|---|---|---|
-| `window` | 131072 | physical context of the engine (llama-server: taken from `n_ctx` if unset) | 131072 |
+| `window` | 131072 | physical context of the engine (llama-server, ds4, Strata, Ollama, LM Studio: taken from the engine if unset; Ollama / LM Studio: a bigger value refuses the start) | 131072 |
+| `window_margin`, `window_estimate_slack` | 64, 0 (0.15 on generic engines without `--tokenizer`) | window guard: tokens kept free; inflation of the estimated prompt count | default |
 | `mask_trigger` / `mask_target` | 80000 / 40000 | start a masking block above trigger, mask oldest-first down to target | default |
 | `min_batch_tokens` | 16000 | a block runs only if it frees at least this much | default |
 | `keep_recent_tokens` / `min_age_turns` | 16000 / 2 | protected tail, never masked | default |
@@ -707,12 +1088,15 @@ with an engine that has saved state (`examples/config.example.json` plus the opt
 | `slot_dir` | "" | the engine's `--slot-save-path` folder, same host (anchor cleanup, autosave, `kv_archive`) | set |
 | `segments_enabled`, `tail_max`, `notes_max_tokens` | true, 24000, 8192 | segment switch with handoff notes | true, 24000, **12288** |
 | `slot_save` / `seal_experimental` | false / false | save the old segment to disk before switching (saved state) | **true / true** |
-| `response_floor` | 0 | minimum response size assumed in the window check | **16384** |
+| `response_floor` | 0 | minimum useful response budget for segment switching (bounded to a quarter of the window) | **16384** |
 | `inject_recall`, `max_recall_rounds`, `recall_max_tokens` | true, 4, 6000 | the `recall` tool | default |
 | `recall_tool_name` / `tools_tool_name` | "recall" / "tools" | names of the proxy's tools for new conversations | default |
 | `prompt_language` | "en" | language of the texts the proxy writes into the prompt, for NEW conversations: `"en"` or `"it"`; stored per conversation at first use; conversations from before 0.3.1 stay Italian | default |
 | `recall_struct`, `recall_flex`, `recall_multi` | false, false, false | structured recall: file timelines, links, flexible words, several queries per call (the benchmark above) | **true, true, true** |
 | `auto_recall` / `auto_recall_hint` | false / false | on each new user message, add relevant hidden pieces / only a one-line hint with ids | false / **true** |
+| `recall_leaf_select` | false | on structured outputs (yaml/toml (Python ≥ 3.11)/xml/json, tables, diffs, logs, git log, pytest) `recall` returns only the matching record and the key lines of its ancestors, verbatim, with `⋮` markers and offsets for the gaps; falls back to the normal recall when unsure ([dev note](docs/dev-notes/LEAF-SELECT-RESULT.md)) | default (experimental) |
+| `answer_thinking_after_recall`, `answer_thinking_budget` | false, 512 | turn thinking on only for the answer that follows a recall (fixes answers cut in the middle of a value) | default (experimental) |
+| `truncation_guard` | false | replay once, with thinking, a short answer that ends on a prefix of a recalled token | default (experimental) |
 | `tools_paging`, `tools_core` | false, read/bash/edit/write/grep/find/ls | tools on demand: only core tools + `recall` + `tools` in the prompt | **true** |
 | `typed_receipts`, `receipt_guard` | true, true | typed one-line receipts for hidden outputs; block write/edit/shell calls (pi, Claude Code, Codex names) that copy a receipt or the shortened form of an old argument | default |
 | `pins_max_tokens` | 8192 | cap of the carried-over 📌 block | default |
@@ -720,6 +1104,7 @@ with an engine that has saved state (`examples/config.example.json` plus the opt
 | `autorestore`, `autorestore_min_gain` | true, 8192 | restore before forwarding when it saves at least this many tokens | default |
 | `kv_archive`, `kv_archive_idle_s`, `kv_archive_min_age_s` | false, 600, 1800 | cold archive of state files (needs Python ≥ 3.14 for `compression.zstd`) | **true** |
 | `engine`, `slot_id`, `engine_tokenize` | "auto", 0, false | engine type, slot, token counting via the engine's `/tokenize` | auto |
+| `router_model`, `router_load_timeout_s`, `router_keepalive_s` | "", 600, 10 | llama-server router mode: model used when the client's `model` is not served (empty = refuse), how long a model may take to load, seconds between two SSE comments to a streaming client while the model loads | default |
 | `live_dump` | false | write `data/live/last_request.json` for the dashboard | **true** |
 A config for llama-server with saved state, as recommended above:
@@ -755,8 +1140,9 @@ alone and the proxy uses `history_recall` (or `load_tools`) instead.
   exactly, two partially (one value missed, one timeline mixed up). None were invented.
 - The model does not attend to 448K tokens. It sees at most one window; the rest is recallable, not "in context".
 - Segment switches cost ≈2 minutes of note-writing on a 12 GB GPU.
-- Saved state on Strata: measured on our 0.1.38 + PR #668 build; the released Strata 0.1.40 (which includes it) has not
-  been re-measured with the proxy.
+- Saved state on Strata: restore times measured on our 0.1.38 + PR #668 build; on the released Strata 0.1.40 (which
+  includes it) the proxy ran the AgentLongBench benchmark and the recall tests, but restore times were not
+  re-measured.
 - Claude Code and Codex are tested live only on Strata; on llama-server and ds4-server only against mock engines.
 - Copied shortened arguments (found live with Claude Code: 3 writes out of 140 ended with the `…` line of a masked
   old argument, one caused a real `NameError`). Since 0.3.0 the proxy blocks a write/edit/shell call whose text
@@ -769,8 +1155,15 @@ alone and the proxy uses `history_recall` (or `load_tools`) instead.
   original estimate. Questions about visible content: 7/7 either way. Measured without the proxy's recall loop.
 - Facts scattered in a long prompt: 0 wrong out of 160 on Strata, but 9 omissions, all from 64K up (two seeds).
 - Recall search is lexical (FTS5 + structure), not semantic: a paraphrase with no shared words can miss.
+- Unprompted recall is measured with Claude Code, pi and omp on one model; with Cline a user saw recall only when asking
+  about the past, and we could not reproduce that setup.
 - One sequence: the proxy serialises requests and assumes it is the engine's only client; another client on the
   same engine costs re-reads.
+- llama-server router mode: tested on CPU with two tiny models (stories260K, stories15M) and on AI395 (Vulkan) with
+  Qwen3.6-35B-A3B (64K) and Gemma 4 E2B (32K), `--models-max 1`, one slot per model. Not tested: `--parallel` > 1,
+  `--models-max` > 1 with evictions under load, `--sleep-idle-seconds`, `/models/sse` (not used: the proxy reads
+  `/models` at every request), `check --live`. A client that goes away while a model loads frees the proxy only when
+  the load ends (the proxy does not stop a load in progress).
 - Numbers come from single runs on one machine and one kind of workload; treat them as indicative.
 ## Security & privacy
@@ -798,6 +1191,51 @@ alone and the proxy uses `history_recall` (or `load_tools`) instead.
      `python3 -m ctxproxy.kvarchive --root /path/to/kv-archive gc` to drop unreferenced blocks;
   5. the journal is append-only: delete or filter `data/journal.jsonl` (lines with `"conv": "<id>"`).
+## Tools: audit the archive and measure restore
+Two stand-alone tools, Strata-only for now. Both are read-only against the archive and
+never restart the engine; run them with the Python that has the archive support
+(`.venv314`).
+**`tools/check_kv_archive.py`** — is the cold archive healthy, and what is it saving?
+```sh
+.venv314/bin/python tools/check_kv_archive.py                 # snapshot now
+.venv314/bin/python tools/check_kv_archive.py --restore-check  # rebuild each archive, compare sha256
+.venv314/bin/python tools/check_kv_archive.py --watch          # follow until it archives
+```
+It reports the `sessions/` files, the `kv-archive/` usage with dedup, and how far the
+archiver is from firing (engine idle, file age, file size). It never creates the archive
+directories and never writes to disk: `--restore-check` restores into a private temp dir.
+Exit code 0 when everything is fine, 1 on a problem (a corrupt block, a failed restore,
+a `--watch` that never archived), 2 on a usage error.
+**`tools/measure_restore.py`** — how much does the automatic restore save?
+```sh
+.venv314/bin/python tools/measure_restore.py --allow-shared-engine
+```
+It sends **the same request twice**, each time after B has evicted A from the engine:
+**control** (`autorestore` off) gives the real re-read, **restore** (`autorestore` on)
+gives `restore_ms` plus the few new tokens — both read from the journal, so the speed-up
+is `control_ms / (restore_ms + new_ms)`, not an extrapolation. `--two-phases` is the
+earlier method: it runs a whole A/B cycle per phase and re-sends A with one extra
+message, so its restore path also carries those new tokens — a slightly different
+question, measured with the same journal timings.
+Measured with **Strata 0.1.41** and **swift-1.5-iq3_xxs** (window 131072, prompt
+~31.4K tokens) on an **RTX 5060 Ti 16 GB** (PCIe 3.0 x8), an **Intel Xeon E5-2699 v3**
+and **64 GB DDR4-2667 running at 2133 MT/s** (the CPU's memory limit), with three runs
+of each method (six runs in total): the same request gave **~10–12×** (median 11.6×) —
+re-read ~30 s vs restore + new tokens ~2.7 s — and the two-phase method gave ~6.7×
+under the same conditions.
+It needs `--allow-shared-engine` because filling the engine with conversation B evicts
+other sessions; it removes only the files it created itself. Exit code 0 when a restore
+was measured, 1 when nothing was restored, 2 on a usage error.
 ## Tests
 ```sh
@@ -812,6 +1250,12 @@ through the context manager offline. `tools/state_e2e.py --server PATH/llama-ser
 checks saved state end to end with a real llama-server on any OS (restart, restore from disk vs re-read, and
 `kv_archive` with `--kv-archive`). CI runs the unit tests on Linux and on native Windows.
+The two Strata tools have their own tests: `tests/test_kv_archive.py` and
+`tests/test_check_kv_archive.py` build a real archive with the same in-memory helper (no engine
+needed) and check that the audit detects a corrupt block, exits non-zero, and touches nothing;
+`tests/test_measure_restore.py` covers the pure logic (journal filtering, the speed-up from
+`prompt_ms` / `restore_ms`, the CLI guards). The `kv_archive` group needs Python ≥ 3.14.
 Partly in Italian: code comments, the dashboard UI and the developer notes in `docs/dev-notes/`. Since 0.3.1 every
 text the proxy writes into the model's prompt — receipts and placeholders of hidden content, the handoff-note
 request, the segment and pinned-points blocks, automatic recall, recall results, guard and limit messages — is in

mnemonic-proxy 0.3.2__tar.gz → 0.3.9__tar.gz

mnemonic-proxy 0.3.2tar.gz → 0.3.9tar.gz