minnimemory 1.0.0-beta.1

package/README.md

@@ -0,0 +1,824 @@
+# MinniMemoryMCP
+
+**Curated memory for AI agents, served over MCP.** Your AI coding agent re-reads its entire
+project memory file on every turn. MinniMemoryMCP is an MCP server whose tools audit that file,
+compile it into a small AlwaysOnMemory body plus OnDemandMemory files loaded only when
+relevant, and serve it back on demand. The `minnimemory` command line underneath is the build layer those tools call;
+it is documented further down for maintainers.
+
+Built by MinniAI.
+
+```
+BEFORE                                  AFTER
+CLAUDE.md     689 tokens                CLAUDE.md      282 tokens   <- every turn
+   x every turn, all of it              .minnimemory/OnDemandMemory/
+                                            963 tokens  <- only when relevant
+```
+
+Those are real numbers from `examples/CLAUDE.md` in this package, default settings, offline
+tokenizer (`approx-v2`, calibrated against a real BPE encoder, see
+[Measurement policy](#measurement-policy)). Rerun them yourself (after the build in
+[Install](#install)): `npx minnimemory init node_modules/minnimemory/examples/CLAUDE.md`.
+
+The classifier was validated on 23 real `CLAUDE.md` and `AGENTS.md` files from public
+repositories on 2026-09-02: every one compiled losslessly, and every one landed inside the
+2,000-token always-loaded budget afterwards.
+
+Deterministic, lossless, offline. No LLM, no API key, no network. `doctor`, `init`, and `bench`
+have no runtime dependencies; only `mcp` pulls in the MCP SDK.
+
+> **Status: 1.0.0-beta.1, prepared for a public beta; not yet on npm (2026-09-16).** `doctor`, `init`, and `bench` are implemented and tested from the
+> CLI. `mcp` is the product and still early: it works, is tested in-process, and `recall` (only, not the other tools)
+> has been exercised in a second host (Claude Code loading it as an external MCP server). The
+> default client surface is `--profile basic`: `check` before the workspace is compiled and
+> `recall` after, chosen from the workspace at launch; `optimize` and `sync` (same split) are
+> opt-in behind `--allow-write`, back up before every write, and have not yet been exercised
+> outside this project's own test corpus. The original nine tools stay
+> available, unchanged, under `--profile full` for maintainers and for an agent driving a folder
+> reorganization directly. `bench` reports exact session bounds and, given a workload, an
+> explicitly-labelled model of a session; it does not measure a live agent by itself, though a
+> separate internal harness has now measured the memory layer live on two real project files (see
+> [Measurement policy](#measurement-policy)). See [Roadmap](#roadmap).
+
+---
+
+## Contents
+
+- [The problem](#the-problem)
+- [What it does](#what-it-does)
+- [Install](#install)
+- [Use it](#use-it)
+- [What gets generated](#what-gets-generated)
+- [Commands](#commands)
+- [Profiles](#profiles)
+- [Keeping it current](#keeping-it-current)
+- [What doctor counts as always loaded](#what-doctor-counts-as-always-loaded)
+- [Kinds, episodic JSON, and the write protocol](#kinds-episodic-json-and-the-write-protocol)
+- [Security](#security)
+- [Rule set](#rule-set) (moved to `docs/DESIGN.md`)
+- [Design principles](#design-principles) (moved to `docs/DESIGN.md`)
+- [What it does not do](#what-it-does-not-do)
+- [Measurement policy](#measurement-policy) (moved to `docs/DESIGN.md`)
+- [FAQ](#faq)
+- [Roadmap](#roadmap) (moved to `docs/DESIGN.md`)
+- [License](#license)
+
+---
+
+## The problem
+
+Every turn of an agentic coding session re-sends the whole prefix to the model: the system
+prompt, your project memory file (`CLAUDE.md`, `AGENTS.md`, `.cursorrules`), the conversation so
+far, and every tool result.
+
+The conversation grows because work is happening. That is fair. **The memory file does not grow
+for any good reason, and it is charged on every turn regardless.**
+
+```
+a 12,000 token memory file
+x 200 turns in a working session
+= 2,400,000 tokens of re-sent, identical text
+```
+
+Most of it is irrelevant to any given task. Your deploy runbook is being re-sent while you write
+a unit test. Your changelog is being re-sent while you debug auth.
+
+There is a second, quieter cost. Prompt caching keys on an **exact prefix match**, so one edited
+character near the top invalidates the cache for everything after it. The usual instinct, putting
+"Current status" and open TODOs at the top of the memory file where they are most visible, is
+therefore exactly backwards: it puts the most-edited content in the most cache-sensitive
+position, busting the cache on every session where anything moved.
+
+Memory files also rot in a specific way. They grow by accretion, because appending a paragraph is
+easier than restructuring, so six months in you are paying every turn for a stale directory tree,
+three copies of the test command, and a changelog nobody reads.
+
+## What it does
+
+Two commands do the work, plus an optional read-only MCP server for hosts that need one.
+
+**`doctor`** audits your current setup and tells you what it is costing. It changes nothing, needs
+no install, and takes about a second.
+
+**`init`** compiles your memory file into a small AlwaysOnMemory body plus OnDemandMemory files
+the agent loads only when they are relevant, and rewrites your memory file as a compact stub.
+Your original is backed up first, and every line of it survives into the output.
+
+Splitting is by **how often content is needed** and **how often it changes**:
+
+| tier | always loaded? | holds | churn |
+|---|---|---|---|
+| AlwaysOnMemory | yes | identity, invariants, hard constraints, conventions, plus one routing line per OnDemandMemory file | rarely edited |
+| OnDemandMemory | **no** | architecture, testing, deploy, gotchas, status, changelog | edited freely |
+
+## Install
+
+Requires Node 18 or later. The package is `minnimemory` on npm; nothing else to download.
+
+**Claude Code**, from inside the repo whose memory you want served (read-only: `check` until
+the memory is compiled, `recall` once it is):
+
+```bash
+claude mcp add minnimemory -- npx -y minnimemory mcp
+```
+
+Add `--allow-write` to let the agent also compile and maintain the file (`optimize` before the
+compile, `sync` after; every write is backed up first), and `--include-auto-memory` to include Claude Code's own
+auto-memory folder for this project:
+
+```bash
+claude mcp add minnimemory -- npx -y minnimemory mcp --allow-write --include-auto-memory
+```
+
+Use `-s user` to register it once for every project instead of the current one. Restart Claude
+Code, then ask it to run `check`.
+
+**Any other MCP host** (Cursor, Windsurf, Claude Desktop, an SDK client), in its MCP config:
+
+```json
+{
+  "mcpServers": {
+    "minnimemory": {
+      "command": "npx",
+      "args": ["-y", "minnimemory", "mcp", "/path/to/your/repo"]
+    }
+  }
+}
+```
+
+The path is optional; without it the server uses its working directory.
+
+**Command line**, for the `doctor`/`init`/`bench` commands below: `npx minnimemory doctor`, or
+`npm install -g minnimemory` for a `minnimemory` binary. Maintainers working on the compiler
+itself clone the repo, `npm ci && npm run build`, and run `node dist/cli.js` in place of `npx
+minnimemory`.
+
+## Use it
+
+Every block below is the real output of that command against
+`examples/CLAUDE.md`, a 75-line memory file for a fictional bookkeeping API, shipped in this
+package.
+
+### 1. Find out what your setup costs
+
+```
+$ npx minnimemory doctor
+
+  MM003  "Current status" (lines 6-11) holds a status heading and   high
+         dated entries and status language in the always-loaded
+         prefix
+         at CLAUDE.md:6-11
+         -> move it to an OnDemandMemory file; editing it here invalidates
+         the prompt cache every time
+
+  MM003  "Changelog" (lines 70-76) holds a status heading and dated high
+         entries and a version string in the always-loaded prefix
+         at CLAUDE.md:70-76
+         -> move it to an OnDemandMemory file; editing it here invalidates
+         the prompt cache every time
+
+  MM002  lines 28-36 are a directory tree derivable from the repo   med
+         itself
+         at CLAUDE.md:28-36
+         -> delete it; the agent can list the directory, and this copy goes
+         stale silently
+
+  3 findings (2 high, 1 med)
+  always-loaded prefix: 689 tokens across 1 file (budget 2,000, within)
+  tokenizer: approx-v2, an offline estimate
+```
+
+Every finding names a rule id, a file, a line range, and what to do about it. Nothing is changed.
+
+### 2. Preview the compile
+
+`init` is a **dry run by default**. It prints the full plan and writes nothing. The plan carries
+the decision as well as the shape: exact session bounds, a plain verdict, and the `doctor`
+findings `init` will not act on. If the verdict is "leave this file as it is", `--write` refuses
+unless you pass `--force`: the tool will not quietly make a file more expensive.
+
+```
+$ npx minnimemory init
+
+  source: CLAUDE.md (689 tokens)
+
+  AlwaysOnMemory.md                      282 tokens  always loaded
+  OnDemandMemory/                        963 tokens  on demand
+
+    OnDemandMemory/current_status_to_deployment.md     438  test, status, current, request
+    OnDemandMemory/common_tasks.md        75  common, tasks, edit, endpoint
+    OnDemandMemory/changelog.md           64  changelog, centralised, draft, flag
+    OnDemandMemory/memory_write_protocol.md     386  memory, remember, save, write
+
+  always-loaded prefix: 689 -> 282 tokens (-59.1%)
+  of which 0 tokens are the added token-discipline block (--profile none)
+  auto: source is under the 2,000 token budget, so no discipline block is embedded
+  tokenizer: approx-v2, an offline estimate
+
+  over a 50-turn session, exact bounds, no workload assumed:
+    best case (no OnDemandMemory file opened)   34,450 -> 14,100, saves 20,350 (59.1%)
+    break-even                     407 tokens of OnDemandMemory can stay open all session
+
+  verdict: compile. The prefix shrinks on every turn; OnDemandMemory files cost only when opened.
+
+  doctor findings init does not apply (init moves content, never deletes it):
+    MM002  lines 28-36 are a directory tree derivable from the repo itself (lines 28-36)
+           -> delete it; the agent can list the directory, and this copy goes stale silently
+
+  files:
+    backup    .minnimemory/original/CLAUDE.md.bak
+    create    .minnimemory/original/.gitignore
+    create    .minnimemory/AlwaysOnMemory.md
+    create    .minnimemory/manifest.json
+    create    .minnimemory/OnDemandMemory/current_status_to_deployment.md
+    create    .minnimemory/OnDemandMemory/common_tasks.md
+    create    .minnimemory/OnDemandMemory/changelog.md
+    create    .minnimemory/OnDemandMemory/memory_write_protocol.md
+    overwrite CLAUDE.md
+
+  dry run, nothing written. Pass --write to apply.
+```
+
+### 3. Apply it
+
+```bash
+npx minnimemory init --write
+```
+
+Your original is copied verbatim to `.minnimemory/original/` before anything is touched.
+
+### 4. Check the result
+
+```bash
+npx minnimemory doctor
+```
+
+### 5. Keep it honest in CI
+
+`doctor` exits non-zero when findings reach the threshold, so memory bloat fails the build the
+same way a lint error does.
+
+```yaml
+- run: npx minnimemory doctor --ci --fail-on med
+```
+
+Exit codes: `0` clean, `1` findings at or above `--fail-on`, `2` execution error.
+
+## What gets generated
+
+```
+your-repo/
+  CLAUDE.md                    compact stub: AlwaysOnMemory + the OnDemandMemory list
+  .minnimemory/
+    AlwaysOnMemory.md          always loaded: always-on content, then the OnDemandMemory list
+    OnDemandMemory/
+      <topic>.md               loaded only when relevant
+    manifest.json              token counts, hashes, triggers, provenance
+    original/
+      CLAUDE.md.bak            verbatim copy of what was there before
+```
+
+The OnDemandMemory list inside `AlwaysOnMemory.md` (and mirrored into the stub) is a routing
+manifest, not a summary, and it says nothing the discipline block already says:
+
+```markdown
+<!-- minnimemory:ondemand-list -->
+# OnDemandMemory
+- `OnDemandMemory/current_status_to_deployment.md`: test, status, current, request, token, integration
+- `OnDemandMemory/common_tasks.md`: common, tasks, edit, endpoint, generated, hour
+- `OnDemandMemory/changelog.md`: changelog, centralised, draft, flag, introduced, map
+- `OnDemandMemory/memory_write_protocol.md`: memory, remember, save, write, note, changelog
+
+Read an OnDemandMemory file only when the task needs it, and answer from it, never from this list.
+<!-- /minnimemory:ondemand-list -->
+```
+
+The closing line appears only with the `none` profile; with `routing` or `full` the same
+instruction is already in the discipline block, so it is not repeated. Assembly order lives in
+`manifest.json` (`order`), not in the prefix.
+
+OnDemandMemory file names are slugs of the source headings each one spans, so the output is
+traceable back to the original file without opening the manifest.
+
+`manifest.json` records what came from where: per-file token counts, content hashes for
+staleness detection, the keyword triggers, and the source line range each OnDemandMemory file
+came from.
+
+## Commands
+
+### `minnimemory doctor [path]`
+
+Audits. Never modifies anything. Works on a repo, a directory of memory files, or a single file.
+
+| flag | effect |
+|---|---|
+| `--json` | machine-readable output |
+| `--ci` | terse output, no colour |
+| `--fail-on <sev>` | exit 1 at this severity or above: `low`, `med`, `high`. Default `high` |
+| `--budget <n>` | always-loaded token budget for MM001. Default 2000 |
+| `--only <ids>` | run only these rules, e.g. `--only MM001,MM003` |
+| `--ignore <ids>` | skip these rules |
+| `--rules` | list the rule set and exit |
+
+### `minnimemory init [path]`
+
+Compiles. **Dry run by default.**
+
+| flag | effect |
+|---|---|
+| `--write` | actually apply. Without it, prints the plan and exits |
+| `--force` | overwrite an existing `.minnimemory/`, and compile a file the plan's verdict says to leave alone |
+| `--update` | recompile a compiled workspace from its stub and the OnDemandMemory files on disk, keeping hand edits. See [Keeping it current](#keeping-it-current) |
+| `--budget <n>` | always-loaded token budget AlwaysOnMemory must fit. AlwaysOnMemory-like sections that do not fit are routed to OnDemandMemory and listed in the plan. Default 2000 |
+| `--profile <name>` | how much guidance to embed: `auto`, `none`, `routing`, `full`. Default `auto`: `none` when the source is under the budget, `routing` at or over it |
+| `--allow-secrets` | compile even when the source holds a credential-shaped string. Without it, `--write` refuses |
+
+`init` follows a pointer: if `CLAUDE.md` is only `@AGENTS.md` or "See `AGENTS.md`", it compiles
+`AGENTS.md`. It never follows a symlink, in either direction.
+
+### `minnimemory mcp [path]`
+
+Starts an MCP server, `--profile basic` (four verbs, two registered per launch) by default, for hosts that cannot hold the
+OnDemandMemory list in their own prefix or that have no memory file at all. `mcp`'s `--profile`
+picks the tool surface (`basic`/`full`); it is a different flag from `init`'s `--profile`
+(`auto`/`none`/`routing`/`full`, [below](#profiles)), which picks how much discipline guidance
+gets embedded in a compile - same word, two unrelated knobs on two different commands. Register
+it with your agent rather than running it by hand:
+
+```bash
+claude mcp add minnimemory -- npx -y minnimemory mcp
+```
+
+Other hosts: see [Install](#install).
+
+#### The basic profile (default): `check`, `optimize`, `sync`, `recall`
+
+Four verbs, two registered per launch. Which two follows from the workspace on disk, with no
+flag to set: the server looks once, at launch, for a `.minnimemory/` directory under its root.
+
+| workspace at launch | read-only | with `--allow-write` |
+|---|---|---|
+| not compiled (no `.minnimemory/`) | `check` | `check`, `optimize` |
+| compiled | `recall` | `recall`, `sync` |
+| a memory directory (Layout B, no manifest) | `check`, `recall` | `check`, `recall`, `optimize` |
+
+`recall` cannot answer without a manifest, and `check`/`optimize` have no first compile left to
+plan once one exists, so registering the other half would charge its definition every turn for
+a tool that session could not use. The surface is fixed for the life of the process: a session
+that compiles mid-way (`optimize` says so in its response) relaunches to get `recall` and `sync`.
+One consequence: compiling a second, uncompiled file under a root that already holds a
+`.minnimemory/` directory is not reachable over MCP on this profile; the CLI's `init` covers it.
+
+| tool | returns |
+|---|---|
+| `check(target?, budget?, format?)` | audit findings, always-loaded token count against budget, drift since init for an already-compiled target, and - when `target` is a single compilable file - the compile plan: AlwaysOnMemory/OnDemandMemory split, session bounds, compile-or-leave verdict. When `target` is a memory directory, already compiled, or an index file, the plan half is left out and one line says why plus what to call instead. `format: "json"` returns `{ doctor, plan, planUnavailableReason }` |
+| `optimize(target?, budget?, force?, ops?)` | write. A compilable file: compiles it, or - already compiled - re-applies it instead and says so. A memory directory: with no `ops`, returns `scan`-shape facts plus an instruction block describing the op shapes to propose; with `ops`, applies them (`write_file`/`delete_file`/`rename_file`), backing up every `.md` file first. `force` forwards to the compile path's own force. Needs `--allow-write` |
+| `sync(target?, budget?, force?)` | write. Checks a compiled `target` for drift (MM010) and, only if something drifted, re-applies it, keeping hand edits. Changes nothing when there is no drift unless `force: true`, a deliberate re-apply (on a compiled target this is the same operation as `optimize`, which is not registered on the compiled surface); never compiles a fresh target. Needs `--allow-write` |
+| `recall(query?, file?, headingsOnly?, maxTokens?)` | `query` given: the memory sections most relevant to it, content inline, best first, under a token cap, restricted to `file` when given. `query` omitted: the OnDemandMemory routing list, no content. `headingsOnly: true` with `file`: that file's headings only. `headingsOnly` without `file` is an error. When its index rebuild finds an OnDemandMemory file changed since `init` wrote it (MM010, the same hash `check`/`doctor` use), the response names the drifted files and points at `sync()`, or at the CLI when `--allow-write` is off |
+
+A `recall()` query that matches nothing does not just say so: the response inlines the same
+routing list `query` omitted returns, so the caller lands on the right `file` without a second
+round trip, followed by "No file matched; pick from the list above and call recall() with file
+set." (`module` on the full profile's `recall`).
+
+`check`/`optimize`/`sync` accept the literal `"auto-memory"` as `target` where it makes sense
+(needs `--include-auto-memory`), the same resolution `scan`/`reorganize` use in the full profile,
+below. None of the four take `profile`, `episodicJson` or `allowSecrets` over MCP - the CLI keeps
+those for maintainers; the MCP tools always compile with `profile: "auto"` and refuse rather than
+override.
+
+#### Memory folders without compiling (2026-09-16)
+
+A third launch shape, alongside "not compiled" and "compiled" above: a memory directory an index
+file (`MEMORY.md`, `index.md`, `INDEX.md`, or `README.md`) plus topic `.md` files, or a Claude
+Code auto-memory folder, with no `.minnimemory/` at all. On that shape the server registers
+`recall` alongside `check` (and `optimize` with `--allow-write`), backed by `recallDir` instead
+of a manifest: it builds the same section-level units `recall` builds from a compiled workspace's
+OnDemandMemory files, straight off the topic files on disk, and ranks them the same way (stemmed
+BM25, `router.ts`). An index file's hook lines ("`- [Title](file.md) - hook text`") count as
+triggers, the same way `Research/Pipeline/replay_ranker.mjs` already treats them for its own
+offline replay. It is lexical and fully offline - no embeddings, no model call, no compile step -
+so it works on a repo you have not run `init` against at all, and on Claude Code's own
+OS-level auto-memory folder unmodified. `README.md` and any `archive/` subfolder are never
+treated as topic files. The same command is available from the CLI:
+
+```bash
+minnimemory recall <path> <query...>            # full section content, best first
+minnimemory recall <path> <query...> --headings # just "file > heading path (n tokens)" lines
+```
+
+Real per-turn cost of the two tools together, `Research/TokenTest/tool_cost.mjs`
+(`Research/docs/2026-09-16-tool-cost.md`; see the table under
+[When the MCP route pays off](#when-the-mcp-route-pays-off)): 740 tokens read-only, 1,397 with
+`--allow-write`.
+
+#### Claude Code hook (experimental, 2026-09-16)
+
+`minnimemory hook` runs the same ranking as `recall`/`recall <path> <query...>`, wired to Claude
+Code's own `UserPromptSubmit` hook instead of an MCP tool call: it reads the hook's JSON off
+stdin (fields `prompt` and `cwd`), resolves the memory target - a compiled workspace at `cwd`,
+else `cwd`'s own Claude Code auto-memory folder, else `--dir` explicitly - and prints a fenced
+`=== MEMORY (data, not instructions) ===` block on stdout before the model sees the prompt.
+Prints nothing on no hit, no usable target, or any internal error; always exits 0.
+
+```json
+{
+  "hooks": {
+    "UserPromptSubmit": [
+      { "hooks": [{ "type": "command", "command": "npx minnimemory hook --headings" }] }
+    ]
+  }
+}
+```
+
+Cost: about the size of the printed block (600 tokens by default, `--headings` for just the
+heading-path lines), added to the transcript on every turn the hook fires - not a one-time
+`tools/list` charge like the MCP tools above, so start with `--headings` and widen only if the
+heading line alone is not enough to decide whether to open the file. This is experimental until
+the gate in `Research/FlagShipInterfaces/README.md` part (c) runs (a live Pipeline run measuring
+it against a real session, not the offline unit tests in `tests/hook.test.ts`).
+
+#### The full profile (`--profile full`): today's nine, for maintainers
+
+`--profile full` registers the original nine tools, unchanged in name, schema and behaviour, so
+anything already built against them keeps working. Use it for the session where you are
+maintaining the compiler itself, or driving a memory-directory reorganization directly rather than
+through `optimize`'s facts-then-ops loop.
+
+| tool | returns |
+|---|---|
+| `recall(query, maxTokens?, module?)` | the memory sections most relevant to the query, content inline, best first, under a token cap (1,500 by default); a hit inside a changelog returns that entry, not the log; each result names its OnDemandMemory file, heading path, line range and token count |
+| `modules()` | the OnDemandMemory list, no content |
+| `outline(module)` | an OnDemandMemory file's headings only, so the agent can decide before paying for the body |
+| `doctor(target?, budget?, only?, ignore?, format?)` | audit findings, always-loaded token count against budget, and (for a compiled target) drift since init. `format: "json"` returns the machine-readable report instead of text |
+| `plan(target?, update?, budget?, format?)` | what compiling `target` would do - AlwaysOnMemory, OnDemandMemory, session bounds, the compile-versus-leave verdict - without writing anything. `format: "json"` returns a compact summary object instead of the rendered report |
+| `scan(target?, detail?, files?)` | structured inventory of memory files: frontmatter, sections, volatility evidence, keywords, cross-file duplicates. `target` is a path, or the literal `"auto-memory"` (needs `--include-auto-memory`). `detail: "summary"` (default) returns per-file counts and rollups; `"full"` returns every section. `files` restricts to exact rel matches, with unmatched names in `unknownFiles`. |
+| `apply(target?, budget?)` | compiles `target`, same as `init --write`. Refuses an already-compiled target - call `update()` instead - and a compile that would not shrink the prefix or holds a credential-shaped string, with no override over MCP. Needs `--allow-write` |
+| `update(target?, budget?)` | recompiles an already-compiled `target` from its stub and OnDemandMemory files, same as `init --update`. Needs `--allow-write` |
+| `reorganize(target?, ops)` | applies `write_file` / `delete_file` / `rename_file` operations to `target` (default: this server's root, or `"auto-memory"`, needs `--include-auto-memory`). Needs `--allow-write` |
+
+`recall`/`modules`/`outline` need a compiled workspace. `doctor`/`plan`/`scan` do not: they work
+on any target, compiled or not. `apply`/`update` are the single-file compile path (what the CLI's
+`init` and `init --update` do); `reorganize` is the multi-file path.
+
+#### The loop
+
+Point a client at the basic profile and the intended flow is: **check, then optimize, then
+sync**, with **recall** in between turns once something is compiled.
+
+```
+check()      - what does this file/directory cost, and what would compiling it do?
+             - (show the person the result before doing anything)
+optimize()   - compile it (or apply a reorganize plan you both approved)
+recall()     - per-turn retrieval against the compiled workspace, in later turns
+sync()       - after someone hand-edits a memory file, catch the workspace back up
+```
+
+**The server only advertises the tools you asked for, and the basic profile only the half the
+workspace can use.** Every tool definition is charged to the client's prefix on every turn,
+called or not. Real costs (`Research/TokenTest/tool_cost.mjs`, `approx-v2` tokenizer; full table
+and what the number is: `Research/docs/2026-09-16-tool-cost.md`):
+
+| launch | workspace | tools advertised | prefix cost |
+|---|---|---|---|
+| `mcp` (default, basic, read-only) | not compiled | `check` | 364 tokens |
+| `mcp --allow-write` (basic) | not compiled | `check`, `optimize` | 1,021 tokens |
+| `mcp` (default, basic, read-only) | compiled | `recall` | 329 tokens |
+| `mcp --allow-write` (basic) | compiled | `recall`, `sync` | 620 tokens |
+| `mcp` (default, basic, read-only) | memory directory | `recall`, `check` | 740 tokens |
+| `mcp --allow-write` (basic) | memory directory | `recall`, `check`, `optimize` | 1,397 tokens |
+| `mcp --profile full` (read-only) | any | `recall`, `modules`, `outline`, `scan`, `doctor`, `plan` | 1,666 tokens |
+| `mcp --profile full --allow-write` | any | + `apply`, `update`, `reorganize` | 2,668 tokens |
+
+The compiled rows are what a working session holds - well under this tool's own 2,000-token
+always-loaded budget. A tool that was not registered is absent from `tools/list` and answers
+"tool not found" if called; nothing is present-and-refusing, because a refusing tool still costs
+its definition every turn.
+
+Ranking (`recall`) is whole-word keyword overlap against the triggers `init` computed. No
+embeddings, no model call, no network. A query that shares no vocabulary with an OnDemandMemory
+file's triggers will not find it, and that is a deliberate trade for determinism, not a bug.
+
+#### When the MCP route pays off
+
+The cost that matters is the compiled one, because setup happens once: a working session on a
+compiled workspace holds `recall` for 329 tokens of definitions per turn, or `recall` and `sync`
+for 620 with `--allow-write`. The setup surface (`check` 364, `check` and `optimize` 1,021) is
+paid only by the session that does the first compile, and that session relaunches afterwards.
+The shipped `examples/CLAUDE.md` breaks even at 407 tokens per turn (see [Use it](#use-it)). The
+read-only definitions alone sit under that, but they are not the whole bill: every `recall`
+answer adds the sections it returns, up to 1,500 tokens by default, so one routed answer on that
+example costs more than the compile saved on that turn. The MCP route pays for itself when the
+per-turn prefix saving is bigger than the definition cost plus whatever `recall` returns for a
+typical query - in practice, a memory whose always-loaded prefix sits well above the 2,000-token
+budget, where the alternative `recall` is replacing is an agent reading the whole file on every
+task rather than a routed section. A small memory file is cheaper served the way this README's
+own examples are: compiled once, read directly by the agent, no server attached.
+
+#### Single-file compile (`check` + `optimize`, or `doctor` + `plan` + `apply` + `update` under `--profile full`)
+
+The CLI's `init`/`init --update`, over MCP, so an agent applies the interface without shelling
+out. `target` is a path relative to the server's root (default: the root itself); it is confined
+to the root the same way `reorganize`'s paths are - no absolute paths, no `..` segments, no
+escaping via a symlink.
+
+- `check` never writes; it is registered whenever the root is not yet compiled, with or
+  without `--allow-write`. It audits (`doctor`) and, when `target` is a compilable file,
+  previews the compile (`plan`) in the same response. Accepts `format: "json"`
+  for a machine-readable shape instead of the rendered report.
+- `optimize` is gated behind `--allow-write`, same as `sync`. On a compilable file it refuses a
+  compile that would not shrink the always-loaded prefix, refuses a source holding a
+  credential-shaped string, and - on an already-compiled target - re-applies it instead of
+  refusing outright, saying so in the response (`--profile full`'s `apply` refuses that case
+  and points at `update()`; `optimize` just does what `update()` would). `sync` only ever
+  re-applies, and only when `doctor`'s MM010 drift check finds something to re-apply or the
+  call passes `force: true` (a deliberate recompile, reachable on the compiled surface where
+  `optimize` is not registered); it keeps hand edits to AlwaysOnMemory text and OnDemandMemory
+  file content, and backs up the pre-update stub first.
+
+#### Multi-file reorganization (`optimize` with `ops`, or `scan` + `reorganize` under `--profile full`)
+
+`init` compiles one host file. A Claude Code auto-memory folder is a different shape: an
+OnDemandMemory-style list plus dozens of topic files, already multi-file, needing content moved
+*between* files rather than split out of one. `optimize`, pointed at a memory directory, splits
+the work along the same deliberate line the full profile's `scan`/`reorganize` pair always has:
+
+- **Facts first, and only facts.** With no `ops`, `optimize` returns `scan`-shape structure:
+  frontmatter (`name`/`description`/`metadata.type`), section boundaries, which sections carry
+  volatility evidence and why, candidate routing keywords, and blocks duplicated across files. It
+  never classifies, merges, or decides on its own.
+- **The calling agent makes the judgment.** Which sections are semantic vs episodic vs
+  procedural, what merges, what is stale, where content belongs under O1-O5. That reasoning is
+  the agent's, done in the agent's own session, from the facts `optimize` just handed it.
+- **Then `optimize` again, with `ops`, executes.** The same explicit `write_file` / `delete_file`
+  / `rename_file` operations `reorganize` takes, applied the same way.
+
+Both take a `target`: a path relative to the server root (default: the root itself), or the
+literal `"auto-memory"`, which resolves to the operator's OS-level Claude Code auto-memory folder
+for this root - a real directory outside the server root, so it only resolves when the server is
+launched with `--include-auto-memory`; without the flag, `"auto-memory"` is refused with a clear
+error rather than silently falling back to the server root.
+
+No LLM is embedded in this package and none is called by it. `init`, `doctor` and `scan` stay
+fully deterministic and offline. The intelligence is the agent already driving the tools, not an
+API key this package holds.
+
+Safety, non-negotiable and not skippable:
+
+- **Every `.md` file under the resolved target is backed up verbatim** to
+  `<target>/.minnimemory-backup/<timestamp>/` before the first operation of a plan runs - the only
+  files this tool can touch; `node_modules`, `.git`, `dist`, `build` are skipped. For an
+  `"auto-memory"` target, the backup lands inside that folder, not the server root.
+- **Every path in a plan is validated before any of it is applied**, so a bad path fails the
+  whole plan rather than leaving it half-applied.
+- Paths are confined to the resolved target: absolute paths, `..` segments, symlinked escapes,
+  and writes into the backup directory itself are all refused.
+- **Every `write_file`/`delete_file`/`rename_file` path must end in `.md`.** Containment alone
+  still let a write land on an extensionless file like `.git/hooks/pre-commit` inside a confined
+  root; this operates on memory files, not arbitrary files a confined write could otherwise reach.
+- **Write access is opt-in per launch.** Without `--allow-write` the server is read-only:
+  `optimize`/`sync` (basic) and `apply`/`update`/`reorganize` (full) are not registered at all,
+  so a client cannot call them.
+
+In Claude Code the file-based route is usually enough, because the agent can read an
+OnDemandMemory file directly. MCP tool definitions themselves cost prefix tokens on every turn, which is why the
+server registers only the profile you launch it with; weigh even the default two-tool basic
+profile before registering the server on a repo with only a few OnDemandMemory files.
+
+### `minnimemory bench [path]`
+
+What the compiled shape costs over a whole session, rather than on one turn.
+
+The figure `init` prints is tokens *moved* out of the always-loaded prefix. That is not the same
+as tokens saved. An OnDemandMemory file the agent opens on turn 1 becomes a tool result in the
+conversation and is re-sent on every turn after it, so it costs the rest of the session, not one
+turn. Real savings are the OnDemandMemory files a session never opens, and `bench` is what makes
+that visible.
+
+It reports two different kinds of claim, and labels which is which every time.
+
+**Bounds. Exact arithmetic, no workload, nothing to assume:**
+
+```
+$ npx minnimemory bench --turns 50
+
+    original memory file           689 tokens per turn
+    compiled always-loaded         282 tokens per turn
+    OnDemandMemory, all of it        963 tokens, on demand
+
+    best case, no OnDemandMemory file ever opened
+      34,450 -> 14,100 tokens, saves 20,350 (59.1%)
+
+    worst case, every OnDemandMemory file opened on turn 1 and carried
+      34,450 -> 62,250 tokens, costs 27,800 more
+
+    break-even: 407 tokens of OnDemandMemory can sit in
+    context from turn 1 to the end and come out even.
+```
+
+The worst case is a real number and it is a loss. On a small memory file a compiled layout can
+cost more than the file it replaced, and this command will say so rather than quietly reporting
+the prefix reduction and stopping there.
+
+**A session model. Conditional on a workload you supply:**
+
+```bash
+npx minnimemory bench --workload tasks.txt    # one task description per line
+```
+
+It routes each task at section level with the same deterministic ranker and token cap `recall` ships (`--whole-files` models an agent reading entire files instead), carries every opened section
+forward for the rest of the session, and reports the total, which OnDemandMemory files were never
+opened, what one further turn costs at the steady state, and whether the workload ever breaks even.
+
+`minnimemory` ships **no default task list**. A number produced from tasks the tool invented for
+itself would say more about the tool than about your repo, so without `--workload` only the bounds
+are reported.
+
+**This is a model, not a measurement, and the output says so.** It compares memory tokens only,
+excluding reasoning, tool results and conversation from both sides, so it is not a session bill.
+Observed usage against a live multi-turn agent is measured by a separate internal harness (the
+`claude-cli` provider) and is the only path allowed to call a number measured.
+
+## Profiles
+
+`init` embeds a short block of token-discipline guidance into AlwaysOnMemory, because an
+AlwaysOnMemory-plus-OnDemandMemory layout does nothing unless the agent is actually told to route
+rather than read everything.
+
+That block costs tokens too, so it is adjustable. Real numbers from the same 689-token
+`examples/CLAUDE.md`:
+
+| profile | what it embeds | result on that file |
+|---|---|---|
+| `none` | nothing but the one-line routing note, structure only | 689 -> 276 tokens (**-59.9%**) |
+| `routing` | the rules without which routing does not happen, plus the one that keeps new memory from bypassing them, about 200 tokens | 689 -> 476 tokens (**-30.9%**) |
+| `full` | the complete discipline set, about 690 tokens | 689 -> 969 tokens (**+40.6%**) |
+
+`auto`, the default, picks `none` for a source under the budget and `routing` at or over it, so
+a small file never pays for guidance it does not need. On this file `auto` chose `none`.
+
+`full` is a net loss on a file this size, and `init` says so rather than reporting a fake win:
+
+```
+  warning: this compile does not shrink the always-loaded prefix.
+  The source is small enough that the added instruction block costs more than routing saves.
+```
+
+That is why `auto` never picks `full` on its own: it embeds `routing`, the smallest set that
+makes the structure work, and only once the source is large enough to pay for it. `full` is an
+explicit opt-in that earns its keep on large memory files, and on high-volume agents where
+output-token discipline matters more than prefix size.
+
+## Keeping it current
+
+After the first compile, the source of truth is the stub's own AlwaysOnMemory text plus the
+OnDemandMemory files on disk. Edit either freely.
+
+**Coming back to a memory you optimised earlier? Start with `doctor`.** On a compiled workspace
+it reports whether anything has drifted from what `init` wrote, and names the command to fix it
+(over MCP, `check` does the same on a compiled target, and on the basic profile's compiled
+surface, where `check` is absent, `recall` reports drift itself when it rebuilds its index):
+
+```
+$ npx minnimemory doctor
+
+  ...findings...
+
+  compiled workspace: 4 OnDemandMemory files in .minnimemory/
+  drift: 1 file changed since init (MM010). Re-apply with: minnimemory init --update
+```
+
+When there is nothing to do it says so plainly ("no drift since init ... Nothing to re-apply"),
+so a re-run costs you one second and no thinking. `doctor` never writes. To apply:
+
+```bash
+npx minnimemory init --update
+```
+
+It strips the generated parts of the stub, routes anything you appended to it the same way a
+fresh section would be routed, keeps every OnDemandMemory file exactly as it is on disk,
+recomputes triggers and hashes, and rewrites the OnDemandMemory list, manifest, and stub. The previous stub is kept in
+`.minnimemory/previous/`. Nothing is merged against the original file, because the original is
+no longer what anyone edits.
+
+## What doctor counts as always loaded
+
+The host file, plus everything it pulls in with Claude Code's `@path` import syntax, resolved
+the way Claude Code resolves it: relative to the importing file, recursive to a depth of five,
+ignoring anything inside code spans and fenced blocks. If the host file is only a pointer at
+`AGENTS.md`, the pointer and its target are both counted. For a project that Claude Code has an
+auto-memory folder for, `MEMORY.md` from that folder counts too; its topic files are read on
+demand and are audited but not charged. This is the CLI's `doctor`/`bench` behavior; over MCP,
+`scan`/`doctor`/`plan` leave the auto-memory folder out unless launched with
+`--follow-external-imports` (see [Security](#security)).
+
+## Kinds, episodic JSON, and the write protocol
+
+Every OnDemandMemory file carries a **kind** in the manifest (O2 of the interface): `semantic`
+(facts true now, edited in place), `episodic` (things that happened, append-only: changelogs,
+logs) or `procedural` (a learned rule). The kind is decided per section from its heading and
+body, so a `project` file that blends current state with a changelog compiles into separate
+OnDemandMemory files, and an episodic section is never merged into a neighbour. `scan` reports the kind for every section
+of a memory directory too.
+
+With **`init --episodic-json`** (O3, opt in), an episodic OnDemandMemory file is written as
+`OnDemandMemory/<name>.json`: the heading, any preamble, and one entry per dated bullet with its
+continuation lines, all verbatim, so the markdown rebuilds byte for byte. `recall`, `outline`,
+`bench`, `doctor` and `init --update` read JSON OnDemandMemory files as the markdown they stand for. The
+point is Anthropic's documented finding that a model is less likely to rewrite or summarize
+JSON it was only meant to append to. Off by default until it has run in a second host.
+
+`init` also emits one OnDemandMemory file the source did not contain:
+**`OnDemandMemory/memory_write_protocol.md`**, the eleven-item memory write protocol (O9),
+routed by the OnDemandMemory list under `memory, remember, save, write, note, changelog`, so the
+agent opens it when a task is about saving memory and is never charged for it otherwise.
+`init --update` re-emits it, so a protocol change reaches every workspace on the next update. It
+accounts for one OnDemandMemory list line in the prefix figures above.
+
+OnDemandMemory paths are relative to the file the list sits in: `OnDemandMemory/<name>.md`
+inside `.minnimemory/AlwaysOnMemory.md`, and `.minnimemory/OnDemandMemory/<name>.md` in the stub
+the agent actually reads from. The first session measurement (2026-09-05) caught an agent reading
+the bare path from the stub, missing, and giving up; the stub has named the full path since.
+
+## Security
+
+- `doctor`, `init`, and `mcp` make no network calls and never run a subprocess.
+- `init` refuses a symlinked source and refuses to write through a symlink, so a cloned repo
+  cannot point `CLAUDE.md` or an OnDemandMemory file at a file outside the checkout.
+- `init` refuses `--write` when the source holds a credential-shaped string, unless you pass
+  `--allow-secrets`. The verbatim backup in `.minnimemory/original/` ships with a `.gitignore`
+  so it stays local.
+- The MCP server validates `manifest.json` and only serves files that resolve inside
+  `.minnimemory/OnDemandMemory/`, whatever the manifest says.
+- `doctor --json` includes the absolute workspace path and the headings of your auto-memory
+  files. Read it before pasting it somewhere public.
+
+## Rule set
+
+Moved to [`docs/DESIGN.md` section 12](docs/DESIGN.md#12-rule-set-for-doctor) (2026-09-16): the
+full, current MM001-MM010 table, one copy instead of two that had drifted apart.
+
+## Design principles
+
+Moved to [`docs/DESIGN.md` section 1.1](docs/DESIGN.md#11-design-principles) (2026-09-16):
+deterministic, lossless, volatile content never in AlwaysOnMemory, OnDemandMemory files emitted
+verbatim, the original never destroyed.
+
+## What it does not do
+
+See [`docs/DESIGN.md` section 2, "What it is not"](docs/DESIGN.md#2-what-it-is-not): not a
+conversation compressor, not a RAG index over your code, not a proxy or model router, not a
+summariser. One addition specific to this package: **not a compiler for a memory routing list.**
+`init` refuses `MEMORY.md`, or any file that is mostly links to topic files beside it, with no
+override - compiling a routing list re-files its hook lines as content and orphans every topic
+file. A memory directory is `check`'s job to audit and `optimize`'s (`mcp --allow-write`: facts
+first, then `ops`, backup first) to restructure. Auditing alone needs no write access - `check`
+is registered by default.
+
+## Measurement policy
+
+Moved to [`docs/DESIGN.md` section 8](docs/DESIGN.md#8-measurement-and-honesty-rules) (2026-09-16),
+including the live-run measurement table (section 8.1) and the tokenizer calibration figures.
+
+## FAQ
+
+**Does it send my code or memory files anywhere?**
+No. `doctor` and `init` make no network calls at all. There is no telemetry and no API key.
+
+**Will it destroy my `CLAUDE.md`?**
+No. `init` is dry run by default. With `--write`, your original is copied verbatim to
+`.minnimemory/original/` before anything is modified.
+
+**Why not just ask an LLM to summarise my memory file?**
+Lossy, non-reproducible, and it costs tokens to save tokens. See
+[Design principles](#design-principles).
+
+**Does this fight with prompt caching?**
+The opposite. Splitting volatile content out of the always-loaded prefix is precisely what keeps
+the cache valid across sessions.
+
+**What if the agent ignores the OnDemandMemory list and never opens an OnDemandMemory file?**
+That is the real failure mode of routing, which is why OnDemandMemory list rows are written as
+hooks rather than summaries, why `auto` embeds the `routing` profile as soon as the source is at
+the budget, and why `MM004` flags OnDemandMemory files nothing reachable ever references.
+
+**Do I have to keep the generated structure?**
+No. It is plain markdown. Edit it, restructure it, delete OnDemandMemory files. `doctor` checks budgets and
+routing, not prose style.
+
+**Does it work with Cursor or Codex?**
+`doctor` reads `AGENTS.md` and `.cursorrules` as well as `CLAUDE.md`. Only the Claude Code path
+has been exercised end to end, and this README will not claim otherwise until that changes.
+
+## Roadmap
+
+Moved to [`docs/DESIGN.md` section 11, "Milestones"](docs/DESIGN.md#11-milestones) (2026-09-16):
+merged with that section's own milestone table, which had drifted out of date. Publish state
+lives there too (as of 2026-09-16 the package is prepared but not yet on npm).
+
+## License
+
+Proprietary, free to use. Copyright MinniAI. You may install and run the package, unmodified,
+for your own use, including at work. You may not redistribute, modify or reverse engineer it. The
+files it reads and writes on your machine stay yours, and it sends nothing off your machine. Full
+terms in the `LICENSE` file shipped with the package.