memkernel 0.6.0__tar.gz → 0.8.0__tar.gz

{memkernel-0.6.0 → memkernel-0.8.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: memkernel
-Version: 0.6.0
+Version: 0.8.0
 Summary: Code-aware memory server for AI coding copilots
 License: Apache-2.0
 License-File: LICENSE
@@ -150,11 +150,33 @@ Copy the hook scripts from [`templates/hooks/`](templates/hooks/) to `.claude/ho
 | `session-start.py` | `SessionStart` | By `source`: on `startup` and `clear`, consolidates, recovers the latest checkpoint and long-term context, lists promotion candidates; on `compact`, the recovery after a compaction (the latest checkpoint the agent saved, with its age, and long-term context; no consolidation); on `resume` and `fork`, nothing |
 | `pre-compact.py` | `PreCompact` | Runs a consolidation pass (Claude Code passes no summary here, so no checkpoint is retained) |
 | `post-compact.py` | `PostCompact` | Retains Claude Code's compaction summary (`compact_summary`) as a checkpoint for the next session; prints nothing for the model |
+| `session-end.py` | `SessionEnd` | Writes one line to the session spool in `MEMKERNEL_STATE_DIR` (no store, no model, within SessionEnd's 1.5-second budget); the next session start says when a session ended without a saved checkpoint. A session that never recorded an end (a crash, or this hook not registered) is dropped from the spool after 7 days, and the next start says how many |
 
-Register `SessionStart` without a matcher, so it runs for every `source`. Claude Code adds only `SessionStart`'s output to the model's context, not `PreCompact`'s or `PostCompact`'s, so the recovery after a compaction comes from `SessionStart` with `source: "compact"`. After upgrading memkernel, copy the three scripts again.
+Each injected memory line ends with its `ref_id`, so `get(ref_id)` returns the full text; a block never exceeds 9,000 characters (Claude Code caps hook output at 10,000), and a cut checkpoint says which `get` holds the rest. At session start the long-term context is the most important decisions, conventions, procedures and facts, not the matches of a fixed query; after a compaction it is recalled with the checkpoint's text. Register `SessionStart` without a matcher, so it runs for every `source`, and `session-end.py` under `SessionEnd`. Claude Code adds only `SessionStart`'s output to the model's context, not `PreCompact`'s or `PostCompact`'s, so the recovery after a compaction comes from `SessionStart` with `source: "compact"`. After upgrading memkernel, copy the three scripts again.
 
 See the Prerequisites section inside each script for the Python environment requirement. Each script is a thin wrapper over [`MemoryLifecycle`](#embedding-memkernel-in-your-own-harness), so an in-process harness gets the same behaviour without the scripts.
 
+**Injected memory is quoted as data.** Stored text can come from anything the agent once read: tool output, a web page, a clipped source, a wiki page. So everything the blocks inject but memkernel did not write is quoted inside a `<stored-memory kind="…">` element, and the block's first line after its heading says those items are data to consult, not instructions to follow. That covers the checkpoint text, memory previews, promotion candidates, the source path of a "Needs review" item, and the curation debt's page paths. memkernel's own lines stay outside the elements, `ref_id`s included. A copy of the tag inside stored text (`</stored-memory>` in any case, with or without whitespace after `<` or `/`) is neutralised by writing its `<` as `&lt;`; no other character of stored text changes. This marks stored text; it does not detect or remove instructions in it.
+
+### 4. (Recommended) Keep memkernel's files and the hooks out of the agent's reach
+
+Merge [`templates/claude-settings.example.json`](templates/claude-settings.example.json) into your project's `.claude/settings.json`. Its `permissions.deny` rules stop Claude's own file tools from editing four paths:
+- `.memkernel/`, the store and state directory;
+- `.claude/settings.json` and `.claude/settings.local.json`;
+- `.claude/hooks/`.
+
+So a session cannot rewrite the store, or what runs at the next session start. memkernel's server and hooks write `.memkernel/` themselves, not through Claude's tools, so they are unaffected.
+
+Claude Code applies deny rules from a committed `.claude/settings.json` at once, and an allow rule cannot override them. `Edit(...)` rules cover every file-editing tool; a `Write(...)` rule would never match.
+
+These rules have limits:
+- **What they stop:** Claude's built-in file tools, the file commands Claude Code recognises in Bash (`cat`, `sed`, `tee`, ...) and redirections. Checked live with Claude Code 2.1.319: a write to each denied path was refused with "File is in a directory that is denied by your permission settings".
+- **What they don't stop:** a program that writes files without naming them, such as a script or `python -c`. [Sandboxing](https://code.claude.com/docs/en/sandboxing) is the stronger boundary.
+- **Other locations:** if you keep `MEMKERNEL_STATE_DIR` or the Chroma path elsewhere, change the first rule to match. A `/` rule anchors at the project, so a directory outside it needs an absolute `//` or a home-relative `~/` pattern, such as `Edit(~/.memkernel/**)`.
+- **The rest of `.claude/`** (agents, commands, skills) stays editable, because teams often have the agent maintain them; it is a protected path, so edits there still prompt. Add `Edit(/.claude/**)` if yours should not be editable at all.
+
+A mid-session "always allow" prompt, especially one asking for broad Bash or file access, deserves a second read: injected text can be what asked for it.
+
 ---
 
 ## Add to Kiro
@@ -176,14 +198,14 @@ Kiro (and the Kiro CLI, formerly Amazon Q Developer CLI) connects to MemKernel a
 | Tool | What it does |
 |------|-------------|
 | `retain(content, type, session_id?, checkpoint?, source?, importance?, tags?, supersedes?)` | Store a memory. Returns `ref_id`. Idempotent — identical content + type returns the same `ref_id`. `supersedes` hides the memory it replaces. |
-| `recall(query, type?, tier?, session_id?, checkpoint_only?, top_k?, include_needs_review?)` | Search by natural language or identifier. Returns lightweight `ArtifactRef` list ranked by relevance + recency + importance. Records an access (reinforces retention). Code memories flagged `needs_review` are hidden unless `include_needs_review=true`. |
+| `recall(query, type?, tier?, session_id?, checkpoint_only?, top_k?, include_needs_review?, order?)` | Search by natural language or identifier. Returns lightweight `ArtifactRef` list ranked by relevance + recency + importance. Records an access (reinforces retention). Code memories flagged `needs_review` are hidden unless `include_needs_review=true`. |
 | `get(ref_id)` | Fetch full content of a specific memory. Records an access. |
 | `forget(ref_id)` | Delete a memory from all indexes. |
 | `promote(ref_id, type, content?, importance?, tags?)` | Turn a short-term/working memory into a long-term one (you supply the distilled text). Keeps provenance via `promoted_from`. |
 | `consolidate(session_id?, limit?, synthesize?, revalidate?)` | Purge expired short-term memories and list the ones most worth promoting (ranked by importance × how often they were recalled). `synthesize=true` also asks an external reasoner to draft long-term statements (`suggested`); `revalidate=true` first checks every code memory against its source file (`revalidated`). The report always carries a `health` summary. |
 | `ask(ref_id, question, mode?)` | Answer a question about one memory without loading it — for a `document` the full body is consulted. `mode`: `direct` \| `rag` \| `rlm` \| `auto`, passed to the reasoner unchanged; the built-in reasoner answers in `rag` mode with the best-matching chunks. Returns `answer`, `mode`, `citations`, `usage`. |
 | `wiki_export(wiki_dir?)` | Compile long-term memories without a wiki page into draft markdown pages (`index.md`, `log.md` updated). |
-| `wiki_import(wiki_dir?)` | Index wiki pages as long-term memories: approved revisions only with wiki-studio ≥ 0.7.0, every page without it (idempotent; changed pages supersede; `corrects` supersedes the memories a page corrects). |
+| `wiki_import(wiki_dir?, adopt_legacy?, adopt_pages?, adopt_manifest?)` | Index wiki pages as long-term memories, each wiki keeping to its own: approved revisions only with wiki-studio ≥ 0.7.0, every page without it (idempotent; changed pages supersede; `corrects` supersedes the memories a page corrects). |
 
 ### Memory types and tiers
 
@@ -203,7 +225,7 @@ Retention: `ttl = base_ttl × (1 + access_boost × access_count)`, measured from
 
 ### Session protocol
 
-1. **Session start** — `recall(tier="long_term", query=<task>)`, plus the latest checkpoint (`checkpoint_only=true`). The `session-start.py` hook does this and prints promotion candidates; after a compaction it prints a shorter recovery block instead.
+1. **Session start** — `recall(tier="long_term", query=<task>)`, plus the latest checkpoint (`checkpoint_only=true, order="newest"`: newest first, not the most relevant). The `session-start.py` hook does this and prints promotion candidates; after a compaction it prints a shorter recovery block instead.
 2. **While working** — `retain(type="task")` when the plan changes; `retain(type="observation")` for findings; durable knowledge as `decision` / `convention` / `fact` / `procedure`.
 3. **Before compaction** — `retain(type="episode", checkpoint=true)` with your own summary of where the work stands; the `pre-compact.py` hook runs `consolidate()`, and the `post-compact.py` hook keeps Claude Code's compaction summary as a checkpoint.
 4. **After compaction / end of phase** — `promote(ref_id, type, content)` for the candidates worth keeping.
@@ -222,7 +244,7 @@ retain(
 
 recall(query="database choice", top_k=5)
 
-recall(query="session checkpoint", checkpoint_only=True)
+recall(query="session checkpoint", checkpoint_only=True, order="newest", top_k=1)  # the latest checkpoint
 
 retain(content="pytest hangs unless MEMKERNEL_PROJECT_ID is set", type="observation", tags=["ci"])
 
@@ -279,7 +301,10 @@ setting is logged as a warning at startup.
 | `MEMKERNEL_RECALL_WEIGHT_RECENCY` | `0.3` | Weight of recency in the final recall score |
 | `MEMKERNEL_RECALL_WEIGHT_IMPORTANCE` | `0.3` | Weight of importance in the final recall score |
 | `MEMKERNEL_WIKI_DIR` | `` | Directory of the LLM wiki used by `wiki_export` / `wiki_import` (empty = pass `wiki_dir` explicitly) |
+| `MEMKERNEL_STATE_DIR` | `.memkernel/state` | memkernel's own state outside ChromaDB: the wiki sync ledger. Resolved like `MEMKERNEL_CHROMA_PERSIST_PATH` (a relative path is relative to the process's working directory, so set it absolute when the persist path is); keep it out of git |
 | `MEMKERNEL_WIKI_PROPOSALS_DIR` | `""` | Where wiki-studio patch-sets are written before promote (`memkernel[wiki]`); empty means `<project root>/.memkernel/wiki-proposals`. Setting it requests the gate: without wiki-studio installed, a warning is logged. |
+| `MEMKERNEL_WIKI_MODE` | `auto` | `auto`: wiki-studio's gate and review state when wiki-studio 0.7.0 or later is installed, the direct writer and file-reading import when it is not. `governed`: `wiki_export` and `wiki_import` refuse, writing nothing, unless wiki-studio is installed with every capability memkernel uses and the wiki's manifest is `governed` (`approval_required: true`) without schema drift. `ungated`: wiki-studio is never loaded. Responses report `mode` (`gated` or `ungated`, the path that ran) and `configured_mode` (this setting). |
+| `MEMKERNEL_WIKI_EXPORT_BATCH` | `10` | New pages one `wiki_export` writes at most, the most important memories first; the response's `remaining` counts the rest (`null` when the gate's outcome is uncertain or `governed` refused before reading the wiki). `0` = no limit; a negative value is refused |
 | `MEMKERNEL_TRANSPORT` | `stdio` | `stdio` or `streamable-http` |
 | `MEMKERNEL_HOST` | `127.0.0.1` | Host for `streamable-http` transport |
 | `MEMKERNEL_PORT` | `8765` | Port for `streamable-http` transport |
@@ -321,7 +346,9 @@ for ref in svc.recall("database"):
 report = svc.consolidate()                 # {"purged": n, "candidates": [...]}
 ```
 
-`build_service()` is the supported entry point; the MCP server, the admin API and the hook templates all use it. Integration notes for `code-copilot-team` and `haa` are in [`docs/integration/harness-integration.md`](docs/integration/harness-integration.md).
+`build_service()` is the supported entry point; the MCP server, the admin API and the hook templates all use it. If you pass a `PersistentClient` as `chroma_client` and also set `MEMKERNEL_CHROMA_PERSIST_PATH` (in the environment or in the settings), both must name the same directory, or `ValueError` is raised; with the default setting, the client's own directory is used.
+
+**Store locks.** Processes that share a store (the MCP server, a hook, a script) coordinate through locks keyed by the store itself: lock files in the persist directory, `.memkernel-<name>-<key>.lock`, whatever each process's `MEMKERNEL_STATE_DIR` is; an in-memory store's locks are per process. Only one writer changes a record's tags, under the store's `tags` lock, so a tag added by one process is never dropped by another's write. Locks are not re-entrant (taking one the thread already holds raises `LockReentryError`), a wait longer than 10 seconds is logged once, and spelling the persist directory with different letter case in two processes gives two locks, so use one spelling. The locks use `fcntl` and are verified on POSIX (Linux, macOS) only; on Windows they run unlocked across processes, with one logged warning per lock. Integration notes for `code-copilot-team` and `haa` are in [`docs/integration/harness-integration.md`](docs/integration/harness-integration.md).
 
 ### The session protocol as a component
 
@@ -360,13 +387,41 @@ memory.capture_compact_summary(compact_summary, session_id=..., trigger=...)  #
 
 Long-term memory has a human-readable twin: a markdown wiki in the [LLM Wiki](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f) pattern (`index.md`, `log.md`, typed pages with cited sources — the same schema as `code-copilot-team/knowledge/wiki`).
 
+To see the whole loop on a clean install, follow wiki-studio's [memkernel composition quickstart](https://github.com/gosha70/wiki-studio/blob/main/docs/quickstart-memkernel.md): retain a memory, export it through wiki-studio's gate, correct and approve the page, import only the approved revision, and see the next `recall` return it. CI runs the quickstart against memkernel **0.6.0**.
+
 - `wiki_export` compiles every long-term memory that has no page yet into a **draft** page under `decisions/`, `concepts/` or `workflows/`, with the required sections and a `memkernel://<project>/<ref_id>` citation. Existing pages are never overwritten.
 - `wiki_import` indexes pages as long-term memories so `recall` finds the reviewed version; a changed page supersedes its earlier memory. With wiki-studio 0.7.0 or later installed it is **governed** (`"governed": true`): it imports only pages wiki-studio's review state reports approved, and only when the page's bytes are still the approved revision, so drafts, self-declared approvals and pages edited after approval are not imported (`not_approved` says why); an edited page keeps its last approved memory until it is approved again. memkernel never approves a page; a curator does, with `wiki-studio approve`. Without wiki-studio the import reads the page files, drafts included, and reports `"governed": false`. An installed wiki-studio that is broken or older than 0.7.0 imports nothing and reports `import_error`.
+- **Withdrawal (governed import, 0.7).** A page's memory leaves the current answers, its record kept for history, when:
+  - the page is **deleted**. Only memories this wiki's sync ledger recorded as imported are withdrawn this way; a memory imported before 0.7, or without wiki-studio, never is.
+  - the page is **deprecated**, once the `status: deprecated` revision is approved.
+  - the page is **superseded** (`superseded_by: <page path>`). Its memory is replaced by the successor page's memory; if the successor has none yet, the old memory is withdrawn and linked once it does (`pending` until then).
+
+  The response lists `withdrawn`, `superseded`, `pending` and `sync_failed` (retried at the next import). The ledger is one JSON file per project and wiki under `MEMKERNEL_STATE_DIR/wiki-sync/`; a corrupt one is kept aside, reported as `ledger_error`, and rebuilt.
 - **Corrections.** An approved page can name the memories it corrects in its frontmatter: `corrects: <ref_id>, <ref_id>` (one line; tokens of letters, digits, `-` and `_`). On import each named memory that did not come from the wiki is superseded by the page's memory: it stops being a current answer, and its record is kept. References that cannot be applied are listed in `unresolved` with a reason (unknown, another project's, already superseded, or a wiki memory: correct that page instead); a value that breaks the grammar is listed in `invalid_corrects`, and the page is still imported. An approved page that is already current is listed in `unchanged`, and its corrections are reconciled again on every import, so a reference that could not be applied earlier takes effect once it can; corrections already applied are not repeated.
 
+- **Each wiki keeps to its own memories (0.7).** Every memory `wiki_import` writes, or finds current, carries its wiki's identity: `wiki-path:<hash>` (the wiki directory's path relative to the project root) and, when the wiki has a wiki-studio manifest, `wiki-id:<uuid>`. A page's previous memory, its corrections, its supersession and its deprecation are only that wiki's, so two wikis in one store may share page paths and `page_id`s. These tags are reserved: `retain` refuses them, and an ordinary `retain` whose text matches a wiki memory gets its own record instead of merging into it; `promote` drops every wiki tag from its copy.
+  - **A wiki that may move, or that shares a store with another wiki, needs a manifest.** The path a wiki is known by is relative to its project root: two project roots sharing one store, with wikis at the same relative path, get the same `wiki-path:`, so the second import is refused (memories at this path belong to another wiki) even with manifests. Use one store per project, or different relative paths. Without one, a wiki *is* its path: moved, it starts afresh, and whatever wiki is later placed at the old path continues its memories. A copied wiki directory (two directories, one `wiki_id`) is one wiki: each import moves its memories to the directory imported last.
+  - **Upgrading a 0.6 store:** memories written before 0.7 carry no identity, so the first import writes nothing for their pages and lists them under `ambiguous` (`{"path", "ref_ids", "reason"}`). Import once with `adopt_legacy=true` to adopt each page's sole such memory; a page with several stays `ambiguous` (forget the ones that are not its own). With several wikis in one store, name the records instead: `adopt_pages=[{"path", "ref_id"}, …]` copied from `ambiguous`; a named record that changed meanwhile is not adopted (`candidate changed`).
+  - **Identity changes stop the import before any write, with the remedy:** a wiki that gained a manifest needs `adopt_manifest=true` once (its sync ledger moves to the `wiki_id` key; if both ledger files exist, delete the stale one); memories at this path that belong to another `wiki_id` (the manifest removed or replaced, or a moved wiki not yet imported from its new place) stop it until the manifest is restored, the moved wiki is imported first, or those memories are forgotten. A wiki with a manifest that moves keeps its memories; the next import moves their path.
+  - One import runs at a time per store (the `wiki-import` store lock). On Windows the import runs unlocked with one warning, as in 0.6: two concurrent imports of one wiki can leave two records for a page (the next import reports it as `duplicate records`; forget one), and two wikis adopting the same record at once can tag it for one while the other supersedes it. Give processes that share a store the same `MEMKERNEL_STATE_DIR`, so they share the sync ledger.
 Set `MEMKERNEL_WIKI_DIR` (for example `knowledge/wiki`) or pass `wiki_dir` to the tools. The standalone maintainer that owns this page schema is [wiki-studio](https://github.com/gosha70/wiki-studio). With the `memkernel[wiki]` extra installed, every `wiki_export` response carries wiki-studio's lint and health findings for the whole wiki after the export, curated pages included, under `lint` (`violations`, `audit_errors`, `pages_checked`, `health` items with `kind`, `severity`, `pages`, `description`); without it `lint` is `null` and `lint_hint` says how to enable it; if the lint pass itself fails, `lint` is `null` and `lint_error` says why. The files written are the same in every case.
 
-With wiki-studio 0.7.0 or later installed, `wiki_export` writes through wiki-studio's promote gate: wiki-studio's `init` creates `index.md` and `log.md` if they are missing (wiki-studio's own scaffold, `status: stable`), then every new page goes with its index and log lines into one patch-set that is proposed and promoted. Other writers can work in the same wiki: wiki-studio's commit writes only the paths the patch-set names, under its lock, and refuses a page someone else created in the meantime. A rejected export writes no page and the response carries `gate_error` (a new wiki keeps the `index.md` and `log.md` that `init` created); so does an installed wiki-studio that is broken or older than 0.7.0, rather than falling back to the direct writer. A promoted export's response adds `proposal`, `submission_digest` and `archived_dir` from wiki-studio's receipt. If `archived_dir` is `null`, the pages landed but wiki-studio did not finish: it could not archive the proposal, or it failed after committing the pages (memkernel checks wiki-studio's receipt: while it still says `committing`, every planned write, index and log included, must be at its post-write sha256; a `committed` receipt needs memkernel's pages at theirs; an unusable receipt falls back to comparing the pages' bytes). `archive_hint` gives the `promote` command that tries to finish it, with absolute paths, so it runs from any directory. It applies nothing twice, but if the wiki or the proposal changed since, for example another writer appended to `index.md`, it may refuse with `uncertain_outcome` or `payload_changed` and write nothing; then compare the wiki with the proposal's `receipt.json` by hand. memkernel never runs it itself. If wiki-studio failed during its commit and its receipt shows neither a commit nor a rollback (say the page was written, the `index.md` write failed and removing the page failed too), the response carries `gate_error` with `uncertain_outcome` and `proposal`: `written` is empty, but the pages it names may be on disk; compare the wiki with the `receipt.json` it names before exporting again. `receipt_error` says the receipt could not be read or used. A failed export leaves its patch-set under the proposals directory for inspection; delete it once the cause is fixed. Patch-sets go to `MEMKERNEL_WIKI_PROPOSALS_DIR`, by default `.memkernel/wiki-proposals/` under the project root. Commit `<wiki>/.audit/`, wiki-studio's audit trail of promoted proposals; ignore `.memkernel/wiki-proposals/` and wiki-studio's lock file next to the wiki directory, `.<wiki-dir>.wiki-studio.lock` (for `knowledge/wiki`, `knowledge/.wiki.wiki-studio.lock`).
+With wiki-studio 0.7.0 or later installed, `wiki_export` writes through wiki-studio's promote gate: wiki-studio's `init` creates `index.md` and `log.md` if they are missing (wiki-studio's own scaffold, `status: stable`), then every new page goes with its index and log lines into one patch-set that is proposed and promoted. Other writers can work in the same wiki: wiki-studio's commit writes only the paths the patch-set names, under its lock, and refuses a page someone else created in the meantime. A rejected export writes no page and the response carries `gate_error` (a new wiki keeps the `index.md` and `log.md` that `init` created); so does an installed wiki-studio that is broken or older than 0.7.0, rather than falling back to the direct writer. A promoted export's response adds `proposal`, `submission_digest` and `archived_dir` from wiki-studio's receipt. If `archived_dir` is `null`, the pages landed but wiki-studio did not finish: it could not archive the proposal, or it failed after committing the pages (memkernel checks wiki-studio's receipt: while it still says `committing`, every planned write, index and log included, must be at its post-write sha256; a `committed` receipt needs memkernel's pages at theirs; an unusable receipt falls back to comparing the pages' bytes). `archive_hint` gives the `promote` command that tries to finish it, with absolute paths, so it runs from any directory. It applies nothing twice, but if the wiki or the proposal changed since, for example another writer appended to `index.md`, it may refuse with `uncertain_outcome` or `payload_changed` and write nothing; then compare the wiki with the proposal's `receipt.json` by hand. memkernel never runs it itself. If wiki-studio failed during its commit and its receipt shows neither a commit nor a rollback (say the page was written, the `index.md` write failed and removing the page failed too), the response carries `gate_error` with `uncertain_outcome` and `proposal`: `written` is empty, but the pages it names may be on disk; compare the wiki with the `receipt.json` it names before exporting again. `receipt_error` says the receipt could not be read or used. A failed export leaves its patch-set under the proposals directory for inspection; delete it once the cause is fixed. Patch-sets go to `MEMKERNEL_WIKI_PROPOSALS_DIR`, by default `.memkernel/wiki-proposals/` under the project root. Commit `<wiki>/.audit/`, wiki-studio's audit trail of promoted proposals; ignore `.memkernel/wiki-proposals/`, `.memkernel/state/` (memkernel's sync ledger) and wiki-studio's files next to the wiki directory, `.<wiki-dir>.wiki-studio.*` (for `knowledge/wiki`, `knowledge/.wiki.wiki-studio.*`: its lock file and, from wiki-studio 0.8, its search cache).
+
+**Wiki modes.** `MEMKERNEL_WIKI_MODE` is a property of the deployment, not of a call. `governed` runs one preflight before any page is written or read, for export and import alike: wiki-studio 0.7.0 or later installed and working; its `info` listing the capabilities `wiki-lock`, `receipts`, `root-manifest`, `page-id`, `approval-record`, `superseded-by` and `findings`; the wiki's manifest mode `governed`; no schema drift. A `managed` or `legacy` wiki is refused with the remedy: run `wiki-studio init` if there is no manifest, then set `approval_required: true` in the manifest's policy. A refusal reports `gate_error` (export) or `import_error` (import) and writes nothing; `update_index=False` is refused too, since the direct writer is never used. A gate rejection after the preflight commits no drafted page, but a new wiki keeps the `index.md` and `log.md` that wiki-studio's `init` created: memkernel keeps no outer rollback. `ungated` never loads wiki-studio (no gate, lint, review state or findings): `lint` is `null` and `lint_hint` names the mode.
+
+**Export batches and grounding.** One `wiki_export` writes at most `MEMKERNEL_WIKI_EXPORT_BATCH` new pages (by importance, highest first, then oldest, then `ref_id`), so a curator reviews a small batch at a time; `remaining` counts the eligible memories still without a page, all of them after a rejected batch. Every exported page carries `memkernel_sha256`, the memory's content hash, beside `memkernel_ref`; a `path:` source gains `sha:`, the commit it is grounded at (`git log -1 --format=%h`, wiki-studio's citation rule), only when the project's `git ls-files` lists exactly that file and `git status` shows no change to it: a modified, staged or untracked file, a file in a submodule, or no git gets none. Pages created through the gate carry wiki-studio's `page_id`, so a page renamed or moved under its `page_id` keeps its memory: the next import supersedes the memory at the old path with one at the new path, also when the moved page's bytes are unchanged.
+
+**Curation debt at session start.** With `MEMKERNEL_WIKI_DIR` set and wiki-studio available (in `governed`, only after the preflight passes), the session-start block (`startup`, `clear`) gains `### Curation debt`: one line of wiki-studio's `findings` counts in a fixed order (review kinds first) plus the sync ledger's unfinished actions (`sync pending`, `sync failed`), up to three example pages, and a hint to run `wiki-studio findings`. It uses at most 500 characters, taken from the long-term context's share, so the block stays under 9,000 characters. The check runs in a child process bounded to 2.5 seconds once it has started (a fork or exec the operating system holds up is not covered); it reads the sync ledger without changing it and never waits for an import that holds it. Any failure is one `Curation debt unavailable: <reason>` line. Not on `compact`, `resume` or `fork`, nor in `ungated`.
+
+**wiki-studio's files beside the wiki, its ingest log, and wiki-studio 0.8.** When memkernel exports through the gate, wiki-studio creates its lock file, `.<wiki-dir>.wiki-studio.lock`, beside the wiki directory, and from 0.8 a curator's `find` or `query` adds a search cache, `.<wiki-dir>.wiki-studio.search.sqlite`, there too: for a wiki kept in git, ignore both with `.<wiki-dir>.wiki-studio.*` (for `knowledge/wiki`, `knowledge/.wiki.wiki-studio.*`). memkernel creates no file of its own there. memkernel reads no wiki-studio ingest log, so the log's `hash_recipe` has no consumer in memkernel. From wiki-studio 0.8.0 a page that is not valid UTF-8 fails lint for the whole wiki, so every promote, memkernel's gated export included, is refused with a `gate_error` naming that page until it is re-encoded or removed. From 0.8.0 an unknown `WIKI_STUDIO_RANKER` value makes wiki-studio's `info` fail, so a `governed` export or import, a gated import and the curation debt refuse with wiki-studio's message naming the variable. memkernel needs wiki-studio 0.7.0 or later and is tested at 0.7.0; an informational CI job runs the wiki tests against the newest release below 1.0 on every pull request and weekly.
+
+wiki-studio's MCP server (`wiki-studio-mcp`, from wiki-studio 0.8.0) needs `mcp>=2.3`, which memkernel's `mcp<2` excludes. Run it in its own environment, at the same wiki-studio version as memkernel's:
+
+1. In memkernel's environment, read the version: `python -c "import importlib.metadata as m; print(m.version('wiki-studio'))"`. If it is below 0.8.0, upgrade it there first: `pip install 'wiki-studio>=0.8,<1'`.
+2. Install exactly that version for the server, for example `pipx install 'wiki-studio[mcp]==0.8.1'`, and register it beside memkernel's server.
+
+On one machine and one filesystem, wiki-studio's lock serializes the two servers' writes to the same wiki. It does not coordinate other hosts, and network filesystems are outside its scope.
 
 Rationale: [`docs/adr/ADR-008-llm-wiki-as-semantic-layer.md`](docs/adr/ADR-008-llm-wiki-as-semantic-layer.md) and [`docs/adr/ADR-010-wiki-studio-as-wiki-maintainer.md`](docs/adr/ADR-010-wiki-studio-as-wiki-maintainer.md).
 

{memkernel-0.6.0 → memkernel-0.8.0}/README.md

@@ -119,11 +119,33 @@ Copy the hook scripts from [`templates/hooks/`](templates/hooks/) to `.claude/ho
 | `session-start.py` | `SessionStart` | By `source`: on `startup` and `clear`, consolidates, recovers the latest checkpoint and long-term context, lists promotion candidates; on `compact`, the recovery after a compaction (the latest checkpoint the agent saved, with its age, and long-term context; no consolidation); on `resume` and `fork`, nothing |
 | `pre-compact.py` | `PreCompact` | Runs a consolidation pass (Claude Code passes no summary here, so no checkpoint is retained) |
 | `post-compact.py` | `PostCompact` | Retains Claude Code's compaction summary (`compact_summary`) as a checkpoint for the next session; prints nothing for the model |
+| `session-end.py` | `SessionEnd` | Writes one line to the session spool in `MEMKERNEL_STATE_DIR` (no store, no model, within SessionEnd's 1.5-second budget); the next session start says when a session ended without a saved checkpoint. A session that never recorded an end (a crash, or this hook not registered) is dropped from the spool after 7 days, and the next start says how many |
 
-Register `SessionStart` without a matcher, so it runs for every `source`. Claude Code adds only `SessionStart`'s output to the model's context, not `PreCompact`'s or `PostCompact`'s, so the recovery after a compaction comes from `SessionStart` with `source: "compact"`. After upgrading memkernel, copy the three scripts again.
+Each injected memory line ends with its `ref_id`, so `get(ref_id)` returns the full text; a block never exceeds 9,000 characters (Claude Code caps hook output at 10,000), and a cut checkpoint says which `get` holds the rest. At session start the long-term context is the most important decisions, conventions, procedures and facts, not the matches of a fixed query; after a compaction it is recalled with the checkpoint's text. Register `SessionStart` without a matcher, so it runs for every `source`, and `session-end.py` under `SessionEnd`. Claude Code adds only `SessionStart`'s output to the model's context, not `PreCompact`'s or `PostCompact`'s, so the recovery after a compaction comes from `SessionStart` with `source: "compact"`. After upgrading memkernel, copy the three scripts again.
 
 See the Prerequisites section inside each script for the Python environment requirement. Each script is a thin wrapper over [`MemoryLifecycle`](#embedding-memkernel-in-your-own-harness), so an in-process harness gets the same behaviour without the scripts.
 
+**Injected memory is quoted as data.** Stored text can come from anything the agent once read: tool output, a web page, a clipped source, a wiki page. So everything the blocks inject but memkernel did not write is quoted inside a `<stored-memory kind="…">` element, and the block's first line after its heading says those items are data to consult, not instructions to follow. That covers the checkpoint text, memory previews, promotion candidates, the source path of a "Needs review" item, and the curation debt's page paths. memkernel's own lines stay outside the elements, `ref_id`s included. A copy of the tag inside stored text (`</stored-memory>` in any case, with or without whitespace after `<` or `/`) is neutralised by writing its `<` as `&lt;`; no other character of stored text changes. This marks stored text; it does not detect or remove instructions in it.
+
+### 4. (Recommended) Keep memkernel's files and the hooks out of the agent's reach
+
+Merge [`templates/claude-settings.example.json`](templates/claude-settings.example.json) into your project's `.claude/settings.json`. Its `permissions.deny` rules stop Claude's own file tools from editing four paths:
+- `.memkernel/`, the store and state directory;
+- `.claude/settings.json` and `.claude/settings.local.json`;
+- `.claude/hooks/`.
+
+So a session cannot rewrite the store, or what runs at the next session start. memkernel's server and hooks write `.memkernel/` themselves, not through Claude's tools, so they are unaffected.
+
+Claude Code applies deny rules from a committed `.claude/settings.json` at once, and an allow rule cannot override them. `Edit(...)` rules cover every file-editing tool; a `Write(...)` rule would never match.
+
+These rules have limits:
+- **What they stop:** Claude's built-in file tools, the file commands Claude Code recognises in Bash (`cat`, `sed`, `tee`, ...) and redirections. Checked live with Claude Code 2.1.319: a write to each denied path was refused with "File is in a directory that is denied by your permission settings".
+- **What they don't stop:** a program that writes files without naming them, such as a script or `python -c`. [Sandboxing](https://code.claude.com/docs/en/sandboxing) is the stronger boundary.
+- **Other locations:** if you keep `MEMKERNEL_STATE_DIR` or the Chroma path elsewhere, change the first rule to match. A `/` rule anchors at the project, so a directory outside it needs an absolute `//` or a home-relative `~/` pattern, such as `Edit(~/.memkernel/**)`.
+- **The rest of `.claude/`** (agents, commands, skills) stays editable, because teams often have the agent maintain them; it is a protected path, so edits there still prompt. Add `Edit(/.claude/**)` if yours should not be editable at all.
+
+A mid-session "always allow" prompt, especially one asking for broad Bash or file access, deserves a second read: injected text can be what asked for it.
+
 ---
 
 ## Add to Kiro
@@ -145,14 +167,14 @@ Kiro (and the Kiro CLI, formerly Amazon Q Developer CLI) connects to MemKernel a
 | Tool | What it does |
 |------|-------------|
 | `retain(content, type, session_id?, checkpoint?, source?, importance?, tags?, supersedes?)` | Store a memory. Returns `ref_id`. Idempotent — identical content + type returns the same `ref_id`. `supersedes` hides the memory it replaces. |
-| `recall(query, type?, tier?, session_id?, checkpoint_only?, top_k?, include_needs_review?)` | Search by natural language or identifier. Returns lightweight `ArtifactRef` list ranked by relevance + recency + importance. Records an access (reinforces retention). Code memories flagged `needs_review` are hidden unless `include_needs_review=true`. |
+| `recall(query, type?, tier?, session_id?, checkpoint_only?, top_k?, include_needs_review?, order?)` | Search by natural language or identifier. Returns lightweight `ArtifactRef` list ranked by relevance + recency + importance. Records an access (reinforces retention). Code memories flagged `needs_review` are hidden unless `include_needs_review=true`. |
 | `get(ref_id)` | Fetch full content of a specific memory. Records an access. |
 | `forget(ref_id)` | Delete a memory from all indexes. |
 | `promote(ref_id, type, content?, importance?, tags?)` | Turn a short-term/working memory into a long-term one (you supply the distilled text). Keeps provenance via `promoted_from`. |
 | `consolidate(session_id?, limit?, synthesize?, revalidate?)` | Purge expired short-term memories and list the ones most worth promoting (ranked by importance × how often they were recalled). `synthesize=true` also asks an external reasoner to draft long-term statements (`suggested`); `revalidate=true` first checks every code memory against its source file (`revalidated`). The report always carries a `health` summary. |
 | `ask(ref_id, question, mode?)` | Answer a question about one memory without loading it — for a `document` the full body is consulted. `mode`: `direct` \| `rag` \| `rlm` \| `auto`, passed to the reasoner unchanged; the built-in reasoner answers in `rag` mode with the best-matching chunks. Returns `answer`, `mode`, `citations`, `usage`. |
 | `wiki_export(wiki_dir?)` | Compile long-term memories without a wiki page into draft markdown pages (`index.md`, `log.md` updated). |
-| `wiki_import(wiki_dir?)` | Index wiki pages as long-term memories: approved revisions only with wiki-studio ≥ 0.7.0, every page without it (idempotent; changed pages supersede; `corrects` supersedes the memories a page corrects). |
+| `wiki_import(wiki_dir?, adopt_legacy?, adopt_pages?, adopt_manifest?)` | Index wiki pages as long-term memories, each wiki keeping to its own: approved revisions only with wiki-studio ≥ 0.7.0, every page without it (idempotent; changed pages supersede; `corrects` supersedes the memories a page corrects). |
 
 ### Memory types and tiers
 
@@ -172,7 +194,7 @@ Retention: `ttl = base_ttl × (1 + access_boost × access_count)`, measured from
 
 ### Session protocol
 
-1. **Session start** — `recall(tier="long_term", query=<task>)`, plus the latest checkpoint (`checkpoint_only=true`). The `session-start.py` hook does this and prints promotion candidates; after a compaction it prints a shorter recovery block instead.
+1. **Session start** — `recall(tier="long_term", query=<task>)`, plus the latest checkpoint (`checkpoint_only=true, order="newest"`: newest first, not the most relevant). The `session-start.py` hook does this and prints promotion candidates; after a compaction it prints a shorter recovery block instead.
 2. **While working** — `retain(type="task")` when the plan changes; `retain(type="observation")` for findings; durable knowledge as `decision` / `convention` / `fact` / `procedure`.
 3. **Before compaction** — `retain(type="episode", checkpoint=true)` with your own summary of where the work stands; the `pre-compact.py` hook runs `consolidate()`, and the `post-compact.py` hook keeps Claude Code's compaction summary as a checkpoint.
 4. **After compaction / end of phase** — `promote(ref_id, type, content)` for the candidates worth keeping.
@@ -191,7 +213,7 @@ retain(
 
 recall(query="database choice", top_k=5)
 
-recall(query="session checkpoint", checkpoint_only=True)
+recall(query="session checkpoint", checkpoint_only=True, order="newest", top_k=1)  # the latest checkpoint
 
 retain(content="pytest hangs unless MEMKERNEL_PROJECT_ID is set", type="observation", tags=["ci"])
 
@@ -248,7 +270,10 @@ setting is logged as a warning at startup.
 | `MEMKERNEL_RECALL_WEIGHT_RECENCY` | `0.3` | Weight of recency in the final recall score |
 | `MEMKERNEL_RECALL_WEIGHT_IMPORTANCE` | `0.3` | Weight of importance in the final recall score |
 | `MEMKERNEL_WIKI_DIR` | `` | Directory of the LLM wiki used by `wiki_export` / `wiki_import` (empty = pass `wiki_dir` explicitly) |
+| `MEMKERNEL_STATE_DIR` | `.memkernel/state` | memkernel's own state outside ChromaDB: the wiki sync ledger. Resolved like `MEMKERNEL_CHROMA_PERSIST_PATH` (a relative path is relative to the process's working directory, so set it absolute when the persist path is); keep it out of git |
 | `MEMKERNEL_WIKI_PROPOSALS_DIR` | `""` | Where wiki-studio patch-sets are written before promote (`memkernel[wiki]`); empty means `<project root>/.memkernel/wiki-proposals`. Setting it requests the gate: without wiki-studio installed, a warning is logged. |
+| `MEMKERNEL_WIKI_MODE` | `auto` | `auto`: wiki-studio's gate and review state when wiki-studio 0.7.0 or later is installed, the direct writer and file-reading import when it is not. `governed`: `wiki_export` and `wiki_import` refuse, writing nothing, unless wiki-studio is installed with every capability memkernel uses and the wiki's manifest is `governed` (`approval_required: true`) without schema drift. `ungated`: wiki-studio is never loaded. Responses report `mode` (`gated` or `ungated`, the path that ran) and `configured_mode` (this setting). |
+| `MEMKERNEL_WIKI_EXPORT_BATCH` | `10` | New pages one `wiki_export` writes at most, the most important memories first; the response's `remaining` counts the rest (`null` when the gate's outcome is uncertain or `governed` refused before reading the wiki). `0` = no limit; a negative value is refused |
 | `MEMKERNEL_TRANSPORT` | `stdio` | `stdio` or `streamable-http` |
 | `MEMKERNEL_HOST` | `127.0.0.1` | Host for `streamable-http` transport |
 | `MEMKERNEL_PORT` | `8765` | Port for `streamable-http` transport |
@@ -290,7 +315,9 @@ for ref in svc.recall("database"):
 report = svc.consolidate()                 # {"purged": n, "candidates": [...]}
 ```
 
-`build_service()` is the supported entry point; the MCP server, the admin API and the hook templates all use it. Integration notes for `code-copilot-team` and `haa` are in [`docs/integration/harness-integration.md`](docs/integration/harness-integration.md).
+`build_service()` is the supported entry point; the MCP server, the admin API and the hook templates all use it. If you pass a `PersistentClient` as `chroma_client` and also set `MEMKERNEL_CHROMA_PERSIST_PATH` (in the environment or in the settings), both must name the same directory, or `ValueError` is raised; with the default setting, the client's own directory is used.
+
+**Store locks.** Processes that share a store (the MCP server, a hook, a script) coordinate through locks keyed by the store itself: lock files in the persist directory, `.memkernel-<name>-<key>.lock`, whatever each process's `MEMKERNEL_STATE_DIR` is; an in-memory store's locks are per process. Only one writer changes a record's tags, under the store's `tags` lock, so a tag added by one process is never dropped by another's write. Locks are not re-entrant (taking one the thread already holds raises `LockReentryError`), a wait longer than 10 seconds is logged once, and spelling the persist directory with different letter case in two processes gives two locks, so use one spelling. The locks use `fcntl` and are verified on POSIX (Linux, macOS) only; on Windows they run unlocked across processes, with one logged warning per lock. Integration notes for `code-copilot-team` and `haa` are in [`docs/integration/harness-integration.md`](docs/integration/harness-integration.md).
 
 ### The session protocol as a component
 
@@ -329,13 +356,41 @@ memory.capture_compact_summary(compact_summary, session_id=..., trigger=...)  #
 
 Long-term memory has a human-readable twin: a markdown wiki in the [LLM Wiki](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f) pattern (`index.md`, `log.md`, typed pages with cited sources — the same schema as `code-copilot-team/knowledge/wiki`).
 
+To see the whole loop on a clean install, follow wiki-studio's [memkernel composition quickstart](https://github.com/gosha70/wiki-studio/blob/main/docs/quickstart-memkernel.md): retain a memory, export it through wiki-studio's gate, correct and approve the page, import only the approved revision, and see the next `recall` return it. CI runs the quickstart against memkernel **0.6.0**.
+
 - `wiki_export` compiles every long-term memory that has no page yet into a **draft** page under `decisions/`, `concepts/` or `workflows/`, with the required sections and a `memkernel://<project>/<ref_id>` citation. Existing pages are never overwritten.
 - `wiki_import` indexes pages as long-term memories so `recall` finds the reviewed version; a changed page supersedes its earlier memory. With wiki-studio 0.7.0 or later installed it is **governed** (`"governed": true`): it imports only pages wiki-studio's review state reports approved, and only when the page's bytes are still the approved revision, so drafts, self-declared approvals and pages edited after approval are not imported (`not_approved` says why); an edited page keeps its last approved memory until it is approved again. memkernel never approves a page; a curator does, with `wiki-studio approve`. Without wiki-studio the import reads the page files, drafts included, and reports `"governed": false`. An installed wiki-studio that is broken or older than 0.7.0 imports nothing and reports `import_error`.
+- **Withdrawal (governed import, 0.7).** A page's memory leaves the current answers, its record kept for history, when:
+  - the page is **deleted**. Only memories this wiki's sync ledger recorded as imported are withdrawn this way; a memory imported before 0.7, or without wiki-studio, never is.
+  - the page is **deprecated**, once the `status: deprecated` revision is approved.
+  - the page is **superseded** (`superseded_by: <page path>`). Its memory is replaced by the successor page's memory; if the successor has none yet, the old memory is withdrawn and linked once it does (`pending` until then).
+
+  The response lists `withdrawn`, `superseded`, `pending` and `sync_failed` (retried at the next import). The ledger is one JSON file per project and wiki under `MEMKERNEL_STATE_DIR/wiki-sync/`; a corrupt one is kept aside, reported as `ledger_error`, and rebuilt.
 - **Corrections.** An approved page can name the memories it corrects in its frontmatter: `corrects: <ref_id>, <ref_id>` (one line; tokens of letters, digits, `-` and `_`). On import each named memory that did not come from the wiki is superseded by the page's memory: it stops being a current answer, and its record is kept. References that cannot be applied are listed in `unresolved` with a reason (unknown, another project's, already superseded, or a wiki memory: correct that page instead); a value that breaks the grammar is listed in `invalid_corrects`, and the page is still imported. An approved page that is already current is listed in `unchanged`, and its corrections are reconciled again on every import, so a reference that could not be applied earlier takes effect once it can; corrections already applied are not repeated.
 
+- **Each wiki keeps to its own memories (0.7).** Every memory `wiki_import` writes, or finds current, carries its wiki's identity: `wiki-path:<hash>` (the wiki directory's path relative to the project root) and, when the wiki has a wiki-studio manifest, `wiki-id:<uuid>`. A page's previous memory, its corrections, its supersession and its deprecation are only that wiki's, so two wikis in one store may share page paths and `page_id`s. These tags are reserved: `retain` refuses them, and an ordinary `retain` whose text matches a wiki memory gets its own record instead of merging into it; `promote` drops every wiki tag from its copy.
+  - **A wiki that may move, or that shares a store with another wiki, needs a manifest.** The path a wiki is known by is relative to its project root: two project roots sharing one store, with wikis at the same relative path, get the same `wiki-path:`, so the second import is refused (memories at this path belong to another wiki) even with manifests. Use one store per project, or different relative paths. Without one, a wiki *is* its path: moved, it starts afresh, and whatever wiki is later placed at the old path continues its memories. A copied wiki directory (two directories, one `wiki_id`) is one wiki: each import moves its memories to the directory imported last.
+  - **Upgrading a 0.6 store:** memories written before 0.7 carry no identity, so the first import writes nothing for their pages and lists them under `ambiguous` (`{"path", "ref_ids", "reason"}`). Import once with `adopt_legacy=true` to adopt each page's sole such memory; a page with several stays `ambiguous` (forget the ones that are not its own). With several wikis in one store, name the records instead: `adopt_pages=[{"path", "ref_id"}, …]` copied from `ambiguous`; a named record that changed meanwhile is not adopted (`candidate changed`).
+  - **Identity changes stop the import before any write, with the remedy:** a wiki that gained a manifest needs `adopt_manifest=true` once (its sync ledger moves to the `wiki_id` key; if both ledger files exist, delete the stale one); memories at this path that belong to another `wiki_id` (the manifest removed or replaced, or a moved wiki not yet imported from its new place) stop it until the manifest is restored, the moved wiki is imported first, or those memories are forgotten. A wiki with a manifest that moves keeps its memories; the next import moves their path.
+  - One import runs at a time per store (the `wiki-import` store lock). On Windows the import runs unlocked with one warning, as in 0.6: two concurrent imports of one wiki can leave two records for a page (the next import reports it as `duplicate records`; forget one), and two wikis adopting the same record at once can tag it for one while the other supersedes it. Give processes that share a store the same `MEMKERNEL_STATE_DIR`, so they share the sync ledger.
 Set `MEMKERNEL_WIKI_DIR` (for example `knowledge/wiki`) or pass `wiki_dir` to the tools. The standalone maintainer that owns this page schema is [wiki-studio](https://github.com/gosha70/wiki-studio). With the `memkernel[wiki]` extra installed, every `wiki_export` response carries wiki-studio's lint and health findings for the whole wiki after the export, curated pages included, under `lint` (`violations`, `audit_errors`, `pages_checked`, `health` items with `kind`, `severity`, `pages`, `description`); without it `lint` is `null` and `lint_hint` says how to enable it; if the lint pass itself fails, `lint` is `null` and `lint_error` says why. The files written are the same in every case.
 
-With wiki-studio 0.7.0 or later installed, `wiki_export` writes through wiki-studio's promote gate: wiki-studio's `init` creates `index.md` and `log.md` if they are missing (wiki-studio's own scaffold, `status: stable`), then every new page goes with its index and log lines into one patch-set that is proposed and promoted. Other writers can work in the same wiki: wiki-studio's commit writes only the paths the patch-set names, under its lock, and refuses a page someone else created in the meantime. A rejected export writes no page and the response carries `gate_error` (a new wiki keeps the `index.md` and `log.md` that `init` created); so does an installed wiki-studio that is broken or older than 0.7.0, rather than falling back to the direct writer. A promoted export's response adds `proposal`, `submission_digest` and `archived_dir` from wiki-studio's receipt. If `archived_dir` is `null`, the pages landed but wiki-studio did not finish: it could not archive the proposal, or it failed after committing the pages (memkernel checks wiki-studio's receipt: while it still says `committing`, every planned write, index and log included, must be at its post-write sha256; a `committed` receipt needs memkernel's pages at theirs; an unusable receipt falls back to comparing the pages' bytes). `archive_hint` gives the `promote` command that tries to finish it, with absolute paths, so it runs from any directory. It applies nothing twice, but if the wiki or the proposal changed since, for example another writer appended to `index.md`, it may refuse with `uncertain_outcome` or `payload_changed` and write nothing; then compare the wiki with the proposal's `receipt.json` by hand. memkernel never runs it itself. If wiki-studio failed during its commit and its receipt shows neither a commit nor a rollback (say the page was written, the `index.md` write failed and removing the page failed too), the response carries `gate_error` with `uncertain_outcome` and `proposal`: `written` is empty, but the pages it names may be on disk; compare the wiki with the `receipt.json` it names before exporting again. `receipt_error` says the receipt could not be read or used. A failed export leaves its patch-set under the proposals directory for inspection; delete it once the cause is fixed. Patch-sets go to `MEMKERNEL_WIKI_PROPOSALS_DIR`, by default `.memkernel/wiki-proposals/` under the project root. Commit `<wiki>/.audit/`, wiki-studio's audit trail of promoted proposals; ignore `.memkernel/wiki-proposals/` and wiki-studio's lock file next to the wiki directory, `.<wiki-dir>.wiki-studio.lock` (for `knowledge/wiki`, `knowledge/.wiki.wiki-studio.lock`).
+With wiki-studio 0.7.0 or later installed, `wiki_export` writes through wiki-studio's promote gate: wiki-studio's `init` creates `index.md` and `log.md` if they are missing (wiki-studio's own scaffold, `status: stable`), then every new page goes with its index and log lines into one patch-set that is proposed and promoted. Other writers can work in the same wiki: wiki-studio's commit writes only the paths the patch-set names, under its lock, and refuses a page someone else created in the meantime. A rejected export writes no page and the response carries `gate_error` (a new wiki keeps the `index.md` and `log.md` that `init` created); so does an installed wiki-studio that is broken or older than 0.7.0, rather than falling back to the direct writer. A promoted export's response adds `proposal`, `submission_digest` and `archived_dir` from wiki-studio's receipt. If `archived_dir` is `null`, the pages landed but wiki-studio did not finish: it could not archive the proposal, or it failed after committing the pages (memkernel checks wiki-studio's receipt: while it still says `committing`, every planned write, index and log included, must be at its post-write sha256; a `committed` receipt needs memkernel's pages at theirs; an unusable receipt falls back to comparing the pages' bytes). `archive_hint` gives the `promote` command that tries to finish it, with absolute paths, so it runs from any directory. It applies nothing twice, but if the wiki or the proposal changed since, for example another writer appended to `index.md`, it may refuse with `uncertain_outcome` or `payload_changed` and write nothing; then compare the wiki with the proposal's `receipt.json` by hand. memkernel never runs it itself. If wiki-studio failed during its commit and its receipt shows neither a commit nor a rollback (say the page was written, the `index.md` write failed and removing the page failed too), the response carries `gate_error` with `uncertain_outcome` and `proposal`: `written` is empty, but the pages it names may be on disk; compare the wiki with the `receipt.json` it names before exporting again. `receipt_error` says the receipt could not be read or used. A failed export leaves its patch-set under the proposals directory for inspection; delete it once the cause is fixed. Patch-sets go to `MEMKERNEL_WIKI_PROPOSALS_DIR`, by default `.memkernel/wiki-proposals/` under the project root. Commit `<wiki>/.audit/`, wiki-studio's audit trail of promoted proposals; ignore `.memkernel/wiki-proposals/`, `.memkernel/state/` (memkernel's sync ledger) and wiki-studio's files next to the wiki directory, `.<wiki-dir>.wiki-studio.*` (for `knowledge/wiki`, `knowledge/.wiki.wiki-studio.*`: its lock file and, from wiki-studio 0.8, its search cache).
+
+**Wiki modes.** `MEMKERNEL_WIKI_MODE` is a property of the deployment, not of a call. `governed` runs one preflight before any page is written or read, for export and import alike: wiki-studio 0.7.0 or later installed and working; its `info` listing the capabilities `wiki-lock`, `receipts`, `root-manifest`, `page-id`, `approval-record`, `superseded-by` and `findings`; the wiki's manifest mode `governed`; no schema drift. A `managed` or `legacy` wiki is refused with the remedy: run `wiki-studio init` if there is no manifest, then set `approval_required: true` in the manifest's policy. A refusal reports `gate_error` (export) or `import_error` (import) and writes nothing; `update_index=False` is refused too, since the direct writer is never used. A gate rejection after the preflight commits no drafted page, but a new wiki keeps the `index.md` and `log.md` that wiki-studio's `init` created: memkernel keeps no outer rollback. `ungated` never loads wiki-studio (no gate, lint, review state or findings): `lint` is `null` and `lint_hint` names the mode.
+
+**Export batches and grounding.** One `wiki_export` writes at most `MEMKERNEL_WIKI_EXPORT_BATCH` new pages (by importance, highest first, then oldest, then `ref_id`), so a curator reviews a small batch at a time; `remaining` counts the eligible memories still without a page, all of them after a rejected batch. Every exported page carries `memkernel_sha256`, the memory's content hash, beside `memkernel_ref`; a `path:` source gains `sha:`, the commit it is grounded at (`git log -1 --format=%h`, wiki-studio's citation rule), only when the project's `git ls-files` lists exactly that file and `git status` shows no change to it: a modified, staged or untracked file, a file in a submodule, or no git gets none. Pages created through the gate carry wiki-studio's `page_id`, so a page renamed or moved under its `page_id` keeps its memory: the next import supersedes the memory at the old path with one at the new path, also when the moved page's bytes are unchanged.
+
+**Curation debt at session start.** With `MEMKERNEL_WIKI_DIR` set and wiki-studio available (in `governed`, only after the preflight passes), the session-start block (`startup`, `clear`) gains `### Curation debt`: one line of wiki-studio's `findings` counts in a fixed order (review kinds first) plus the sync ledger's unfinished actions (`sync pending`, `sync failed`), up to three example pages, and a hint to run `wiki-studio findings`. It uses at most 500 characters, taken from the long-term context's share, so the block stays under 9,000 characters. The check runs in a child process bounded to 2.5 seconds once it has started (a fork or exec the operating system holds up is not covered); it reads the sync ledger without changing it and never waits for an import that holds it. Any failure is one `Curation debt unavailable: <reason>` line. Not on `compact`, `resume` or `fork`, nor in `ungated`.
+
+**wiki-studio's files beside the wiki, its ingest log, and wiki-studio 0.8.** When memkernel exports through the gate, wiki-studio creates its lock file, `.<wiki-dir>.wiki-studio.lock`, beside the wiki directory, and from 0.8 a curator's `find` or `query` adds a search cache, `.<wiki-dir>.wiki-studio.search.sqlite`, there too: for a wiki kept in git, ignore both with `.<wiki-dir>.wiki-studio.*` (for `knowledge/wiki`, `knowledge/.wiki.wiki-studio.*`). memkernel creates no file of its own there. memkernel reads no wiki-studio ingest log, so the log's `hash_recipe` has no consumer in memkernel. From wiki-studio 0.8.0 a page that is not valid UTF-8 fails lint for the whole wiki, so every promote, memkernel's gated export included, is refused with a `gate_error` naming that page until it is re-encoded or removed. From 0.8.0 an unknown `WIKI_STUDIO_RANKER` value makes wiki-studio's `info` fail, so a `governed` export or import, a gated import and the curation debt refuse with wiki-studio's message naming the variable. memkernel needs wiki-studio 0.7.0 or later and is tested at 0.7.0; an informational CI job runs the wiki tests against the newest release below 1.0 on every pull request and weekly.
+
+wiki-studio's MCP server (`wiki-studio-mcp`, from wiki-studio 0.8.0) needs `mcp>=2.3`, which memkernel's `mcp<2` excludes. Run it in its own environment, at the same wiki-studio version as memkernel's:
+
+1. In memkernel's environment, read the version: `python -c "import importlib.metadata as m; print(m.version('wiki-studio'))"`. If it is below 0.8.0, upgrade it there first: `pip install 'wiki-studio>=0.8,<1'`.
+2. Install exactly that version for the server, for example `pipx install 'wiki-studio[mcp]==0.8.1'`, and register it beside memkernel's server.
+
+On one machine and one filesystem, wiki-studio's lock serializes the two servers' writes to the same wiki. It does not coordinate other hosts, and network filesystems are outside its scope.
 
 Rationale: [`docs/adr/ADR-008-llm-wiki-as-semantic-layer.md`](docs/adr/ADR-008-llm-wiki-as-semantic-layer.md) and [`docs/adr/ADR-010-wiki-studio-as-wiki-maintainer.md`](docs/adr/ADR-010-wiki-studio-as-wiki-maintainer.md).
 

{memkernel-0.6.0 → memkernel-0.8.0}/pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "memkernel"
-version = "0.6.0"
+version = "0.8.0"
 description = "Code-aware memory server for AI coding copilots"
 authors = ["EGOGE"]
 license = "Apache-2.0"

{memkernel-0.6.0 → memkernel-0.8.0}/src/memkernel/__init__.py

@@ -2,4 +2,4 @@
 # SPDX-License-Identifier: Apache-2.0
 """MemKernel — Code-aware memory server for AI coding copilots."""
 
-__version__ = "0.6.0"
+__version__ = "0.8.0"

{memkernel-0.6.0 → memkernel-0.8.0}/src/memkernel/config.py

@@ -7,6 +7,7 @@ from __future__ import annotations
 import logging
 from typing import TYPE_CHECKING, Any, Literal
 
+from pydantic import Field
 from pydantic_settings import BaseSettings, EnvSettingsSource, PydanticBaseSettingsSource, SettingsConfigDict
 
 if TYPE_CHECKING:
@@ -34,6 +35,10 @@ class MemKernelSettings(BaseSettings):
     chroma_persist_path: str = ".memkernel/chroma"
     chroma_collection_prefix: str = "memkernel"
 
+    # memkernel's own state outside Chroma (memkernel-0-7): the wiki sync ledger. Resolved like
+    # chroma_persist_path: a relative path is relative to the working directory of the process.
+    state_dir: str = ".memkernel/state"
+
     # Embedding
     embedding_model: str = "all-MiniLM-L6-v2"
     embedding_dimension: int = 384
@@ -79,6 +84,10 @@ class MemKernelSettings(BaseSettings):
     # Where wiki-studio patch-sets are written before promote (memkernel[wiki], ADR-010).
     # Empty = <project_root>/.memkernel/wiki-proposals; set it to request the gate explicitly.
     wiki_proposals_dir: str = ""
+    # Release B part 2c (G1): auto = the gate when wiki-studio is installed, the direct writer otherwise; governed =
+    # wiki-studio and a governed wiki required, refused otherwise; ungated = wiki-studio is never loaded.
+    wiki_mode: Literal["auto", "governed", "ungated"] = "auto"
+    wiki_export_batch: int = Field(default=10, ge=0)  # new pages per wiki_export, most important first (G3); 0 = none
 
     # Server
     transport: Literal["stdio", "streamable-http"] = "stdio"

{memkernel-0.6.0 → memkernel-0.8.0}/src/memkernel/core/models.py

@@ -102,11 +102,34 @@ CODE_CHUNK_KEY = "code_chunk"  # boolean metadata so adapters can skip chunk rec
 PARENT_REF_KEY = "parent_ref"
 # Durable liveness marker (0.4): set on a record that another record replaced.
 SUPERSEDED_BY_KEY = "superseded_by"
+# Durable liveness marker (0.7, memkernel-0-7 D1): set on a record taken out of the current answers with no
+# replacing record (a deleted, deprecated or superseded wiki page); the reason and the time it happened.
+WITHDRAWN_REASON_KEY = "withdrawn_reason"
+WITHDRAWN_AT_KEY = "withdrawn_at"
+WITHDRAWN_DELETED = "deleted"
+WITHDRAWN_DEPRECATED = "deprecated"
+WITHDRAWN_SUPERSEDED = "superseded"
+WITHDRAWN_REASONS = frozenset({WITHDRAWN_DELETED, WITHDRAWN_DEPRECATED, WITHDRAWN_SUPERSEDED})
 # Revalidation flags (0.4).
 NEEDS_REVIEW_KEY = "needs_review"
 REVIEW_REASON_KEY = "review_reason"
 REVIEW_REASON_CHANGED = "changed"
 REVIEW_REASON_MISSING = "missing"
+# Wiki-import tags (memkernel-0-6, memkernel-0-7-wiki-scope): what the wiki import writes on a memory. The two
+# identity tags name the wiki a memory belongs to; only the import sets them (S8).
+WIKI_TAG = "wiki"
+WIKI_PAGE_TAG = "page:"
+WIKI_PAGE_TYPE_TAG = "page_type:"
+PAGE_KEY_TAG = "page_key:"
+PAGE_SHA_TAG = "page_sha:"
+WIKI_PATH_TAG = "wiki-path:"
+WIKI_ID_TAG = "wiki-id:"
+WIKI_IDENTITY_TAGS = (WIKI_PATH_TAG, WIKI_ID_TAG)
+WIKI_IMPORT_TAG_PREFIXES = (WIKI_PAGE_TAG, WIKI_PAGE_TYPE_TAG, PAGE_KEY_TAG, PAGE_SHA_TAG, *WIKI_IDENTITY_TAGS)
+# Store locks (memkernel-0-7-store-locks L2): the names a caller may take with ``VectorStorePort.exclusive``.
+STORE_LOCK_WIKI_IMPORT = "wiki-import"
+STORE_LOCK_TAGS = "tags"  # held by ``merge_tags``, the only writer of a record's tags
+STORE_LOCK_NAMES = frozenset({STORE_LOCK_WIKI_IMPORT, STORE_LOCK_TAGS})
 
 
 def tier_of(memory_type: MemoryType) -> MemoryTier:
@@ -169,6 +192,13 @@ class Memory(BaseModel):
     parent_ref: str | None = None
     needs_review: bool = False
     review_reason: str = ""
+    # 0.7: withdrawn without a replacement (memkernel-0-7 FR-001); absent, so live, in older stores.
+    withdrawn_reason: str | None = None
+    withdrawn_at: datetime | None = None
+
+    def retired(self) -> bool:
+        """True for history: superseded by another record or withdrawn (memkernel-0-7 FR-002)."""
+        return self.superseded_by is not None or self.withdrawn_reason is not None
 
 
 class HealthItem(BaseModel):
@@ -206,6 +236,23 @@ class ReasonerError(RuntimeError):
     """A ReasonerPort implementation ran but did not produce an answer (provider error, timeout, budget)."""
 
 
+class AmbiguousDedupError(ValueError):
+    """Two or more live records qualify as one retain's dedup hit (memkernel-0-7-wiki-scope S2): none is chosen."""
+
+    def __init__(self, ref_ids: list[str]) -> None:
+        super().__init__(f"several records qualify as the dedup hit: {', '.join(ref_ids)}")
+        self.ref_ids = ref_ids
+
+
+def has_wiki_identity(tags: list[str]) -> bool:
+    """Whether a memory carries a wiki identity tag (``wiki-path:`` or ``wiki-id:``)."""
+    return any(tag.startswith(WIKI_IDENTITY_TAGS) for tag in tags)
+
+
+class LockReentryError(RuntimeError):
+    """A thread took a store lock it already holds (memkernel-0-7-store-locks L2): store locks are not re-entrant."""
+
+
 class Answer(BaseModel):
     """Result of ``ask``: the answer text, the reasoning mode actually used, and provenance."""
 

{memkernel-0.6.0 → memkernel-0.8.0}/src/memkernel/core/ports.py

@@ -11,6 +11,9 @@ from __future__ import annotations
 from typing import TYPE_CHECKING, Any, Protocol, runtime_checkable
 
 if TYPE_CHECKING:
+    from collections.abc import Sequence
+    from contextlib import AbstractContextManager
+
     from memkernel.core.models import Answer, Memory, SearchResult
 
 
@@ -50,7 +53,27 @@ class VectorStorePort(Protocol):
         ...
 
     def update_metadata(self, ref_id: str, metadata: dict[str, Any]) -> None:
-        """Fetch-and-merge: only overwrites caller-specified keys."""
+        """Overwrite only the caller's keys; every other key is kept.
+
+        Raises ``ValueError`` for a ``tags`` key: ``merge_tags`` is the only writer of tags (memkernel-0-7-store-locks
+        L3), so a write of other keys can never put back a stale tag list.
+        """
+        ...
+
+    def merge_tags(self, ref_id: str, add: Sequence[str] = (), remove: Sequence[str] = ()) -> bool:
+        """Drop ``remove``, then append the missing ``add`` tags, read and written under the store's tags lock.
+
+        A tag in both ends up present, after the kept ones. Returns False when the record is unknown or its tags
+        are already as asked. Raises ``LockReentryError`` when the calling thread already holds the tags lock.
+        """
+        ...
+
+    def exclusive(self, name: str) -> AbstractContextManager[None]:
+        """Hold the store's lock ``name`` (one of ``STORE_LOCK_NAMES``) until the context exits.
+
+        Keyed by the store itself, not by settings; blocks until acquired; not re-entrant (``LockReentryError``).
+        Where the platform has no lock, it runs unlocked with one logged warning.
+        """
         ...
 
     def count(self, where: dict[str, Any] | None = None) -> int: