memkernel 0.3.0__tar.gz → 0.4.1__tar.gz

{memkernel-0.3.0 → memkernel-0.4.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: memkernel
-Version: 0.3.0
+Version: 0.4.1
 Summary: Code-aware memory server for AI coding copilots
 License: Apache-2.0
 License-File: LICENSE
@@ -27,7 +27,10 @@ Project-URL: Homepage, https://memkernel.dev
 Project-URL: Repository, https://github.com/gosha70/memkernel
 Description-Content-Type: text/markdown
 
-# MemKernel
+<h1>
+  <img src="https://raw.githubusercontent.com/gosha70/memkernel/main/docs/assets/memkernel-logo.png" alt="MemKernel logo: a glowing memory core on three stacked tiers" width="110" align="absmiddle">
+  MemKernel
+</h1>
 
 [![CI](https://github.com/gosha70/memkernel/actions/workflows/ci.yml/badge.svg)](https://github.com/gosha70/memkernel/actions/workflows/ci.yml)
 [![PyPI](https://img.shields.io/pypi/v/memkernel)](https://pypi.org/project/memkernel/)
@@ -44,27 +47,55 @@ MemKernel gives Claude Code (and any MCP-compatible copilot) persistent memory a
 
 **Tiered memory (0.2)**: memories live in three tiers with different persistence rules — *working* (the current task, hours), *short-term* (episodes and observations, decays unless recalled) and *long-term* (decisions, conventions, facts, procedures, code; never decays, replaced by supersession). Recall ranks by relevance, recency and importance; `consolidate` and `promote` move what proved useful into long-term memory; an LLM-wiki bridge keeps a human-readable, git-versioned copy of long-term knowledge. Design: [`docs/architecture/memory-tiers.md`](docs/architecture/memory-tiers.md).
 
+## What MemKernel is, and what it is not
+
+**It is:**
+
+- **Persistent memory for a coding agent.** What the agent retains survives context compaction and the end of a session, in a local ChromaDB store per project.
+- **Explicit.** The agent decides what to retain (decisions, conventions, facts, code snapshots, tasks) and what to recall. The lifecycle hooks only automate the protocol around that: checkpoint before compaction, recovery after it, consolidation and revalidation at session start.
+- **Code-aware.** AST chunking, hybrid search that finds identifiers as well as meaning, and revalidation of `code` memories against the files they came from.
+- **No mandatory generative LLM.** Default retrieval uses a local sentence-transformer embedding model plus BM25 and RRF; nothing generates text inside MemKernel. `ask` with the built-in reasoner returns the best-matching chunks; synthesis is the agent's job, or an optional external reasoner's.
+
+**It is not:**
+
+- **A token optimizer or context compressor.** MemKernel does not shrink the context window, compress prompts, or lower model cost by itself. It adds tool calls and returned content to a session. Any token saving comes from the agent re-reading fewer files or documents after a resume; that is a side effect to measure, not a promise.
+- **A replacement for files, specs and git.** Progress notes, task lists and review records already carry state between sessions. MemKernel adds retrieval by meaning and identifier, tiers with decay, supersession and revalidation on top. When a file already holds everything the next session needs, expect no gain.
+- **An index of the whole repository.** `code` memories are the snapshots the agent chose to retain, not an automatic scan of every file.
+- **A cross-project knowledge base.** One store per project id. Sharing is deliberate: a second server with a shared id, or the wiki bridge.
+
+**Evidence.** `eval/run_eval.py` measures retrieval (recall@k, precision@k, MRR). An agent-level benchmark that runs the same agent with and without MemKernel, including a negative control where files already carry all state, is tracked in [#17](https://github.com/gosha70/memkernel/issues/17). Results will be published per minor release, including the cases with no gain.
+
 ---
 
 ## Requirements
 
-- Python 3.11 – 3.13
+- Python 3.11 – 3.13 (the package declares `Requires-Python >=3.11,<3.14`; pip reports "from versions: none" when the installing interpreter is outside that range)
 - [Poetry](https://python-poetry.org) (for development installs)
 
 ---
 
 ## Install
 
+MemKernel is a Python package on PyPI. Its `memkernel` command is the MCP server. There is no npm package: an `npx` install someone suggests is a hallucination.
+
+**Recommended: an isolated install with [pipx](https://pipx.pypa.io)**, which puts the `memkernel` command on your PATH without touching any other Python environment:
+
 ```bash
-pip install memkernel
+pipx install memkernel
+# if pipx's default interpreter is outside 3.11–3.13:
+pipx install --python python3.11 memkernel
 ```
 
-Or with Poetry:
+**As a library inside a project**, install it into that project's virtual environment:
 
 ```bash
-poetry add memkernel
+pip install memkernel        # or: poetry add memkernel
 ```
 
+Avoid a plain `pip install memkernel` into a shared or system interpreter (Homebrew's `python3`, for example). MemKernel pins `chromadb` 1.x and recent `cffi`, so pip will replace older copies and can break unrelated tools that need them.
+
+Verify the install with `pipx list` or `which memkernel`. Running `memkernel` with no arguments starts the MCP server on stdio and waits for a client; `memkernel --version` and `memkernel --help` print and exit.
+
 ---
 
 ## Add to Claude Code
@@ -77,17 +108,29 @@ Add MemKernel to `.claude/settings.json` in your project:
 {
   "mcpServers": {
     "memkernel": {
-      "command": "memkernel",
+      "command": "/Users/you/.local/bin/memkernel",
       "args": [],
       "env": {
-        "MEMKERNEL_PROJECT_ID": "my-project"
+        "MEMKERNEL_PROJECT_ID": "my-project",
+        "MEMKERNEL_CHROMA_PERSIST_PATH": "/Users/you/.memkernel/my-project"
       }
     }
   }
 }
 ```
 
-Replace `my-project` with a short identifier for the project — memories are isolated per project.
+Replace `my-project` with a short identifier for the project — memories are isolated per project. Use the absolute path of the command (`which memkernel` prints it) because Claude Code does not necessarily see the PATH changes your shell profile makes, and an absolute `MEMKERNEL_CHROMA_PERSIST_PATH` keeps the store outside the repository. On first server initialization, MemKernel downloads the embedding model (about 90 MB) if it is not already cached: `build_service()` loads the sentence-transformers model when the MCP server starts, before any tool call, so the first connection can take a minute. Later starts and every tool call are fast.
+
+<details>
+<summary>Troubleshooting the install</summary>
+
+- **`pipx install memkernel` says "No matching distribution found"** — pipx used an interpreter outside 3.11–3.13. Pass one explicitly: `pipx install --python /opt/homebrew/opt/python@3.11/bin/python3.11 memkernel`.
+- **`bash: /some/old/path/memkernel: No such file or directory`** right after reinstalling — the shell cached the old location. Run `hash -r` (bash) or `rehash` (zsh), then `which memkernel`.
+- **`memkernel --help` seems to hang** on versions before 0.4.0 — the command had no flags and started the server; press Ctrl-C. From 0.4.0 on, `--help` and `--version` print and exit.
+- **The first MCP connection is slow or times out** — the server is downloading the embedding model at startup (see above). Wait for it once, or pre-warm it with `python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('all-MiniLM-L6-v2')"` in the same environment.
+- **A global pip install broke another tool** (for example `langchain-chroma` wanting `chromadb < 0.6`) — uninstall memkernel from that interpreter, reinstall the versions the other tool needs, and install memkernel with pipx instead.
+
+</details>
 
 ### 2. Add memory instructions to CLAUDE.md
 
@@ -107,16 +150,30 @@ See the Prerequisites section inside each script for the Python environment requ
 
 ---
 
+## Add to Kiro
+
+Kiro (and the Kiro CLI, formerly Amazon Q Developer CLI) connects to MemKernel as a stdio MCP server. [`templates/kiro/`](templates/kiro/) holds the three files to copy into a workspace:
+
+| Template | Destination | Role |
+|----------|-------------|------|
+| `settings/mcp.json` | `.kiro/settings/mcp.json` | Server entry with `env` and `autoApprove` |
+| `steering/memory.md` | `.kiro/steering/memory.md` | Memory instructions, always included |
+| `hooks/memkernel.json` | `.kiro/hooks/memkernel.json` | `SessionStart` recall, `Stop` retain, manual checkpoint |
+
+[`templates/kiro/README.md`](templates/kiro/README.md) has the setup steps, the Kiro CLI custom-agent variant and the caveats (absolute persist path, hook format by Kiro version).
+
+---
+
 ## MCP Tools
 
 | Tool | What it does |
 |------|-------------|
 | `retain(content, type, session_id?, checkpoint?, source?, importance?, tags?, supersedes?)` | Store a memory. Returns `ref_id`. Idempotent — identical content + type returns the same `ref_id`. `supersedes` hides the memory it replaces. |
-| `recall(query, type?, tier?, session_id?, checkpoint_only?, top_k?)` | Search by natural language or identifier. Returns lightweight `ArtifactRef` list ranked by relevance + recency + importance. Records an access (reinforces retention). |
+| `recall(query, type?, tier?, session_id?, checkpoint_only?, top_k?, include_needs_review?)` | Search by natural language or identifier. Returns lightweight `ArtifactRef` list ranked by relevance + recency + importance. Records an access (reinforces retention). Code memories flagged `needs_review` are hidden unless `include_needs_review=true`. |
 | `get(ref_id)` | Fetch full content of a specific memory. Records an access. |
 | `forget(ref_id)` | Delete a memory from all indexes. |
 | `promote(ref_id, type, content?, importance?, tags?)` | Turn a short-term/working memory into a long-term one (you supply the distilled text). Keeps provenance via `promoted_from`. |
-| `consolidate(session_id?, limit?, synthesize?)` | Purge expired short-term memories and list the ones most worth promoting (ranked by importance × how often they were recalled). `synthesize=true` also asks an external reasoner to draft long-term statements (`suggested`). |
+| `consolidate(session_id?, limit?, synthesize?, revalidate?)` | Purge expired short-term memories and list the ones most worth promoting (ranked by importance × how often they were recalled). `synthesize=true` also asks an external reasoner to draft long-term statements (`suggested`); `revalidate=true` first checks every code memory against its source file (`revalidated`). The report always carries a `health` summary. |
 | `ask(ref_id, question, mode?)` | Answer a question about one memory without loading it — for a `document` the full body is consulted. `mode`: `direct` \| `rag` \| `rlm` \| `auto`, passed to the reasoner unchanged; the built-in reasoner answers in `rag` mode with the best-matching chunks. Returns `answer`, `mode`, `citations`, `usage`. |
 | `wiki_export(wiki_dir?)` | Compile long-term memories without a wiki page into draft markdown pages (`index.md`, `log.md` updated). |
 | `wiki_import(wiki_dir?)` | Index curated wiki pages as long-term memories (idempotent; changed pages supersede). |
@@ -129,7 +186,7 @@ See the Prerequisites section inside each script for the Python environment requ
 | `convention` | long-term | Naming rules, code style, project patterns | 0.9 | Never |
 | `fact` | long-term | Durable project facts: build commands, environments, contracts | 0.7 | Never |
 | `procedure` | long-term | How-to that worked: workflows, recipes, verified skills | 0.8 | Never |
-| `code` | long-term | Function/class snapshots, API contracts | 0.6 | Never |
+| `code` | long-term | Function/class snapshots, API contracts; identified by `(source, content)`, a changed file supersedes the old snapshot, a drifted source flags `needs_review` | 0.6 | Never (superseded on re-retain) |
 | `document` | long-term | Large artefacts (specs, transcripts, files): the body is stored and chunk-indexed, `get` returns a summary, `ask` answers from the indexed chunks (local reasoner) or the body (RLM Studio) | 0.6 | Never |
 | `episode` | short-term | Session events, compaction checkpoints | 0.4 | 30 days, extended by every recall |
 | `observation` | short-term | Findings, tool outputs, things noticed | 0.3 | 7 days, extended by every recall |
@@ -144,6 +201,7 @@ Retention: `ttl = base_ttl × (1 + access_boost × access_count)`, measured from
 3. **Before compaction** — `retain(type="episode", checkpoint=true)` (the `pre-compact.py` hook) and `consolidate()`.
 4. **After compaction / end of phase** — `promote(ref_id, type, content)` for the candidates worth keeping.
 5. **Large artefacts** — `retain(type="document", content=<body>, source=<name>)` then `ask(ref_id, question)`; never `get` a document to read it.
+7. **Code snapshots** — `retain(type="code", content=<file>, source=<path>)`; retain the file again after editing it (the new snapshot supersedes the old one). The session-start hook revalidates snapshots against their files and lists the ones that need re-retaining.
 6. **Curate** — `wiki_export` → review the drafts → `wiki_import`.
 
 ### Quick examples
@@ -181,6 +239,8 @@ ask(ref_id="<document ref_id>", question="Which endpoints need rate limiting?",
 ## Configuration
 
 All settings are read from environment variables with the `MEMKERNEL_` prefix, or from a `.env` file.
+Entries in `.env` without the prefix (keys for other tools) are ignored; a `MEMKERNEL_` key that matches no
+setting is logged as a warning at startup.
 
 | Variable | Default | Description |
 |----------|---------|-------------|
@@ -199,6 +259,10 @@ All settings are read from environment variables with the `MEMKERNEL_` prefix, o
 | `MEMKERNEL_DOCUMENT_SUMMARY_CHARS` | `1000` | Characters of the body kept in a `document` memory's summary record |
 | `MEMKERNEL_REASONER` | `local` | `ReasonerPort` behind `ask`: `local` (hybrid search over chunks, no LLM) or `rlmstudio` (falls back to `local` when `rlmstudio` is not importable) |
 | `MEMKERNEL_ASK_TOP_K` | `5` | Chunks the local reasoner returns per `ask` |
+| `MEMKERNEL_PROJECT_ROOT` | `.` | Directory that relative `source` paths of `code` memories resolve under; nothing outside it is ever read. Set it whenever the server or a hook is not started from the project directory |
+| `MEMKERNEL_HEALTH_DUPLICATE_THRESHOLD` | `0.95` | Cosine similarity in [0, 1] from which two long-term memories count as near-duplicates in the health pass |
+| `MEMKERNEL_HEALTH_DUPLICATE_LIMIT` | `500` | Most recent long-term memories the near-duplicate check covers |
+| `MEMKERNEL_HEALTH_REPORT_LIMIT` | `20` | Items listed per category in a health report |
 | `MEMKERNEL_EPISODE_TTL_DAYS` | `30` | Base TTL for `episode` memories (short-term) |
 | `MEMKERNEL_OBSERVATION_TTL_DAYS` | `7` | Base TTL for `observation` memories (short-term) |
 | `MEMKERNEL_TASK_TTL_HOURS` | `24` | Base TTL for `task` memories (working) |
@@ -287,7 +351,13 @@ Long-term memory has a human-readable twin: a markdown wiki in the [LLM Wiki](ht
 - `wiki_export` compiles every long-term memory that has no page yet into a **draft** page under `decisions/`, `concepts/` or `workflows/`, with the required sections and a `memkernel://<project>/<ref_id>` citation. Existing pages are never overwritten.
 - `wiki_import` indexes curated pages as long-term memories so `recall` finds the reviewed version; a changed page supersedes its earlier memory.
 
-Set `MEMKERNEL_WIKI_DIR` (for example `knowledge/wiki`) or pass `wiki_dir` to the tools. Rationale and the proposed standalone `ai-wiki` repo: [`docs/adr/ADR-008-llm-wiki-as-semantic-layer.md`](docs/adr/ADR-008-llm-wiki-as-semantic-layer.md).
+Set `MEMKERNEL_WIKI_DIR` (for example `knowledge/wiki`) or pass `wiki_dir` to the tools. The standalone maintainer that owns this page schema is [wiki-studio](https://github.com/gosha70/wiki-studio) (`pip install wiki-studio`; `wiki-studio lint --health` checks what `wiki_export` writes). Rationale: [`docs/adr/ADR-008-llm-wiki-as-semantic-layer.md`](docs/adr/ADR-008-llm-wiki-as-semantic-layer.md).
+
+### Revalidation and store health
+
+A `code` memory is a snapshot of a file. `MemoryService.revalidate()` hashes each live snapshot's whole source file and compares it with the stored content hash: a changed file flags the memory `needs_review` (`changed`), a vanished file flags it `missing`, a matching file clears the flag. Flagged memories leave default `recall` until the file is retained again (which supersedes the stale snapshot) or they are asked for with `include_needs_review=true`. Revalidation runs from `consolidate(revalidate=true)`, at every session start through `MemoryLifecycle.on_session_start()` (a `### Needs review` section lists what to re-retain), and from `POST /revalidate`. Sources resolve under `MEMKERNEL_PROJECT_ROOT` (default: the current directory); set it whenever the server or a hook is not started from the project directory. Nothing outside that root is ever read. 0.4 hashes whole files; per-chunk invalidation is a follow-up.
+
+`MemoryService.health()` audits the store without changing it: orphaned chunk and document records, dangling `supersedes` / `superseded_by` / `promoted_from` / `parent_ref` links, supersessions from 0.2/0.3 that lack the durable marker, `code` memories whose source file is gone, memories flagged for review, expired short-term memories not yet purged and, in full mode only, near-duplicate pairs among the most recent long-term memories (cosine ≥ `MEMKERNEL_HEALTH_DUPLICATE_THRESHOLD`). The quick report (counts only, no embeddings, no content loads) is part of every `consolidate()` report and of the `memory://status` resource; the full report is `GET /health/store`. The pass never fixes anything: a fix mode is a later feature.
 
 ## Admin API
 
@@ -302,6 +372,8 @@ uvicorn memkernel.api.app:app --host 127.0.0.1 --port 8766
 | `GET /health` | Server health and memory count |
 | `GET /stats` | Total memory count, per-type and per-tier breakdown, growth rate, last recall latency |
 | `GET /collections` | List ChromaDB collections |
+| `GET /health/store` | Full store health report (orphans, dangling links, stale sources, near-duplicates, …) |
+| `POST /revalidate` | Check every code memory against its source file; flag changed or missing ones |
 
 ---
 
@@ -324,7 +396,7 @@ Releases are published to PyPI by [`.github/workflows/release.yml`](.github/work
 
 1. Set `version` in `pyproject.toml` and move the `[Unreleased]` entries in [`CHANGELOG.md`](CHANGELOG.md) under the new version.
 2. Merge to `main` with CI green.
-3. Tag and push: `git tag -a v0.3.0 -m "memkernel 0.3.0" && git push origin v0.3.0`.
+3. Tag and push: `git tag -a v0.4.0 -m "memkernel 0.4.0" && git push origin v0.4.0`.
 
 The workflow refuses a tag that does not match the `pyproject.toml` version, builds the sdist and wheel, smoke-installs the wheel, and uploads through the `pypi` environment.
 
@@ -359,7 +431,7 @@ MemKernel is the memory component of a small family of projects by the same auth
 
 | Project | Role | Relationship |
 |---|---|---|
-| [ai-wiki](https://github.com/gosha70/ai-wiki) | LLM-maintained markdown wiki (ingest / promote / query / lint) | the human-readable form of long-term memory; `wiki_export` / `wiki_import` speak its page schema (ADR-008) |
+| [wiki-studio](https://github.com/gosha70/wiki-studio) | LLM-wiki maintainer (ingest / promote / query / lint / audit), stdlib-only | the human-readable form of long-term memory; `wiki_export` / `wiki_import` speak its page schema (ADR-008), and CI lints exporter output with it |
 | [rlm-studio](https://github.com/gosha70/rlm-studio) | Recursive Language Models workbench: Direct / RAG / RLM / auto over large documents | the document layer behind `document` memories and the `ask()` tool (`MEMKERNEL_REASONER=rlmstudio`, ADR-009) |
 | [code-copilot-team](https://github.com/gosha70/code-copilot-team) | Enforceable harness configuration for Claude Code, Pi and other copilots | consumes MemKernel through MCP and hooks |
 | [haa](https://github.com/gosha70/haa) | Home-built AI coding harness, future component assembler | consumes MemKernel in-process through `build_service()` |

{memkernel-0.3.0 → memkernel-0.4.1}/README.md

@@ -1,4 +1,7 @@
-# MemKernel
+<h1>
+  <img src="https://raw.githubusercontent.com/gosha70/memkernel/main/docs/assets/memkernel-logo.png" alt="MemKernel logo: a glowing memory core on three stacked tiers" width="110" align="absmiddle">
+  MemKernel
+</h1>
 
 [![CI](https://github.com/gosha70/memkernel/actions/workflows/ci.yml/badge.svg)](https://github.com/gosha70/memkernel/actions/workflows/ci.yml)
 [![PyPI](https://img.shields.io/pypi/v/memkernel)](https://pypi.org/project/memkernel/)
@@ -15,27 +18,55 @@ MemKernel gives Claude Code (and any MCP-compatible copilot) persistent memory a
 
 **Tiered memory (0.2)**: memories live in three tiers with different persistence rules — *working* (the current task, hours), *short-term* (episodes and observations, decays unless recalled) and *long-term* (decisions, conventions, facts, procedures, code; never decays, replaced by supersession). Recall ranks by relevance, recency and importance; `consolidate` and `promote` move what proved useful into long-term memory; an LLM-wiki bridge keeps a human-readable, git-versioned copy of long-term knowledge. Design: [`docs/architecture/memory-tiers.md`](docs/architecture/memory-tiers.md).
 
+## What MemKernel is, and what it is not
+
+**It is:**
+
+- **Persistent memory for a coding agent.** What the agent retains survives context compaction and the end of a session, in a local ChromaDB store per project.
+- **Explicit.** The agent decides what to retain (decisions, conventions, facts, code snapshots, tasks) and what to recall. The lifecycle hooks only automate the protocol around that: checkpoint before compaction, recovery after it, consolidation and revalidation at session start.
+- **Code-aware.** AST chunking, hybrid search that finds identifiers as well as meaning, and revalidation of `code` memories against the files they came from.
+- **No mandatory generative LLM.** Default retrieval uses a local sentence-transformer embedding model plus BM25 and RRF; nothing generates text inside MemKernel. `ask` with the built-in reasoner returns the best-matching chunks; synthesis is the agent's job, or an optional external reasoner's.
+
+**It is not:**
+
+- **A token optimizer or context compressor.** MemKernel does not shrink the context window, compress prompts, or lower model cost by itself. It adds tool calls and returned content to a session. Any token saving comes from the agent re-reading fewer files or documents after a resume; that is a side effect to measure, not a promise.
+- **A replacement for files, specs and git.** Progress notes, task lists and review records already carry state between sessions. MemKernel adds retrieval by meaning and identifier, tiers with decay, supersession and revalidation on top. When a file already holds everything the next session needs, expect no gain.
+- **An index of the whole repository.** `code` memories are the snapshots the agent chose to retain, not an automatic scan of every file.
+- **A cross-project knowledge base.** One store per project id. Sharing is deliberate: a second server with a shared id, or the wiki bridge.
+
+**Evidence.** `eval/run_eval.py` measures retrieval (recall@k, precision@k, MRR). An agent-level benchmark that runs the same agent with and without MemKernel, including a negative control where files already carry all state, is tracked in [#17](https://github.com/gosha70/memkernel/issues/17). Results will be published per minor release, including the cases with no gain.
+
 ---
 
 ## Requirements
 
-- Python 3.11 – 3.13
+- Python 3.11 – 3.13 (the package declares `Requires-Python >=3.11,<3.14`; pip reports "from versions: none" when the installing interpreter is outside that range)
 - [Poetry](https://python-poetry.org) (for development installs)
 
 ---
 
 ## Install
 
+MemKernel is a Python package on PyPI. Its `memkernel` command is the MCP server. There is no npm package: an `npx` install someone suggests is a hallucination.
+
+**Recommended: an isolated install with [pipx](https://pipx.pypa.io)**, which puts the `memkernel` command on your PATH without touching any other Python environment:
+
 ```bash
-pip install memkernel
+pipx install memkernel
+# if pipx's default interpreter is outside 3.11–3.13:
+pipx install --python python3.11 memkernel
 ```
 
-Or with Poetry:
+**As a library inside a project**, install it into that project's virtual environment:
 
 ```bash
-poetry add memkernel
+pip install memkernel        # or: poetry add memkernel
 ```
 
+Avoid a plain `pip install memkernel` into a shared or system interpreter (Homebrew's `python3`, for example). MemKernel pins `chromadb` 1.x and recent `cffi`, so pip will replace older copies and can break unrelated tools that need them.
+
+Verify the install with `pipx list` or `which memkernel`. Running `memkernel` with no arguments starts the MCP server on stdio and waits for a client; `memkernel --version` and `memkernel --help` print and exit.
+
 ---
 
 ## Add to Claude Code
@@ -48,17 +79,29 @@ Add MemKernel to `.claude/settings.json` in your project:
 {
   "mcpServers": {
     "memkernel": {
-      "command": "memkernel",
+      "command": "/Users/you/.local/bin/memkernel",
       "args": [],
       "env": {
-        "MEMKERNEL_PROJECT_ID": "my-project"
+        "MEMKERNEL_PROJECT_ID": "my-project",
+        "MEMKERNEL_CHROMA_PERSIST_PATH": "/Users/you/.memkernel/my-project"
       }
     }
   }
 }
 ```
 
-Replace `my-project` with a short identifier for the project — memories are isolated per project.
+Replace `my-project` with a short identifier for the project — memories are isolated per project. Use the absolute path of the command (`which memkernel` prints it) because Claude Code does not necessarily see the PATH changes your shell profile makes, and an absolute `MEMKERNEL_CHROMA_PERSIST_PATH` keeps the store outside the repository. On first server initialization, MemKernel downloads the embedding model (about 90 MB) if it is not already cached: `build_service()` loads the sentence-transformers model when the MCP server starts, before any tool call, so the first connection can take a minute. Later starts and every tool call are fast.
+
+<details>
+<summary>Troubleshooting the install</summary>
+
+- **`pipx install memkernel` says "No matching distribution found"** — pipx used an interpreter outside 3.11–3.13. Pass one explicitly: `pipx install --python /opt/homebrew/opt/python@3.11/bin/python3.11 memkernel`.
+- **`bash: /some/old/path/memkernel: No such file or directory`** right after reinstalling — the shell cached the old location. Run `hash -r` (bash) or `rehash` (zsh), then `which memkernel`.
+- **`memkernel --help` seems to hang** on versions before 0.4.0 — the command had no flags and started the server; press Ctrl-C. From 0.4.0 on, `--help` and `--version` print and exit.
+- **The first MCP connection is slow or times out** — the server is downloading the embedding model at startup (see above). Wait for it once, or pre-warm it with `python -c "from sentence_transformers import SentenceTransformer; SentenceTransformer('all-MiniLM-L6-v2')"` in the same environment.
+- **A global pip install broke another tool** (for example `langchain-chroma` wanting `chromadb < 0.6`) — uninstall memkernel from that interpreter, reinstall the versions the other tool needs, and install memkernel with pipx instead.
+
+</details>
 
 ### 2. Add memory instructions to CLAUDE.md
 
@@ -78,16 +121,30 @@ See the Prerequisites section inside each script for the Python environment requ
 
 ---
 
+## Add to Kiro
+
+Kiro (and the Kiro CLI, formerly Amazon Q Developer CLI) connects to MemKernel as a stdio MCP server. [`templates/kiro/`](templates/kiro/) holds the three files to copy into a workspace:
+
+| Template | Destination | Role |
+|----------|-------------|------|
+| `settings/mcp.json` | `.kiro/settings/mcp.json` | Server entry with `env` and `autoApprove` |
+| `steering/memory.md` | `.kiro/steering/memory.md` | Memory instructions, always included |
+| `hooks/memkernel.json` | `.kiro/hooks/memkernel.json` | `SessionStart` recall, `Stop` retain, manual checkpoint |
+
+[`templates/kiro/README.md`](templates/kiro/README.md) has the setup steps, the Kiro CLI custom-agent variant and the caveats (absolute persist path, hook format by Kiro version).
+
+---
+
 ## MCP Tools
 
 | Tool | What it does |
 |------|-------------|
 | `retain(content, type, session_id?, checkpoint?, source?, importance?, tags?, supersedes?)` | Store a memory. Returns `ref_id`. Idempotent — identical content + type returns the same `ref_id`. `supersedes` hides the memory it replaces. |
-| `recall(query, type?, tier?, session_id?, checkpoint_only?, top_k?)` | Search by natural language or identifier. Returns lightweight `ArtifactRef` list ranked by relevance + recency + importance. Records an access (reinforces retention). |
+| `recall(query, type?, tier?, session_id?, checkpoint_only?, top_k?, include_needs_review?)` | Search by natural language or identifier. Returns lightweight `ArtifactRef` list ranked by relevance + recency + importance. Records an access (reinforces retention). Code memories flagged `needs_review` are hidden unless `include_needs_review=true`. |
 | `get(ref_id)` | Fetch full content of a specific memory. Records an access. |
 | `forget(ref_id)` | Delete a memory from all indexes. |
 | `promote(ref_id, type, content?, importance?, tags?)` | Turn a short-term/working memory into a long-term one (you supply the distilled text). Keeps provenance via `promoted_from`. |
-| `consolidate(session_id?, limit?, synthesize?)` | Purge expired short-term memories and list the ones most worth promoting (ranked by importance × how often they were recalled). `synthesize=true` also asks an external reasoner to draft long-term statements (`suggested`). |
+| `consolidate(session_id?, limit?, synthesize?, revalidate?)` | Purge expired short-term memories and list the ones most worth promoting (ranked by importance × how often they were recalled). `synthesize=true` also asks an external reasoner to draft long-term statements (`suggested`); `revalidate=true` first checks every code memory against its source file (`revalidated`). The report always carries a `health` summary. |
 | `ask(ref_id, question, mode?)` | Answer a question about one memory without loading it — for a `document` the full body is consulted. `mode`: `direct` \| `rag` \| `rlm` \| `auto`, passed to the reasoner unchanged; the built-in reasoner answers in `rag` mode with the best-matching chunks. Returns `answer`, `mode`, `citations`, `usage`. |
 | `wiki_export(wiki_dir?)` | Compile long-term memories without a wiki page into draft markdown pages (`index.md`, `log.md` updated). |
 | `wiki_import(wiki_dir?)` | Index curated wiki pages as long-term memories (idempotent; changed pages supersede). |
@@ -100,7 +157,7 @@ See the Prerequisites section inside each script for the Python environment requ
 | `convention` | long-term | Naming rules, code style, project patterns | 0.9 | Never |
 | `fact` | long-term | Durable project facts: build commands, environments, contracts | 0.7 | Never |
 | `procedure` | long-term | How-to that worked: workflows, recipes, verified skills | 0.8 | Never |
-| `code` | long-term | Function/class snapshots, API contracts | 0.6 | Never |
+| `code` | long-term | Function/class snapshots, API contracts; identified by `(source, content)`, a changed file supersedes the old snapshot, a drifted source flags `needs_review` | 0.6 | Never (superseded on re-retain) |
 | `document` | long-term | Large artefacts (specs, transcripts, files): the body is stored and chunk-indexed, `get` returns a summary, `ask` answers from the indexed chunks (local reasoner) or the body (RLM Studio) | 0.6 | Never |
 | `episode` | short-term | Session events, compaction checkpoints | 0.4 | 30 days, extended by every recall |
 | `observation` | short-term | Findings, tool outputs, things noticed | 0.3 | 7 days, extended by every recall |
@@ -115,6 +172,7 @@ Retention: `ttl = base_ttl × (1 + access_boost × access_count)`, measured from
 3. **Before compaction** — `retain(type="episode", checkpoint=true)` (the `pre-compact.py` hook) and `consolidate()`.
 4. **After compaction / end of phase** — `promote(ref_id, type, content)` for the candidates worth keeping.
 5. **Large artefacts** — `retain(type="document", content=<body>, source=<name>)` then `ask(ref_id, question)`; never `get` a document to read it.
+7. **Code snapshots** — `retain(type="code", content=<file>, source=<path>)`; retain the file again after editing it (the new snapshot supersedes the old one). The session-start hook revalidates snapshots against their files and lists the ones that need re-retaining.
 6. **Curate** — `wiki_export` → review the drafts → `wiki_import`.
 
 ### Quick examples
@@ -152,6 +210,8 @@ ask(ref_id="<document ref_id>", question="Which endpoints need rate limiting?",
 ## Configuration
 
 All settings are read from environment variables with the `MEMKERNEL_` prefix, or from a `.env` file.
+Entries in `.env` without the prefix (keys for other tools) are ignored; a `MEMKERNEL_` key that matches no
+setting is logged as a warning at startup.
 
 | Variable | Default | Description |
 |----------|---------|-------------|
@@ -170,6 +230,10 @@ All settings are read from environment variables with the `MEMKERNEL_` prefix, o
 | `MEMKERNEL_DOCUMENT_SUMMARY_CHARS` | `1000` | Characters of the body kept in a `document` memory's summary record |
 | `MEMKERNEL_REASONER` | `local` | `ReasonerPort` behind `ask`: `local` (hybrid search over chunks, no LLM) or `rlmstudio` (falls back to `local` when `rlmstudio` is not importable) |
 | `MEMKERNEL_ASK_TOP_K` | `5` | Chunks the local reasoner returns per `ask` |
+| `MEMKERNEL_PROJECT_ROOT` | `.` | Directory that relative `source` paths of `code` memories resolve under; nothing outside it is ever read. Set it whenever the server or a hook is not started from the project directory |
+| `MEMKERNEL_HEALTH_DUPLICATE_THRESHOLD` | `0.95` | Cosine similarity in [0, 1] from which two long-term memories count as near-duplicates in the health pass |
+| `MEMKERNEL_HEALTH_DUPLICATE_LIMIT` | `500` | Most recent long-term memories the near-duplicate check covers |
+| `MEMKERNEL_HEALTH_REPORT_LIMIT` | `20` | Items listed per category in a health report |
 | `MEMKERNEL_EPISODE_TTL_DAYS` | `30` | Base TTL for `episode` memories (short-term) |
 | `MEMKERNEL_OBSERVATION_TTL_DAYS` | `7` | Base TTL for `observation` memories (short-term) |
 | `MEMKERNEL_TASK_TTL_HOURS` | `24` | Base TTL for `task` memories (working) |
@@ -258,7 +322,13 @@ Long-term memory has a human-readable twin: a markdown wiki in the [LLM Wiki](ht
 - `wiki_export` compiles every long-term memory that has no page yet into a **draft** page under `decisions/`, `concepts/` or `workflows/`, with the required sections and a `memkernel://<project>/<ref_id>` citation. Existing pages are never overwritten.
 - `wiki_import` indexes curated pages as long-term memories so `recall` finds the reviewed version; a changed page supersedes its earlier memory.
 
-Set `MEMKERNEL_WIKI_DIR` (for example `knowledge/wiki`) or pass `wiki_dir` to the tools. Rationale and the proposed standalone `ai-wiki` repo: [`docs/adr/ADR-008-llm-wiki-as-semantic-layer.md`](docs/adr/ADR-008-llm-wiki-as-semantic-layer.md).
+Set `MEMKERNEL_WIKI_DIR` (for example `knowledge/wiki`) or pass `wiki_dir` to the tools. The standalone maintainer that owns this page schema is [wiki-studio](https://github.com/gosha70/wiki-studio) (`pip install wiki-studio`; `wiki-studio lint --health` checks what `wiki_export` writes). Rationale: [`docs/adr/ADR-008-llm-wiki-as-semantic-layer.md`](docs/adr/ADR-008-llm-wiki-as-semantic-layer.md).
+
+### Revalidation and store health
+
+A `code` memory is a snapshot of a file. `MemoryService.revalidate()` hashes each live snapshot's whole source file and compares it with the stored content hash: a changed file flags the memory `needs_review` (`changed`), a vanished file flags it `missing`, a matching file clears the flag. Flagged memories leave default `recall` until the file is retained again (which supersedes the stale snapshot) or they are asked for with `include_needs_review=true`. Revalidation runs from `consolidate(revalidate=true)`, at every session start through `MemoryLifecycle.on_session_start()` (a `### Needs review` section lists what to re-retain), and from `POST /revalidate`. Sources resolve under `MEMKERNEL_PROJECT_ROOT` (default: the current directory); set it whenever the server or a hook is not started from the project directory. Nothing outside that root is ever read. 0.4 hashes whole files; per-chunk invalidation is a follow-up.
+
+`MemoryService.health()` audits the store without changing it: orphaned chunk and document records, dangling `supersedes` / `superseded_by` / `promoted_from` / `parent_ref` links, supersessions from 0.2/0.3 that lack the durable marker, `code` memories whose source file is gone, memories flagged for review, expired short-term memories not yet purged and, in full mode only, near-duplicate pairs among the most recent long-term memories (cosine ≥ `MEMKERNEL_HEALTH_DUPLICATE_THRESHOLD`). The quick report (counts only, no embeddings, no content loads) is part of every `consolidate()` report and of the `memory://status` resource; the full report is `GET /health/store`. The pass never fixes anything: a fix mode is a later feature.
 
 ## Admin API
 
@@ -273,6 +343,8 @@ uvicorn memkernel.api.app:app --host 127.0.0.1 --port 8766
 | `GET /health` | Server health and memory count |
 | `GET /stats` | Total memory count, per-type and per-tier breakdown, growth rate, last recall latency |
 | `GET /collections` | List ChromaDB collections |
+| `GET /health/store` | Full store health report (orphans, dangling links, stale sources, near-duplicates, …) |
+| `POST /revalidate` | Check every code memory against its source file; flag changed or missing ones |
 
 ---
 
@@ -295,7 +367,7 @@ Releases are published to PyPI by [`.github/workflows/release.yml`](.github/work
 
 1. Set `version` in `pyproject.toml` and move the `[Unreleased]` entries in [`CHANGELOG.md`](CHANGELOG.md) under the new version.
 2. Merge to `main` with CI green.
-3. Tag and push: `git tag -a v0.3.0 -m "memkernel 0.3.0" && git push origin v0.3.0`.
+3. Tag and push: `git tag -a v0.4.0 -m "memkernel 0.4.0" && git push origin v0.4.0`.
 
 The workflow refuses a tag that does not match the `pyproject.toml` version, builds the sdist and wheel, smoke-installs the wheel, and uploads through the `pypi` environment.
 
@@ -330,7 +402,7 @@ MemKernel is the memory component of a small family of projects by the same auth
 
 | Project | Role | Relationship |
 |---|---|---|
-| [ai-wiki](https://github.com/gosha70/ai-wiki) | LLM-maintained markdown wiki (ingest / promote / query / lint) | the human-readable form of long-term memory; `wiki_export` / `wiki_import` speak its page schema (ADR-008) |
+| [wiki-studio](https://github.com/gosha70/wiki-studio) | LLM-wiki maintainer (ingest / promote / query / lint / audit), stdlib-only | the human-readable form of long-term memory; `wiki_export` / `wiki_import` speak its page schema (ADR-008), and CI lints exporter output with it |
 | [rlm-studio](https://github.com/gosha70/rlm-studio) | Recursive Language Models workbench: Direct / RAG / RLM / auto over large documents | the document layer behind `document` memories and the `ask()` tool (`MEMKERNEL_REASONER=rlmstudio`, ADR-009) |
 | [code-copilot-team](https://github.com/gosha70/code-copilot-team) | Enforceable harness configuration for Claude Code, Pi and other copilots | consumes MemKernel through MCP and hooks |
 | [haa](https://github.com/gosha70/haa) | Home-built AI coding harness, future component assembler | consumes MemKernel in-process through `build_service()` |

{memkernel-0.3.0 → memkernel-0.4.1}/pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "memkernel"
-version = "0.3.0"
+version = "0.4.1"
 description = "Code-aware memory server for AI coding copilots"
 authors = ["EGOGE"]
 license = "Apache-2.0"
@@ -40,6 +40,7 @@ ruff = ">=0.9"
 mypy = ">=1.13"
 httpx = ">=0.27"
 pip-audit = "^2.10.0"
+wiki-studio = ">=0.1,<1"
 
 [build-system]
 requires = ["poetry-core"]

{memkernel-0.3.0 → memkernel-0.4.1}/src/memkernel/__init__.py

@@ -2,4 +2,4 @@
 # SPDX-License-Identifier: Apache-2.0
 """MemKernel — Code-aware memory server for AI coding copilots."""
 
-__version__ = "0.3.0"
+__version__ = "0.4.1"

{memkernel-0.3.0 → memkernel-0.4.1}/src/memkernel/api/routes/admin.py

@@ -1,6 +1,6 @@
 # Copyright (c) 2026 EGOGE
 # SPDX-License-Identifier: Apache-2.0
-"""Admin API route handlers: /health, /stats, /collections."""
+"""Admin API route handlers: /health, /health/store, /stats, /collections, /revalidate."""
 
 from __future__ import annotations
 
@@ -30,6 +30,14 @@ async def health(request: Request) -> dict[str, Any]:
     }
 
 
+@router.get("/health/store")
+async def health_store(request: Request) -> dict[str, Any]:
+    """Read-only store health report: orphans, dangling links, stale sources, near-duplicates and more."""
+    svc = _svc(request)
+    report = svc.health()
+    return {**report.model_dump(mode="json"), "summary": report.summary}
+
+
 @router.get("/stats")
 async def stats(request: Request) -> dict[str, Any]:
     """Memory metrics: per-type breakdown, growth by time window, last recall latency."""
@@ -42,3 +50,10 @@ async def collections(request: Request) -> list[dict[str, Any]]:
     """List all project collections with record counts."""
     svc = _svc(request)
     return svc.list_collections()
+
+
+@router.post("/revalidate")
+async def revalidate(request: Request) -> dict[str, Any]:
+    """Check every code memory against its source file; flag changed or missing ones (needs_review)."""
+    svc = _svc(request)
+    return svc.revalidate()

memkernel-0.4.1/src/memkernel/config.py

@@ -0,0 +1,129 @@
+# Copyright (c) 2026 EGOGE
+# SPDX-License-Identifier: Apache-2.0
+"""MemKernel configuration via environment variables."""
+
+from __future__ import annotations
+
+import logging
+from typing import TYPE_CHECKING, Any, Literal
+
+from pydantic_settings import BaseSettings, EnvSettingsSource, PydanticBaseSettingsSource, SettingsConfigDict
+
+if TYPE_CHECKING:
+    from pydantic.fields import FieldInfo
+
+logger = logging.getLogger(__name__)
+
+ENV_PREFIX = "MEMKERNEL_"
+ENV_FILE = ".env"
+
+
+class MemKernelSettings(BaseSettings):
+    """All settings are configurable via MEMKERNEL_ prefixed env vars."""
+
+    # Unknown keys stay forbidden (constructor typos raise), but the .env source
+    # is filtered to known settings: a project .env usually holds keys for other
+    # tools and must not stop the server from starting (issue #15). Misspelled
+    # MEMKERNEL_ keys are reported by settings_customise_sources().
+    model_config = SettingsConfigDict(env_prefix=ENV_PREFIX, env_file=ENV_FILE)
+
+    project_id: str = "default"
+    default_session_id: str = "default"
+
+    # ChromaDB
+    chroma_persist_path: str = ".memkernel/chroma"
+    chroma_collection_prefix: str = "memkernel"
+
+    # Embedding
+    embedding_model: str = "all-MiniLM-L6-v2"
+    embedding_dimension: int = 384
+
+    # Retrieval
+    search_top_k: int = 5
+    rrf_k: int = 60
+    preview_length: int = 200
+
+    # Chunking
+    chunk_size: int = 1500  # max chars per chunk
+    chunk_overlap: int = 200  # char overlap between token chunks (fallback only)
+
+    # Documents and ask() (ADR-009)
+    document_store: Literal["local"] = "local"  # where `document` bodies live
+    document_summary_chars: int = 1000  # characters of the body kept in the document's summary record
+    reasoner: Literal["local", "rlmstudio"] = "local"  # ReasonerPort behind ask(); falls back to local
+    ask_top_k: int = 5  # chunks the local reasoner returns per ask()
+
+    # Sources, revalidation and store health (0.4)
+    project_root: str = "."  # relative `source` paths resolve under this directory; nothing outside it is read
+    health_duplicate_threshold: float = (
+        0.95  # cosine similarity in [0, 1] from which two long-term memories are near-duplicates
+    )
+    health_duplicate_limit: int = 500  # most recent long-term memories checked for near-duplicates
+    health_report_limit: int = 20  # items listed per health category
+
+    # Lifecycle / decay (see docs/architecture/memory-tiers.md)
+    episode_ttl_days: int = 30  # base TTL for short-term `episode` memories
+    observation_ttl_days: int = 7  # base TTL for short-term `observation` memories
+    task_ttl_hours: int = 24  # base TTL for working-memory `task` records
+    access_boost: float = 0.5  # each recall extends TTL by this fraction of the base TTL
+    recency_half_life_days: float = 7.0  # recency decay half-life for short-term memories
+
+    # Recall scoring: final = w_relevance * fused + w_recency * recency + w_importance * importance
+    recall_weight_relevance: float = 1.0
+    recall_weight_recency: float = 0.3
+    recall_weight_importance: float = 0.3
+
+    # LLM wiki bridge: directory holding the markdown wiki (index.md, decisions/, ...).
+    # Empty = no wiki configured; wiki_export/wiki_import then require an explicit path.
+    wiki_dir: str = ""
+
+    # Server
+    transport: Literal["stdio", "streamable-http"] = "stdio"
+    host: str = "127.0.0.1"
+    port: int = 8765
+
+    @classmethod
+    def settings_customise_sources(
+        cls,
+        settings_cls: type[BaseSettings],
+        init_settings: PydanticBaseSettingsSource,
+        env_settings: PydanticBaseSettingsSource,
+        dotenv_settings: PydanticBaseSettingsSource,
+        file_secret_settings: PydanticBaseSettingsSource,
+    ) -> tuple[PydanticBaseSettingsSource, ...]:
+        """Default source order with the .env source limited to known settings.
+
+        Warns about MEMKERNEL_ keys in the environment or in .env that match no setting.
+        """
+        for source, origin in ((env_settings, "the environment"), (dotenv_settings, ENV_FILE)):
+            for key in unknown_prefixed_keys(settings_cls, source):
+                logger.warning(
+                    "memkernel: unknown setting %s in %s is ignored; check the spelling against the README", key, origin
+                )
+        return init_settings, env_settings, KnownFieldsOnly(dotenv_settings), file_secret_settings
+
+
+class KnownFieldsOnly(PydanticBaseSettingsSource):
+    """Pass through only the values of ``inner`` that belong to a settings field."""
+
+    def __init__(self, inner: PydanticBaseSettingsSource) -> None:
+        super().__init__(inner.settings_cls)
+        self._inner = inner
+
+    def get_field_value(self, field: FieldInfo, field_name: str) -> tuple[Any, str, bool]:
+        return self._inner.get_field_value(field, field_name)
+
+    def __call__(self) -> dict[str, Any]:
+        return {key: value for key, value in self._inner().items() if key in self.settings_cls.model_fields}
+
+
+def unknown_prefixed_keys(settings_cls: type[BaseSettings], source: PydanticBaseSettingsSource) -> list[str]:
+    """Return the MEMKERNEL_ keys ``source`` holds that are not fields of ``settings_cls``, upper-cased."""
+    if not isinstance(source, EnvSettingsSource):
+        return []
+    prefix = ENV_PREFIX.lower()  # case-insensitive sources expose lower-cased keys
+    return sorted(
+        key.upper()
+        for key in source.env_vars
+        if key.lower().startswith(prefix) and key.lower()[len(prefix) :] not in settings_cls.model_fields
+    )