memkernel 0.2.0__tar.gz → 0.3.0__tar.gz

{memkernel-0.2.0 → memkernel-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: memkernel
-Version: 0.2.0
+Version: 0.3.0
 Summary: Code-aware memory server for AI coding copilots
 License: Apache-2.0
 License-File: LICENSE
@@ -22,6 +22,8 @@ Requires-Dist: rank-bm25 (>=0.2)
 Requires-Dist: sentence-transformers (>=4.0,<6)
 Requires-Dist: tree-sitter (>=0.22,<1)
 Requires-Dist: tree-sitter-python (>=0.23)
+Project-URL: Documentation, https://memkernel.dev
+Project-URL: Homepage, https://memkernel.dev
 Project-URL: Repository, https://github.com/gosha70/memkernel
 Description-Content-Type: text/markdown
 
@@ -32,6 +34,8 @@ Description-Content-Type: text/markdown
 [![Python](https://img.shields.io/badge/python-3.11%20%7C%203.12%20%7C%203.13-blue)](pyproject.toml)
 [![License: Apache-2.0](https://img.shields.io/badge/license-Apache--2.0-green)](LICENSE)
 
+**Site:** [memkernel.dev](https://memkernel.dev) · **Package:** [pypi.org/project/memkernel](https://pypi.org/project/memkernel/)
+
 Code-aware memory server for AI coding copilots.
 
 MemKernel gives Claude Code (and any MCP-compatible copilot) persistent memory across sessions. When Claude compacts context or a session ends, project decisions, conventions, and code understanding are normally lost. MemKernel stores them in a local vector database and retrieves them on demand.
@@ -99,7 +103,7 @@ Copy the hook scripts from [`templates/hooks/`](templates/hooks/) to `.claude/ho
 | `pre-compact.py` | `PreCompact` | Retains a checkpoint episode, runs a consolidation pass |
 | `post-compact.py` | `PostCompact` | Re-injects the checkpoint, long-term context and promotion candidates |
 
-See the Prerequisites section inside each script for the Python environment requirement.
+See the Prerequisites section inside each script for the Python environment requirement. Each script is a thin wrapper over [`MemoryLifecycle`](#embedding-memkernel-in-your-own-harness), so an in-process harness gets the same behaviour without the scripts.
 
 ---
 
@@ -112,8 +116,8 @@ See the Prerequisites section inside each script for the Python environment requ
 | `get(ref_id)` | Fetch full content of a specific memory. Records an access. |
 | `forget(ref_id)` | Delete a memory from all indexes. |
 | `promote(ref_id, type, content?, importance?, tags?)` | Turn a short-term/working memory into a long-term one (you supply the distilled text). Keeps provenance via `promoted_from`. |
-| `consolidate(session_id?, limit?)` | Purge expired short-term memories and list the ones most worth promoting (ranked by importance × how often they were recalled). |
-| `forget_expired()` | Delete every short-term/working memory whose retention window lapsed. |
+| `consolidate(session_id?, limit?, synthesize?)` | Purge expired short-term memories and list the ones most worth promoting (ranked by importance × how often they were recalled). `synthesize=true` also asks an external reasoner to draft long-term statements (`suggested`). |
+| `ask(ref_id, question, mode?)` | Answer a question about one memory without loading it — for a `document` the full body is consulted. `mode`: `direct` \| `rag` \| `rlm` \| `auto`, passed to the reasoner unchanged; the built-in reasoner answers in `rag` mode with the best-matching chunks. Returns `answer`, `mode`, `citations`, `usage`. |
 | `wiki_export(wiki_dir?)` | Compile long-term memories without a wiki page into draft markdown pages (`index.md`, `log.md` updated). |
 | `wiki_import(wiki_dir?)` | Index curated wiki pages as long-term memories (idempotent; changed pages supersede). |
 
@@ -126,6 +130,7 @@ See the Prerequisites section inside each script for the Python environment requ
 | `fact` | long-term | Durable project facts: build commands, environments, contracts | 0.7 | Never |
 | `procedure` | long-term | How-to that worked: workflows, recipes, verified skills | 0.8 | Never |
 | `code` | long-term | Function/class snapshots, API contracts | 0.6 | Never |
+| `document` | long-term | Large artefacts (specs, transcripts, files): the body is stored and chunk-indexed, `get` returns a summary, `ask` answers from the indexed chunks (local reasoner) or the body (RLM Studio) | 0.6 | Never |
 | `episode` | short-term | Session events, compaction checkpoints | 0.4 | 30 days, extended by every recall |
 | `observation` | short-term | Findings, tool outputs, things noticed | 0.3 | 7 days, extended by every recall |
 | `task` | working | The current goal, plan state, open questions | 0.5 | 24 hours |
@@ -138,7 +143,8 @@ Retention: `ttl = base_ttl × (1 + access_boost × access_count)`, measured from
 2. **While working** — `retain(type="task")` when the plan changes; `retain(type="observation")` for findings; durable knowledge as `decision` / `convention` / `fact` / `procedure`.
 3. **Before compaction** — `retain(type="episode", checkpoint=true)` (the `pre-compact.py` hook) and `consolidate()`.
 4. **After compaction / end of phase** — `promote(ref_id, type, content)` for the candidates worth keeping.
-5. **Curate** — `wiki_export` → review the drafts → `wiki_import`.
+5. **Large artefacts** — `retain(type="document", content=<body>, source=<name>)` then `ask(ref_id, question)`; never `get` a document to read it.
+6. **Curate** — `wiki_export` → review the drafts → `wiki_import`.
 
 ### Quick examples
 
@@ -164,6 +170,10 @@ retain(content="Use arq, not Celery", type="decision", supersedes="<old decision
 get(ref_id="<ref_id from recall>")
 
 forget(ref_id="<ref_id>")
+
+retain(content=<the whole spec>, type="document", source="specs/auth/spec.md")
+
+ask(ref_id="<document ref_id>", question="Which endpoints need rate limiting?", mode="auto")
 ```
 
 ---
@@ -185,6 +195,10 @@ All settings are read from environment variables with the `MEMKERNEL_` prefix, o
 | `MEMKERNEL_PREVIEW_LENGTH` | `200` | Max characters in the `preview` field of `ArtifactRef` |
 | `MEMKERNEL_CHUNK_SIZE` | `1500` | Max characters per AST chunk (code memories) |
 | `MEMKERNEL_CHUNK_OVERLAP` | `200` | Character overlap between token-based fallback chunks |
+| `MEMKERNEL_DOCUMENT_STORE` | `local` | Where `document` bodies live (`local` = the project's own ChromaDB collection) |
+| `MEMKERNEL_DOCUMENT_SUMMARY_CHARS` | `1000` | Characters of the body kept in a `document` memory's summary record |
+| `MEMKERNEL_REASONER` | `local` | `ReasonerPort` behind `ask`: `local` (hybrid search over chunks, no LLM) or `rlmstudio` (falls back to `local` when `rlmstudio` is not importable) |
+| `MEMKERNEL_ASK_TOP_K` | `5` | Chunks the local reasoner returns per `ask` |
 | `MEMKERNEL_EPISODE_TTL_DAYS` | `30` | Base TTL for `episode` memories (short-term) |
 | `MEMKERNEL_OBSERVATION_TTL_DAYS` | `7` | Base TTL for `observation` memories (short-term) |
 | `MEMKERNEL_TASK_TTL_HOURS` | `24` | Base TTL for `task` memories (working) |
@@ -237,6 +251,35 @@ report = svc.consolidate()                 # {"purged": n, "candidates": [...]}
 
 `build_service()` is the supported entry point; the MCP server, the admin API and the hook templates all use it. Integration notes for `code-copilot-team` and `haa` are in [`docs/integration/harness-integration.md`](docs/integration/harness-integration.md).
 
+### The session protocol as a component
+
+`MemoryLifecycle` packages what the hook scripts do, so a harness calls four methods instead of re-implementing the protocol. Every method returns markdown ready for injection and does nothing but talk to the service:
+
+```python
+from memkernel.factory import build_service
+from memkernel.lifecycle import MemoryLifecycle
+
+memory = MemoryLifecycle(build_service())
+system_context = memory.on_session_start()       # checkpoint, long-term context, promotion candidates, wiki pointer
+...
+memory.on_pre_compact(summary)                   # checkpoint episode + consolidation, before the context is compacted
+recovered = memory.on_post_compact()             # what to re-inject after compaction
+memory.on_session_end(summary)                   # final checkpoint + consolidation
+```
+
+### Documents and `ask`
+
+`retain(type="document")` keeps large artefacts out of the context window: the body goes to the document store (built-in: the project's own ChromaDB collection), its chunks are indexed for hybrid search, and the memkernel record is a summary. `ask(ref_id, question, mode)` answers through a `ReasonerPort`:
+
+- `MEMKERNEL_REASONER=local` (default) — no LLM: the question is matched against the document's chunks with dense + BM25 + RRF and the best chunks come back as the answer, `mode="rag"`.
+- `MEMKERNEL_REASONER=rlmstudio` — delegates to [rlm-studio](https://github.com/gosha70/rlm-studio)'s `interact(content, question, mode=mode)`, so `direct`, `rag`, `rlm` and `auto` select how the document is read (ADR-009). `rlm-studio` is not a memkernel dependency and is not on PyPI yet; install it from git into the same environment:
+
+  ```bash
+  pip install "rlm-studio @ git+https://github.com/gosha70/rlm-studio"
+  ```
+
+  When it is not importable, `build_service()` logs a warning and uses the local reasoner. With an external reasoner, `consolidate(synthesize=True)` also drafts long-term statements from the session's short-term memories.
+
 ## LLM wiki bridge
 
 Long-term memory has a human-readable twin: a markdown wiki in the [LLM Wiki](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f) pattern (`index.md`, `log.md`, typed pages with cited sources — the same schema as `code-copilot-team/knowledge/wiki`).
@@ -281,7 +324,7 @@ Releases are published to PyPI by [`.github/workflows/release.yml`](.github/work
 
 1. Set `version` in `pyproject.toml` and move the `[Unreleased]` entries in [`CHANGELOG.md`](CHANGELOG.md) under the new version.
 2. Merge to `main` with CI green.
-3. Tag and push: `git tag -a v0.2.0 -m "memkernel 0.2.0" && git push origin v0.2.0`.
+3. Tag and push: `git tag -a v0.3.0 -m "memkernel 0.3.0" && git push origin v0.3.0`.
 
 The workflow refuses a tag that does not match the `pyproject.toml` version, builds the sdist and wheel, smoke-installs the wheel, and uploads through the `pypi` environment.
 
@@ -293,12 +336,14 @@ The workflow refuses a tag that does not match the `pyproject.toml` version, bui
 src/memkernel/
 ├── mcp/           # MCP server + tool definitions (memory tools, wiki tools)
 ├── core/          # Domain models (tiers, types), MemoryService, ports (interfaces)
-├── storage/       # ChromaDB adapter
-├── retrieval/     # AST chunking, embeddings, hybrid search (dense + BM25 + RRF + rescoring)
+├── adapters/      # Optional external systems (RLM Studio reasoner), imported lazily
+├── storage/       # ChromaDB adapter, local document store
+├── retrieval/     # AST/token chunking, embeddings, hybrid search (dense + BM25 + RRF + rescoring), local reasoner
 ├── memory/        # Lifecycle: TTL/decay policies, retention, deduplication
 ├── wiki/          # LLM-wiki bridge: page model, export, import
 ├── api/           # FastAPI admin endpoints
 ├── factory.py     # build_service() — the supported embedding entry point
+├── lifecycle.py   # MemoryLifecycle — the session protocol as a component
 └── config.py      # Pydantic settings
 ```
 
@@ -315,7 +360,7 @@ MemKernel is the memory component of a small family of projects by the same auth
 | Project | Role | Relationship |
 |---|---|---|
 | [ai-wiki](https://github.com/gosha70/ai-wiki) | LLM-maintained markdown wiki (ingest / promote / query / lint) | the human-readable form of long-term memory; `wiki_export` / `wiki_import` speak its page schema (ADR-008) |
-| [rlm-studio](https://github.com/gosha70/rlm-studio) | Recursive Language Models workbench: Direct / RAG / RLM / auto over large documents | the document layer; planned `document` memory type and `ask()` tool (ADR-009) |
+| [rlm-studio](https://github.com/gosha70/rlm-studio) | Recursive Language Models workbench: Direct / RAG / RLM / auto over large documents | the document layer behind `document` memories and the `ask()` tool (`MEMKERNEL_REASONER=rlmstudio`, ADR-009) |
 | [code-copilot-team](https://github.com/gosha70/code-copilot-team) | Enforceable harness configuration for Claude Code, Pi and other copilots | consumes MemKernel through MCP and hooks |
 | [haa](https://github.com/gosha70/haa) | Home-built AI coding harness, future component assembler | consumes MemKernel in-process through `build_service()` |
 

{memkernel-0.2.0 → memkernel-0.3.0}/README.md

@@ -5,6 +5,8 @@
 [![Python](https://img.shields.io/badge/python-3.11%20%7C%203.12%20%7C%203.13-blue)](pyproject.toml)
 [![License: Apache-2.0](https://img.shields.io/badge/license-Apache--2.0-green)](LICENSE)
 
+**Site:** [memkernel.dev](https://memkernel.dev) · **Package:** [pypi.org/project/memkernel](https://pypi.org/project/memkernel/)
+
 Code-aware memory server for AI coding copilots.
 
 MemKernel gives Claude Code (and any MCP-compatible copilot) persistent memory across sessions. When Claude compacts context or a session ends, project decisions, conventions, and code understanding are normally lost. MemKernel stores them in a local vector database and retrieves them on demand.
@@ -72,7 +74,7 @@ Copy the hook scripts from [`templates/hooks/`](templates/hooks/) to `.claude/ho
 | `pre-compact.py` | `PreCompact` | Retains a checkpoint episode, runs a consolidation pass |
 | `post-compact.py` | `PostCompact` | Re-injects the checkpoint, long-term context and promotion candidates |
 
-See the Prerequisites section inside each script for the Python environment requirement.
+See the Prerequisites section inside each script for the Python environment requirement. Each script is a thin wrapper over [`MemoryLifecycle`](#embedding-memkernel-in-your-own-harness), so an in-process harness gets the same behaviour without the scripts.
 
 ---
 
@@ -85,8 +87,8 @@ See the Prerequisites section inside each script for the Python environment requ
 | `get(ref_id)` | Fetch full content of a specific memory. Records an access. |
 | `forget(ref_id)` | Delete a memory from all indexes. |
 | `promote(ref_id, type, content?, importance?, tags?)` | Turn a short-term/working memory into a long-term one (you supply the distilled text). Keeps provenance via `promoted_from`. |
-| `consolidate(session_id?, limit?)` | Purge expired short-term memories and list the ones most worth promoting (ranked by importance × how often they were recalled). |
-| `forget_expired()` | Delete every short-term/working memory whose retention window lapsed. |
+| `consolidate(session_id?, limit?, synthesize?)` | Purge expired short-term memories and list the ones most worth promoting (ranked by importance × how often they were recalled). `synthesize=true` also asks an external reasoner to draft long-term statements (`suggested`). |
+| `ask(ref_id, question, mode?)` | Answer a question about one memory without loading it — for a `document` the full body is consulted. `mode`: `direct` \| `rag` \| `rlm` \| `auto`, passed to the reasoner unchanged; the built-in reasoner answers in `rag` mode with the best-matching chunks. Returns `answer`, `mode`, `citations`, `usage`. |
 | `wiki_export(wiki_dir?)` | Compile long-term memories without a wiki page into draft markdown pages (`index.md`, `log.md` updated). |
 | `wiki_import(wiki_dir?)` | Index curated wiki pages as long-term memories (idempotent; changed pages supersede). |
 
@@ -99,6 +101,7 @@ See the Prerequisites section inside each script for the Python environment requ
 | `fact` | long-term | Durable project facts: build commands, environments, contracts | 0.7 | Never |
 | `procedure` | long-term | How-to that worked: workflows, recipes, verified skills | 0.8 | Never |
 | `code` | long-term | Function/class snapshots, API contracts | 0.6 | Never |
+| `document` | long-term | Large artefacts (specs, transcripts, files): the body is stored and chunk-indexed, `get` returns a summary, `ask` answers from the indexed chunks (local reasoner) or the body (RLM Studio) | 0.6 | Never |
 | `episode` | short-term | Session events, compaction checkpoints | 0.4 | 30 days, extended by every recall |
 | `observation` | short-term | Findings, tool outputs, things noticed | 0.3 | 7 days, extended by every recall |
 | `task` | working | The current goal, plan state, open questions | 0.5 | 24 hours |
@@ -111,7 +114,8 @@ Retention: `ttl = base_ttl × (1 + access_boost × access_count)`, measured from
 2. **While working** — `retain(type="task")` when the plan changes; `retain(type="observation")` for findings; durable knowledge as `decision` / `convention` / `fact` / `procedure`.
 3. **Before compaction** — `retain(type="episode", checkpoint=true)` (the `pre-compact.py` hook) and `consolidate()`.
 4. **After compaction / end of phase** — `promote(ref_id, type, content)` for the candidates worth keeping.
-5. **Curate** — `wiki_export` → review the drafts → `wiki_import`.
+5. **Large artefacts** — `retain(type="document", content=<body>, source=<name>)` then `ask(ref_id, question)`; never `get` a document to read it.
+6. **Curate** — `wiki_export` → review the drafts → `wiki_import`.
 
 ### Quick examples
 
@@ -137,6 +141,10 @@ retain(content="Use arq, not Celery", type="decision", supersedes="<old decision
 get(ref_id="<ref_id from recall>")
 
 forget(ref_id="<ref_id>")
+
+retain(content=<the whole spec>, type="document", source="specs/auth/spec.md")
+
+ask(ref_id="<document ref_id>", question="Which endpoints need rate limiting?", mode="auto")
 ```
 
 ---
@@ -158,6 +166,10 @@ All settings are read from environment variables with the `MEMKERNEL_` prefix, o
 | `MEMKERNEL_PREVIEW_LENGTH` | `200` | Max characters in the `preview` field of `ArtifactRef` |
 | `MEMKERNEL_CHUNK_SIZE` | `1500` | Max characters per AST chunk (code memories) |
 | `MEMKERNEL_CHUNK_OVERLAP` | `200` | Character overlap between token-based fallback chunks |
+| `MEMKERNEL_DOCUMENT_STORE` | `local` | Where `document` bodies live (`local` = the project's own ChromaDB collection) |
+| `MEMKERNEL_DOCUMENT_SUMMARY_CHARS` | `1000` | Characters of the body kept in a `document` memory's summary record |
+| `MEMKERNEL_REASONER` | `local` | `ReasonerPort` behind `ask`: `local` (hybrid search over chunks, no LLM) or `rlmstudio` (falls back to `local` when `rlmstudio` is not importable) |
+| `MEMKERNEL_ASK_TOP_K` | `5` | Chunks the local reasoner returns per `ask` |
 | `MEMKERNEL_EPISODE_TTL_DAYS` | `30` | Base TTL for `episode` memories (short-term) |
 | `MEMKERNEL_OBSERVATION_TTL_DAYS` | `7` | Base TTL for `observation` memories (short-term) |
 | `MEMKERNEL_TASK_TTL_HOURS` | `24` | Base TTL for `task` memories (working) |
@@ -210,6 +222,35 @@ report = svc.consolidate()                 # {"purged": n, "candidates": [...]}
 
 `build_service()` is the supported entry point; the MCP server, the admin API and the hook templates all use it. Integration notes for `code-copilot-team` and `haa` are in [`docs/integration/harness-integration.md`](docs/integration/harness-integration.md).
 
+### The session protocol as a component
+
+`MemoryLifecycle` packages what the hook scripts do, so a harness calls four methods instead of re-implementing the protocol. Every method returns markdown ready for injection and does nothing but talk to the service:
+
+```python
+from memkernel.factory import build_service
+from memkernel.lifecycle import MemoryLifecycle
+
+memory = MemoryLifecycle(build_service())
+system_context = memory.on_session_start()       # checkpoint, long-term context, promotion candidates, wiki pointer
+...
+memory.on_pre_compact(summary)                   # checkpoint episode + consolidation, before the context is compacted
+recovered = memory.on_post_compact()             # what to re-inject after compaction
+memory.on_session_end(summary)                   # final checkpoint + consolidation
+```
+
+### Documents and `ask`
+
+`retain(type="document")` keeps large artefacts out of the context window: the body goes to the document store (built-in: the project's own ChromaDB collection), its chunks are indexed for hybrid search, and the memkernel record is a summary. `ask(ref_id, question, mode)` answers through a `ReasonerPort`:
+
+- `MEMKERNEL_REASONER=local` (default) — no LLM: the question is matched against the document's chunks with dense + BM25 + RRF and the best chunks come back as the answer, `mode="rag"`.
+- `MEMKERNEL_REASONER=rlmstudio` — delegates to [rlm-studio](https://github.com/gosha70/rlm-studio)'s `interact(content, question, mode=mode)`, so `direct`, `rag`, `rlm` and `auto` select how the document is read (ADR-009). `rlm-studio` is not a memkernel dependency and is not on PyPI yet; install it from git into the same environment:
+
+  ```bash
+  pip install "rlm-studio @ git+https://github.com/gosha70/rlm-studio"
+  ```
+
+  When it is not importable, `build_service()` logs a warning and uses the local reasoner. With an external reasoner, `consolidate(synthesize=True)` also drafts long-term statements from the session's short-term memories.
+
 ## LLM wiki bridge
 
 Long-term memory has a human-readable twin: a markdown wiki in the [LLM Wiki](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f) pattern (`index.md`, `log.md`, typed pages with cited sources — the same schema as `code-copilot-team/knowledge/wiki`).
@@ -254,7 +295,7 @@ Releases are published to PyPI by [`.github/workflows/release.yml`](.github/work
 
 1. Set `version` in `pyproject.toml` and move the `[Unreleased]` entries in [`CHANGELOG.md`](CHANGELOG.md) under the new version.
 2. Merge to `main` with CI green.
-3. Tag and push: `git tag -a v0.2.0 -m "memkernel 0.2.0" && git push origin v0.2.0`.
+3. Tag and push: `git tag -a v0.3.0 -m "memkernel 0.3.0" && git push origin v0.3.0`.
 
 The workflow refuses a tag that does not match the `pyproject.toml` version, builds the sdist and wheel, smoke-installs the wheel, and uploads through the `pypi` environment.
 
@@ -266,12 +307,14 @@ The workflow refuses a tag that does not match the `pyproject.toml` version, bui
 src/memkernel/
 ├── mcp/           # MCP server + tool definitions (memory tools, wiki tools)
 ├── core/          # Domain models (tiers, types), MemoryService, ports (interfaces)
-├── storage/       # ChromaDB adapter
-├── retrieval/     # AST chunking, embeddings, hybrid search (dense + BM25 + RRF + rescoring)
+├── adapters/      # Optional external systems (RLM Studio reasoner), imported lazily
+├── storage/       # ChromaDB adapter, local document store
+├── retrieval/     # AST/token chunking, embeddings, hybrid search (dense + BM25 + RRF + rescoring), local reasoner
 ├── memory/        # Lifecycle: TTL/decay policies, retention, deduplication
 ├── wiki/          # LLM-wiki bridge: page model, export, import
 ├── api/           # FastAPI admin endpoints
 ├── factory.py     # build_service() — the supported embedding entry point
+├── lifecycle.py   # MemoryLifecycle — the session protocol as a component
 └── config.py      # Pydantic settings
 ```
 
@@ -288,7 +331,7 @@ MemKernel is the memory component of a small family of projects by the same auth
 | Project | Role | Relationship |
 |---|---|---|
 | [ai-wiki](https://github.com/gosha70/ai-wiki) | LLM-maintained markdown wiki (ingest / promote / query / lint) | the human-readable form of long-term memory; `wiki_export` / `wiki_import` speak its page schema (ADR-008) |
-| [rlm-studio](https://github.com/gosha70/rlm-studio) | Recursive Language Models workbench: Direct / RAG / RLM / auto over large documents | the document layer; planned `document` memory type and `ask()` tool (ADR-009) |
+| [rlm-studio](https://github.com/gosha70/rlm-studio) | Recursive Language Models workbench: Direct / RAG / RLM / auto over large documents | the document layer behind `document` memories and the `ask()` tool (`MEMKERNEL_REASONER=rlmstudio`, ADR-009) |
 | [code-copilot-team](https://github.com/gosha70/code-copilot-team) | Enforceable harness configuration for Claude Code, Pi and other copilots | consumes MemKernel through MCP and hooks |
 | [haa](https://github.com/gosha70/haa) | Home-built AI coding harness, future component assembler | consumes MemKernel in-process through `build_service()` |
 

{memkernel-0.2.0 → memkernel-0.3.0}/pyproject.toml

@@ -1,11 +1,13 @@
 [tool.poetry]
 name = "memkernel"
-version = "0.2.0"
+version = "0.3.0"
 description = "Code-aware memory server for AI coding copilots"
 authors = ["EGOGE"]
 license = "Apache-2.0"
 readme = "README.md"
 repository = "https://github.com/gosha70/memkernel"
+homepage = "https://memkernel.dev"
+documentation = "https://memkernel.dev"
 classifiers = [
     "Programming Language :: Python :: 3",
     "Programming Language :: Python :: 3.11",

{memkernel-0.2.0 → memkernel-0.3.0}/src/memkernel/__init__.py

@@ -2,4 +2,4 @@
 # SPDX-License-Identifier: Apache-2.0
 """MemKernel — Code-aware memory server for AI coding copilots."""
 
-__version__ = "0.2.0"
+__version__ = "0.3.0"

memkernel-0.3.0/src/memkernel/adapters/__init__.py

@@ -0,0 +1,3 @@
+# Copyright (c) 2026 EGOGE
+# SPDX-License-Identifier: Apache-2.0
+"""Adapters to optional external systems. Each imports its package lazily and degrades cleanly."""

memkernel-0.3.0/src/memkernel/adapters/rlmstudio.py

@@ -0,0 +1,123 @@
+# Copyright (c) 2026 EGOGE
+# SPDX-License-Identifier: Apache-2.0
+"""RLM Studio adapter implementing ReasonerPort (ADR-009).
+
+``rlmstudio`` is imported inside :class:`RLMStudioReasoner.__init__`, never at
+package import time, because it pulls LiteLLM in. It is not a dependency of
+memkernel (FR-016); install it from git::
+
+    pip install "rlm-studio @ git+https://github.com/gosha70/rlm-studio"
+
+and set ``MEMKERNEL_REASONER=rlmstudio``. When the package is missing the
+adapter raises :class:`ReasonerUnavailable` and ``build_service()`` falls
+back to the LLM-free :class:`~memkernel.retrieval.reasoner.LocalReasoner`.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Mapping
+from typing import Any
+
+from memkernel.core.models import Answer, ReasonerError
+from memkernel.retrieval.reasoner import ReasonerUnavailable
+
+INSTALL_HINT = 'pip install "rlm-studio @ git+https://github.com/gosha70/rlm-studio"'
+
+# Field names tried, in order, when mapping an ``interact`` result onto Answer.
+# ``InteractResult`` (rlmstudio.api) carries ``answer``, ``mode_used``,
+# ``metrics`` and ``raw_result`` (a RunResultDTO with ``success`` / ``error``);
+# the other names cover dict results and older shapes.
+_ANSWER_FIELDS = ("answer", "response", "text", "content", "output")
+_MODE_FIELDS = ("mode_used", "mode", "strategy")
+_CITATION_FIELDS = ("citations", "sources", "references")
+_USAGE_FIELDS = ("metrics", "usage", "cost", "budget")
+_RAW_RESULT_FIELDS = ("raw_result",)
+_SUCCESS_FIELDS = ("success",)
+_ERROR_FIELDS = ("error",)
+
+
+def _field(result: Any, names: tuple[str, ...]) -> Any:
+    """Read the first present field from a mapping or an object; None when absent."""
+    for name in names:
+        if isinstance(result, Mapping):
+            if name in result:
+                return result[name]
+        elif hasattr(result, name):
+            return getattr(result, name)
+    return None
+
+
+def _usage(raw: Any) -> dict[str, float | int]:
+    """Keep the numeric entries of a usage mapping (tokens, cost, steps, ...)."""
+    usage: dict[str, float | int] = {}
+    items = raw.items() if isinstance(raw, Mapping) else vars(raw).items() if hasattr(raw, "__dict__") else ()
+    for key, value in items:
+        if isinstance(value, bool):
+            continue
+        if isinstance(value, int | float):
+            usage[str(key)] = value
+    return usage
+
+
+def _failure(result: Any) -> str | None:
+    """Return the error text when the result (or its raw result) reports ``success=False``."""
+    for candidate in (result, _field(result, _RAW_RESULT_FIELDS)):
+        if candidate is None:
+            continue
+        if _field(candidate, _SUCCESS_FIELDS) is False:
+            error = _field(candidate, _ERROR_FIELDS)
+            return str(error) if error else "execution reported success=False"
+    return None
+
+
+def to_answer(result: Any, requested_mode: str) -> Answer:
+    """Map whatever ``rlmstudio.interact`` returned onto :class:`Answer`.
+
+    A result whose ``success`` (or ``raw_result.success``) is False raises
+    :class:`ReasonerError`: RLM Studio puts formatted error text in ``answer``
+    on failure, and that text must never be taken for an answer. The execution
+    ``trace`` is not provenance and is never turned into a citation.
+    """
+    if result is None:
+        return Answer(answer="", mode=requested_mode)
+    if isinstance(result, str):
+        return Answer(answer=result, mode=requested_mode)
+    failure = _failure(result)
+    if failure is not None:
+        raise ReasonerError(f"rlmstudio {requested_mode} run failed: {failure}")
+    raw_answer = _field(result, _ANSWER_FIELDS)
+    answer = str(raw_answer) if raw_answer is not None else str(result)
+    raw_mode = _field(result, _MODE_FIELDS)
+    mode = str(raw_mode) if raw_mode else requested_mode
+    raw_citations = _field(result, _CITATION_FIELDS)
+    citations = [str(c) for c in raw_citations] if isinstance(raw_citations, list | tuple) else []
+    raw_usage = _field(result, _USAGE_FIELDS)
+    usage = _usage(raw_usage) if raw_usage is not None else {}
+    return Answer(answer=answer, mode=mode, citations=citations, usage=usage)
+
+
+class RLMStudioReasoner:
+    """``ReasonerPort`` delegating to ``rlmstudio.interact(content, question, mode=mode)``."""
+
+    def __init__(self) -> None:
+        try:
+            import rlmstudio  # type: ignore[import-not-found]  # optional, never a dependency (FR-016)
+        except ImportError as exc:
+            raise ReasonerUnavailable(f"rlmstudio is not importable ({exc}); {INSTALL_HINT}") from exc
+        self._rlmstudio: Any = rlmstudio
+
+    def ask(self, content: str, question: str, mode: str) -> Answer:
+        """Pass ``mode`` through unchanged (direct | rag | rlm | auto); report the mode actually used.
+
+        Raises :class:`ReasonerError` when RLM Studio reports a failed run or
+        when the call itself raises (provider down, timeout, budget exceeded):
+        callers degrade on that one exception type instead of on whatever the
+        provider stack throws.
+        """
+        try:
+            result = self._rlmstudio.interact(content, question, mode=mode)
+        except ReasonerError:
+            raise
+        except Exception as exc:
+            raise ReasonerError(f"rlmstudio {mode} run raised {type(exc).__name__}: {exc}") from exc
+        return to_answer(result, mode)

{memkernel-0.2.0 → memkernel-0.3.0}/src/memkernel/config.py

@@ -34,6 +34,12 @@ class MemKernelSettings(BaseSettings):
     chunk_size: int = 1500  # max chars per chunk
     chunk_overlap: int = 200  # char overlap between token chunks (fallback only)
 
+    # Documents and ask() (ADR-009)
+    document_store: Literal["local"] = "local"  # where `document` bodies live
+    document_summary_chars: int = 1000  # characters of the body kept in the document's summary record
+    reasoner: Literal["local", "rlmstudio"] = "local"  # ReasonerPort behind ask(); falls back to local
+    ask_top_k: int = 5  # chunks the local reasoner returns per ask()
+
     # Lifecycle / decay (see docs/architecture/memory-tiers.md)
     episode_ttl_days: int = 30  # base TTL for short-term `episode` memories
     observation_ttl_days: int = 7  # base TTL for short-term `observation` memories

{memkernel-0.2.0 → memkernel-0.3.0}/src/memkernel/core/models.py

@@ -42,6 +42,8 @@ class MemoryType(StrEnum):
     CODE = "code"
     # Long-term, procedural
     PROCEDURE = "procedure"
+    # Long-term, a large artefact whose body lives in the document store (ADR-009)
+    DOCUMENT = "document"
     # Short-term, episodic
     EPISODE = "episode"
     OBSERVATION = "observation"
@@ -55,6 +57,7 @@ TYPE_TIER: dict[MemoryType, MemoryTier] = {
     MemoryType.FACT: MemoryTier.LONG_TERM,
     MemoryType.CODE: MemoryTier.LONG_TERM,
     MemoryType.PROCEDURE: MemoryTier.LONG_TERM,
+    MemoryType.DOCUMENT: MemoryTier.LONG_TERM,
     MemoryType.EPISODE: MemoryTier.SHORT_TERM,
     MemoryType.OBSERVATION: MemoryTier.SHORT_TERM,
     MemoryType.TASK: MemoryTier.WORKING,
@@ -69,12 +72,30 @@ DEFAULT_IMPORTANCE: dict[MemoryType, float] = {
     MemoryType.PROCEDURE: 0.8,
     MemoryType.FACT: 0.7,
     MemoryType.CODE: 0.6,
+    MemoryType.DOCUMENT: 0.6,
     MemoryType.TASK: 0.5,
     MemoryType.EPISODE: 0.4,
     MemoryType.OBSERVATION: 0.3,
 }
 
 
+# Tags that mark the records a document store keeps beside the memkernel summary
+# record: the canonical body and its indexed chunks. Both are hidden from
+# top-level recall / list results; a chunk hit resolves to the document summary.
+DOCUMENT_BODY_TAG = "document_body"
+DOCUMENT_CHUNK_TAG = "document_chunk"
+# Boolean metadata key set on body/chunk records so search can exempt them
+# from owner-specific filters (session, type, tier, checkpoint): one body may
+# be owned by several summaries in different sessions, so the filters are
+# applied to the owning summary a hit resolves to, not to the shared part.
+DOCUMENT_PART_KEY = "document_part"
+
+# Reasoning modes accepted by ask(): RLM Studio's Direct / RAG / RLM / auto
+# selection, passed through to the ReasonerPort unchanged (ADR-009).
+ASK_MODES: tuple[str, ...] = ("direct", "rag", "rlm", "auto")
+DEFAULT_ASK_MODE = "auto"
+
+
 def tier_of(memory_type: MemoryType) -> MemoryTier:
     """Return the tier a memory type belongs to."""
     return TYPE_TIER[memory_type]
@@ -103,6 +124,22 @@ class Memory(BaseModel):
     tags: list[str] = []
     supersedes: str | None = None
     promoted_from: str | None = None
+    # Set on a ``document`` summary record and on the body/chunk records the
+    # document store keeps for it (ADR-009).
+    document_id: str | None = None
+
+
+class ReasonerError(RuntimeError):
+    """A ReasonerPort implementation ran but did not produce an answer (provider error, timeout, budget)."""
+
+
+class Answer(BaseModel):
+    """Result of ``ask``: the answer text, the reasoning mode actually used, and provenance."""
+
+    answer: str
+    mode: str
+    citations: list[str] = []
+    usage: dict[str, float | int] = {}
 
 
 class ArtifactRef(BaseModel):

{memkernel-0.2.0 → memkernel-0.3.0}/src/memkernel/core/ports.py

@@ -11,7 +11,7 @@ from __future__ import annotations
 from typing import TYPE_CHECKING, Any, Protocol, runtime_checkable
 
 if TYPE_CHECKING:
-    from memkernel.core.models import Memory, SearchResult
+    from memkernel.core.models import Answer, Memory, SearchResult
 
 
 @runtime_checkable
@@ -47,7 +47,9 @@ class VectorStorePort(Protocol):
         """Fetch-and-merge: only overwrites caller-specified keys."""
         ...
 
-    def count(self) -> int: ...
+    def count(self, where: dict[str, Any] | None = None) -> int:
+        """Number of records, optionally only those matching a metadata filter (ids-only fetch)."""
+        ...
 
     def list_collections(self) -> list[dict[str, Any]]:
         """List all collections visible to this adapter with their record counts."""
@@ -93,3 +95,35 @@ class ChunkerPort(Protocol):
     """Code/text chunking strategy (stub for future US3)."""
 
     def chunk(self, content: str, metadata: dict[str, Any]) -> list[str]: ...
+
+
+@runtime_checkable
+class DocumentStorePort(Protocol):
+    """Document bodies: stored whole, chunk-indexed, searchable per document (ADR-009)."""
+
+    def put(self, content: str, name: str) -> str:
+        """Store a document body and return its doc_id (idempotent for identical content)."""
+        ...
+
+    def get(self, doc_id: str) -> str | None:
+        """Return the stored body, or None if ``doc_id`` is unknown."""
+        ...
+
+    def index(self, doc_id: str, chunks: list[str]) -> int:
+        """Index the chunks of a stored document; returns the number of chunks indexed."""
+        ...
+
+    def indexed(self, doc_id: str) -> int:
+        """Number of chunks currently indexed for ``doc_id`` (0 for an unknown or un-indexed body); ids-only."""
+        ...
+
+    def search(self, doc_id: str, query: str, top_k: int) -> list[str]:
+        """Return the top_k chunk texts of one document ranked against ``query``."""
+        ...
+
+
+@runtime_checkable
+class ReasonerPort(Protocol):
+    """Answers a question about a body of text in a given mode (direct | rag | rlm | auto)."""
+
+    def ask(self, content: str, question: str, mode: str) -> Answer: ...