keepm 0.2.0__tar.gz → 0.3.0__tar.gz

{keepm-0.2.0/src/keepm.egg-info → keepm-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: keepm
-Version: 0.2.0
+Version: 0.3.0
 Summary: Local Markdown memory MCP for coding agents
 Requires-Python: >=3.11
 License-File: LICENSE

keepm-0.3.0/README.md

@@ -0,0 +1,184 @@
+# KeepM
+
+KeepM is a local Markdown memory MCP for coding agents. It stores durable memories in one Obsidian Vault, keeps Markdown as the source of truth, and uses a rebuildable SQLite FTS5 index for bounded context, search, WikiLinks, and backlinks.
+
+[中文文档](README-zh.md)
+
+KeepM is local-only: no embeddings, vector database, cloud service, HTTP server, required Obsidian plugin, or local model. Obsidian may be closed while KeepM runs.
+
+## Install and setup
+
+Requirements:
+
+- Python 3.11+
+- [uv](https://docs.astral.sh/uv/) for the recommended `uvx` flow
+- One local Obsidian Vault reachable from the machine running KeepM
+
+From WSL, a Windows Vault is normally addressed through its mount, for example:
+
+```text
+/mnt/c/Users/alice/Documents/ObsidianVault
+```
+
+Install and configure one Vault:
+
+```bash
+uvx keepm setup
+```
+
+The colored setup wizard asks for the Vault, default memory language (`zh` or `en`), one Agent (Codex or Claude Code), and either user or project scope. Use arrow keys and Enter to select options. Non-interactive setup is available with:
+
+```bash
+keepm setup --yes --agent codex --scope project
+```
+
+Add `--audit` or `--no-audit` to set the optional local audit preference without editing the config file.
+
+The Agent registration scope determines where setup installs the managed instruction entry:
+
+| Agent | User scope | Project scope |
+| --- | --- | --- |
+| Codex | `~/.codex/AGENTS.md` | `<project>/AGENTS.md` |
+| Claude Code | `~/.claude/CLAUDE.md` | `<project>/CLAUDE.md` |
+
+Setup also installs the detailed policy referenced by that entry:
+
+```text
+User scope:    ~/.config/keepm/policies/keepm-memory-policy.md
+Project scope: <project>/.keepm/keepm-memory-policy.md
+```
+
+KeepM manages only content between `<!-- keepm:start -->` and `<!-- keepm:end -->`. Re-running setup replaces that block without changing surrounding user instructions. Broken, reversed, or duplicated markers are rejected rather than overwritten.
+
+The single-Vault configuration is stored at `~/.config/keepm/config.toml`; it is not a Codex or Claude configuration file. A typical file is:
+
+```toml
+vault_path = "/mnt/c/Users/alice/Documents/ObsidianVault"
+memory_dir = "Memory"
+stale_after_seconds = 30
+language = "zh"
+audit_enabled = false
+```
+
+Run `keepm` without a subcommand only as the MCP stdio server. Vault configuration is deliberately performed by `keepm setup`, not by a normal Agent MCP tool.
+
+## Storage and V2 format
+
+```text
+<Vault>/Memory/
+├── global/
+├── projects/
+│   └── <project-id>/
+└── .trash/
+
+~/.local/share/keepm/
+├── indexes/<vault-id>.sqlite3
+└── audit/<vault-id>.jsonl
+```
+
+- `global/` contains cross-project preferences, workflows, feedback, and references.
+- `projects/<project-id>/` contains repository-specific knowledge.
+- `.trash/` holds soft-deleted memories and is never indexed.
+- SQLite is derived data. It can be removed and rebuilt from Markdown without losing memory.
+
+The current project ID comes from `.keepm.toml`, Git origin, or the workspace directory name. A project can make the mapping stable with:
+
+```toml
+# <repository>/.keepm.toml
+project = "keepm"
+```
+
+A V2 memory has explicit range and content semantics:
+
+```markdown
+---
+id: 00000000-0000-0000-0000-000000000001
+name: markdown-source-of-truth
+description: Markdown is the canonical KeepM memory record
+scope: project
+project_id: keepm
+kind: constraint
+language: en
+tags: [storage, obsidian]
+pinned: true
+created: 2026-07-18
+updated: 2026-07-18
+---
+
+Why: Users must be able to inspect, edit, back up, and recover every memory in Obsidian.
+
+How to apply: Store durable facts in Markdown first. Treat SQLite only as a rebuildable index.
+
+## Relations
+
+- depends_on [[local-first]]
+```
+
+`scope` is `global` or `project`; a project memory must include `project_id`, while a global memory must not. New documents also record `language: zh|en`. KeepM preserves non-KeepM frontmatter added manually in Obsidian when an Agent updates the note.
+
+| kind | Typical use | Required non-empty labels |
+| --- | --- | --- |
+| `preference` | User preference | `How to apply:` |
+| `workflow` | Repeatable workflow | `How to apply:` |
+| `feedback` | Corrective feedback | `Why:`, `How to apply:` |
+| `overview` | Project purpose and boundaries | None |
+| `capability` | Stable completed capability | None |
+| `decision` | Technical or product decision | `Why:`, `How to apply:` |
+| `constraint` | Invariant or prohibition | `Why:`, `How to apply:` |
+| `fix` | Reusable diagnosis and repair | `Why:`, `How to apply:` |
+| `reference` | Stable terminology or external material | None |
+
+V1 documents using `type: user|feedback|project|reference` are intentionally not compatible with V2 and are left untouched but excluded from the index. Rewrite them as V2 Markdown if they should remain active memories.
+
+## MCP tools
+
+| Tool | Purpose |
+| --- | --- |
+| `memory_status` | Report configuration, active project, watcher, index, dirty state, and audit status |
+| `memory_context` | Return bounded pinned and query-relevant global/project memories |
+| `memory_search` | Search the SQLite FTS5 index |
+| `memory_read` | Read one current Markdown memory, checksum, and an update token when output is complete |
+| `memory_create` | Create a new memory; it never overwrites an existing logical name |
+| `memory_update` | Replace mutable fields after a full read, checksum check, and update token |
+| `memory_delete` | Soft-delete a full-read, checksum-verified memory into `.trash` |
+| `memory_links` | Return forward WikiLinks and backlinks |
+| `memory_doctor` | Diagnose Markdown/index problems and return a short-lived full-rebuild token |
+| `memory_sync` | Run quick reconciliation or an explicitly authorized full SQLite rebuild |
+
+Creating a memory needs no checksum. Updating or deleting one requires a complete `memory_read`, then its `expected_checksum` and one-time, ten-minute `update_token`. A truncated read intentionally has no update token. If the user edited the note in Obsidian after the read, the operation is rejected instead of overwritten.
+
+The setup policy tells Agents when to use these tools. The Agent decides whether a verified result has durable value; the server enforces document format, version protection, body size, duplicate prevention, and sensitive-content protection.
+
+## Synchronization, safety, and recovery
+
+KeepM reconciles the Vault at MCP startup. It compares path, mtime, and size first, hashes only changed files, and updates only the affected index rows. While the MCP process runs, `watchfiles` provides lower-latency external-edit handling; startup and stale reconciliation remain the correctness fallback, including Windows-to-WSL event loss.
+
+`memory_create`, `memory_update`, and `memory_delete` immediately reconcile their affected paths. Do not call `memory_sync` as routine task cleanup.
+
+For external bulk edits or suspected inconsistency:
+
+1. Call `memory_status` and `memory_doctor`.
+2. Use `memory_sync(full=false)` for a quick reconcile if needed.
+3. Use `memory_sync(full=true, confirm=true, repair_token=...)` only with the recent token from `memory_doctor` to rebuild SQLite.
+
+The full rebuild never rewrites or deletes Markdown.
+
+KeepM limits a memory body to 16 KiB and rejects high-confidence private keys, vendor tokens, and AWS credentials passed through MCP writes. It does not use fuzzy raw-log detection. A file edited directly in Obsidian that is invalid, oversized, or sensitive remains on disk but is excluded from the index and reported by `memory_doctor`.
+
+## Optional audit log
+
+Auditing is disabled by default. Set `audit_enabled = true` in KeepM configuration to enable a local, rotating JSONL log under `~/.local/share/keepm/audit/`.
+
+The log is for the local owner, not normal Agent context. It records only operation metadata, IDs, scope/kind, checksums, result, and stable error codes—never body text, titles, descriptions, tags, raw exceptions, or secrets.
+
+```bash
+keepm audit --limit 50
+```
+
+## Development
+
+```bash
+uv sync --extra dev
+uv run pytest -q
+uv run keepm setup
+```

{keepm-0.2.0 → keepm-0.3.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "keepm"
-version = "0.2.0"
+version = "0.3.0"
 description = "Local Markdown memory MCP for coding agents"
 requires-python = ">=3.11"
 dependencies = [

keepm-0.3.0/src/keepm/__init__.py

@@ -0,0 +1 @@
+__version__ = "0.3.0"

keepm-0.3.0/src/keepm/audit.py

@@ -0,0 +1,140 @@
+from __future__ import annotations
+
+import datetime as dt
+import argparse
+import json
+import os
+from collections.abc import Mapping
+from pathlib import Path
+from typing import Any
+
+from keepm.config import ConfigStore, default_data_root, vault_id
+
+
+MAX_AUDIT_BYTES = 10 * 1024 * 1024
+MAX_AUDIT_HISTORY = 5
+SAFE_FIELDS = frozenset(
+    {
+        "time",
+        "operation",
+        "result",
+        "memory_id",
+        "scope",
+        "project_id",
+        "kind",
+        "before_checksum",
+        "after_checksum",
+        "code",
+    }
+)
+
+
+class AuditLogger:
+    """Best-effort, metadata-only audit sink for the local Vault owner."""
+
+    def __init__(self, path: Path, *, enabled: bool) -> None:
+        self.path = path.expanduser().resolve()
+        self.enabled = enabled
+
+    def record(
+        self,
+        *,
+        operation: str,
+        result: str,
+        memory_id: str | None,
+        scope: str | None,
+        project_id: str | None,
+        kind: str | None,
+        before_checksum: str | None,
+        after_checksum: str | None,
+        code: str | None,
+    ) -> None:
+        if not self.enabled:
+            return
+        event = {
+            "time": dt.datetime.now(dt.timezone.utc).isoformat(),
+            "operation": operation,
+            "result": result,
+            "memory_id": memory_id,
+            "scope": scope,
+            "project_id": project_id,
+            "kind": kind,
+            "before_checksum": before_checksum,
+            "after_checksum": after_checksum,
+            "code": code,
+        }
+        try:
+            payload = (json.dumps(event, ensure_ascii=False, separators=(",", ":")) + "\n").encode(
+                "utf-8"
+            )
+            self.path.parent.mkdir(parents=True, exist_ok=True)
+            if self.path.exists() and self.path.stat().st_size + len(payload) > MAX_AUDIT_BYTES:
+                self._rotate()
+            with self.path.open("ab") as handle:
+                handle.write(payload)
+                handle.flush()
+        except OSError:
+            # Auditing must never make an otherwise valid memory operation fail.
+            return
+
+    def _rotate(self) -> None:
+        try:
+            oldest = self.path.with_name(f"{self.path.name}.{MAX_AUDIT_HISTORY}")
+            oldest.unlink(missing_ok=True)
+            for index in range(MAX_AUDIT_HISTORY - 1, 0, -1):
+                source = self.path.with_name(f"{self.path.name}.{index}")
+                if source.exists():
+                    os.replace(source, self.path.with_name(f"{self.path.name}.{index + 1}"))
+            if self.path.exists():
+                os.replace(self.path, self.path.with_name(f"{self.path.name}.1"))
+        except OSError:
+            # A later append can still succeed if a platform disallows rotation.
+            return
+
+
+def read_events(path: Path, *, limit: int = 100) -> tuple[dict[str, Any], ...]:
+    """Read a bounded, schema-filtered audit history without exposing arbitrary JSON."""
+    if limit < 1:
+        return ()
+    events: list[dict[str, Any]] = []
+    paths = [
+        path.with_name(f"{path.name}.{index}")
+        for index in range(MAX_AUDIT_HISTORY, 0, -1)
+    ] + [path]
+    for candidate in paths:
+        if not candidate.is_file():
+            continue
+        try:
+            lines = candidate.read_text(encoding="utf-8").splitlines()
+        except OSError:
+            continue
+        for line in lines:
+            try:
+                decoded = json.loads(line)
+            except json.JSONDecodeError:
+                continue
+            if not isinstance(decoded, Mapping):
+                continue
+            events.append(
+                {key: decoded.get(key) for key in SAFE_FIELDS if key in decoded}
+            )
+    return tuple(events[-limit:])
+
+
+def main(argv: list[str] | None = None) -> None:
+    parser = argparse.ArgumentParser(
+        prog="keepm audit", description="Show local KeepM audit metadata for the Vault owner."
+    )
+    parser.add_argument("--limit", type=int, default=100, help="maximum recent events")
+    arguments = parser.parse_args(argv)
+    if arguments.limit < 1:
+        parser.error("--limit must be positive")
+    try:
+        config = ConfigStore().load()
+    except FileNotFoundError as error:
+        parser.error(str(error))
+    if not config.audit_enabled:
+        parser.error("KeepM audit is disabled; set audit_enabled = true in KeepM config")
+    path = default_data_root() / "audit" / f"{vault_id(config)}.jsonl"
+    for event in read_events(path, limit=arguments.limit):
+        print(json.dumps(event, ensure_ascii=False, sort_keys=True))

{keepm-0.2.0 → keepm-0.3.0}/src/keepm/cli.py

@@ -12,6 +12,11 @@ def main(argv: Sequence[str] | None = None) -> None:
 
         setup_main(arguments[1:])
         return
+    if arguments[:1] == ["audit"]:
+        from keepm.audit import main as audit_main
+
+        audit_main(arguments[1:])
+        return
 
     from keepm.server import main as server_main
 

{keepm-0.2.0 → keepm-0.3.0}/src/keepm/config.py

@@ -2,6 +2,7 @@ from __future__ import annotations
 
 import json
 import os
+import hashlib
 import tomllib
 from dataclasses import dataclass
 from pathlib import Path
@@ -21,12 +22,19 @@ def default_data_root() -> Path:
     return root / "keepm"
 
 
+def vault_id(config: "KeepMConfig") -> str:
+    """Return a stable, non-secret identifier for one configured memory root."""
+    identity = f"{config.vault_path}\0{config.memory_root.resolve()}"
+    return hashlib.sha256(identity.encode("utf-8")).hexdigest()[:24]
+
+
 @dataclass(frozen=True, slots=True)
 class KeepMConfig:
     vault_path: Path
     memory_dir: str = "Memory"
     stale_after_seconds: int = 30
     language: MemoryLanguage = "en"
+    audit_enabled: bool = False
 
     def __post_init__(self) -> None:
         memory_path = Path(self.memory_dir)
@@ -40,6 +48,8 @@ class KeepMConfig:
             raise ValueError("stale_after_seconds must be non-negative")
         if self.language not in ("zh", "en"):
             raise ValueError("language must be 'zh' or 'en'")
+        if not isinstance(self.audit_enabled, bool):
+            raise ValueError("audit_enabled must be true or false")
         object.__setattr__(self, "vault_path", self.vault_path.expanduser().resolve())
 
     @property
@@ -67,11 +77,15 @@ class ConfigStore:
         if not self.path.is_file():
             raise FileNotFoundError(f"KeepM is not configured: {self.path}")
         data = tomllib.loads(self.path.read_text(encoding="utf-8"))
+        audit_enabled = data.get("audit_enabled", False)
+        if not isinstance(audit_enabled, bool):
+            raise ValueError("audit_enabled must be true or false")
         return KeepMConfig(
             vault_path=Path(data["vault_path"]),
             memory_dir=str(data.get("memory_dir", "Memory")),
             stale_after_seconds=int(data.get("stale_after_seconds", 30)),
             language=cast(MemoryLanguage, str(data.get("language", "en"))),
+            audit_enabled=audit_enabled,
         )
 
     def load_optional(self) -> KeepMConfig | None:
@@ -90,6 +104,7 @@ class ConfigStore:
             f"memory_dir = {json.dumps(config.memory_dir)}\n"
             f"stale_after_seconds = {config.stale_after_seconds}\n"
             f"language = {json.dumps(config.language)}\n"
+            f"audit_enabled = {str(config.audit_enabled).lower()}\n"
         )
         temporary = self.path.with_suffix(".tmp")
         temporary.write_text(content, encoding="utf-8", newline="\n")