memorymaster 3.13.0__tar.gz → 3.15.0__tar.gz

{memorymaster-3.13.0/memorymaster.egg-info → memorymaster-3.15.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: memorymaster
-Version: 3.13.0
+Version: 3.15.0
 Summary: Production-grade memory reliability system for AI coding agents. Lifecycle-managed claims with citations, conflict detection, steward governance, and MCP integration.
 Author: wolverin0
 License: MIT
@@ -17,6 +17,8 @@ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
 Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE
+Requires-Dist: requests>=2.31
+Requires-Dist: tenacity>=8.2
 Provides-Extra: postgres
 Requires-Dist: psycopg[binary]>=3.2; extra == "postgres"
 Provides-Extra: security
@@ -61,29 +63,23 @@ MemoryMaster prevents the #1 problem with agent memory: **drift, stale assumptio
 
 ## Architecture
 
+MemoryMaster is layered around MCP/CLI entry points, the `MemoryService` facade, SQLite/Postgres
+storage, optional Qdrant vector search, scheduled jobs, and the Obsidian wiki/vault layer. The
+canonical ingest path is:
+
+```text
+MCP/CLI -> sensitivity filter -> MemoryService.ingest -> store write -> FTS5 index
 ```
-┌─────────────────────────────────────────────────────────────────┐
-│                        Agent Runtime                            │
-│  (Claude Code / Codex / any MCP-compatible agent)               │
-└────────────┬────────────────────────────────┬───────────────────┘
-             │ MCP (24 tools)                 │ CLI (86 commands)
-             v                                v
-┌─────────────────────────────────────────────────────────────────┐
-│                      MemoryMaster Core                          │
-│  ┌──────────┐  ┌───────────┐  ┌──────────┐  ┌───────────────┐  │
-│  │ Ingestor │  │ Extractor │  │ Validator │  │ State Engine  │  │
-│  │ (events) │->│ (claims)  │->│ (probes)  │->│ (6-state FSM) │  │
-│  └──────────┘  └───────────┘  └──────────┘  └───────────────┘  │
-│  ┌──────────┐  ┌───────────┐  ┌──────────┐  ┌───────────────┐  │
-│  │ Retrieval│  │ Compactor │  │ Steward  │  │  Dashboard    │  │
-│  │ (hybrid) │  │ (archive) │  │ (govern) │  │  (HTML+SSE)   │  │
-│  └──────────┘  └───────────┘  └──────────┘  └───────────────┘  │
-└────────┬──────────────┬──────────────┬──────────┬───────────────┘
-         v              v              v          v
-   SQLite/Postgres   Qdrant      Ollama/CLI    Claude Code
-                    (vectors)   (LLM stack)   Auto Dream + Vault
+
+The query path is:
+
+```text
+query_memory -> MemoryService.query -> storage reads + optional Qdrant candidates -> ranked context
 ```
 
+See [docs/architecture.md](docs/architecture.md) for the current module map, data-flow details,
+recent PR status, and sensitivity-filter invariants.
+
 ## Key features
 
 - **6-state lifecycle**: `candidate` → `confirmed` → `stale` → `superseded` → `conflicted` → `archived`
@@ -104,18 +100,20 @@ Full feature index lives in [`docs/handbook.md`](docs/handbook.md).
 
 ## Prerequisites
 
-**Required**
+**Required (the package won't function without these)**
 
 - Python **3.10+** with `pip`
 - Claude Code, Codex, or any MCP-compatible agent
+- **An LLM provider** — pick one: Claude Code OAuth (free if you're a subscriber, set `MEMORYMASTER_LLM_PROVIDER=claude_cli`), a free Gemini API key from [aistudio.google.com](https://aistudio.google.com), OpenAI, Anthropic API, or local Ollama. The steward, auto-ingest, and wiki-absorb cycles all need an LLM — without one, claims pile up as `candidate` and never get validated, deduped, or compiled into the wiki.
+
+**Strongly recommended (you'll lose ~80% of the value without these)**
+
+- **Node.js 18+** for [graphify](https://github.com/wolverin0/graphify) and [GitNexus](https://github.com/wolverin0/gitnexus) — these are the cached intelligence layers that make MemoryMaster cheap to query. Without them, every "what does this codebase do?" question burns tokens cold-exploring files the graph already mapped. The `intelligence-first` workflow in `CLAUDE.md` assumes both are installed.
+- **Obsidian 1.6+** with the [Bases](https://help.obsidian.md/Plugins/Bases) core plugin — the wiki engine writes plain Markdown so any editor works, but Obsidian's backlinks, graph view, and Bases dashboards are how you actually navigate `wiki-absorb` output. Without Obsidian, the wiki is just a folder of files.
 
-**Optional**
+**Optional (nice to have)**
 
-- **Already a Claude Code subscriber?** No API key needed — set `MEMORYMASTER_LLM_PROVIDER=claude_cli` and the steward + auto-ingest hooks will use your existing OAuth via the local `claude --print` binary
-- A free Gemini API key from [aistudio.google.com](https://aistudio.google.com) — powers the auto-ingest hook at ~zero cost
-- **Node.js 18+** for graphify and GitNexus
-- **Obsidian 1.6+** with the Bases core plugin (for visual wiki browsing)
-- **Docker** for Qdrant (SQLite FTS5 is the default and works out of the box)
+- **Docker** for Qdrant — vector retrieval. SQLite FTS5 is the default and works out of the box; add Qdrant when you want semantic recall on top of keyword search.
 
 ## Quick start
 

{memorymaster-3.13.0 → memorymaster-3.15.0}/README.md

@@ -17,29 +17,23 @@ MemoryMaster prevents the #1 problem with agent memory: **drift, stale assumptio
 
 ## Architecture
 
+MemoryMaster is layered around MCP/CLI entry points, the `MemoryService` facade, SQLite/Postgres
+storage, optional Qdrant vector search, scheduled jobs, and the Obsidian wiki/vault layer. The
+canonical ingest path is:
+
+```text
+MCP/CLI -> sensitivity filter -> MemoryService.ingest -> store write -> FTS5 index
 ```
-┌─────────────────────────────────────────────────────────────────┐
-│                        Agent Runtime                            │
-│  (Claude Code / Codex / any MCP-compatible agent)               │
-└────────────┬────────────────────────────────┬───────────────────┘
-             │ MCP (24 tools)                 │ CLI (86 commands)
-             v                                v
-┌─────────────────────────────────────────────────────────────────┐
-│                      MemoryMaster Core                          │
-│  ┌──────────┐  ┌───────────┐  ┌──────────┐  ┌───────────────┐  │
-│  │ Ingestor │  │ Extractor │  │ Validator │  │ State Engine  │  │
-│  │ (events) │->│ (claims)  │->│ (probes)  │->│ (6-state FSM) │  │
-│  └──────────┘  └───────────┘  └──────────┘  └───────────────┘  │
-│  ┌──────────┐  ┌───────────┐  ┌──────────┐  ┌───────────────┐  │
-│  │ Retrieval│  │ Compactor │  │ Steward  │  │  Dashboard    │  │
-│  │ (hybrid) │  │ (archive) │  │ (govern) │  │  (HTML+SSE)   │  │
-│  └──────────┘  └───────────┘  └──────────┘  └───────────────┘  │
-└────────┬──────────────┬──────────────┬──────────┬───────────────┘
-         v              v              v          v
-   SQLite/Postgres   Qdrant      Ollama/CLI    Claude Code
-                    (vectors)   (LLM stack)   Auto Dream + Vault
+
+The query path is:
+
+```text
+query_memory -> MemoryService.query -> storage reads + optional Qdrant candidates -> ranked context
 ```
 
+See [docs/architecture.md](docs/architecture.md) for the current module map, data-flow details,
+recent PR status, and sensitivity-filter invariants.
+
 ## Key features
 
 - **6-state lifecycle**: `candidate` → `confirmed` → `stale` → `superseded` → `conflicted` → `archived`
@@ -60,18 +54,20 @@ Full feature index lives in [`docs/handbook.md`](docs/handbook.md).
 
 ## Prerequisites
 
-**Required**
+**Required (the package won't function without these)**
 
 - Python **3.10+** with `pip`
 - Claude Code, Codex, or any MCP-compatible agent
+- **An LLM provider** — pick one: Claude Code OAuth (free if you're a subscriber, set `MEMORYMASTER_LLM_PROVIDER=claude_cli`), a free Gemini API key from [aistudio.google.com](https://aistudio.google.com), OpenAI, Anthropic API, or local Ollama. The steward, auto-ingest, and wiki-absorb cycles all need an LLM — without one, claims pile up as `candidate` and never get validated, deduped, or compiled into the wiki.
+
+**Strongly recommended (you'll lose ~80% of the value without these)**
+
+- **Node.js 18+** for [graphify](https://github.com/wolverin0/graphify) and [GitNexus](https://github.com/wolverin0/gitnexus) — these are the cached intelligence layers that make MemoryMaster cheap to query. Without them, every "what does this codebase do?" question burns tokens cold-exploring files the graph already mapped. The `intelligence-first` workflow in `CLAUDE.md` assumes both are installed.
+- **Obsidian 1.6+** with the [Bases](https://help.obsidian.md/Plugins/Bases) core plugin — the wiki engine writes plain Markdown so any editor works, but Obsidian's backlinks, graph view, and Bases dashboards are how you actually navigate `wiki-absorb` output. Without Obsidian, the wiki is just a folder of files.
 
-**Optional**
+**Optional (nice to have)**
 
-- **Already a Claude Code subscriber?** No API key needed — set `MEMORYMASTER_LLM_PROVIDER=claude_cli` and the steward + auto-ingest hooks will use your existing OAuth via the local `claude --print` binary
-- A free Gemini API key from [aistudio.google.com](https://aistudio.google.com) — powers the auto-ingest hook at ~zero cost
-- **Node.js 18+** for graphify and GitNexus
-- **Obsidian 1.6+** with the Bases core plugin (for visual wiki browsing)
-- **Docker** for Qdrant (SQLite FTS5 is the default and works out of the box)
+- **Docker** for Qdrant — vector retrieval. SQLite FTS5 is the default and works out of the box; add Qdrant when you want semantic recall on top of keyword search.
 
 ## Quick start
 

{memorymaster-3.13.0 → memorymaster-3.15.0}/memorymaster/_storage_lifecycle.py

@@ -10,6 +10,7 @@ import json
 import logging
 import sqlite3
 from datetime import datetime, timedelta, timezone
+from typing import TYPE_CHECKING, Any
 
 from memorymaster.embeddings import EmbeddingProvider, cosine_similarity
 from memorymaster.models import (
@@ -23,16 +24,37 @@ from memorymaster.models import (
     validate_transition_event_type,
 )
 
-logger = logging.getLogger(__name__)
-
 from memorymaster._storage_shared import (
     EVENT_HASH_ALGO,
     ConcurrentModificationError,
     utc_now,
 )
 
+logger = logging.getLogger(__name__)
+
 
 class _LifecycleMixin:
+    if TYPE_CHECKING:
+        def connect(self) -> sqlite3.Connection: ...
+
+        def get_claim(self, claim_id: int, include_citations: bool = True) -> Claim | None: ...
+
+        def _ensure_event_integrity_schema(self, conn: sqlite3.Connection) -> None: ...
+
+        def _ensure_embeddings_schema(self, conn: sqlite3.Connection) -> None: ...
+
+        def _insert_event_row(
+            self,
+            conn: sqlite3.Connection,
+            *,
+            claim_id: int | None,
+            event_type: str,
+            from_status: str | None,
+            to_status: str | None,
+            details: str | None,
+            payload_json: str | None,
+            created_at: str,
+        ) -> int: ...
 
     def apply_status_transition(
         self,
@@ -102,14 +124,64 @@ class _LifecycleMixin:
         old_claim = self.get_claim(old_claim_id, include_citations=False)
         if old_claim is None:
             return
-        self.apply_status_transition(
-            old_claim,
-            to_status="superseded",
-            reason=reason,
-            event_type="supersession",
-            replaced_by_claim_id=new_claim_id,
-        )
-        self.set_supersedes(new_claim_id, old_claim_id)
+        if old_claim.status == "superseded" or old_claim.replaced_by_claim_id is not None:
+            raise ConcurrentModificationError(
+                f"Claim {old_claim_id} was already superseded. Reload and retry."
+            )
+
+        now = utc_now()
+        with self.connect() as conn:
+            target = conn.execute(
+                "SELECT supersedes_claim_id FROM claims WHERE id = ?",
+                (new_claim_id,),
+            ).fetchone()
+            if target is None:
+                raise ValueError(f"Replacement claim {new_claim_id} does not exist.")
+            if target["supersedes_claim_id"] not in {None, old_claim_id}:
+                raise ConcurrentModificationError(
+                    f"Claim {new_claim_id} already supersedes another claim. Reload and retry."
+                )
+
+            cur = conn.execute(
+                """
+                UPDATE claims
+                SET status = 'superseded', updated_at = ?, replaced_by_claim_id = ?,
+                    version = version + 1, valid_until = COALESCE(?, valid_until)
+                WHERE id = ? AND version = ? AND status != 'superseded'
+                    AND replaced_by_claim_id IS NULL
+                """,
+                (now, new_claim_id, now, old_claim_id, old_claim.version),
+            )
+            if cur.rowcount == 0:
+                raise ConcurrentModificationError(
+                    f"Claim {old_claim_id} was modified by another writer. Reload and retry."
+                )
+
+            cur = conn.execute(
+                """
+                UPDATE claims
+                SET supersedes_claim_id = ?, updated_at = ?
+                WHERE id = ? AND (supersedes_claim_id IS NULL OR supersedes_claim_id = ?)
+                """,
+                (old_claim_id, now, new_claim_id, old_claim_id),
+            )
+            if cur.rowcount == 0:
+                conn.rollback()
+                raise ConcurrentModificationError(
+                    f"Claim {new_claim_id} was modified by another writer. Reload and retry."
+                )
+
+            self._insert_event_row(
+                conn,
+                claim_id=old_claim.id,
+                event_type="supersession",
+                from_status=old_claim.status,
+                to_status="superseded",
+                details=reason,
+                payload_json=json.dumps({"replaced_by_claim_id": new_claim_id}),
+                created_at=now,
+            )
+            conn.commit()
 
 
     def delete_old_events(self, retain_days: int) -> int:
@@ -117,8 +189,8 @@ class _LifecycleMixin:
         return 0
 
 
-    def reconcile_integrity(self, *, fix: bool = False, limit: int = 500) -> dict[str, object]:
-        report: dict[str, object] = {
+    def reconcile_integrity(self, *, fix: bool = False, limit: int = 500) -> dict[str, Any]:
+        report: dict[str, Any] = {
             "checked_at": utc_now(),
             "fix_mode": bool(fix),
             "issues": {},

{memorymaster-3.13.0 → memorymaster-3.15.0}/memorymaster/_storage_read.py

@@ -9,6 +9,7 @@ from __future__ import annotations
 import logging
 import sqlite3
 from datetime import datetime, timedelta, timezone
+from typing import TYPE_CHECKING
 
 from memorymaster.models import (
     Citation,
@@ -22,6 +23,10 @@ logger = logging.getLogger(__name__)
 
 
 class _ReadMixin:
+    if TYPE_CHECKING:
+        def connect(self) -> sqlite3.Connection: ...
+
+        def init_db(self) -> None: ...
 
     def _check_idempotency(self, conn: sqlite3.Connection, idempotency_key: str | None) -> Claim | None:
         """Check if a claim with this idempotency key already exists. Returns existing claim or None."""

{memorymaster-3.13.0 → memorymaster-3.15.0}/memorymaster/_storage_schema.py

@@ -338,7 +338,8 @@ class _SchemaMixin:
                 predicate,
                 object_value,
                 content='claims',
-                content_rowid='id'
+                content_rowid='id',
+                tokenize='porter unicode61'
             )
             """
         )

{memorymaster-3.13.0 → memorymaster-3.15.0}/memorymaster/_storage_sources.py

@@ -8,7 +8,7 @@ from __future__ import annotations
 
 import json
 import sqlite3
-from typing import Any
+from typing import TYPE_CHECKING, Any
 
 from memorymaster._storage_shared import utc_now
 from memorymaster.models import (
@@ -51,6 +51,22 @@ def _bounded_confidence(confidence: float | None) -> float | None:
 
 
 class _SourceItemsMixin:
+    if TYPE_CHECKING:
+        def connect(self) -> sqlite3.Connection: ...
+
+        def _insert_event_row(
+            self,
+            conn: sqlite3.Connection,
+            *,
+            claim_id: int | None,
+            event_type: str,
+            from_status: str | None,
+            to_status: str | None,
+            details: str | None,
+            payload_json: str | None,
+            created_at: str,
+        ) -> int: ...
+
     def upsert_external_source(
         self,
         *,

{memorymaster-3.13.0 → memorymaster-3.15.0}/memorymaster/_storage_write_claims.py

@@ -9,21 +9,42 @@ from __future__ import annotations
 import json
 import logging
 import sqlite3
+from typing import TYPE_CHECKING
 
 from memorymaster.models import (
     CitationInput,
     Claim,
     validate_event_payload,
 )
-
-logger = logging.getLogger(__name__)
-
 from memorymaster._storage_shared import (
     utc_now,
 )
 
+logger = logging.getLogger(__name__)
+
 
 class _WriteClaimsMixin:
+    if TYPE_CHECKING:
+        def connect(self) -> sqlite3.Connection: ...
+
+        def _check_idempotency(self, conn: sqlite3.Connection, idempotency_key: str | None) -> Claim | None: ...
+
+        def get_claim(self, claim_id: int, include_citations: bool = True) -> Claim | None: ...
+
+        def _allocate_human_id(self, conn: sqlite3.Connection, subject: str | None, text: str, claim_id: int) -> str: ...
+
+        def _insert_event_row(
+            self,
+            conn: sqlite3.Connection,
+            *,
+            claim_id: int | None,
+            event_type: str,
+            from_status: str | None,
+            to_status: str | None,
+            details: str | None,
+            payload_json: str | None,
+            created_at: str,
+        ) -> int: ...
 
     def create_claim(
         self,

{memorymaster-3.13.0 → memorymaster-3.15.0}/memorymaster/auto_resolver.py

@@ -3,27 +3,25 @@
 When two claims contradict (same subject/predicate, different object_value),
 asks an LLM to evaluate which one has stronger evidence and should be kept.
 
-Uses Ollama or any OpenAI-compatible endpoint. The loser gets superseded,
-not deleted — preserving full audit trail.
+Provider routing goes through `memorymaster.llm_provider.call_llm`, which
+honors `MEMORYMASTER_LLM_PROVIDER` (claude_cli / google / openai / anthropic
+/ ollama) instead of the previous hardcoded Ollama-only path. The loser
+gets superseded, not deleted — preserving full audit trail.
 """
 
 from __future__ import annotations
 
 import json
 import logging
-import os
-import urllib.error
-import urllib.request
 from typing import Any
 
+from memorymaster._storage_shared import ConcurrentModificationError
 from memorymaster.lifecycle import transition_claim
+from memorymaster.llm_provider import call_llm
 from memorymaster.models import Claim
 
 logger = logging.getLogger(__name__)
 
-DEFAULT_OLLAMA_URL = "http://localhost:11434"
-DEFAULT_MODEL = "deepseek-coder-v2:16b"
-
 RESOLUTION_PROMPT = """You are a memory quality evaluator. Two claims contradict each other.
 Decide which one should be KEPT based on:
 1. Recency (newer information is usually more accurate)
@@ -47,49 +45,34 @@ Return JSON only: {{"winner": "A" or "B", "reason": "brief explanation"}}"""
 
 
 def _llm_evaluate(prompt: str, model: str = "", base_url: str = "") -> dict:
-    """Call LLM and parse JSON response.
+    """Ask the configured LLM to pick the winner and parse its JSON response.
 
-    Returns empty dict {} if LLM returns invalid JSON or connection fails.
+    Returns empty dict {} if the LLM returns invalid JSON or the call fails.
+    The `model` and `base_url` kwargs are kept for backwards compatibility but
+    are no longer consulted — provider routing is centralized in llm_provider.
     """
-    url = (base_url or os.environ.get("OLLAMA_URL") or DEFAULT_OLLAMA_URL).rstrip("/")
-    mdl = model or os.environ.get("RESOLVER_LLM_MODEL") or DEFAULT_MODEL
-
-    body = json.dumps({
-        "model": mdl,
-        "messages": [{"role": "user", "content": prompt}],
-        "stream": False,
-        "options": {"temperature": 0.1, "num_predict": 200},
-    }).encode()
-
-    req = urllib.request.Request(
-        f"{url}/api/chat",
-        data=body,
-        headers={"Content-Type": "application/json"},
-        method="POST",
-    )
     try:
-        with urllib.request.urlopen(req, timeout=30) as resp:
-            result = json.loads(resp.read())
-            raw = result.get("message", {}).get("content", "")
-            # Parse JSON from response
-            text = raw.strip()
-            if text.startswith("```"):
-                lines = text.split("\n")
-                lines = [line for line in lines if not line.strip().startswith("```")]
-                text = "\n".join(lines)
-            try:
-                parsed = json.loads(text)
-                if not isinstance(parsed, dict):
-                    logger.warning("LLM returned non-dict JSON: %s", type(parsed))
-                    return {}
-                return parsed
-            except json.JSONDecodeError as exc:
-                logger.warning("LLM returned invalid JSON: %s (text: %s)", exc, text[:100])
-                return {}
+        raw = call_llm(prompt, "")
     except Exception as exc:
         logger.warning("LLM conflict evaluation failed: %s", exc)
         return {}
 
+    text = (raw or "").strip()
+    if text.startswith("```"):
+        lines = [line for line in text.split("\n") if not line.strip().startswith("```")]
+        text = "\n".join(lines)
+
+    try:
+        parsed = json.loads(text)
+    except json.JSONDecodeError as exc:
+        logger.warning("LLM returned invalid JSON: %s (text: %s)", exc, text[:100])
+        return {}
+
+    if not isinstance(parsed, dict):
+        logger.warning("LLM returned non-dict JSON: %s", type(parsed))
+        return {}
+    return parsed
+
 
 def _cite_summary(claim: Claim) -> str:
     """Summarize citations for LLM prompt."""
@@ -132,7 +115,7 @@ def resolve_conflict_pair(
     loser = claim_b if winner_letter == "A" else claim_a
 
     try:
-        transition_claim(
+        updated = transition_claim(
             store,
             claim_id=loser.id,
             to_status="superseded",
@@ -140,12 +123,35 @@ def resolve_conflict_pair(
             event_type="validator",
             replaced_by_claim_id=winner.id,
         )
+        current_replacement_id = getattr(updated, "replaced_by_claim_id", None)
+        if current_replacement_id != winner.id:
+            return {
+                "resolved": False,
+                "reason": "lost_race",
+                "winner_id": winner.id,
+                "loser_id": loser.id,
+                "current_replacement_id": current_replacement_id,
+            }
+        if hasattr(store, "set_supersedes"):
+            store.set_supersedes(winner.id, loser.id)
         return {
             "resolved": True,
             "winner_id": winner.id,
             "loser_id": loser.id,
             "reason": reason,
         }
+    except ConcurrentModificationError as exc:
+        current = store.get_claim(loser.id, include_citations=False)
+        if current is not None and current.status == "superseded":
+            return {
+                "resolved": False,
+                "reason": "lost_race",
+                "winner_id": winner.id,
+                "loser_id": loser.id,
+                "current_replacement_id": current.replaced_by_claim_id,
+            }
+        logger.warning("Failed to resolve conflict %d vs %d: %s", claim_a.id, claim_b.id, exc)
+        return {"resolved": False, "reason": str(exc)}
     except Exception as exc:
         logger.warning("Failed to resolve conflict %d vs %d: %s", claim_a.id, claim_b.id, exc)
         return {"resolved": False, "reason": str(exc)}

{memorymaster-3.13.0 → memorymaster-3.15.0}/memorymaster/claim_verifier.py

@@ -16,7 +16,6 @@ import os
 import re
 import sqlite3
 import subprocess
-from pathlib import Path
 from typing import Any
 
 logger = logging.getLogger(__name__)