memorymaster 3.4.0__tar.gz → 3.5.0__tar.gz

{memorymaster-3.4.0 → memorymaster-3.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: memorymaster
-Version: 3.4.0
+Version: 3.5.0
 Summary: Production-grade memory reliability system for AI coding agents. Lifecycle-managed claims with citations, conflict detection, steward governance, and MCP integration.
 Author: wolverin0
 License: MIT
@@ -27,11 +27,19 @@ Provides-Extra: gemini
 Requires-Dist: google-genai>=1.0; extra == "gemini"
 Provides-Extra: qdrant
 Requires-Dist: httpx>=0.27; extra == "qdrant"
+Provides-Extra: vector
+Requires-Dist: sentence-transformers>=3.0; extra == "vector"
+Requires-Dist: qdrant-client>=1.9; extra == "vector"
+Provides-Extra: graph
+Requires-Dist: kuzu>=0.4; extra == "graph"
 Provides-Extra: dev
 Requires-Dist: pytest>=8.2; extra == "dev"
 Requires-Dist: pytest-cov>=6.0; extra == "dev"
 Provides-Extra: mcp
 Requires-Dist: mcp>=1.2; extra == "mcp"
+Provides-Extra: ml
+Requires-Dist: scikit-learn>=1.3; extra == "ml"
+Requires-Dist: joblib>=1.3; extra == "ml"
 Dynamic: license-file
 
 # MemoryMaster
@@ -143,6 +151,48 @@ MemoryMaster gives AI coding agents **persistent, verifiable memory** with a ful
 - **Obsidian 1.6+** with the **Bases** core plugin — only if you want to browse the wiki visually
 - **Docker** — only if you want Qdrant for hybrid vector search (SQLite FTS5 is the default and works out of the box)
 
+## Setup
+
+A minimal path for new users. Every env var mentioned here is documented in [`.env.example`](.env.example) — copy that file and uncomment the lines you need.
+
+### 1. Minimum viable setup
+
+```bash
+pip install "memorymaster[mcp]"
+python -m memorymaster --db memorymaster.db init-db
+cp .env.example .env
+# Then set ONE of:
+#   GEMINI_API_KEY=...    (free from https://aistudio.google.com)
+#   OPENAI_API_KEY=...
+#   ANTHROPIC_API_KEY=...
+# Or run Ollama locally (no key needed) — see below.
+```
+
+That's enough to use the CLI, the MCP server, and the auto-ingest Stop hook.
+
+### 2. Pick your LLM provider
+
+| Provider | Env vars | Model (default) | Cost |
+|----------|----------|-----------------|------|
+| Google Gemini (default) | `MEMORYMASTER_LLM_PROVIDER=google` + `GEMINI_API_KEY=...` | `gemini-3.1-flash-lite-preview` | ~free |
+| OpenAI | `MEMORYMASTER_LLM_PROVIDER=openai` + `OPENAI_API_KEY=...` | `gpt-4o-mini` | ~$0.001/call |
+| Anthropic | `MEMORYMASTER_LLM_PROVIDER=anthropic` + `ANTHROPIC_API_KEY=...` | `claude-haiku-4-5-20251001` | ~$0.001/call |
+| Ollama (local) | `MEMORYMASTER_LLM_PROVIDER=ollama` + `OLLAMA_URL=http://localhost:11434` | `llama3.2:3b` | free |
+
+For zero-cost offline use, install [Ollama](https://ollama.com), `ollama pull llama3.2:3b`, and set `MEMORYMASTER_LLM_PROVIDER=ollama`. No API key required.
+
+### 3. Enable the v3 classifier + cadence policy (optional)
+
+The v3 statistical classifier + cadence policy are off by default so fresh installs behave like legacy steward. To opt in, set `MEMORYMASTER_STEWARD_CLASSIFIER_ENABLED=1` (or point `MEMORYMASTER_STEWARD_CLASSIFIER_PATH` at a trained `.pkl`) and `MEMORYMASTER_POLICY_MODE=cadence`. Full details, the training workflow, and the back-test harness live in [`docs/enabling-v2-systems.md`](docs/enabling-v2-systems.md).
+
+### 4. First run
+
+```bash
+python -m memorymaster --db memorymaster.db run-cycle
+```
+
+Expect output summarising `ingest / validate / decay / supersession / archive` counts. A fresh DB prints all zeroes — that's normal. After one or two sessions of the auto-ingest hook feeding candidates, the next cycle starts promoting `candidate` → `confirmed`.
+
 ## Install via Agent (One-Prompt) ⚡
 
 **The fastest way to install MemoryMaster end-to-end is to let an AI agent do it.** Open Claude Code, Codex, Cursor, or any agent with shell access in the project directory you want to instrument, and paste the prompt below. The agent handles pip install, MCP wiring, all 7 hooks, steward cron, LLM provider selection, and verification — you only approve steps and provide an API key when asked.

memorymaster-3.4.0/memorymaster.egg-info/PKG-INFO → memorymaster-3.5.0/README.md

@@ -1,39 +1,3 @@
-Metadata-Version: 2.4
-Name: memorymaster
-Version: 3.4.0
-Summary: Production-grade memory reliability system for AI coding agents. Lifecycle-managed claims with citations, conflict detection, steward governance, and MCP integration.
-Author: wolverin0
-License: MIT
-Keywords: memory,ai-agents,claims,lifecycle,mcp,sqlite,postgres,coding-agents
-Classifier: Development Status :: 5 - Production/Stable
-Classifier: Intended Audience :: Developers
-Classifier: License :: OSI Approved :: MIT License
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Classifier: Programming Language :: Python :: 3.12
-Classifier: Topic :: Software Development :: Libraries
-Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
-Requires-Python: >=3.10
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Provides-Extra: postgres
-Requires-Dist: psycopg[binary]>=3.2; extra == "postgres"
-Provides-Extra: security
-Requires-Dist: cryptography>=42; extra == "security"
-Provides-Extra: embeddings
-Requires-Dist: sentence-transformers>=3.0; extra == "embeddings"
-Provides-Extra: gemini
-Requires-Dist: google-genai>=1.0; extra == "gemini"
-Provides-Extra: qdrant
-Requires-Dist: httpx>=0.27; extra == "qdrant"
-Provides-Extra: dev
-Requires-Dist: pytest>=8.2; extra == "dev"
-Requires-Dist: pytest-cov>=6.0; extra == "dev"
-Provides-Extra: mcp
-Requires-Dist: mcp>=1.2; extra == "mcp"
-Dynamic: license-file
-
 # MemoryMaster
 
 **Production-grade memory reliability system for AI coding agents.**
@@ -143,6 +107,48 @@ MemoryMaster gives AI coding agents **persistent, verifiable memory** with a ful
 - **Obsidian 1.6+** with the **Bases** core plugin — only if you want to browse the wiki visually
 - **Docker** — only if you want Qdrant for hybrid vector search (SQLite FTS5 is the default and works out of the box)
 
+## Setup
+
+A minimal path for new users. Every env var mentioned here is documented in [`.env.example`](.env.example) — copy that file and uncomment the lines you need.
+
+### 1. Minimum viable setup
+
+```bash
+pip install "memorymaster[mcp]"
+python -m memorymaster --db memorymaster.db init-db
+cp .env.example .env
+# Then set ONE of:
+#   GEMINI_API_KEY=...    (free from https://aistudio.google.com)
+#   OPENAI_API_KEY=...
+#   ANTHROPIC_API_KEY=...
+# Or run Ollama locally (no key needed) — see below.
+```
+
+That's enough to use the CLI, the MCP server, and the auto-ingest Stop hook.
+
+### 2. Pick your LLM provider
+
+| Provider | Env vars | Model (default) | Cost |
+|----------|----------|-----------------|------|
+| Google Gemini (default) | `MEMORYMASTER_LLM_PROVIDER=google` + `GEMINI_API_KEY=...` | `gemini-3.1-flash-lite-preview` | ~free |
+| OpenAI | `MEMORYMASTER_LLM_PROVIDER=openai` + `OPENAI_API_KEY=...` | `gpt-4o-mini` | ~$0.001/call |
+| Anthropic | `MEMORYMASTER_LLM_PROVIDER=anthropic` + `ANTHROPIC_API_KEY=...` | `claude-haiku-4-5-20251001` | ~$0.001/call |
+| Ollama (local) | `MEMORYMASTER_LLM_PROVIDER=ollama` + `OLLAMA_URL=http://localhost:11434` | `llama3.2:3b` | free |
+
+For zero-cost offline use, install [Ollama](https://ollama.com), `ollama pull llama3.2:3b`, and set `MEMORYMASTER_LLM_PROVIDER=ollama`. No API key required.
+
+### 3. Enable the v3 classifier + cadence policy (optional)
+
+The v3 statistical classifier + cadence policy are off by default so fresh installs behave like legacy steward. To opt in, set `MEMORYMASTER_STEWARD_CLASSIFIER_ENABLED=1` (or point `MEMORYMASTER_STEWARD_CLASSIFIER_PATH` at a trained `.pkl`) and `MEMORYMASTER_POLICY_MODE=cadence`. Full details, the training workflow, and the back-test harness live in [`docs/enabling-v2-systems.md`](docs/enabling-v2-systems.md).
+
+### 4. First run
+
+```bash
+python -m memorymaster --db memorymaster.db run-cycle
+```
+
+Expect output summarising `ingest / validate / decay / supersession / archive` counts. A fresh DB prints all zeroes — that's normal. After one or two sessions of the auto-ingest hook feeding candidates, the next cycle starts promoting `candidate` → `confirmed`.
+
 ## Install via Agent (One-Prompt) ⚡
 
 **The fastest way to install MemoryMaster end-to-end is to let an AI agent do it.** Open Claude Code, Codex, Cursor, or any agent with shell access in the project directory you want to instrument, and paste the prompt below. The agent handles pip install, MCP wiring, all 7 hooks, steward cron, LLM provider selection, and verification — you only approve steps and provide an API key when asked.

memorymaster-3.5.0/artifacts/bm25-per-field-eval-harness.py

@@ -0,0 +1,309 @@
+"""One-off eval harness for roadmap 1.4 BM25 per-field weighting.
+
+The shipped ``scripts/eval_recall_precision_at_5.py`` has its own inline
+``_score`` implementation that reads ``row["lexical_score"]`` (the FTS5 rank
+from retrieval) — it does NOT exercise the BM25 rescorer that lives inside
+``context_hook.recall``. That's fine for the per-weight grid search it was
+built for, but it makes it impossible to measure a BM25-internal change
+(like per-field weighting) through that script.
+
+This harness reuses the same candidate-collection path, then applies the
+EXACT per-field BM25 rescorer from ``memorymaster.context_hook`` so each
+config is a true end-to-end measurement. It writes ``row["lexical_score"]``
+back with the per-field score before delegating to the eval's ``_evaluate``
+helper so the rest of the pipeline (ranker weights, labels, p@5, MAP@5) is
+identical across configs.
+
+Run::
+
+    python artifacts/bm25-per-field-eval-harness.py [--prompts ...] [--db ...]
+
+Does NOT modify the DB. Read-only, like the parent eval.
+"""
+from __future__ import annotations
+
+import argparse
+import math
+import os
+import sys
+from pathlib import Path
+
+# Add repo root and scripts/ to path.
+HERE = Path(__file__).resolve().parent
+REPO = HERE.parent
+sys.path.insert(0, str(REPO))
+sys.path.insert(0, str(REPO / "scripts"))
+
+# Import from the eval script (treat it as a module).
+import importlib.util
+spec = importlib.util.spec_from_file_location(
+    "eval_module", REPO / "scripts" / "eval_recall_precision_at_5.py"
+)
+assert spec is not None and spec.loader is not None
+eval_module = importlib.util.module_from_spec(spec)
+sys.modules["eval_module"] = eval_module  # dataclass needs cls.__module__ resolvable
+spec.loader.exec_module(eval_module)
+
+from memorymaster.context_hook import (
+    _BM25_K1_DEFAULT,
+    _BM25_B_DEFAULT,
+    _BM25_W_SUBJECT_DEFAULT,
+    _BM25_W_TEXT_DEFAULT,
+)
+from memorymaster.recall_tokenizer import _candidate_tokens
+from memorymaster.service import MemoryService
+
+
+def _tokens(raw: str) -> list[str]:
+    if not isinstance(raw, str):
+        return []
+    return [t for t in _candidate_tokens(raw) if len(t) >= 3]
+
+
+def _apply_per_field_bm25(
+    prompt: str,
+    rows: list[dict],
+    w_subject: float,
+    w_text: float,
+    k1: float = _BM25_K1_DEFAULT,
+    b: float = _BM25_B_DEFAULT,
+) -> None:
+    """Overwrite ``row["lexical_score"]`` with per-field BM25 for each row.
+
+    This replicates the logic in context_hook.recall() so the eval
+    harness measures the same scoring code as production.
+    """
+    # Per-field tokenisation + df.
+    subj_tok: dict[int, list[str]] = {}
+    text_tok: dict[int, list[str]] = {}
+    df_s: dict[str, int] = {}
+    df_t: dict[str, int] = {}
+    for r in rows:
+        c = r.get("claim")
+        cid = getattr(c, "id", None)
+        if cid is None or cid in subj_tok:
+            continue
+        st = _tokens(getattr(c, "subject", "") or "")
+        tt = _tokens(getattr(c, "text", "") or "")
+        subj_tok[cid] = st
+        text_tok[cid] = tt
+        for t in set(st):
+            df_s[t] = df_s.get(t, 0) + 1
+        for t in set(tt):
+            df_t[t] = df_t.get(t, 0) + 1
+
+    n_docs = len(subj_tok)
+    non_empty_s = [v for v in subj_tok.values() if v]
+    non_empty_t = [v for v in text_tok.values() if v]
+    avg_s = sum(len(v) for v in non_empty_s) / len(non_empty_s) if non_empty_s else 0.0
+    avg_t = sum(len(v) for v in non_empty_t) / len(non_empty_t) if non_empty_t else 0.0
+
+    q_tokens = [t for t in _candidate_tokens(prompt) if len(t) >= 3]
+
+    def field_score(toks: list[str], df: dict[str, int], avg: float) -> float:
+        if not toks or avg <= 0.0:
+            return 0.0
+        tf: dict[str, int] = {}
+        for t in toks:
+            tf[t] = tf.get(t, 0) + 1
+        dl = len(toks)
+        s = 0.0
+        for qt in q_tokens:
+            f = tf.get(qt, 0)
+            if f == 0:
+                continue
+            n_q = df.get(qt, 0)
+            idf = math.log(((n_docs - n_q + 0.5) / (n_q + 0.5)) + 1.0)
+            norm = 1.0 - b + b * (dl / avg)
+            s += idf * ((f * (k1 + 1.0)) / (f + k1 * norm))
+        return s
+
+    # Write per-field combined score into row["lexical_score"] so the
+    # downstream eval _score() sees it as the lexical signal. This is the
+    # one mutation; everything else is untouched.
+    scores: dict[int, float] = {}
+    if n_docs > 0 and q_tokens:
+        for cid in subj_tok:
+            ss = field_score(subj_tok[cid], df_s, avg_s)
+            ts = field_score(text_tok[cid], df_t, avg_t)
+            scores[cid] = w_subject * ss + w_text * ts
+
+    for r in rows:
+        c = r.get("claim")
+        cid = getattr(c, "id", None)
+        if cid is not None and cid in scores:
+            r["lexical_score"] = scores[cid]
+        else:
+            r["lexical_score"] = 0.0
+
+
+def _apply_concat_bm25(
+    prompt: str,
+    rows: list[dict],
+    k1: float = _BM25_K1_DEFAULT,
+    b: float = _BM25_B_DEFAULT,
+) -> None:
+    """Replicate the pre-change concatenated BM25 scorer for an honest baseline.
+
+    Mirrors the block at context_hook.py commit 3a34b2d:529-582.
+    """
+    tok: dict[int, list[str]] = {}
+    df: dict[str, int] = {}
+    for r in rows:
+        c = r.get("claim")
+        cid = getattr(c, "id", None)
+        if cid is None or cid in tok:
+            continue
+        subject = getattr(c, "subject", "") or ""
+        text = getattr(c, "text", "") or ""
+        if not isinstance(subject, str):
+            subject = ""
+        if not isinstance(text, str):
+            text = ""
+        joined = f"{subject} {text}"
+        toks = [t for t in _candidate_tokens(joined) if len(t) >= 3]
+        tok[cid] = toks
+        for t in set(toks):
+            df[t] = df.get(t, 0) + 1
+    n_docs = len(tok)
+    avg = sum(len(v) for v in tok.values()) / n_docs if n_docs else 0.0
+    q_tokens = [t for t in _candidate_tokens(prompt) if len(t) >= 3]
+    scores: dict[int, float] = {}
+    if n_docs > 0 and avg > 0 and q_tokens:
+        for cid, toks in tok.items():
+            if not toks:
+                continue
+            tf: dict[str, int] = {}
+            for t in toks:
+                tf[t] = tf.get(t, 0) + 1
+            dl = len(toks)
+            s = 0.0
+            for qt in q_tokens:
+                f = tf.get(qt, 0)
+                if f == 0:
+                    continue
+                n_q = df.get(qt, 0)
+                idf = math.log(((n_docs - n_q + 0.5) / (n_q + 0.5)) + 1.0)
+                norm = 1.0 - b + b * (dl / avg)
+                s += idf * ((f * (k1 + 1.0)) / (f + k1 * norm))
+            scores[cid] = s
+    for r in rows:
+        c = r.get("claim")
+        cid = getattr(c, "id", None)
+        if cid is not None and cid in scores:
+            r["lexical_score"] = scores[cid]
+        else:
+            r["lexical_score"] = 0.0
+
+
+def run_config(
+    collected: list[tuple[str, list[dict], object]],
+    label: str,
+    rescorer,
+    *rescorer_args,
+    min_overlap: int = 2,
+) -> tuple[float, float, int]:
+    # Fresh copies per config (rescorer mutates lexical_score).
+    import copy
+    rescored = []
+    for prompt, rows, svc_tokens in collected:
+        fresh = [dict(r) for r in rows]
+        rescorer(prompt, fresh, *rescorer_args)
+        rescored.append((prompt, fresh, svc_tokens))
+    p5, m5, hits = eval_module._evaluate(
+        rescored, eval_module.W0, min_overlap=min_overlap
+    )
+    return p5, m5, hits
+
+
+def main() -> int:
+    ap = argparse.ArgumentParser(description=__doc__)
+    ap.add_argument(
+        "--prompts",
+        default=str(REPO.parent.parent.parent / "artifacts" / "real-prompts.jsonl"),
+    )
+    ap.add_argument(
+        "--db",
+        default=str(REPO.parent.parent.parent / "memorymaster.db"),
+    )
+    ap.add_argument("--top-k", type=int, default=20)
+    ap.add_argument("--min-overlap", type=int, default=2)
+    args = ap.parse_args()
+
+    prompts_path = Path(args.prompts)
+    db_path = Path(args.db)
+    if not prompts_path.exists() or not db_path.exists():
+        print(f"ERROR missing: prompts={prompts_path}  db={db_path}")
+        return 2
+
+    prompts = eval_module._load_prompts(prompts_path)
+    svc = MemoryService(db_target=str(db_path), workspace_root=REPO)
+    svc._record_accesses = lambda *a, **k: None  # type: ignore[assignment]
+    if hasattr(svc, "store") and hasattr(svc.store, "record_accesses_batch"):
+        svc.store.record_accesses_batch = lambda *a, **k: None  # type: ignore[assignment]
+
+    print(f"Loaded {len(prompts)} prompts, collecting top-{args.top_k} candidates...")
+    collected = eval_module._collect_candidates(
+        prompts, svc, str(db_path), top_k=args.top_k,
+        include_entity_fanout=True, include_vector_fallback=False,
+    )
+    cand_counts = [len(r) for _, r, _ in collected]
+    print(f"  mean candidates/prompt: {sum(cand_counts) / max(1, len(cand_counts)):.1f} "
+          f"(min={min(cand_counts, default=0)}, max={max(cand_counts, default=0)})")
+
+    configs = [
+        ("A concat baseline            ", _apply_concat_bm25, ()),
+        ("B per-field W_S=2.0 W_T=1.0  ", _apply_per_field_bm25, (2.0, 1.0)),
+        ("C per-field W_S=3.0 W_T=1.0  ", _apply_per_field_bm25, (3.0, 1.0)),
+        ("D per-field W_S=1.5 W_T=1.0  ", _apply_per_field_bm25, (1.5, 1.0)),
+        ("E per-field W_S=5.0 W_T=1.0  ", _apply_per_field_bm25, (5.0, 1.0)),
+        ("F per-field W_S=10.0 W_T=0.0 ", _apply_per_field_bm25, (10.0, 0.0)),
+        ("G per-field W_S=0.0 W_T=10.0 ", _apply_per_field_bm25, (0.0, 10.0)),
+        ("H per-field W_S=1.0 W_T=1.0  ", _apply_per_field_bm25, (1.0, 1.0)),
+    ]
+
+    print("\n{:<34} {:>10} {:>10} {:>12}".format(
+        "config", "p@5", "MAP@5", "non_empty"))
+    print("-" * 70)
+    results = []
+    for label, fn, args_tuple in configs:
+        p5, m5, hits = run_config(collected, label, fn, *args_tuple,
+                                  min_overlap=args.min_overlap)
+        print(f"{label}  {p5:>8.3f}  {m5:>8.3f}   {hits:>3}/{len(prompts)}")
+        results.append((label, p5, m5, hits))
+
+    # Sample drill-down: find a prompt where concat (A) and per-field
+    # H=(1.0, 1.0) give a DIFFERENT top-1, and print both top-5 lists.
+    for prompt, rows, _ in collected:
+        if len(rows) < 5:
+            continue
+        rows_concat = [dict(r) for r in rows]
+        rows_pf = [dict(r) for r in rows]
+        _apply_concat_bm25(prompt, rows_concat)
+        _apply_per_field_bm25(prompt, rows_pf, 1.0, 1.0)
+        # Rank by the hook's real _relevance proxy (W0).
+        top5_concat = eval_module._rank(rows_concat, eval_module.W0)[:5]
+        top5_pf = eval_module._rank(rows_pf, eval_module.W0)[:5]
+        id0_c = getattr(top5_concat[0].get("claim"), "id", None)
+        id0_p = getattr(top5_pf[0].get("claim"), "id", None)
+        if id0_c != id0_p:
+            print("\n--- sample prompt where top-1 differs ---")
+            print(f"PROMPT: {prompt[:120]!r}")
+            print("concat baseline top-5:")
+            for row in top5_concat:
+                c = row.get("claim")
+                print(f"  cid={getattr(c, 'id', '?')!s:>6}  "
+                      f"subj={str(getattr(c, 'subject', ''))[:40]!r}  "
+                      f"text={str(getattr(c, 'text', ''))[:70]!r}")
+            print("per-field (1.0, 1.0) top-5:")
+            for row in top5_pf:
+                c = row.get("claim")
+                print(f"  cid={getattr(c, 'id', '?')!s:>6}  "
+                      f"subj={str(getattr(c, 'subject', ''))[:40]!r}  "
+                      f"text={str(getattr(c, 'text', ''))[:70]!r}")
+            break
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

{memorymaster-3.4.0 → memorymaster-3.5.0}/memorymaster/__init__.py

@@ -2,4 +2,4 @@
 
 __all__ = ["__version__"]
 
-__version__ = "3.4.0"
+__version__ = "3.4.1"

{memorymaster-3.4.0 → memorymaster-3.5.0}/memorymaster/cli.py

@@ -298,6 +298,14 @@ def build_parser() -> argparse.ArgumentParser:
     )
     wiki_backfill.add_argument("--output", default="obsidian-vault", help="Wiki directory to scan")
 
+    wiki_freshness = sub.add_parser(
+        "wiki-freshness",
+        help="Report per-article freshness (Option A — absorb recency)",
+    )
+    wiki_freshness.add_argument("--vault", default="obsidian-vault/wiki", help="Wiki root (defaults to obsidian-vault/wiki)")
+    wiki_freshness.add_argument("--below", type=float, default=None, help="Only show articles with freshness_score below this threshold (0-1)")
+    wiki_freshness.add_argument("--threshold-days", type=int, default=None, help="Only show articles older than N days since last absorb (alias for --below)")
+
     mine_cmd = sub.add_parser("mine-transcript", help="Parse Claude Code transcripts into claims")
     mine_cmd.add_argument("--input", required=True, help="JSONL transcript file or directory")
     mine_cmd.add_argument("--scope", default="project", help="Scope for ingested claims")
@@ -407,7 +415,7 @@ def main(argv: list[str] | None = None) -> int:
     effective_db = _resolve_db_path(args)
 
     # Commands that don't need MemoryService run first; service is lazy-created once for all others.
-    _NO_SERVICE_COMMANDS = {"stealth-status", "export-metrics"}
+    _NO_SERVICE_COMMANDS = {"stealth-status", "export-metrics", "wiki-freshness"}
 
     try:
         handler = COMMAND_HANDLERS.get(args.command)

{memorymaster-3.4.0 → memorymaster-3.5.0}/memorymaster/cli_handlers_curation.py

@@ -107,6 +107,16 @@ def _handle_lint_vault(args: argparse.Namespace, service, parser: argparse.Argum
             print(f"\n  Stale claims ({len(report['stale'])}):")
             for s in report["stale"][:10]:
                 print(f"    #{s['id']} ({s['age_days']}d old, conf={s['confidence']:.2f}) {s['text'][:50]}")
+        stale_articles = report.get("stale_articles") or []
+        if stale_articles:
+            print(f"\n  Stale articles ({len(stale_articles)}):")
+            for a in stale_articles[:10]:
+                scope = a.get("scope") or ""
+                title = a.get("title") or ""
+                print(
+                    f"    [{scope}] {title} — {a['days_since_absorb']:.0f}d "
+                    f"(freshness={a['freshness_score']:.2f})"
+                )
     return 0
 
 
@@ -217,6 +227,65 @@ def _handle_bases_generate(args: argparse.Namespace, service, parser: argparse.A
     return 0
 
 
+def _handle_wiki_freshness(args: argparse.Namespace, service, parser: argparse.ArgumentParser, effective_db: str) -> int:
+    """Print per-article freshness scores (Option A — absorb recency).
+
+    Service is unused; the metric is a pure filesystem read over the vault.
+    """
+    from memorymaster.wiki_freshness import (
+        as_jsonable,
+        bucket_distribution,
+        scan_vault,
+    )
+
+    vault_root = Path(args.vault)
+    t0 = time.perf_counter()
+    snapshots = scan_vault(vault_root)
+    elapsed_ms = (time.perf_counter() - t0) * 1000
+
+    # Optional filters.
+    threshold_score: float | None = None
+    if args.below is not None:
+        threshold_score = float(args.below)
+    if args.threshold_days is not None:
+        import math as _math
+        # Convert the day threshold into the equivalent score cut-off using the
+        # same decay curve as wiki_freshness.FRESHNESS_SCALE_DAYS.
+        equivalent = _math.exp(-float(args.threshold_days) / 30.0)
+        threshold_score = equivalent if threshold_score is None else min(threshold_score, equivalent)
+
+    filtered = snapshots
+    if threshold_score is not None:
+        filtered = [s for s in snapshots if s.freshness_score < threshold_score]
+
+    dist = bucket_distribution(snapshots)
+
+    if args.json_output:
+        payload = {
+            "vault": str(vault_root),
+            "total_articles": len(snapshots),
+            "distribution": dist,
+            "threshold_score": threshold_score,
+            "articles": as_jsonable(filtered),
+        }
+        print(_json_envelope(payload, total=len(filtered), query_ms=elapsed_ms))
+        return 0
+
+    print(f"wiki-freshness: {len(snapshots)} articles scanned in {elapsed_ms:.0f}ms")
+    print(f"  fresh (>=0.5): {dist['fresh']}  mid (0.2-0.5): {dist['mid']}  stale (<0.2): {dist['stale']}")
+    if threshold_score is not None:
+        print(f"  filter: freshness_score < {threshold_score:.3f}  -> {len(filtered)} matching")
+    if not filtered:
+        return 0
+    print()
+    print(f"  {'score':>6}  {'days':>6}  {'scope':<28} {'title'}")
+    for snap in filtered:
+        title = snap.title[:50]
+        scope = (snap.scope or "")[:28]
+        print(f"  {snap.freshness_score:>6.3f}  {snap.days_since_absorb:>6.1f}  {scope:<28} {title}")
+    return 0
+
+
 def _handle_wiki_cleanup(args: argparse.Namespace, service, parser: argparse.ArgumentParser, effective_db: str) -> int:
     from memorymaster.wiki_engine import cleanup
     t0 = time.perf_counter()
@@ -599,6 +668,7 @@ COMMAND_HANDLERS: dict[str, object] = {
     "wiki-cleanup": _handle_wiki_cleanup,
     "wiki-breakdown": _handle_wiki_breakdown,
     "wiki-backfill-bindings": _handle_wiki_backfill_bindings,
+    "wiki-freshness": _handle_wiki_freshness,
     "bases-generate": _handle_bases_generate,
     "mine-transcript": _handle_mine_transcript,
     "verify-claims": _handle_verify_claims,