memorymaster 3.19.0__tar.gz → 3.21.0__tar.gz

{memorymaster-3.19.0/memorymaster.egg-info → memorymaster-3.21.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: memorymaster
-Version: 3.19.0
+Version: 3.21.0
 Summary: Production-grade memory reliability system for AI coding agents. Lifecycle-managed claims with citations, conflict detection, steward governance, and MCP integration.
 Author: wolverin0
 License: MIT
@@ -52,7 +52,7 @@ Lifecycle-managed claims with citations, conflict detection, steward governance,
 
 [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
 [![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
-[![Tests](https://img.shields.io/badge/tests-1953-green.svg)]()
+[![Tests](https://img.shields.io/badge/tests-2194-green.svg)]()
 [![MCP Tools](https://img.shields.io/badge/MCP%20tools-24-purple.svg)]()
 [![CLI Commands](https://img.shields.io/badge/CLI%20commands-86-orange.svg)]()
 [![PyPI](https://img.shields.io/pypi/v/memorymaster.svg)](https://pypi.org/project/memorymaster/)
@@ -87,6 +87,9 @@ recent PR status, and sensitivity-filter invariants.
 - **Hybrid retrieval**: vector (sentence-transformers / Gemini) + FTS5 + freshness + confidence
 - **Context optimizer**: `query_for_context(budget=4000)` returns auto-curated memory that fits your token budget
 - **Entity graph** with typed relationships and alias resolution
+- **Rule-shaped claims** (new in v3.21.0): prescriptive `when <trigger>, do <action> because <rationale>` claims (`ingest_rule` / `query_rules`) — the shape an agent needs to actually change behaviour next time, not just recall a fact
+- **Correction mining** (new in v3.21.0): `mine-rules` scans the verbatim transcript archive for user corrections and distills them into rule claims; the Stop hook also mines each session's latest correction automatically
+- **Versioned schema migrations** (new in v3.20.0): `migrate` applies SQLite/Postgres migrations with sha256 drift detection; incremental `export-delta` ships small claim deltas for cheap cross-machine sync
 - **Steward governance**: multi-probe validators (filesystem, format, citation, semantic, tool) with proposal review
 - **Conflict resolution**: 5-tier auto (confidence > freshness > citations > LLM > manual)
 - **Auto-redaction** at ingest: JWT, GitHub tokens, Bearer, AWS keys, SSH keys, custom patterns
@@ -187,7 +190,7 @@ For zero-cost offline use, install [Ollama](https://ollama.com), `ollama pull ll
 }
 ```
 
-22 MCP tools: `init_db`, `ingest_claim`, `run_cycle`, `run_steward`, `classify_query`, `query_memory`, `query_for_context`, `list_claims`, `redact_claim_payload`, `pin_claim`, `compact_memory`, `list_events`, `search_verbatim`, `open_dashboard`, `list_steward_proposals`, `resolve_steward_proposal`, `extract_entities`, `entity_stats`, `find_related_claims`, `quality_scores`, `recompute_tiers`, `federated_query`.
+24 MCP tools: `init_db`, `ingest_claim`, `ingest_rule`, `query_rules`, `run_cycle`, `run_steward`, `classify_query`, `query_memory`, `query_for_context`, `list_claims`, `redact_claim_payload`, `pin_claim`, `compact_memory`, `list_events`, `search_verbatim`, `open_dashboard`, `list_steward_proposals`, `resolve_steward_proposal`, `extract_entities`, `entity_stats`, `find_related_claims`, `quality_scores`, `recompute_tiers`, `federated_query`.
 
 See [`.mcp.json.example`](.mcp.json.example) for the full template.
 

{memorymaster-3.19.0 → memorymaster-3.21.0}/README.md

@@ -6,7 +6,7 @@ Lifecycle-managed claims with citations, conflict detection, steward governance,
 
 [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
 [![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
-[![Tests](https://img.shields.io/badge/tests-1953-green.svg)]()
+[![Tests](https://img.shields.io/badge/tests-2194-green.svg)]()
 [![MCP Tools](https://img.shields.io/badge/MCP%20tools-24-purple.svg)]()
 [![CLI Commands](https://img.shields.io/badge/CLI%20commands-86-orange.svg)]()
 [![PyPI](https://img.shields.io/pypi/v/memorymaster.svg)](https://pypi.org/project/memorymaster/)
@@ -41,6 +41,9 @@ recent PR status, and sensitivity-filter invariants.
 - **Hybrid retrieval**: vector (sentence-transformers / Gemini) + FTS5 + freshness + confidence
 - **Context optimizer**: `query_for_context(budget=4000)` returns auto-curated memory that fits your token budget
 - **Entity graph** with typed relationships and alias resolution
+- **Rule-shaped claims** (new in v3.21.0): prescriptive `when <trigger>, do <action> because <rationale>` claims (`ingest_rule` / `query_rules`) — the shape an agent needs to actually change behaviour next time, not just recall a fact
+- **Correction mining** (new in v3.21.0): `mine-rules` scans the verbatim transcript archive for user corrections and distills them into rule claims; the Stop hook also mines each session's latest correction automatically
+- **Versioned schema migrations** (new in v3.20.0): `migrate` applies SQLite/Postgres migrations with sha256 drift detection; incremental `export-delta` ships small claim deltas for cheap cross-machine sync
 - **Steward governance**: multi-probe validators (filesystem, format, citation, semantic, tool) with proposal review
 - **Conflict resolution**: 5-tier auto (confidence > freshness > citations > LLM > manual)
 - **Auto-redaction** at ingest: JWT, GitHub tokens, Bearer, AWS keys, SSH keys, custom patterns
@@ -141,7 +144,7 @@ For zero-cost offline use, install [Ollama](https://ollama.com), `ollama pull ll
 }
 ```
 
-22 MCP tools: `init_db`, `ingest_claim`, `run_cycle`, `run_steward`, `classify_query`, `query_memory`, `query_for_context`, `list_claims`, `redact_claim_payload`, `pin_claim`, `compact_memory`, `list_events`, `search_verbatim`, `open_dashboard`, `list_steward_proposals`, `resolve_steward_proposal`, `extract_entities`, `entity_stats`, `find_related_claims`, `quality_scores`, `recompute_tiers`, `federated_query`.
+24 MCP tools: `init_db`, `ingest_claim`, `ingest_rule`, `query_rules`, `run_cycle`, `run_steward`, `classify_query`, `query_memory`, `query_for_context`, `list_claims`, `redact_claim_payload`, `pin_claim`, `compact_memory`, `list_events`, `search_verbatim`, `open_dashboard`, `list_steward_proposals`, `resolve_steward_proposal`, `extract_entities`, `entity_stats`, `find_related_claims`, `quality_scores`, `recompute_tiers`, `federated_query`.
 
 See [`.mcp.json.example`](.mcp.json.example) for the full template.
 

{memorymaster-3.19.0 → memorymaster-3.21.0}/memorymaster/__init__.py

@@ -2,4 +2,4 @@
 
 __all__ = ["__version__"]
 
-__version__ = "3.4.1"
+__version__ = "3.21.0"

{memorymaster-3.19.0 → memorymaster-3.21.0}/memorymaster/cli.py

@@ -23,7 +23,9 @@ from memorymaster.cli_handlers_curation import COMMAND_HANDLERS
 from memorymaster.cli_handlers_basic import (
     _handle_decay,
     _handle_entity_graph_export,
+    _handle_export_delta,
     _handle_ingest_daydream,
+    _handle_migrate,
     _handle_recompute_confidence_priors,
     _handle_wiki_suggest_links,
     handle_mcp_usage_report,
@@ -38,6 +40,8 @@ COMMAND_HANDLERS["ingest-daydream"] = _handle_ingest_daydream
 COMMAND_HANDLERS["mcp-usage-report"] = (
     lambda args, service, parser, effective_db: handle_mcp_usage_report(args, effective_db)
 )
+COMMAND_HANDLERS["migrate"] = _handle_migrate
+COMMAND_HANDLERS["export-delta"] = _handle_export_delta
 
 
 def build_parser() -> argparse.ArgumentParser:
@@ -51,6 +55,11 @@ def build_parser() -> argparse.ArgumentParser:
 
     sub.add_parser("init-db", help="Create schema in SQLite database")
 
+    migrate = sub.add_parser("migrate", help="Apply pending versioned schema migrations (v3.20.0+)")
+    migrate_mode = migrate.add_mutually_exclusive_group()
+    migrate_mode.add_argument("--list", action="store_true", help="List known migrations without touching the DB")
+    migrate_mode.add_argument("--status", action="store_true", help="Report applied vs pending per migration")
+
     sub.add_parser("stealth-status", help="Show whether stealth mode is active and which DB is in use")
 
     ingest = sub.add_parser("ingest", help="Ingest a raw claim with citations")
@@ -447,6 +456,13 @@ def build_parser() -> argparse.ArgumentParser:
     mine_cmd.add_argument("--scope", default="project", help="Scope for ingested claims")
     mine_cmd.add_argument("--max", type=int, default=100, help="Max claims to ingest")
 
+    mine_rules_cmd = sub.add_parser("mine-rules", help="Mine verbatim corrections into rule-shaped claims (v3.21.0-R1b)")
+    mine_rules_cmd.add_argument("--since-id", dest="since_id", type=int, default=None, help="Override the stored watermark; start scanning after this verbatim id")
+    mine_rules_cmd.add_argument("--limit", type=int, default=None, help="Max candidate windows to examine this run (caps LLM calls)")
+    mine_rules_cmd.add_argument("--batch-size", dest="batch_size", type=int, default=200, help="Rows fetched per SQL pre-filter page (default: 200)")
+    mine_rules_cmd.add_argument("--provider", default="claude_cli", help="LLM provider for this run (default: claude_cli)")
+    mine_rules_cmd.add_argument("--reset", action="store_true", help="Clear the stored watermark before running (re-scan from the start)")
+
     verify_cmd = sub.add_parser("verify-claims", help="Cross-check claims against current codebase")
     verify_cmd.add_argument("--scope", default="", help="Scope filter")
     verify_cmd.add_argument("--limit", type=int, default=200, help="Max claims to check")
@@ -504,6 +520,13 @@ def build_parser() -> argparse.ArgumentParser:
     merge_cmd = sub.add_parser("merge-db", help="Merge claims from a remote memorymaster DB (bidirectional sync)")
     merge_cmd.add_argument("--source", required=True, help="Path to source DB file to merge from")
 
+    delta_cmd = sub.add_parser(
+        "export-delta",
+        help="Export claims changed since a watermark into a small SQLite delta file (incremental sync)",
+    )
+    delta_cmd.add_argument("--since", default="", help="ISO-8601 watermark; export claims with updated_at after this (empty = full export)")
+    delta_cmd.add_argument("--output", required=True, help="Path to write the delta SQLite file (overwritten if it exists)")
+
     daily = sub.add_parser("daily-note", help="Generate a daily note summarizing today's activity")
     daily.add_argument("--date", default="", help="Date to generate for (YYYY-MM-DD, default: today)")
     daily.add_argument("--output", default="", help="Directory to save .md file (default: print to stdout)")
@@ -556,7 +579,7 @@ def main(argv: list[str] | None = None) -> int:
     effective_db = _resolve_db_path(args)
 
     # Commands that don't need MemoryService run first; service is lazy-created once for all others.
-    _NO_SERVICE_COMMANDS = {"stealth-status", "export-metrics", "wiki-freshness", "mcp-usage-report"}
+    _NO_SERVICE_COMMANDS = {"stealth-status", "export-metrics", "wiki-freshness", "mcp-usage-report", "export-delta"}
 
     try:
         handler = COMMAND_HANDLERS.get(args.command)

{memorymaster-3.19.0 → memorymaster-3.21.0}/memorymaster/cli_handlers_basic.py

@@ -1357,3 +1357,95 @@ def _handle_check_staleness(args: argparse.Namespace, service, parser: argparse.
     return 0
 
 
+def _handle_migrate(args: argparse.Namespace, service, parser: argparse.ArgumentParser, effective_db: str) -> int:
+    """v3.20.0-S1: apply pending schema migrations, or report status.
+
+    Default (no flags): apply every pending migration in version order.
+    --list: dump known migrations (version + description) without touching the DB.
+    --status: query the DB and show applied vs pending per migration.
+    """
+    from memorymaster.migrations import (
+        MigrationRunner,
+        discover_migrations,
+    )
+    from memorymaster.store_factory import is_postgres_dsn
+
+    # --list works without a DB connection at all.
+    if getattr(args, "list", False):
+        migrations = discover_migrations()
+        if args.json_output:
+            payload = [{"version": m.version, "description": m.description} for m in migrations]
+            print(_json_envelope(payload))
+        else:
+            print(f"known migrations ({len(migrations)}):")
+            for m in migrations:
+                print(f"  v{m.version:04d}  {m.description}")
+        return 0
+
+    backend = "postgres" if is_postgres_dsn(effective_db) else "sqlite"
+    store = service.store
+    with store.connect() as conn:
+        runner = MigrationRunner(conn, backend=backend)
+
+        if getattr(args, "status", False):
+            entries = runner.status()
+            if args.json_output:
+                payload = [
+                    {
+                        "version": e.version,
+                        "description": e.description,
+                        "applied": e.applied,
+                        "applied_at": e.applied_at,
+                    }
+                    for e in entries
+                ]
+                print(_json_envelope(payload))
+            else:
+                print(f"backend={backend} db={effective_db}")
+                for e in entries:
+                    marker = "[applied]" if e.applied else "[pending]"
+                    when = f" applied_at={e.applied_at}" if e.applied_at else ""
+                    print(f"  v{e.version:04d}  {marker} {e.description}{when}")
+            return 0
+
+        # Default: apply pending
+        newly = runner.apply_pending()
+        if args.json_output:
+            print(_json_envelope({"applied": newly, "backend": backend}))
+        else:
+            if not newly:
+                print(f"migrate: nothing to apply (backend={backend}, db={effective_db})")
+            else:
+                print(f"migrate: applied {len(newly)} migration(s) on backend={backend}:")
+                for v in newly:
+                    print(f"  v{v:04d}")
+    return 0
+
+
+def _handle_export_delta(args: argparse.Namespace, service, parser: argparse.ArgumentParser, effective_db: str) -> int:
+    """Export claims changed since a watermark into a small SQLite delta file.
+
+    The delta file is a valid `merge-db --source` input. Prints (or JSON-emits)
+    the export counts and the new watermark — callers should record
+    `max_updated_at` and pass it as `--since` on the next run.
+    """
+    from memorymaster.delta_sync import export_delta
+
+    t0 = time.perf_counter()
+    result = export_delta(effective_db, args.since, args.output)
+    elapsed_ms = (time.perf_counter() - t0) * 1000
+    if args.json_output:
+        print(_json_envelope(result, query_ms=elapsed_ms))
+    else:
+        since_label = result["since"] or "(full export)"
+        print(
+            f"export-delta: {result['exported']} claims + {result['citations']} citations "
+            f"since {since_label} -> {args.output}"
+        )
+        if result["max_updated_at"]:
+            print(f"  next watermark (--since): {result['max_updated_at']}")
+        else:
+            print("  delta is empty — nothing changed since the watermark")
+    return 0
+
+

{memorymaster-3.19.0 → memorymaster-3.21.0}/memorymaster/cli_handlers_curation.py

@@ -343,6 +343,31 @@ def _handle_mine_transcript(args: argparse.Namespace, service, parser: argparse.
     return 0
 
 
+def _handle_mine_rules(args: argparse.Namespace, service, parser: argparse.ArgumentParser, effective_db: str) -> int:
+    from memorymaster.rule_miner import mine_rules
+    t0 = time.perf_counter()
+    result = mine_rules(
+        effective_db,
+        service,
+        since_id=getattr(args, "since_id", None),
+        limit=getattr(args, "limit", None),
+        batch_size=getattr(args, "batch_size", 200),
+        provider=getattr(args, "provider", "claude_cli"),
+        reset=getattr(args, "reset", False),
+    )
+    elapsed_ms = (time.perf_counter() - t0) * 1000
+    if args.json_output:
+        print(_json_envelope(result, query_ms=elapsed_ms))
+    else:
+        abort = f", ABORTED ({result['aborted_reason']})" if result.get("aborted_reason") else ""
+        print(
+            f"Mined rules: {result['candidates']} candidates, {result['llm_calls']} llm calls, "
+            f"{result['ingested']} ingested, {result['duplicates']} dupes, {result['skipped']} skipped "
+            f"(watermark={result['last_id']}{abort}, {elapsed_ms:.0f}ms)"
+        )
+    return 0
+
+
 def _handle_wiki_breakdown(args: argparse.Namespace, service, parser: argparse.ArgumentParser, effective_db: str) -> int:
     from memorymaster.wiki_engine import breakdown
     t0 = time.perf_counter()
@@ -703,6 +728,7 @@ COMMAND_HANDLERS: dict[str, object] = {
     "wiki-freshness": _handle_wiki_freshness,
     "bases-generate": _handle_bases_generate,
     "mine-transcript": _handle_mine_transcript,
+    "mine-rules": _handle_mine_rules,
     "verify-claims": _handle_verify_claims,
     "extract-entities": _handle_extract_entities,
     "entity-stats": _handle_entity_stats,

{memorymaster-3.19.0 → memorymaster-3.21.0}/memorymaster/config_templates/hooks/memorymaster-auto-ingest.py

@@ -180,6 +180,27 @@ Only: bug root causes, decisions, gotchas, constraints. Never: credentials, IPs,
         pass
 
 
+def _run_rule_extraction(transcript_path, cwd):
+    """R1b ongoing: mine the latest correction in this session into a rule claim.
+
+    Reuses memorymaster.rule_miner.mine_transcript_rules (single source of truth
+    for the correction->rule prompt + ingest path). Bounded to one window per
+    stop to keep the hook fast; rules land as low-confidence candidates."""
+    try:
+        if not transcript_path or not os.path.exists(transcript_path) or not os.path.exists(DB_PATH):
+            return
+        from memorymaster.rule_miner import mine_transcript_rules
+        from memorymaster.service import MemoryService
+
+        scope = "project:" + os.path.basename(cwd).lower().replace(" ", "-") if cwd else "global"
+        svc = MemoryService(DB_PATH, workspace_root=Path(cwd or PROJECT_ROOT))
+        stats = mine_transcript_rules(transcript_path, svc, scope=scope, max_windows=1)
+        if stats.get("ingested"):
+            sys.stderr.write(f"[MemoryMaster] mined {stats['ingested']} rule(s) from corrections\n")
+    except Exception:
+        pass
+
+
 def main():
     try:
         data = json.loads(sys.stdin.read() or "{}")
@@ -232,6 +253,9 @@ def main():
     # Not time to block — run passive Gemini extraction
     _run_gemini_extraction(transcript_path, cwd)
 
+    # R1b: mine the latest correction in this session into a rule claim
+    _run_rule_extraction(transcript_path, cwd)
+
     sys.stdout.write(json.dumps({"decision": "approve"}))
 
 

memorymaster-3.21.0/memorymaster/delta_sync.py

@@ -0,0 +1,172 @@
+"""Delta export for incremental memory sync.
+
+Pre-existing sync (``scripts/openclaw-sync.sh``) copied the WHOLE database
+file across the network twice per cycle — fine when the DB was small,
+wasteful at 2.5 GB. The ``merge-db`` step was always append-only (it dedups
+on ``idempotency_key`` + text-hash), but the *transport* moved everything.
+
+``export_delta`` closes that gap: it writes a small SQLite file containing
+ONLY the claims (and their citations) changed since a watermark timestamp.
+That small file is a valid merge source — ``merge-db --source delta.db``
+consumes it unchanged, because the merge engine only reads ``claims`` and
+``citations`` and ignores everything else.
+
+Sync loop with deltas:
+
+    1. each side: ``export-delta --since <last_sync> --output delta.db``
+    2. ship the small delta.db over the network (KB, not GB)
+    3. each side: ``merge-db --source <other-side-delta.db>``
+    4. record the new watermark (``max_updated_at`` from the export result)
+
+The whole-DB file never crosses the network — which also removes the
+SQLite-over-network-mount corruption risk entirely.
+"""
+from __future__ import annotations
+
+import logging
+import sqlite3
+from pathlib import Path
+
+logger = logging.getLogger(__name__)
+
+
+# Tables a merge source must carry. The merge engine (db_merge.py) reads
+# `claims` (SELECT *) and `citations` (source/locator/excerpt/created_at).
+# Nothing else is needed — FTS5, events, entity tables are all rebuilt or
+# ignored on the merge side.
+_DELTA_TABLES = ("claims", "citations")
+
+
+def _copy_table_ddl(src: sqlite3.Connection, out: sqlite3.Connection, table: str) -> None:
+    """Copy a table's CREATE statement verbatim from src into out."""
+    row = src.execute(
+        "SELECT sql FROM sqlite_master WHERE type='table' AND name=?",
+        (table,),
+    ).fetchone()
+    if row is None or not row[0]:
+        raise ValueError(f"source DB has no '{table}' table — not a memorymaster DB?")
+    out.execute(row[0])
+
+
+def export_delta(
+    source_db: str | Path,
+    since: str,
+    output_path: str | Path,
+) -> dict[str, object]:
+    """Write claims changed since ``since`` into a small SQLite delta file.
+
+    Args:
+        source_db: path to the full memorymaster DB to export from.
+        since: ISO-8601 timestamp watermark. Claims with ``updated_at`` strictly
+            greater than this are exported. Pass an empty string or a very
+            early date to export everything (full bootstrap).
+        output_path: where to write the delta SQLite file. Overwritten if it
+            exists.
+
+    Returns:
+        dict with ``exported`` (claim count), ``citations`` (citation count),
+        ``since`` (echoed watermark), and ``max_updated_at`` (the newest
+        ``updated_at`` seen — use this as the next watermark; None when the
+        delta is empty).
+
+    Raises:
+        FileNotFoundError: source DB missing.
+        ValueError: source DB lacks the expected tables.
+    """
+    source_db = str(source_db)
+    output_path = Path(output_path)
+    if not Path(source_db).exists():
+        raise FileNotFoundError(f"Source DB not found: {source_db}")
+
+    # Fresh output file every time — a stale delta would merge old rows again
+    # (harmless thanks to idempotent merge, but wasteful).
+    if output_path.exists():
+        output_path.unlink()
+    output_path.parent.mkdir(parents=True, exist_ok=True)
+
+    src = sqlite3.connect(source_db)
+    src.row_factory = sqlite3.Row
+    out = sqlite3.connect(str(output_path))
+    try:
+        for table in _DELTA_TABLES:
+            _copy_table_ddl(src, out, table)
+
+        # `since` empty => full export. SQLite string comparison on ISO-8601
+        # timestamps is chronological, so it works as a watermark.
+        #
+        # We use `>=`, not `>`. Strictly-after would SKIP any claim whose
+        # updated_at exactly equals the watermark — and multiple claims can
+        # share a timestamp (same-second ingest). Skipping a claim is silent
+        # data loss. `>=` instead re-exports the boundary claim(s); the merge
+        # engine is idempotent (dedups on idempotency_key + text-hash), so a
+        # re-export costs nothing but a few rows. Safe beats clean.
+        watermark = since.strip()
+        if watermark:
+            claim_rows = src.execute(
+                "SELECT * FROM claims WHERE updated_at >= ? ORDER BY updated_at",
+                (watermark,),
+            ).fetchall()
+        else:
+            claim_rows = src.execute(
+                "SELECT * FROM claims ORDER BY updated_at"
+            ).fetchall()
+
+        if not claim_rows:
+            out.commit()
+            return {
+                "exported": 0,
+                "citations": 0,
+                "since": watermark,
+                "max_updated_at": None,
+            }
+
+        claim_cols = [c[1] for c in src.execute("PRAGMA table_info(claims)").fetchall()]
+        placeholders = ",".join("?" for _ in claim_cols)
+        col_list = ",".join(claim_cols)
+        insert_claim = f"INSERT INTO claims ({col_list}) VALUES ({placeholders})"
+
+        exported_ids: list[int] = []
+        max_updated = ""
+        for row in claim_rows:
+            out.execute(insert_claim, tuple(row[c] for c in claim_cols))
+            exported_ids.append(int(row["id"]))
+            updated = str(row["updated_at"] or "")
+            if updated > max_updated:
+                max_updated = updated
+
+        # Citations for exactly the exported claims. claim_id linkage is
+        # preserved because we keep original claim ids in the delta file.
+        cit_cols = [c[1] for c in src.execute("PRAGMA table_info(citations)").fetchall()]
+        cit_placeholders = ",".join("?" for _ in cit_cols)
+        cit_col_list = ",".join(cit_cols)
+        insert_cit = f"INSERT INTO citations ({cit_col_list}) VALUES ({cit_placeholders})"
+
+        citation_count = 0
+        # Chunk the IN clause — SQLite caps host parameters at 999.
+        for start in range(0, len(exported_ids), 900):
+            batch = exported_ids[start : start + 900]
+            qmarks = ",".join("?" for _ in batch)
+            cit_rows = src.execute(
+                f"SELECT * FROM citations WHERE claim_id IN ({qmarks})",
+                batch,
+            ).fetchall()
+            for cit in cit_rows:
+                out.execute(insert_cit, tuple(cit[c] for c in cit_cols))
+                citation_count += 1
+
+        out.commit()
+        logger.info(
+            "export_delta: %d claims, %d citations since %r",
+            len(exported_ids),
+            citation_count,
+            watermark or "(full)",
+        )
+        return {
+            "exported": len(exported_ids),
+            "citations": citation_count,
+            "since": watermark,
+            "max_updated_at": max_updated or None,
+        }
+    finally:
+        src.close()
+        out.close()

{memorymaster-3.19.0 → memorymaster-3.21.0}/memorymaster/mcp_server.py

@@ -1113,6 +1113,57 @@ if FastMCP is not None:
         )
         return {"ok": True, "rows": len(claims), "claims": [_claim_to_dict(c) for c in claims]}
 
+    @mcp.tool()
+    def ingest_rule(
+        trigger: str,
+        action: str,
+        rationale: str = "",
+        db: str = "memorymaster.db",
+        workspace: str = ".",
+        scope: str = "project",
+        source_agent: str = "mcp",
+    ) -> dict[str, Any]:
+        """Ingest a prescriptive rule-shaped claim (v3.21.0-R1).
+
+        A rule captures "when <trigger>, do <action> because <rationale>" —
+        the behavioural shape, distinct from descriptive fact claims. Stored
+        as a claim_type='rule' claim; retrieve via query_rules.
+        """
+        from memorymaster.rules import build_rule_fields
+
+        svc = _service(db, workspace)
+        fields = build_rule_fields(trigger, action, rationale)
+        # Auto-citation: a rule's provenance is the session that taught it.
+        claim = svc.ingest(
+            **fields,
+            citations=[CitationInput(source=f"agent://{source_agent}", locator="rule", excerpt=action[:200])],
+            scope=scope,
+            source_agent=source_agent,
+        )
+        return {"ok": True, "claim_id": claim.id, "human_id": claim.human_id, "rule": fields["text"]}
+
+    @mcp.tool()
+    def query_rules(
+        query: str,
+        db: str = "memorymaster.db",
+        workspace: str = ".",
+        limit: int = 10,
+        allow_sensitive: bool = False,
+    ) -> dict[str, Any]:
+        """Retrieve rule-shaped claims matching a query, in prescriptive form.
+
+        Returns each rule's trigger / action / rationale / text, ranked by the
+        hybrid retriever. Use this when you want only behavioural rules, not
+        descriptive fact claims.
+        """
+        resolve_allow_sensitive_access(
+            allow_sensitive=allow_sensitive,
+            context="mcp.query_rules",
+        )
+        svc = _service(db, workspace)
+        rules = svc.query_rules(query, limit=limit, allow_sensitive=allow_sensitive)
+        return {"ok": True, "rows": len(rules), "rules": rules}
+
     @mcp.tool()
     def redact_claim_payload(
         claim_id: int,

memorymaster-3.21.0/memorymaster/migrations/0001_initial.py

@@ -0,0 +1,25 @@
+"""0001_initial — baseline marker for the v3.20.0 migration framework.
+
+This migration is intentionally a no-op. As of v3.20.0-S1 the pre-existing
+schema (from ``schema.sql`` / ``schema_postgres.sql`` and the inline
+``_ensure_*_schema`` helpers in storage layers) is treated as the baseline
+v0001. The runner stamps this version into ``schema_versions`` on first
+run so subsequent NEW migrations (v0002+) apply cleanly on top.
+
+Future migrations starting at v0002 will contain real ``ALTER TABLE`` /
+``CREATE INDEX`` / data-backfill DDL.
+"""
+from __future__ import annotations
+
+VERSION = 1
+DESCRIPTION = "baseline (existing schema as of v3.20.0)"
+
+
+def apply_sqlite(conn) -> None:  # noqa: ARG001 — intentional no-op baseline
+    """No-op: existing schema is the baseline."""
+    return None
+
+
+def apply_postgres(conn) -> None:  # noqa: ARG001 — intentional no-op baseline
+    """No-op: existing schema is the baseline."""
+    return None

memorymaster-3.21.0/memorymaster/migrations/0002_miner_state.py

@@ -0,0 +1,38 @@
+"""0002_miner_state — KV table for resumable miner watermarks (v3.21.0-R1b).
+
+The verbatim rule-miner (:mod:`memorymaster.rule_miner`) needs to remember
+how far through ``verbatim_memories`` it has scanned so re-runs are
+incremental. The watermark is a single integer, not a per-row fact, so it
+lives in a small key-value table rather than a column on the 744k-row
+verbatim table (a column would force a full backfill and contend with the
+Stop hook's inserts under WAL).
+
+A KV table also generalizes: future miners reuse it with different keys.
+"""
+from __future__ import annotations
+
+VERSION = 2
+DESCRIPTION = "miner_state KV table for resumable verbatim mining watermarks"
+
+_DDL = """
+CREATE TABLE IF NOT EXISTS miner_state (
+    key TEXT PRIMARY KEY,
+    value TEXT,
+    updated_at TEXT
+)
+""".strip()
+
+
+def apply_sqlite(conn) -> None:
+    conn.execute(_DDL)
+    commit = getattr(conn, "commit", None)
+    if callable(commit):
+        commit()
+
+
+def apply_postgres(conn) -> None:
+    cur = conn.cursor()
+    cur.execute(_DDL)
+    commit = getattr(conn, "commit", None)
+    if callable(commit):
+        commit()

memorymaster-3.21.0/memorymaster/migrations/__init__.py

@@ -0,0 +1,41 @@
+"""Versioned schema migrations for MemoryMaster (v3.20.0-S1).
+
+Pre-v3.20 schema evolved via opportunistic ``ALTER TABLE`` with try/except
+sprinkled across storage modules — no version tracking, no rollback, silent
+drift risk between SQLite and Postgres. This package replaces that pattern
+with explicit versioned migrations.
+
+Each migration is a Python module under this package named
+``NNNN_short_description.py`` with four module-level attributes:
+
+    VERSION: int                            # 1, 2, 3, ...
+    DESCRIPTION: str                        # one-line human summary
+    def apply_sqlite(conn) -> None: ...     # SQLite-backend DDL
+    def apply_postgres(conn) -> None: ...   # Postgres-backend DDL
+
+The ``MigrationRunner`` discovers all migration modules, sorts by VERSION,
+and applies only the ones not yet recorded in the ``schema_versions``
+table. Each applied migration's file source is sha256-checksummed and
+stored alongside its version; on subsequent runs, a mismatch raises
+``MigrationDriftError`` — migrations are immutable once applied.
+
+Public API:
+
+    from memorymaster.migrations import MigrationRunner, MigrationDriftError
+    runner = MigrationRunner(conn, backend="sqlite")
+    runner.apply_pending()
+    runner.status()  # -> list of (version, description, applied_at, status)
+"""
+from __future__ import annotations
+
+from memorymaster.migrations.runner import (
+    MigrationDriftError,
+    MigrationRunner,
+    discover_migrations,
+)
+
+__all__ = [
+    "MigrationDriftError",
+    "MigrationRunner",
+    "discover_migrations",
+]