mnemonics 0.1.0__tar.gz

mnemonics-0.1.0/PKG-INFO

@@ -0,0 +1,152 @@
+Metadata-Version: 2.4
+Name: mnemonics
+Version: 0.1.0
+Summary: Verified AI memory — retrieval that doesn't hallucinate.
+Author: atakan
+License-Expression: MIT
+Keywords: llm,memory,retrieval,rag,hallucination,mcp,agent
+Classifier: Development Status :: 3 - Alpha
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: numpy>=1.24
+Requires-Dist: hnswlib>=0.8
+Requires-Dist: sentence-transformers>=3.0
+Requires-Dist: requests>=2.28
+Provides-Extra: verify
+Requires-Dist: halluguard>=0.3; extra == "verify"
+Provides-Extra: server
+Requires-Dist: halluguard>=0.3; extra == "server"
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0; extra == "dev"
+Requires-Dist: pytest-cov>=5.0; extra == "dev"
+Requires-Dist: ruff>=0.5; extra == "dev"
+
+# Mnemonics
+
+**Your memory doesn't hallucinate.**
+
+Mnemonics is a local-first AI memory layer that stores, retrieves, and *verifies* what it returns. Built on sentence embeddings + HNSW vector search, with optional [halluguard](https://github.com/nakata-app/halluguard) verification to flag results that drift from the indexed corpus.
+
+## Why
+
+Every RAG pipeline has the same silent failure mode: the retriever returns plausible-looking chunks, the LLM fills in the gaps, and nobody notices the fabrication until it matters. Mnemonics surfaces that problem at retrieval time, not after.
+
+## Install
+
+```bash
+pip install mnemonics
+# with verification support:
+pip install "mnemonics[verify]"
+```
+
+## Quick start
+
+```bash
+# Store something
+mnemonics ingest "The Eiffel Tower is 330 meters tall and located in Paris."
+
+# Retrieve with hallucination check
+mnemonics retrieve "how tall is the Eiffel Tower"
+# trust_score: 1.0  flagged: 0
+#   [0.912] The Eiffel Tower is 330 meters tall and located in Paris.
+```
+
+## Python API
+
+```python
+from mnemonics.store import Store
+from mnemonics.ingest import ingest
+from mnemonics.retrieve import retrieve
+
+store = Store("~/.mnemonics")
+
+ingest(["Paris is the capital of France.", "Rome is the capital of Italy."], store)
+
+result = retrieve("what is the capital of France", store, top_k=3, verify=True)
+for r in result["results"]:
+    flag = " FLAGGED" if r["flagged"] else ""
+    print(f"[{r['score']:.3f}]{flag} {r['text']}")
+
+print(f"trust_score: {result['trust_score']}")
+```
+
+## REST server
+
+```bash
+mnemonics serve --port 7810
+```
+
+| Method | Path | Body |
+|--------|------|------|
+| POST | `/ingest` | `{"texts": [...], "ns": "default"}` |
+| POST | `/retrieve` | `{"query": "...", "top_k": 5, "verify": true}` |
+| GET | `/health` | |
+| GET | `/namespaces` | |
+| GET | `/count?ns=default` | |
+| DELETE | `/memory/<id>` | |
+
+## MCP (Claude Code / Cursor / Metis)
+
+```bash
+mnemonics mcp
+```
+
+Add to your MCP config:
+
+```json
+{
+  "mcpServers": {
+    "mnemonics": {
+      "command": "mnemonics",
+      "args": ["mcp"]
+    }
+  }
+}
+```
+
+Tools exposed: `mnemonics_ingest`, `mnemonics_retrieve`, `mnemonics_forget`
+
+## Namespaces
+
+Isolate memories by project, user, or any key:
+
+```bash
+mnemonics ingest "project notes..." --ns work
+mnemonics retrieve "deadlines" --ns work
+```
+
+## Architecture
+
+```
+texts -> chunk (200w / 40w overlap) -> embed (all-MiniLM-L6-v2)
+      -> hnswlib cosine index (per namespace)
+      -> SQLite metadata store
+
+retrieve -> embed query -> knn search -> halluguard verify -> ranked results
+```
+
+Storage layout under `~/.mnemonics`:
+
+```
+memories.db        SQLite (text, meta, timestamps)
+index_default.bin  hnswlib index for "default" namespace
+index_<ns>.bin     one index per namespace
+```
+
+## Verification
+
+When `verify=True`, retrieved chunks are sent to a local halluguard daemon (port 7801) which cross-checks each result against the full retrieved corpus. Results that diverge get flagged and the aggregate `trust_score` drops.
+
+```bash
+pip install "mnemonics[verify]"
+halluguard serve &
+mnemonics retrieve "your query"  # auto-verifies
+```
+
+Verification is best-effort: if the daemon is not running, retrieval proceeds normally with `trust_score: 1.0`.
+
+## License
+
+MIT

mnemonics-0.1.0/README.md

@@ -0,0 +1,127 @@
+# Mnemonics
+
+**Your memory doesn't hallucinate.**
+
+Mnemonics is a local-first AI memory layer that stores, retrieves, and *verifies* what it returns. Built on sentence embeddings + HNSW vector search, with optional [halluguard](https://github.com/nakata-app/halluguard) verification to flag results that drift from the indexed corpus.
+
+## Why
+
+Every RAG pipeline has the same silent failure mode: the retriever returns plausible-looking chunks, the LLM fills in the gaps, and nobody notices the fabrication until it matters. Mnemonics surfaces that problem at retrieval time, not after.
+
+## Install
+
+```bash
+pip install mnemonics
+# with verification support:
+pip install "mnemonics[verify]"
+```
+
+## Quick start
+
+```bash
+# Store something
+mnemonics ingest "The Eiffel Tower is 330 meters tall and located in Paris."
+
+# Retrieve with hallucination check
+mnemonics retrieve "how tall is the Eiffel Tower"
+# trust_score: 1.0  flagged: 0
+#   [0.912] The Eiffel Tower is 330 meters tall and located in Paris.
+```
+
+## Python API
+
+```python
+from mnemonics.store import Store
+from mnemonics.ingest import ingest
+from mnemonics.retrieve import retrieve
+
+store = Store("~/.mnemonics")
+
+ingest(["Paris is the capital of France.", "Rome is the capital of Italy."], store)
+
+result = retrieve("what is the capital of France", store, top_k=3, verify=True)
+for r in result["results"]:
+    flag = " FLAGGED" if r["flagged"] else ""
+    print(f"[{r['score']:.3f}]{flag} {r['text']}")
+
+print(f"trust_score: {result['trust_score']}")
+```
+
+## REST server
+
+```bash
+mnemonics serve --port 7810
+```
+
+| Method | Path | Body |
+|--------|------|------|
+| POST | `/ingest` | `{"texts": [...], "ns": "default"}` |
+| POST | `/retrieve` | `{"query": "...", "top_k": 5, "verify": true}` |
+| GET | `/health` | |
+| GET | `/namespaces` | |
+| GET | `/count?ns=default` | |
+| DELETE | `/memory/<id>` | |
+
+## MCP (Claude Code / Cursor / Metis)
+
+```bash
+mnemonics mcp
+```
+
+Add to your MCP config:
+
+```json
+{
+  "mcpServers": {
+    "mnemonics": {
+      "command": "mnemonics",
+      "args": ["mcp"]
+    }
+  }
+}
+```
+
+Tools exposed: `mnemonics_ingest`, `mnemonics_retrieve`, `mnemonics_forget`
+
+## Namespaces
+
+Isolate memories by project, user, or any key:
+
+```bash
+mnemonics ingest "project notes..." --ns work
+mnemonics retrieve "deadlines" --ns work
+```
+
+## Architecture
+
+```
+texts -> chunk (200w / 40w overlap) -> embed (all-MiniLM-L6-v2)
+      -> hnswlib cosine index (per namespace)
+      -> SQLite metadata store
+
+retrieve -> embed query -> knn search -> halluguard verify -> ranked results
+```
+
+Storage layout under `~/.mnemonics`:
+
+```
+memories.db        SQLite (text, meta, timestamps)
+index_default.bin  hnswlib index for "default" namespace
+index_<ns>.bin     one index per namespace
+```
+
+## Verification
+
+When `verify=True`, retrieved chunks are sent to a local halluguard daemon (port 7801) which cross-checks each result against the full retrieved corpus. Results that diverge get flagged and the aggregate `trust_score` drops.
+
+```bash
+pip install "mnemonics[verify]"
+halluguard serve &
+mnemonics retrieve "your query"  # auto-verifies
+```
+
+Verification is best-effort: if the daemon is not running, retrieval proceeds normally with `trust_score: 1.0`.
+
+## License
+
+MIT

mnemonics-0.1.0/mnemonics/__init__.py

@@ -0,0 +1,7 @@
+"""mnemonics — verified AI memory. Retrieval that doesn't hallucinate."""
+from mnemonics.store import Store
+from mnemonics.ingest import ingest
+from mnemonics.retrieve import retrieve
+
+__all__ = ["Store", "ingest", "retrieve"]
+__version__ = "0.1.0"

mnemonics-0.1.0/mnemonics/cli.py

@@ -0,0 +1,82 @@
+"""mnemonics CLI."""
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+
+
+def main() -> None:
+    p = argparse.ArgumentParser(prog="mnemonics")
+    sub = p.add_subparsers(dest="cmd")
+
+    # serve
+    s = sub.add_parser("serve", help="Start REST server")
+    s.add_argument("--port", type=int, default=7810)
+    s.add_argument("--path", default="~/.mnemonics")
+
+    # mcp
+    sub.add_parser("mcp", help="Start MCP stdio server")
+
+    # ingest
+    i = sub.add_parser("ingest", help="Add text to memory")
+    i.add_argument("text", nargs="+")
+    i.add_argument("--ns", default="default")
+    i.add_argument("--path", default="~/.mnemonics")
+
+    # retrieve
+    r = sub.add_parser("retrieve", help="Search memory")
+    r.add_argument("query")
+    r.add_argument("--ns", default="default")
+    r.add_argument("--top-k", type=int, default=5)
+    r.add_argument("--no-verify", action="store_true")
+    r.add_argument("--path", default="~/.mnemonics")
+
+    # stats
+    st = sub.add_parser("stats", help="Show memory stats")
+    st.add_argument("--path", default="~/.mnemonics")
+
+    args = p.parse_args()
+
+    if args.cmd == "serve":
+        import os
+        os.environ["MNEMONICS_PATH"] = args.path
+        from mnemonics.server import serve
+        serve(port=args.port)
+
+    elif args.cmd == "mcp":
+        from mnemonics.server import serve
+        serve(mcp=True)
+
+    elif args.cmd == "ingest":
+        from mnemonics.store import Store
+        from mnemonics.ingest import ingest
+        store = Store(args.path)
+        n = ingest(texts=[" ".join(args.text)], store=store, ns=args.ns)
+        print(f"Stored {n} chunk(s).")
+
+    elif args.cmd == "retrieve":
+        from mnemonics.store import Store
+        from mnemonics.retrieve import retrieve
+        store = Store(args.path)
+        result = retrieve(
+            query=args.query,
+            store=store,
+            ns=args.ns,
+            top_k=args.top_k,
+            verify=not args.no_verify,
+        )
+        print(f"trust_score: {result['trust_score']}  flagged: {result['flagged_count']}")
+        for r in result["results"]:
+            flag = " ⚠" if r.get("flagged") else ""
+            print(f"  [{r['score']:.3f}]{flag} {r['text'][:120]}")
+
+    elif args.cmd == "stats":
+        from mnemonics.store import Store
+        store = Store(args.path)
+        for ns in store.list_namespaces():
+            print(f"  {ns}: {store.count(ns)} chunks")
+
+    else:
+        p.print_help()
+        sys.exit(1)

mnemonics-0.1.0/mnemonics/ingest.py

@@ -0,0 +1,63 @@
+"""Ingest text into the store: chunk → embed → save."""
+from __future__ import annotations
+
+import re
+from typing import Any
+
+import numpy as np
+
+from mnemonics.store import Store
+
+_encoder: Any = None
+_encoder_name: str = "all-MiniLM-L6-v2"
+
+
+def _get_encoder(model: str = _encoder_name) -> Any:
+    global _encoder, _encoder_name
+    if _encoder is None or model != _encoder_name:
+        from sentence_transformers import SentenceTransformer
+        _encoder = SentenceTransformer(model)
+        _encoder_name = model
+    return _encoder
+
+
+def _chunk(text: str, size: int = 200, overlap: int = 40) -> list[str]:
+    words = text.split()
+    if len(words) <= size:
+        return [text]
+    chunks = []
+    i = 0
+    while i < len(words):
+        chunk = " ".join(words[i:i + size])
+        chunks.append(chunk)
+        i += size - overlap
+    return chunks
+
+
+def ingest(
+    texts: list[str],
+    store: Store,
+    ns: str = "default",
+    meta: list[dict] | None = None,
+    model: str = "all-MiniLM-L6-v2",
+    chunk_size: int = 200,
+    chunk_overlap: int = 40,
+) -> int:
+    """Chunk, embed and store texts. Returns total chunks stored."""
+    enc = _get_encoder(model)
+    all_chunks: list[str] = []
+    all_meta: list[dict] = []
+
+    for i, text in enumerate(texts):
+        chunks = _chunk(text, chunk_size, chunk_overlap)
+        m = (meta[i] if meta else {}) | {"source_idx": i}
+        all_chunks.extend(chunks)
+        all_meta.extend([m] * len(chunks))
+
+    if not all_chunks:
+        return 0
+
+    vecs = enc.encode(all_chunks, batch_size=64, show_progress_bar=False,
+                      normalize_embeddings=True, convert_to_numpy=True)
+    store.add(all_chunks, vecs, ns=ns, meta=all_meta)
+    return len(all_chunks)

mnemonics-0.1.0/mnemonics/retrieve.py

@@ -0,0 +1,66 @@
+"""Retrieve memories — embed query, search, optionally verify with halluguard."""
+from __future__ import annotations
+
+from typing import Any
+
+import requests
+
+from mnemonics.store import Store
+from mnemonics.ingest import _get_encoder
+
+
+def retrieve(
+    query: str,
+    store: Store,
+    ns: str = "default",
+    top_k: int = 5,
+    model: str = "all-MiniLM-L6-v2",
+    verify: bool = True,
+    verify_threshold: float = 0.45,
+) -> dict[str, Any]:
+    """
+    Search the store for query.
+    If verify=True, runs halluguard on results against the retrieved corpus.
+    Returns: {results, verified, trust_score, flagged_count}
+    """
+    enc = _get_encoder(model)
+    qvec = enc.encode([query], normalize_embeddings=True, convert_to_numpy=True)[0]
+    results = store.search(qvec, ns=ns, top_k=top_k)
+
+    if not results:
+        return {"results": [], "verified": True, "trust_score": 1.0, "flagged_count": 0}
+
+    trust_score = 1.0
+    flagged_count = 0
+    verified = True
+
+    if verify:
+        try:
+            corpus = [r["text"] for r in results]
+            combined = " ".join(corpus)
+            resp = requests.post(
+                "http://127.0.0.1:7801/check",
+                json={"corpus": corpus, "answer": combined, "threshold": verify_threshold},
+                timeout=10,
+            )
+            if resp.ok:
+                data = resp.json()
+                trust_score = data.get("trust_score", 1.0)
+                flagged_count = data.get("n_flagged", 0)
+                verified = data.get("ok", True)
+                # Mark flagged results
+                flagged_texts = {f["text"] for f in data.get("flagged", [])}
+                for r in results:
+                    r["flagged"] = r["text"] in flagged_texts
+        except Exception:
+            pass  # halluguard daemon not running — skip verify
+
+    for r in results:
+        r.setdefault("flagged", False)
+
+    return {
+        "results": results,
+        "verified": verified,
+        "trust_score": round(trust_score, 4),
+        "flagged_count": flagged_count,
+    }