lethe-memory 0.2.0__tar.gz → 0.2.2__tar.gz

lethe_memory-0.2.2/.release-please-manifest.json

@@ -0,0 +1,3 @@
+{
+  ".": "0.2.2"
+}

lethe_memory-0.2.2/ARCHITECTURE.md

@@ -0,0 +1,74 @@
+# Architecture
+
+## Retrieval pipeline
+
+```
+Query
+  │
+  ├── FAISS top-30 (dense vector similarity)
+  ├── BM25 top-30 (sparse keyword match)
+  │
+  └── Merge (RRF)
+        │
+        └── RIF suppression penalty (per-cluster, gap-based)
+              │
+              └── Cross-encoder rerank → top-k
+                    │
+                    └── Update suppression state, affinities, tier
+```
+
+**Optional write-time LLM enrichment layer** (`src/lethe/enrichment.py`): before indexing, each memory can be processed by an LLM (default `claude-haiku-4-5`) to produce a gist, 3 anticipated queries, entities, and temporal markers. All fields index alongside the original text; cross-encoder still scores against original. Attacks the vocabulary-mismatch failure mode.
+
+## RIF: technical details
+
+The formula that made it work (checkpoint 13 of 17):
+
+```
+competition_strength = max(0, xenc_rank - initial_rank) / pool_size × sigmoid(-xenc_score)
+suppression[eid, cluster] += learning_rate × competition_strength
+score_adjusted = base_score - alpha × suppression[eid, cluster]
+```
+
+The rank-gap term only penalises entries that **dropped in rank** between the initial hybrid retrieval and the cross-encoder rerank AND were actively rejected (negative xenc score). Entries that just lost a close race aren't suppressed. That prevents the system from forgetting legitimately useful context because it happened to rank second once.
+
+k-means runs once (30 clusters) on the bi-encoder query embedding at retrieval time; the cluster assignment is the "cue" coordinate. Based on Anderson's inhibition theory (1994) and the SAM competitive-sampling model (Raaijmakers & Shiffrin, 1981).
+
+## Storage layers
+
+| Layer | File | Purpose |
+|-------|------|---------|
+| DuckDB | `lethe.duckdb` | Entries, suppression state, rescue cache, stats. Many project DBs can be `ATTACH`ed simultaneously for cross-project search. |
+| numpy + FAISS | `embeddings.npz`, `faiss.index` | Vector storage |
+| BM25 | In-memory, rebuilt on startup | Sparse keyword index |
+
+## Entry lifecycle (germinal-center inspired)
+
+```
+NAIVE → GC → MEMORY
+              ↓
+         APOPTOTIC
+```
+
+- **Naive**: new entries, unproven
+- **GC**: retrieved 3+ times, actively evaluated
+- **Memory**: high affinity + frequently retrieved, stable, exempt from decay
+- **Apoptotic**: low affinity + idle > 1000 steps, excluded from search
+
+Useful for long-running agents; doesn't directly improve retrieval quality (that's what RIF and enrichment do).
+
+## Deduplication (on add)
+
+1. **Exact**: SHA-256 content hash (free)
+2. **Near-duplicate**: cosine similarity > 0.95 (keeps the longer entry)
+
+## Global search across projects
+
+Every `lethe index` registers the project in `~/.lethe/projects.json`. `lethe search --all` then ATTACHes each registered `lethe.duckdb` as a read-only schema and runs the full hybrid + RIF + cross-encoder pipeline across the union.
+
+```bash
+lethe search "mongodb pool" --all --top-k 5
+lethe projects list
+lethe projects prune
+```
+
+DuckDB has no cap on attached databases, so the 10/125 ceiling in SQLite's `ATTACH DATABASE` doesn't apply.

{lethe_memory-0.2.0 → lethe_memory-0.2.2}/CHANGELOG.md

@@ -1,5 +1,19 @@
 # Changelog
 
+## [0.2.2](https://github.com/teimurjan/lethe/compare/lethe-memory-v0.2.1...lethe-memory-v0.2.2) (2026-04-17)
+
+
+### Bug Fixes
+
+* Remove chunk_map.json, expand reads from DuckDB directly ([5cb8c84](https://github.com/teimurjan/lethe/commit/5cb8c843213d91b5c561e97a5b058d14d52b0652))
+
+## [0.2.1](https://github.com/teimurjan/lethe/compare/lethe-memory-v0.2.0...lethe-memory-v0.2.1) (2026-04-17)
+
+
+### Bug Fixes
+
+* Add README to PyPI, absolute logo URL, MIT license ([78188dd](https://github.com/teimurjan/lethe/commit/78188dd9642694f441f1a42a4ea99ae42a498dd7))
+
 ## [0.2.0](https://github.com/teimurjan/lethe/compare/lethe-memory-v0.1.0...lethe-memory-v0.2.0) (2026-04-17)
 
 

{lethe_memory-0.2.0 → lethe_memory-0.2.2}/CLAUDE.md

@@ -102,3 +102,7 @@ The 0.3680 is from BM25+vector+cross-encoder reranking (standard IR). The GC mec
 17 checkpoints. Checkpoints 1-10 (GC mechanisms): all failed. Checkpoint 11 (global RIF): +1.1%. Checkpoint 12 (clustered RIF): +5.8%. Checkpoint 13 (clustered + rank-gap): +6.5% NDCG, +9.5% recall@30 — retrieval-only best. Checkpoint 14 (exploration + rescue list): negative at full scale. Checkpoint 15 (SDM research prototype in `sdm/`): negative. Checkpoint 16 (extended metrics for checkpoint 13): RIF's gain is primarily from reducing wrong_family (−1.6pp); sibling_confusion and stale_fact unchanged. Checkpoint 17 (LLM enrichment layer, Haiku write-time gist + anticipated queries + entities + temporal markers): **+8.3pp NDCG on covered queries (+21% rel over RIF), largest single-lever gain in the journey**. Overall diluted to +1.2pp by partial coverage (15% of queries). Full corpus enrichment expected ~$16.
 
 Full research journey: [RESEARCH_JOURNEY.md](RESEARCH_JOURNEY.md)
+
+## git commits
+
+Use conventional commits. release-please is configured to only bump on `feat:` (minor) and `fix:` (patch). Use `feat:` or `fix:` only when you want a PyPI release. For everything else use a plain message with no prefix (release-please ignores non-conventional commits entirely).

lethe_memory-0.2.2/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Teimur Gasanov
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

lethe_memory-0.2.2/PKG-INFO

@@ -0,0 +1,97 @@
+Metadata-Version: 2.4
+Name: lethe-memory
+Version: 0.2.2
+Summary: Self-improving memory store for LLM agents: hybrid retrieval, clustered retrieval-induced forgetting, optional LLM enrichment
+License-Expression: MIT
+License-File: LICENSE
+Requires-Python: >=3.11
+Requires-Dist: anthropic>=0.30
+Requires-Dist: duckdb>=0.10
+Requires-Dist: faiss-cpu>=1.7
+Requires-Dist: fastembed>=0.4
+Requires-Dist: numpy>=1.24
+Requires-Dist: rank-bm25>=0.2
+Provides-Extra: benchmarks
+Requires-Dist: beir>=2.0; extra == 'benchmarks'
+Requires-Dist: datasets>=2.0; extra == 'benchmarks'
+Requires-Dist: matplotlib>=3.7; extra == 'benchmarks'
+Requires-Dist: sentence-transformers>=2.0; extra == 'benchmarks'
+Requires-Dist: tqdm>=4.60; extra == 'benchmarks'
+Provides-Extra: dev
+Requires-Dist: mypy>=1.0; extra == 'dev'
+Requires-Dist: pytest-cov>=4.0; extra == 'dev'
+Requires-Dist: pytest>=7.0; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# lethe
+
+<div align="center">
+    <img src="https://raw.githubusercontent.com/teimurjan/lethe/main/assets/logo.png" width="300" height="300" />
+</div>
+
+> *Λήθη: the ancient Greek personification of forgetfulness, and one of the five rivers of the underworld.*
+
+A memory store for LLM agents that **gets better the more you use it**. Hybrid BM25 + dense retrieval, cross-encoder reranking, clustered retrieval-induced forgetting (RIF), and optional LLM enrichment at write time.
+
+Most memory tools are static caches - you put strings in, you get strings back by similarity, and the retrieval function never changes. lethe is different: every retrieval teaches it which entries are chronic distractors for which kinds of queries, and it quietly suppresses them over time. No fine-tuning, no extra LLM calls - just bookkeeping inspired by how human memory actually works ([Anderson, 1994](https://psycnet.apa.org/record/1994-29917-001)).
+
+## Benchmark
+
+Numbers on the full 199,509-turn LongMemEval S corpus, **turn-level retrieval, NDCG@10, no leakage**. Most memory-tool benchmarks use ~50 sessions at session granularity - a ~2000× easier task. Those 99% numbers don't translate to this setup.
+
+| Stage | NDCG@10 | notes |
+|---|---|---|
+| Hybrid BM25 + vector (RRF) | 0.217 | basic retrieval (most popular) |
+| + cross-encoder reranking | 0.293 | +35% from semantic reranking |
+| + clustered+gap RIF (checkpoint 13) | **0.312** | +6.5% from retrieval-induced forgetting |
+| + LLM enrichment, on covered queries | **0.473** | +21% on the 75 queries where the answer turn was Haiku-enriched |
+
+Full methodology in [BENCHMARKS.md](BENCHMARKS.md). 17 checkpoints (10 failed) in [RESEARCH_JOURNEY.md](RESEARCH_JOURNEY.md).
+
+
+https://github.com/user-attachments/assets/ea6e9657-7525-4fed-ba40-df7a14e6b9f4
+
+
+## Install and quick start
+
+```bash
+pip install lethe-memory
+```
+
+```python
+from lethe import MemoryStore
+from sentence_transformers import SentenceTransformer, CrossEncoder
+
+store = MemoryStore(
+    "./my_memories",
+    bi_encoder=SentenceTransformer("all-MiniLM-L6-v2"),
+    cross_encoder=CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2"),
+)
+
+store.add("I prefer window seats on flights", session_id="trip")
+store.add("My wife needs aisle seats", session_id="trip")
+store.add("I work at Google as a software engineer", session_id="work")
+
+results = store.retrieve("What are my travel preferences?", k=5)
+for entry_id, content, score in results:
+    print(f"  [{score:.1f}] {content}")
+
+store.save()
+store.close()
+```
+
+As a CLI: `uv tool install lethe-memory && lethe --version`
+
+As a Claude Code plugin: `/plugin marketplace add teimurjan/lethe`
+
+See [plugins/claude-code/README.md](plugins/claude-code/README.md) for plugin details.
+
+## How it works
+
+[ARCHITECTURE.md](ARCHITECTURE.md) - pipeline diagram, RIF formula, storage layers, entry lifecycle, cross-project search.
+
+[RESEARCH_JOURNEY.md](RESEARCH_JOURNEY.md) - 17 checkpoints from biology-inspired mutation (all failed) through cognitive-science RIF (+6.5%) to LLM enrichment (+21% on covered queries).
+
+## License
+
+MIT

lethe_memory-0.2.2/README.md

@@ -0,0 +1,72 @@
+# lethe
+
+<div align="center">
+    <img src="https://raw.githubusercontent.com/teimurjan/lethe/main/assets/logo.png" width="300" height="300" />
+</div>
+
+> *Λήθη: the ancient Greek personification of forgetfulness, and one of the five rivers of the underworld.*
+
+A memory store for LLM agents that **gets better the more you use it**. Hybrid BM25 + dense retrieval, cross-encoder reranking, clustered retrieval-induced forgetting (RIF), and optional LLM enrichment at write time.
+
+Most memory tools are static caches - you put strings in, you get strings back by similarity, and the retrieval function never changes. lethe is different: every retrieval teaches it which entries are chronic distractors for which kinds of queries, and it quietly suppresses them over time. No fine-tuning, no extra LLM calls - just bookkeeping inspired by how human memory actually works ([Anderson, 1994](https://psycnet.apa.org/record/1994-29917-001)).
+
+## Benchmark
+
+Numbers on the full 199,509-turn LongMemEval S corpus, **turn-level retrieval, NDCG@10, no leakage**. Most memory-tool benchmarks use ~50 sessions at session granularity - a ~2000× easier task. Those 99% numbers don't translate to this setup.
+
+| Stage | NDCG@10 | notes |
+|---|---|---|
+| Hybrid BM25 + vector (RRF) | 0.217 | basic retrieval (most popular) |
+| + cross-encoder reranking | 0.293 | +35% from semantic reranking |
+| + clustered+gap RIF (checkpoint 13) | **0.312** | +6.5% from retrieval-induced forgetting |
+| + LLM enrichment, on covered queries | **0.473** | +21% on the 75 queries where the answer turn was Haiku-enriched |
+
+Full methodology in [BENCHMARKS.md](BENCHMARKS.md). 17 checkpoints (10 failed) in [RESEARCH_JOURNEY.md](RESEARCH_JOURNEY.md).
+
+
+https://github.com/user-attachments/assets/ea6e9657-7525-4fed-ba40-df7a14e6b9f4
+
+
+## Install and quick start
+
+```bash
+pip install lethe-memory
+```
+
+```python
+from lethe import MemoryStore
+from sentence_transformers import SentenceTransformer, CrossEncoder
+
+store = MemoryStore(
+    "./my_memories",
+    bi_encoder=SentenceTransformer("all-MiniLM-L6-v2"),
+    cross_encoder=CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2"),
+)
+
+store.add("I prefer window seats on flights", session_id="trip")
+store.add("My wife needs aisle seats", session_id="trip")
+store.add("I work at Google as a software engineer", session_id="work")
+
+results = store.retrieve("What are my travel preferences?", k=5)
+for entry_id, content, score in results:
+    print(f"  [{score:.1f}] {content}")
+
+store.save()
+store.close()
+```
+
+As a CLI: `uv tool install lethe-memory && lethe --version`
+
+As a Claude Code plugin: `/plugin marketplace add teimurjan/lethe`
+
+See [plugins/claude-code/README.md](plugins/claude-code/README.md) for plugin details.
+
+## How it works
+
+[ARCHITECTURE.md](ARCHITECTURE.md) - pipeline diagram, RIF formula, storage layers, entry lifecycle, cross-project search.
+
+[RESEARCH_JOURNEY.md](RESEARCH_JOURNEY.md) - 17 checkpoints from biology-inspired mutation (all failed) through cognitive-science RIF (+6.5%) to LLM enrichment (+21% on covered queries).
+
+## License
+
+MIT

{lethe_memory-0.2.0 → lethe_memory-0.2.2}/pyproject.toml

@@ -4,8 +4,10 @@ build-backend = "hatchling.build"
 
 [project]
 name = "lethe-memory"
-version = "0.2.0" # x-release-please-version
+version = "0.2.2" # x-release-please-version
 description = "Self-improving memory store for LLM agents: hybrid retrieval, clustered retrieval-induced forgetting, optional LLM enrichment"
+readme = "README.md"
+license = "MIT"
 requires-python = ">=3.11"
 dependencies = [
     "faiss-cpu>=1.7",

{lethe_memory-0.2.0 → lethe_memory-0.2.2}/src/lethe/__init__.py

@@ -1,6 +1,6 @@
 from __future__ import annotations
 
-__version__ = "0.2.0"  # x-release-please-version
+__version__ = "0.2.2"  # x-release-please-version
 
 from lethe.memory_store import MemoryStore
 from lethe.rif import RIFConfig

{lethe_memory-0.2.0 → lethe_memory-0.2.2}/src/lethe/cli.py

@@ -416,10 +416,15 @@ def _snippet(content: str, width: int = 160) -> str:
 
 def cmd_expand(args: argparse.Namespace) -> int:
     paths = resolve_paths(args.root)
-    from lethe.markdown_store import MarkdownStore
+    db_path = paths.index / "lethe.duckdb"
+    if not db_path.exists():
+        print(f"chunk {args.chunk_id!r} not found", file=sys.stderr)
+        return 1
+    from lethe.db import MemoryDB
 
-    md = MarkdownStore(memory_dir=paths.memory, index_dir=paths.index)
-    content = md.get_chunk(args.chunk_id)
+    db = MemoryDB(db_path)
+    content = db.get_content(args.chunk_id)
+    db.close()
     if content is None:
         print(f"chunk {args.chunk_id!r} not found", file=sys.stderr)
         return 1

{lethe_memory-0.2.0 → lethe_memory-0.2.2}/src/lethe/db.py

@@ -142,6 +142,12 @@ class MemoryDB:
         )
         return _rows_as_dicts(cur)
 
+    def get_content(self, entry_id: str) -> str | None:
+        row = self._conn.execute(
+            "SELECT content FROM entries WHERE id=?", [entry_id]
+        ).fetchone()
+        return str(row[0]) if row else None
+
     def delete_entry(self, entry_id: str) -> None:
         self._conn.execute("DELETE FROM entries WHERE id=?", [entry_id])
 

{lethe_memory-0.2.0 → lethe_memory-0.2.2}/src/lethe/markdown_store.py

@@ -16,7 +16,6 @@ on a directory of markdown files and a :class:`~lethe.memory_store.MemoryStore`.
 from __future__ import annotations
 
 import hashlib
-import json
 import re
 from dataclasses import dataclass
 from pathlib import Path
@@ -165,14 +164,11 @@ class MarkdownStore:
     so ``lethe expand`` can return the full section.
     """
 
-    CHUNK_MAP_FILENAME = "chunk_map.json"
-
     def __init__(self, memory_dir: Path, index_dir: Path) -> None:
         self.memory_dir = Path(memory_dir)
         self.index_dir = Path(index_dir)
         self.memory_dir.mkdir(parents=True, exist_ok=True)
         self.index_dir.mkdir(parents=True, exist_ok=True)
-        self._chunk_map_path = self.index_dir / self.CHUNK_MAP_FILENAME
 
     # ---- Scanning ----------------------------------------------------------
 
@@ -190,26 +186,6 @@ class MarkdownStore:
             out.extend(split_into_chunks(text, f))
         return out
 
-    # ---- Chunk map persistence --------------------------------------------
-
-    def _load_chunk_map(self) -> dict[str, str]:
-        if not self._chunk_map_path.exists():
-            return {}
-        try:
-            data = json.loads(self._chunk_map_path.read_text(encoding="utf-8"))
-        except (OSError, json.JSONDecodeError):
-            return {}
-        return {str(k): str(v) for k, v in data.items() if isinstance(v, str)}
-
-    def _save_chunk_map(self, chunk_map: dict[str, str]) -> None:
-        self._chunk_map_path.write_text(
-            json.dumps(chunk_map, ensure_ascii=False, indent=0),
-            encoding="utf-8",
-        )
-
-    def get_chunk(self, chunk_id: str) -> str | None:
-        return self._load_chunk_map().get(chunk_id)
-
     # ---- Reindex -----------------------------------------------------------
 
     def reindex(self, store: MemoryStore) -> dict[str, int]:
@@ -221,7 +197,6 @@ class MarkdownStore:
         """
         chunks = self.scan()
         current_ids = {c.id for c in chunks}
-        previous = self._load_chunk_map()
 
         added = 0
         unchanged = 0
@@ -239,10 +214,9 @@ class MarkdownStore:
             if inserted is not None:
                 added += 1
             else:
-                # Dedup suppressed the insert (exact hash collision or near-dup).
                 unchanged += 1
 
-        for old_id in set(previous) - current_ids:
+        for old_id in set(store.entries) - current_ids:
             if old_id in store.entries:
                 store.db.delete_entry(old_id)
                 store.entries.pop(old_id, None)
@@ -252,9 +226,6 @@ class MarkdownStore:
         if removed:
             store._rebuild_index()
 
-        chunk_map = {c.id: c.content for c in chunks}
-        self._save_chunk_map(chunk_map)
-
         return {
             "added": added,
             "removed": removed,

{lethe_memory-0.2.0 → lethe_memory-0.2.2}/tests/test_cli.py

@@ -170,7 +170,6 @@ def test_expand_prints_chunk(
     assert rc == 0
     out = capsys.readouterr().out
     assert "window seats" in out
-    assert "session:s" in out
 
 
 def test_expand_missing_chunk_returns_error(

{lethe_memory-0.2.0 → lethe_memory-0.2.2}/tests/test_markdown_store.py

@@ -135,7 +135,7 @@ def test_scan_reads_all_md_files_in_order(tmp_path: Path) -> None:
     assert [c.heading for c in chunks] == ["A", "B"]
 
 
-def test_reindex_adds_new_chunks_and_persists_chunk_map(
+def test_reindex_adds_new_chunks(
     tmp_path: Path, mock_bi_encoder, mock_cross_encoder
 ) -> None:
     memory = tmp_path / "memory"
@@ -157,10 +157,7 @@ def test_reindex_adds_new_chunks_and_persists_chunk_map(
         store.close()
 
     assert counts == {"added": 2, "removed": 0, "unchanged": 0, "total": 2}
-    # chunk map file exists and has both chunk ids
-    chunk_map = md._load_chunk_map()
-    assert len(chunk_map) == 2
-    assert all(len(cid) == 16 for cid in chunk_map)
+    assert store.size == 2
 
 
 def test_reindex_is_idempotent(
@@ -251,21 +248,25 @@ def test_get_chunk_returns_full_markdown(
     )
     try:
         md.reindex(store)
+        [chunk] = md.scan()
+        expanded = store.db.get_content(chunk.id)
+        assert expanded is not None
+        assert "bullet" in expanded
     finally:
         store.close()
 
-    [chunk] = md.scan()
-    expanded = md.get_chunk(chunk.id)
-    assert expanded is not None
-    assert "bullet" in expanded
-    assert parse_anchor(expanded) == {
-        "session": "s", "turn": "t", "transcript": "/x",
-    }
-
 
-def test_get_chunk_returns_none_for_unknown_id(tmp_path: Path) -> None:
-    md = MarkdownStore(memory_dir=tmp_path / "memory", index_dir=tmp_path / "index")
-    assert md.get_chunk("deadbeef" * 2) is None
+def test_get_content_returns_none_for_unknown_id(
+    tmp_path: Path, mock_bi_encoder, mock_cross_encoder,
+) -> None:
+    store = MemoryStore(
+        tmp_path / "store",
+        bi_encoder=mock_bi_encoder,
+        cross_encoder=mock_cross_encoder,
+        dim=mock_bi_encoder.dim,
+    )
+    assert store.db.get_content("deadbeef" * 2) is None
+    store.close()
 
 
 def test_chunk_anchor_property_pulls_from_content() -> None:

lethe_memory-0.2.0/.github/workflows/publish-pypi.yml

@@ -1,40 +0,0 @@
-name: publish
-
-on:
-  release:
-    types: [published]
-
-jobs:
-  build:
-    name: build sdist + wheel
-    runs-on: ubuntu-latest
-    steps:
-      - uses: actions/checkout@v4
-      - uses: actions/setup-python@v5
-        with:
-          python-version: "3.12"
-      - name: Install build tooling
-        run: python -m pip install --upgrade build
-      - name: Build distributions
-        run: python -m build
-      - uses: actions/upload-artifact@v4
-        with:
-          name: dist
-          path: dist/
-          if-no-files-found: error
-
-  publish:
-    name: upload to PyPI (trusted publishing)
-    needs: build
-    runs-on: ubuntu-latest
-    environment:
-      name: pypi
-      url: https://pypi.org/p/lethe-memory
-    permissions:
-      id-token: write
-    steps:
-      - uses: actions/download-artifact@v4
-        with:
-          name: dist
-          path: dist/
-      - uses: pypa/gh-action-pypi-publish@release/v1

lethe_memory-0.2.0/.release-please-manifest.json

@@ -1,3 +0,0 @@
-{
-  ".": "0.2.0"
-}

lethe_memory-0.2.0/PKG-INFO

@@ -1,21 +0,0 @@
-Metadata-Version: 2.4
-Name: lethe-memory
-Version: 0.2.0
-Summary: Self-improving memory store for LLM agents: hybrid retrieval, clustered retrieval-induced forgetting, optional LLM enrichment
-Requires-Python: >=3.11
-Requires-Dist: anthropic>=0.30
-Requires-Dist: duckdb>=0.10
-Requires-Dist: faiss-cpu>=1.7
-Requires-Dist: fastembed>=0.4
-Requires-Dist: numpy>=1.24
-Requires-Dist: rank-bm25>=0.2
-Provides-Extra: benchmarks
-Requires-Dist: beir>=2.0; extra == 'benchmarks'
-Requires-Dist: datasets>=2.0; extra == 'benchmarks'
-Requires-Dist: matplotlib>=3.7; extra == 'benchmarks'
-Requires-Dist: sentence-transformers>=2.0; extra == 'benchmarks'
-Requires-Dist: tqdm>=4.60; extra == 'benchmarks'
-Provides-Extra: dev
-Requires-Dist: mypy>=1.0; extra == 'dev'
-Requires-Dist: pytest-cov>=4.0; extra == 'dev'
-Requires-Dist: pytest>=7.0; extra == 'dev'

lethe_memory-0.2.0/README.md

@@ -1,288 +0,0 @@
-# lethe
-
-<div align="center">
-    <img src="./assets/logo.png" width="300" height="300" />
-</div>
-
-> *Λήθη: the ancient Greek personification of forgetfulness, and one of the five rivers of the underworld.*
-
-A memory store for LLM agents. Hybrid BM25 + dense retrieval, cross-encoder reranking, **clustered retrieval-induced forgetting (RIF)**, and an optional LLM enrichment layer at write time.
-
-## Read this before the benchmark numbers
-
-Most memory-tool READMEs advertise NDCG or recall in the 95 to 99% range. Those numbers come from evaluating on a per-query fresh database of ~50 sessions, at session granularity, with recall@5. That's a roughly 2000× easier task than "find this specific turn among 200,000". Some implementations additionally leak ground-truth annotations into the index.
-
-lethe's numbers are reported on the full 199,509-turn LongMemEval S corpus, **turn-level retrieval, NDCG@10, no leakage**. Random baseline is 0.005%. Vector-only search gets 0.14. BM25 alone gets 0.24. When other tools are run on *this* setup, they land in the same 0.1 to 0.4 range. The 99% numbers don't translate.
-
-No 99%. No cherry-picking. These are the honest numbers on a hard benchmark:
-
-| Stage | NDCG@10 | notes |
-|---|---|---|
-| Hybrid BM25 + vector (RRF) | 0.217 | basic retrieval (most popular) |
-| + cross-encoder reranking | 0.293 | +35% from semantic reranking |
-| + clustered+gap RIF (checkpoint 13) | **0.312** | +6.5% from retrieval-induced forgetting |
-| + LLM enrichment, on covered queries | **0.473** | +21% on the 75 queries where the answer turn was Haiku-enriched |
-
-Enrichment only helps where it's been applied; the full-corpus number is diluted by the uncovered 425 queries. Full methodology in [BENCHMARKS.md](BENCHMARKS.md). The research path, including 10 failed approaches before anything worked, is in [RESEARCH_JOURNEY.md](RESEARCH_JOURNEY.md).
-
-
-https://github.com/user-attachments/assets/ea6e9657-7525-4fed-ba40-df7a14e6b9f4
-
-> Lethe full stack vs basic hybrid RRF on the 199,509-turn LongMemEval corpus. NDCG@10 over 5,000 queries.
-
-## What's new here: retrieval-induced forgetting
-
-RIF is a well-studied phenomenon in cognitive psychology (Anderson, 1994). In plain words:
-
-> *When you retrieve one thing, related things that could have been retrieved but weren't get quietly suppressed. Ask someone to name a fruit; they say "apple". A few minutes later, "pear" and "orange" are measurably harder for them to recall, even though nothing asked about them. Remembering is also an act of forgetting.*
-
-lethe brings this mechanism to vector memory:
-
-- Every retrieval surfaces a candidate pool (BM25 ∪ vector top-k).
-- The cross-encoder picks the winners.
-- Entries that reached the pool but lost don't disappear. They accumulate a per-cluster **suppression score**.
-- On future retrievals in the same query cluster ("travel-like queries", "debugging-like queries"), those entries get their scores penalised *before* the cross-encoder sees them, freeing slots for entries that were previously crowded out.
-
-The key detail: suppression is **cue-dependent, not global**. An entry suppressed for travel queries stays fully available for food queries. Global suppression was ~5× weaker in our benchmarks. The clustering is what made it work.
-
-## Why this matters: a memory that evolves
-
-Most memory stores are static caches. You put strings in; you retrieve them by similarity. The retrieval function never changes.
-
-RIF makes retrieval behaviour **adapt over time without retraining any model**:
-
-- Entries that keep losing to more relevant alternatives get quieter in their own cluster.
-- Entries that keep winning get amplified via tier promotion (naive → gc → memory) and exemption from decay.
-- The system learns "in the context of *this kind* of query, you usually mean *that* chunk, not *this* one" from its own retrieval history. No fine-tuning. No extra LLM calls. Just O(n_clusters × n_entries) bookkeeping.
-
-Concrete example: you store both *"I prefer window seats on flights"* and *"my wife needs aisle seats"*. Travel queries keep picking your preference first. The system quietly penalises your wife's preference for travel-sounding queries, not globally, just in that cluster. Ask about your spouse's preferences and both entries remain equally findable.
-
-Checkpoint 13 (clustered + rank-gap RIF) gives +6.5% NDCG and +9.5% recall@30 on LongMemEval over the baseline hybrid pipeline. As far as I can find, this is the first implementation of clustered RIF in a production retrieval system.
-
-## Claude Code plugin
-
-Lethe ships as a Claude Code plugin that persists memory across sessions per-project:
-
-```
-/plugin marketplace add teimurjan/lethe
-/plugin install lethe
-```
-
-Hooks write daily markdown to `.lethe/memory/YYYY-MM-DD.md`, summarize each turn with `claude -p --model haiku` (no extra API key), reindex via the `lethe` CLI, and a `memory-recall` skill surfaces prior-session context on demand.
-
-See [plugins/claude-code/README.md](plugins/claude-code/README.md) for hook behavior, CLI reference, and local-install instructions.
-
-## Quick start
-
-```python
-from lethe import MemoryStore
-from sentence_transformers import SentenceTransformer, CrossEncoder
-
-store = MemoryStore(
-    "./my_memories",
-    bi_encoder=SentenceTransformer("all-MiniLM-L6-v2"),
-    cross_encoder=CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2"),
-)
-
-store.add("I prefer window seats on flights", session_id="trip")
-store.add("My wife needs aisle seats", session_id="trip")
-store.add("I work at Google as a software engineer", session_id="work")
-
-results = store.retrieve("What are my travel preferences?", k=5)
-for entry_id, content, score in results:
-    print(f"  [{score:.1f}] {content}")
-
-store.save()
-store.close()
-```
-
-## Architecture
-
-```
-Query
-  │
-  ├── FAISS top-30 (dense vector similarity)
-  ├── BM25 top-30 (sparse keyword match)
-  │
-  └── Merge (RRF)
-        │
-        └── RIF suppression penalty (per-cluster, gap-based)
-              │
-              └── Cross-encoder rerank → top-k
-                    │
-                    └── Update suppression state, affinities, tier
-```
-
-**Optional write-time LLM enrichment layer** (`src/lethe/enrichment.py`): before indexing, each memory can be processed by an LLM (default `claude-haiku-4-5`) to produce a gist, 3 anticipated queries, entities, and temporal markers. All fields index alongside the original text; cross-encoder still scores against original. Attacks the vocabulary-mismatch failure mode.
-
-### RIF: technical details
-
-The formula that made it work (checkpoint 13 of 17):
-
-```
-competition_strength = max(0, xenc_rank - initial_rank) / pool_size × sigmoid(-xenc_score)
-suppression[eid, cluster] += learning_rate × competition_strength
-score_adjusted = base_score - alpha × suppression[eid, cluster]
-```
-
-The rank-gap term only penalises entries that **dropped in rank** between the initial hybrid retrieval and the cross-encoder rerank AND were actively rejected (negative xenc score). Entries that just lost a close race aren't suppressed. That prevents the system from forgetting legitimately useful context because it happened to rank second once.
-
-k-means runs once (30 clusters) on the bi-encoder query embedding at retrieval time; the cluster assignment is the "cue" coordinate. Based on Anderson's inhibition theory (1994) and the SAM competitive-sampling model (Raaijmakers & Shiffrin, 1981).
-
-### Three storage layers
-
-| Layer | File | Purpose |
-|-------|------|---------|
-| DuckDB | `lethe.duckdb` | Entries, suppression state, rescue cache, stats. Many project DBs can be `ATTACH`ed simultaneously for cross-project search. |
-| numpy + FAISS | `embeddings.npz`, `faiss.index` | Vector storage |
-| BM25 | In-memory, rebuilt on startup | Sparse keyword index |
-
-### Entry lifecycle (germinal-center inspired)
-
-```
-NAIVE → GC → MEMORY
-              ↓
-         APOPTOTIC
-```
-
-- **Naive**: new entries, unproven
-- **GC**: retrieved 3+ times, actively evaluated
-- **Memory**: high affinity + frequently retrieved, stable, exempt from decay
-- **Apoptotic**: low affinity + idle > 1000 steps, excluded from search
-
-Useful for long-running agents; doesn't directly improve retrieval quality (that's what RIF and enrichment do).
-
-### Deduplication (on add)
-
-1. **Exact**: SHA-256 content hash (free)
-2. **Near-duplicate**: cosine similarity > 0.95 (keeps the longer entry)
-
-## Install
-
-From PyPI (the distribution name is `lethe-memory`; the import name is still `lethe`):
-
-```bash
-pip install lethe-memory
-# or
-uv pip install lethe-memory
-```
-
-As the `lethe` CLI on PATH without a venv:
-
-```bash
-uv tool install lethe-memory
-lethe --version
-```
-
-From source, for development:
-
-```bash
-git clone https://github.com/teimurjan/lethe && cd lethe
-uv venv --python 3.12 && uv pip install -e .
-```
-
-As a Claude Code plugin (see [plugin README](plugins/claude-code/README.md)):
-
-```
-/plugin marketplace add teimurjan/lethe
-/plugin install lethe
-```
-
-## Global search across projects
-
-Every `lethe index` registers the project in `~/.lethe/projects.json`. `lethe search --all` then ATTACHes each registered `lethe.duckdb` as a read-only schema and runs the full hybrid + RIF + cross-encoder pipeline across the union.
-
-```bash
-# In each project you care about:
-lethe index
-
-# Search across all registered projects:
-lethe search "mongodb pool" --all --top-k 5
-# [p_site_fix_pentest_8a99] [+4.63] 06e6...  - MongoDB connection pool issue at 100% rollout…
-# [p_lethe_77be]            [+3.41] 4670...  - lethe search returning real hybrid results…
-
-# Restrict to a subset:
-lethe search "auth flow" --projects p_site_fix_pentest_8a99,p_api_server_12ab
-
-# Inspect / manage the registry:
-lethe projects list
-lethe projects add /path/to/other-project
-lethe projects remove <slug-or-path>
-lethe projects prune   # drop entries whose roots no longer exist
-```
-
-Opt out of auto-registration in three ways:
-- Per invocation: `lethe index --no-register`
-- Per project: `lethe config set auto_register false`
-- Globally: `export LETHE_DISABLE_GLOBAL_REGISTRY=1`
-
-The registry only stores absolute project paths, never contents. Cross-project search is read-only; it does not mutate RIF state. Local `lethe search` continues to learn as normal.
-
-DuckDB has no cap on attached databases, so the 10/125 ceiling in SQLite's `ATTACH DATABASE` doesn't apply. Real-world bottleneck before you'd feel it is the per-project BM25 rebuild, which scales linearly.
-
-## Benchmark
-
-```bash
-# prep LongMemEval
-uv run python experiments/data_prep.py --dataset longmemeval
-
-# retrieval-only baseline + RIF variants
-uv run python benchmarks/run_benchmark.py
-uv run python benchmarks/run_rif_benchmark.py
-
-# LLM enrichment layer (needs ANTHROPIC_API_KEY)
-export ANTHROPIC_API_KEY=sk-ant-...
-uv run python experiments/enrich_longmemeval.py     # one-time, ~$16 for 10k entries
-uv run python benchmarks/run_rif_enriched.py         # 3-arm benchmark
-```
-
-See [BENCHMARKS.md](BENCHMARKS.md) for results tables and methodology details.
-
-## Project structure
-
-```
-src/lethe/             # Production library (162 tests, ~95% coverage)
-├── memory_store.py
-├── markdown_store.py  # Markdown chunker for .lethe/memory/*.md
-├── cli.py             # `lethe` console script
-├── union_store.py     # Cross-project read-only search via DuckDB ATTACH
-├── db.py              # DuckDB persistence
-├── vectors.py         # FAISS + BM25 index
-├── reranker.py        # Cross-encoder + adaptive depth
-├── rif.py             # Retrieval-induced forgetting (clustered + gap)
-├── enrichment.py      # Optional LLM write-time enrichment (Anthropic SDK)
-├── dedup.py           # Hash + cosine deduplication
-└── entry.py           # MemoryEntry dataclass + Tier enum
-
-plugins/claude-code/   # Claude Code plugin (hooks + memory-recall skill)
-.claude-plugin/        # Marketplace manifest
-
-benchmarks/            # Per-checkpoint benchmark scripts
-├── run_*.py
-├── _lib/              # Benchmark-only helpers (metrics, etc.)
-└── results/           # Raw per-run output markdowns
-
-scripts/               # Reproducibility utilities (dataset prep, enrichment runner)
-
-research/              # Experimental / research code (not production)
-├── gc_mutation/       # Germinal-center mutation thread (checkpoints 1-10)
-└── sdm/               # Sparse Distributed Memory prototype (checkpoint 15)
-
-tests/                 # Production unit tests
-```
-
-## Research background
-
-This project started as an experiment porting the immune system's germinal-center mechanism to vector memory. 17 checkpoints so far:
-
-1. **Checkpoints 1-10** (GC-mutation approaches): all failed to improve retrieval quality. Useful for lifecycle management, not retrieval.
-2. **Checkpoints 11-13** (RIF): cognitive-science inspired. Clustered + gap-formula variant gives +6.5% NDCG, +9.5% recall@30 on the full 500-query eval. First positive learned-retrieval result.
-3. **Checkpoints 14-15** (exploration/rescue, Sparse Distributed Memory prototype): both negative at full scale.
-4. **Checkpoint 16** (extended behavior metrics for checkpoint 13): RIF's gain is primarily from reducing cross-topic retrieval (−1.6pp wrong_family); sibling confusion and stale-fact rate unchanged.
-5. **Checkpoint 17** (LLM enrichment layer): write-time structured extraction via Haiku. On covered queries: +8.3pp NDCG over checkpoint 13, the largest single-lever gain in the journey. Overall diluted by partial coverage; scaling to full coverage is pending.
-
-Full journey in [RESEARCH_JOURNEY.md](RESEARCH_JOURNEY.md).
-
-## License
-
-MIT.