am-memory 0.1.0__tar.gz

am_memory-0.1.0/PKG-INFO

@@ -0,0 +1,573 @@
+Metadata-Version: 2.4
+Name: am-memory
+Version: 0.1.0
+Summary: Persistent memory for AI coding agents — SQLite-backed knowledge store with BM25+Vector search and MCP integration
+License: MIT
+Keywords: claude,mcp,memory,rag,claude-code
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+Requires-Dist: mcp>=1.0
+Provides-Extra: vector
+Requires-Dist: sqlite-vec; extra == "vector"
+Provides-Extra: dev
+Requires-Dist: pytest; extra == "dev"
+Requires-Dist: pytest-cov; extra == "dev"
+
+# am-memory
+
+Persistent memory for [Claude Code](https://claude.ai/code) — a self-evolving knowledge layer that survives across sessions, grows from every conversation, and surfaces relevant context automatically.
+
+```
+pip install am-memory
+am init
+# Restart Claude Code → done
+```
+
+No external database. No cloud service. A single SQLite file at `~/.am-memory/memory.db`.
+
+---
+
+## What problem does this solve?
+
+Every Claude Code session starts from zero. Context window fills up. You re-explain the same architecture, re-debug the same gotchas, re-type the same config values — session after session.
+
+`am-memory` gives Claude a memory that persists across sessions. Knowledge is captured automatically from your conversations and file edits. A debugging insight from six months ago surfaces when it's relevant today.
+
+---
+
+## Quick Start
+
+**Requirements:** Python 3.10+, Claude Code
+
+```bash
+pip install am-memory
+am init
+```
+
+`am init` does everything in one shot:
+
+```
+✓ Created ~/.am-memory/memory.db
+✓ Registered MCP server in ~/.claude/mcp.json
+✓ Installed hook: ~/.claude/hooks/SessionStart/am-memory.sh
+✓ Installed hook: ~/.claude/hooks/Stop/am-memory.sh
+✓ Appended memory instructions to ~/.claude/CLAUDE.md
+```
+
+Restart Claude Code. Done. Claude can now search and save memory across sessions.
+
+---
+
+## How it works
+
+### The big picture
+
+![System Overview](docs/img/overview.png)
+
+### Session lifecycle
+
+```
+Claude Code opens
+      │
+      ▼
+SessionStart hook
+  → am session start --project myproject
+  → writes session_id to state table
+      │
+      ▼
+Claude works (multiple turns)
+  → calls am_search before answering
+  → calls am_save when it learns something
+      │
+      ▼
+Claude Code closes (Stop hook)
+  → am session end
+  → quality-gate promote:
+      decisions non-empty  → P0 (never expires)
+      key_facts ≥ 3        → P1 (90 days)
+      else                 → P2 (30 days)
+  → session promoted to documents table
+  → raw session + messages deleted
+```
+
+---
+
+## Architecture: memory.db
+
+One SQLite file. Four semantic layers.
+
+```
+~/.am-memory/memory.db
+│
+│  ── SEARCH LAYER (virtual) ──────────────────────────────────────
+│
+├── documents_fts          FTS5 virtual table
+│                          trigram tokenizer (Chinese + English)
+│                          indexes: title, summary, key_facts, decisions
+│                          auto-synced via INSERT/UPDATE/DELETE triggers
+│                          BM25 ranking
+│
+├── vec_documents           sqlite-vec virtual table (optional)
+│                          float32[4096] HNSW index
+│                          cosine similarity
+│                          joined to documents via document_id FK
+│
+│  ── KNOWLEDGE LAYER ─────────────────────────────────────────────
+│
+├── documents              long-term knowledge store
+│   ├── doc_id             INTEGER PK
+│   ├── title              TEXT  — primary search signal, BM25 weight ×3
+│   ├── summary            TEXT  — 2-3 sentence distillation
+│   ├── key_facts          TEXT  — JSON array of extracted facts
+│   ├── decisions          TEXT  — JSON array of architectural decisions
+│   ├── code_sigs          TEXT  — function/class names mentioned
+│   ├── embedding          BLOB  — float32[4096], populated async
+│   ├── priority           P0 | P1 | P2
+│   ├── source             explicit | session_extract | hook | file
+│   ├── file_path          TEXT UNIQUE  — upsert key for file-sourced docs
+│   ├── expires_at         REAL  — NULL = never expires (P0)
+│   └── last_accessed_at   REAL  — updated on every search hit (LRU)
+│
+│  ── CONVERSATION LAYER ──────────────────────────────────────────
+│
+├── sessions               session lifecycle records
+│   ├── session_id         TEXT PK
+│   ├── project, topic     metadata
+│   ├── source             "cli:myproject"
+│   ├── summary            extracted at session end
+│   ├── key_facts          JSON array
+│   └── decisions          JSON array
+│
+├── messages               raw message storage (30-day TTL, auto-pruned)
+│
+│  ── WORKING MEMORY ──────────────────────────────────────────────
+│
+└── state                  K/V working memory
+    ├── current_session_id
+    ├── active_work        JSON: { ticket, topic, ... }
+    └── scratchpad         freeform per-session notes
+```
+
+---
+
+## Architecture: search
+
+Every `am_search` call flows through a 3.5-layer cascade. It stops at the first layer that returns results. The cheapest path is always tried first.
+
+![Search Cascade](docs/img/search.png)
+
+> **Score ranges are not comparable across layers.**
+> Sub-L1: 0.008–0.066 · Sub-L2: 1e-6–0.2 · Sub-L3: 4–500+
+> Detect origin by magnitude.
+
+---
+
+## Architecture: memory lifecycle
+
+Memory is not static. Three automated paths write to it continuously.
+
+![Memory Lifecycle](docs/img/lifecycle.png)
+
+### TTL + LRU
+
+TTL is derived automatically from `source`. Accessed documents have their TTL reset on every search hit — frequently used knowledge never expires.
+
+![TTL and LRU](docs/img/ttl.png)
+
+> **TTL only affects the search index.** Physical files in your knowledge base are never deleted. An expired document is unsearchable, but the file remains. Re-editing it via PostToolUse rebuilds the index entry automatically.
+
+---
+
+## Architecture: dream consolidation
+
+`am dream` runs background memory consolidation in 5 phases:
+
+```
+Phase 1: Orient    — read existing documents (top 100 recent)
+Phase 2: Gather    — collect ended sessions since last dream
+Phase 3: Consolidate — cross-session pattern detection + contradiction resolution
+Phase 3.5: Health Check — proactive knowledge base quality maintenance
+Phase 4: Prune     — remove expired documents
+```
+
+### Phase 3.5: Health Check
+
+The health check operates on the **entire knowledge base**, not just new sessions. It runs three sub-checks:
+
+**1. Staleness Detection**
+
+Scans non-P0 documents for `code_sigs` (function/class references) that no longer exist in the project directory. Documents with >50% dead references get downgraded:
+
+- P1 → `expires_at` set to 7 days (grace period)
+- P2 → `expires_at` set to now (pruned immediately in Phase 4)
+
+**2. Cross-Document Contradiction Scan**
+
+Extends contradiction detection across ALL documents (not just within a single consolidation batch). When two documents in the same project have conflicting facts (e.g., `timeout: 30s` vs `timeout: 600s`), the older fact is removed.
+
+**3. Redundancy Merge**
+
+Finds document pairs with >70% key_facts overlap (fuzzy Jaccard similarity) and merges the smaller into the larger. Unique facts and decisions from the source document transfer to the target before deletion.
+
+When 3+ documents cluster around the same topic, a **concept index** document is generated — a lightweight map linking related documents with shared themes.
+
+### Gating
+
+Dream only runs when:
+- `min_hours` (default 24) have passed since last dream
+- `min_sessions` (default 5) ended sessions exist since last dream
+- Use `--force` to bypass gating
+
+```bash
+am dream --force              # run now
+am dream --force --dry-run    # preview planned actions without writing
+am dream --status             # check gate state
+```
+
+---
+
+## MCP tools reference
+
+Claude calls these tools autonomously based on instructions injected by `am init` into `~/.claude/CLAUDE.md`.
+
+### `am_search`
+
+Search persistent memory for relevant documents.
+
+```
+Parameters:
+  query   string   required   Keywords, technical terms, or concepts
+  limit   integer  default 5  Max results to return
+
+Returns: structured context injection (~200 tokens)
+Latency: ~50ms (BM25 only) · ~500ms (with Ollama vector)
+```
+
+**When Claude calls it:** Before answering questions about past work, architecture decisions, debugging solutions, technical constraints.
+
+### `am_save`
+
+Save a piece of knowledge to persistent memory.
+
+```
+Parameters:
+  title    string  required  One-line title — primary search signal
+  content  string  required  Full context: facts, decisions, rationale
+  source   enum    required  architectural_decision | debug_solution | technical_insight
+                             | session_note | routine
+                             Determines TTL automatically — no priority field needed.
+
+Returns: doc_id of saved document
+```
+
+**When Claude calls it:** After discovering a non-obvious config constraint, an architectural decision, a bug gotcha worth remembering.
+
+### `am_state_get` / `am_state_set`
+
+Read/write K/V working memory (active task, scratchpad notes).
+
+```
+am_state_get  key: string
+am_state_set  key: string, value: any JSON-serializable
+```
+
+---
+
+## CLI reference
+
+```bash
+am init                              # one-time setup (run once after install)
+am update                            # upgrade to latest version
+
+am search --query "kafka rebalance"  # search documents (returns injection text)
+am search --query "..." --format json  # return raw JSON
+
+am doc save \                        # save a document
+  --title "bug title" \
+  --content "description" \
+  --source architectural_decision
+
+am state get --key active_work       # read working memory
+am state set --key scratchpad \
+             --value "current task"  # write working memory
+
+am session start --project myproj   # create session (prints session_id)
+am session end --session-id S       # end session + promote to documents
+am session list                      # list recent sessions (JSON)
+
+am mcp                               # start MCP server (stdio) — called by Claude Code
+```
+
+---
+
+## Optional: vector search with Ollama
+
+By default, am-memory uses BM25 full-text search (Sub-Layer 2). This handles ~80% of queries well — technical terms, config keys, function names, error codes.
+
+For fuzzy / semantic queries ("that kafka thing we fixed"), install Ollama and pull an embedding model:
+
+```bash
+# Install Ollama: https://ollama.com
+ollama pull qwen3-embedding:8b    # 4096-dim, ~5GB
+
+pip install 'am-memory[vector]'
+```
+
+When Ollama is running, searches automatically upgrade to hybrid BM25 + vector (Sub-Layer 1). When Ollama is offline, they fall back to BM25 silently. No configuration required.
+
+---
+
+## Real-world scenario: 8-hour session expiry
+
+Claude Code sessions expire after ~8 hours. Here is what that means in practice, using a real data pipeline debugging workflow.
+
+**Setup:** You are a engineer working on a Python ETL pipeline built on Airflow + Spark/YARN. The pipeline uses a CPL(YAML-driven config system) that generates DAGs at runtime. You are debugging a flaky Spark job submission.
+
+---
+
+### Day 1 — 09:00, session starts
+
+**Without am-memory:**
+```
+You:    The dan-sync Spark job keeps failing on YARN with a timeout.
+        Help me debug it.
+Claude: Sure — what does the error look like? What's your submission
+        config? What version of the orchestration layer are you on?
+(you spend 15 minutes re-explaining the stack)
+```
+
+**With am-memory:**
+```
+[SessionStart hook creates session silently]
+
+You: The dan-sync Spark job keeps failing on YARN with a timeout.
+
+[Claude calls am_search("dan-sync Spark YARN timeout")]
+→ hits:
+  [P0] "Spark/YARN restart API — polling and timeout config"
+       key_facts: ["15s polling interval", "800s timeout default",
+                   "HTTP API at /restart endpoint"]
+  [P1] "cpl-modules pipeline config — dan-sync"
+       key_facts: ["YAML → dtechclipy → cplapi → cplorchestration → Airflow",
+                   "Spark executor config in pipeline YAML"]
+
+Claude: The orchestration layer uses an HTTP restart API with 800s timeout
+        and 15s polling. Is the job hitting the 800s wall, or failing
+        before that? Check /restart endpoint response — it will tell you
+        whether YARN accepted the submission or rejected it upstream.
+```
+
+Zero re-explanation. Claude already knows your stack.
+
+---
+
+### Day 1 — 14:00, root cause found
+
+After 3 hours of debugging you find the issue: the Spark executor memory config in the pipeline YAML was using a soft limit that YARN silently ignores, causing the container to get killed without a clear error.
+
+**Without am-memory:**
+```
+Claude explains the fix. You apply it. The insight lives only in this session.
+```
+
+**With am-memory:**
+```
+Claude explains the fix, then calls am_save automatically:
+
+  title:   "YARN silently ignores Spark soft memory limit in pipeline YAML"
+  content: "executor_memory soft limit is not enforced by YARN —
+            container gets OOM-killed with no error message.
+            Fix: use hard limit field executor_memory_hard.
+            Affected: all cpl-modules Spark pipelines on YARN."
+  priority: P0   ← architectural constraint, never expires
+
+→ written to documents table permanently
+```
+
+---
+
+### Day 1 — 17:00, session expires (8-hour limit)
+
+**Without am-memory:**
+```
+[Session context window compresses / expires]
+
+Day 2, 09:00:
+You:    Continue where we left off on dan-sync.
+Claude: I don't have context from a previous session.
+        Could you describe the issue again?
+
+(15 minutes re-explaining, then another 30 minutes to re-derive
+ what you already found yesterday)
+```
+
+**With am-memory:**
+```
+[Stop hook fires silently]
+→ reads session messages
+→ extracts summary + key_facts + decisions
+→ quality gate: decisions non-empty → P0
+→ promotes to documents:
+    title:    "dan-sync YARN OOM debugging — Day 1"
+    summary:  "Spark executor soft memory limit silently ignored by YARN.
+               Container OOM-killed. Fix: use executor_memory_hard in YAML."
+    decisions: ["executor_memory_hard replaces executor_memory_soft",
+                "applies to all cpl-modules YARN pipelines"]
+→ raw session + messages deleted
+
+Day 2, 09:00 — new session starts:
+You: Continue where we left off on dan-sync.
+
+[Claude calls am_search("dan-sync YARN memory")]
+→ hits yesterday's promoted P0 doc + the fix from 14:00
+
+Claude: Yesterday's finding: YARN silently ignores the soft memory limit.
+        The fix is executor_memory_hard in the pipeline YAML.
+        Did you get to deploy it, or do you need to pick up from there?
+```
+
+Exact continuation. No re-explanation. No re-derivation.
+
+---
+
+### 3 weeks later — different pipeline, same platform
+
+**Without am-memory:**
+```
+A different Spark job starts failing with the same silent OOM pattern.
+→ 45 minutes debugging
+→ re-discover the same YAML config issue
+→ apply the same fix
+(every engineer re-learns the same lesson)
+```
+
+**With am-memory:**
+```
+You: This Airflow-triggered Spark job on YARN keeps dying silently.
+     No error in the logs.
+
+[Claude calls am_search("YARN silent failure Spark")]
+→ hits the P0 doc from 3 weeks ago
+
+Claude: Seen this before — YARN silently kills the container when
+        executor_memory soft limit is exceeded. No error message.
+        Check executor_memory_hard in your pipeline YAML.
+
+2 minutes to resolution instead of 45.
+```
+
+---
+
+### What actually accumulates over time
+
+```
+Week 1:   am init · 0 documents
+          First session — Claude saves 2 pipeline config decisions
+          → 2 documents
+
+Week 2:   You debug YARN, edit 8 pipeline YAML files
+          PostToolUse captures each edit → upsert by file_path
+          → 10 documents
+
+Week 3:   You delete 4 old sessions from dashboard
+          Each promoted with quality gate (P0/P1/P2)
+          → 14 documents
+
+Month 2:  Routine work — edits, sessions, decisions
+          → ~60 documents · growing
+
+Month 6:  ~200 documents · all searchable in <50ms
+          Claude answers "why did we move away from soft memory limits?"
+          with the exact rationale from a session that no longer exists
+          as a raw conversation — only the distilled decision survives.
+```
+
+
+## How knowledge grows over time
+
+```
+Day 1:  am init · 0 documents
+        ↓
+Day 1:  First session · Claude saves 2 decisions via <memory>
+        → 2 documents
+        ↓
+Day 3:  You edit 5 .py files · PostToolUse captures each
+        → 7 documents
+        ↓
+Day 7:  You delete a 30-message session from the dashboard
+        → session distilled → 1 session_extract P1 doc promoted
+        → 8 documents
+        ↓
+Day 30: Routine file edits, session work
+        → ~50 documents · ~40% vectorized (if Ollama installed)
+        ↓
+Day 90: 301 documents · 98% vectorized · <50ms search
+        Claude answers "why did we choose X?" with full rationale
+        from sessions that no longer exist as raw conversations.
+```
+
+---
+
+## How `am init` configures Claude Code
+
+After `am init`, three things change in your `~/.claude/` directory:
+
+**`mcp.json`** — registers the MCP server:
+```json
+{
+  "mcpServers": {
+    "am-memory": { "command": "am", "args": ["mcp"] }
+  }
+}
+```
+
+**`hooks/SessionStart/am-memory.sh`** — creates a session on startup:
+```bash
+SESSION_ID=$(am session start --project "$PROJECT" --source "cli:$PROJECT")
+am state set --key current_session_id --value "\"$SESSION_ID\""
+```
+
+**`hooks/Stop/am-memory.sh`** — ends and promotes session on exit:
+```bash
+SESSION_ID=$(am state get --key current_session_id | tr -d '"')
+am session end --session-id "$SESSION_ID"
+am state set --key current_session_id --value "null"
+```
+
+**`CLAUDE.md`** — tells Claude when to use the MCP tools:
+```markdown
+## Persistent Memory (am-memory)
+You have am_search and am_save MCP tools.
+• Before answering questions about past work → call am_search
+• After learning non-obvious facts → call am_save with P0/P1/P2
+```
+
+---
+
+## Updates
+
+```bash
+# Check current version
+am --version
+
+# Upgrade (pipx recommended)
+pipx upgrade am-memory
+
+# Or via am
+am update
+```
+
+am-memory checks PyPI once every 24 hours and prints a notice when a newer version is available.
+
+---
+
+## License
+
+MIT