@rbalchii/anchor-engine 4.7.0

package/README.md

@@ -0,0 +1,317 @@
+# Anchor Engine ⚓
+
+**Deterministic semantic memory for LLMs – local-first, graph traversal, <1GB RAM**
+
+[![GitHub release](https://img.shields.io/github/v/release/RSBalchII/anchor-engine-node)](https://github.com/RSBalchII/anchor-engine-node/releases)
+[![License: AGPL v3](https://img.shields.io/badge/License-AGPL%20v3-blue.svg)](https://www.gnu.org/licenses/agpl-3.0)
+[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](http://makeapullrequest.com)
+[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.18841399.svg)](https://doi.org/10.5281/zenodo.18841399)
+
+---
+
+## 🌟 Why Anchor Engine?
+
+Modern AI memory is broken.
+
+Vector databases demand GPUs, gigabytes of RAM, and cloud infrastructure. They're opaque, expensive, and fundamentally incompatible with personal, local‑first AI.
+
+**Anchor Engine takes a different path.**
+
+It's a **deterministic, explainable, CPU‑only memory engine** that runs on anything—from a $200 laptop to a workstation—and retrieves context using a physics‑inspired graph algorithm instead of dense vectors.
+
+If you want:
+- **Local‑first AI** – your data stays yours
+- **Explainable retrieval** – know *why* something was returned
+- **Deterministic results** – same query, same answer, every time
+- **Zero cloud dependency** – no API keys, no servers
+- **<1GB RAM usage** – runs alongside your browser
+- **High‑speed ingestion** – 100MB in minutes
+
+…then Anchor Engine is built for you.
+
+---
+
+## ⚡ Quick Start (5 Minutes)
+
+```bash
+# 1. Clone & Install
+git clone https://github.com/RSBalchII/anchor-engine-node.git
+cd anchor-engine-node
+pnpm install
+pnpm build
+
+# 2. Start the engine
+pnpm start
+
+# 3. Open your browser
+open http://localhost:3160
+```
+
+That's it! You now have a sovereign memory system for your LLM.
+
+---
+
+## 📊 Anchor Engine vs. Vector RAG
+
+| Feature | Anchor Engine | Vector RAG |
+|---------|---------------|------------|
+| **Hardware** | CPU‑only | GPU preferred |
+| **RAM Usage** | <1GB | 4–8GB |
+| **Explainability** | Native (tags, hops, decay) | None (black box) |
+| **Deterministic** | ✅ Yes | ❌ No |
+| **Cloud Required** | ❌ No | Often |
+| **Retrieval Complexity** | O(k·d̄) linear | O(n log n) |
+| **License** | AGPL-3.0 (open) | Varies (often proprietary) |
+
+---
+
+## 🧠 The STAR Algorithm
+
+Anchor Engine uses **STAR** (Semantic Temporal Associative Retrieval)—a physics‑inspired scoring equation that combines:
+
+```
+W(q,a) = |T(q) ∩ T(a)| · γ^d(q,a)  ×  e^(−λΔt)  ×  (1 − H(h_q, h_a)/64)
+         ↑ Semantic Gravity         ↑ Temporal Decay   ↑ Structural Gravity
+```
+
+| Component | What It Does |
+|-----------|--------------|
+| **Semantic Gravity** | Shared tags × hop‑distance damping (γ = 0.85) |
+| **Temporal Decay** | Recent memories pull harder (half-life ~115 min) |
+| **Structural Gravity** | SimHash proximity (64-bit fingerprints) |
+
+**Result:** O(k·d̄) retrieval—dramatically faster than vector ANN for personal datasets. Every atom includes **provenance**: shared tags, hop distance, recency, and byte‑range pointers.
+
+---
+
+## ⚓ Why "Anchor"?
+
+> **Drop a query into your memory graph.**  
+> **The Anchor finds the semantic bottom.**  
+> **The chain plays out to your chosen radius.**  
+> **What's retrieved is what's relevant—nothing more, nothing less.**  
+> **Same anchor, same spot, same result. Every time.**  
+> **This isn't a metaphor. It's the algorithm.**
+
+| Anchor Quality | How It Maps to STAR |
+|----------------|---------------------|
+| **Drops to a point** | Query finds seed atoms (anchor discovery) |
+| **Defines a radius** | Chain length = token budget (hop distance) |
+| **Holds against drift** | Deterministic retrieval (no semantic drift) |
+| **Connects to the ship** | Retrieved context → LLM (your ship) |
+| **Works in deep water** | Scales from 10MB to 10GB graphs |
+
+The name isn't marketing. It's architecture.
+
+---
+
+## 📥 Ingest Your Data
+
+### Option 1: Web UI (Easiest)
+1. Open http://localhost:3160
+2. Click **"Manage Paths"** → Add folders to watch
+3. Or use **"Paste & Ingest"** tab for quick text
+
+### Option 2: API
+```bash
+curl -X POST http://localhost:3160/v1/research/upload-raw \
+  -H "Content-Type: application/json" \
+  -d '{
+    "content": "Your text here...",
+    "filename": "notes.md",
+    "bucket": "inbox"
+  }'
+```
+
+### Option 3: MCP (Claude, Cursor, Qwen Code)
+```
+/anchor_ingest_text content="Meeting notes..." filename="meeting.md" bucket="inbox"
+```
+
+---
+
+## 🔍 Search Your Memory
+
+### Web UI
+- Natural language queries
+- Adjustable token budget slider
+- Filter by buckets/tags
+
+### API
+```bash
+curl -X POST http://localhost:3160/v1/memory/search \
+  -H "Content-Type: application/json" \
+  -d '{
+    "query": "What did we discuss about OAuth?",
+    "token_budget": 2048
+  }'
+```
+
+### MCP Tools
+```
+/anchor_query query="OAuth authentication setup"
+/anchor_search_index query="career planning"
+/anchor_fetch_session session_id="abc-123"
+```
+
+---
+
+## 🏛️ Our Philosophy: AI Memory Should Work Like Your Brain
+
+Human memory is remarkably efficient. It runs on ~20 watts, forgets irrelevant details, and over time clarifies core truths rather than drowning in noise. It doesn't store raw experiences—it stores *patterns*, *relationships*, and *meaning*.
+
+Most AI memory systems do the opposite: they hoard data, brute‑force compute similarity, and require massive infrastructure.
+
+**Anchor Engine was built on a different premise: AI memory should work like the human mind—lightweight, connected, and self‑clarifying.**
+
+| Principle | What It Means | How Anchor Implements It |
+|-----------|---------------|--------------------------|
+| **🧠 Forgetting is a feature** | The brain forgets constantly, leaving only what matters | `distill:` command removes redundancy; temporal decay |
+| **🔗 Meaning lives in relationships** | We store how concepts connect, not isolated facts | Graph model with typed edges; STAR algorithm |
+| **⚡ Low power, high efficiency** | The brain achieves its magic on ~20 watts | Pointer‑only database; content on disk; <1GB RAM |
+| **💎 Clarity through distillation** | Memory builds higher‑level abstractions over time | Decision Records v2.0 extract the *why* |
+| **🔍 Explainability builds trust** | You know *why* a memory came to mind | Provenance tracking; receipts with timestamps |
+
+> **Why This Matters:** Most AI memory systems are built for **scale**, not for **sense**. Anchor Engine is designed for **sense‑making**—for agents that need to remember not just *what* happened, but *why*, and to get *clearer* over time.
+
+---
+
+## 🏗️ Architecture at a Glance
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                         YOU                                  │
+└────────────────────┬────────────────────────────────────────┘
+                     │
+                     ▼
+┌─────────────────────────────────────────────────────────────┐
+│              ⚡ ANCHOR ENGINE                                │
+│         (Deterministic Memory Layer)                        │
+│                                                              │
+│  - Graph traversal (STAR algorithm)                         │
+│  - Pointer-only index (<1GB RAM)                            │
+│  - Deterministic retrieval (same query = same result)       │
+└────────────────────┬────────────────────────────────────────┘
+                     │
+         ┌───────────┼───────────┐
+         │           │           │
+         ▼           ▼           ▼
+┌──────────────┐ ┌──────────┐ ┌─────────────┐
+│ PGlite       │ │ mirrored │ │ MCP Clients │
+│ (WASM DB)    │ │ _brain/  │ │ (Claude,    │
+│              │ │ (Content)│ │  Cursor)    │
+└──────────────┘ └──────────┘ └─────────────┘
+```
+
+**Key Insight:** Content lives in `mirrored_brain/` filesystem. Database stores **pointers only** (byte offsets + metadata). This makes the database **disposable and rebuildable**—wipe it and restore in minutes.
+
+---
+
+## 📊 Benchmarks (Real Production Data)
+
+**Dataset:** 91MB chat history (~25M tokens)
+
+| Metric | Result |
+|--------|--------|
+| **Molecules** | 280,000 |
+| **Atoms** | 151,876 |
+| **Files** | 436 |
+| **Ingestion Time** | 178 seconds |
+| **Search Latency (p95)** | <200ms |
+| **Memory (idle)** | ~600MB |
+| **Memory (peak)** | ~1.6GB |
+| **Restore Speed** | 281,690 atoms in 13.8 min |
+
+**Hardware:** AMD Ryzen / Intel i7, 16GB RAM, NVMe SSD, no GPU.
+
+---
+
+## 🛠️ What's New in v4.8.0
+
+### MCP Write Operations
+- **`anchor_ingest_text`** - Ingest raw text directly
+- **`anchor_ingest_file`** - Ingest files from filesystem
+- Security toggle (opt-in via `user_settings.json`)
+
+### Session Index
+- **`anchor_search_index`** - Fast chat session lookup
+- **`anchor_fetch_session`** - Targeted session retrieval
+- Two-tier memory retrieval
+
+### Web UI Improvements
+- **Paste & Ingest** tab - Quick text ingestion
+- Version badge (v4.8.0)
+- Bucket selector (inbox vs external-inbox)
+
+### Documentation Overhaul
+- 5 new docs (API, Deployment, Troubleshooting, Source Overview, Testing)
+- Philosophy embedded throughout
+- 7 redundant files archived
+
+---
+
+## 📚 Documentation
+
+### Getting Started
+- **[Quick Start](#-quick-start-5-minutes)** - Install & first query
+- **[docs/API.md](docs/API.md)** - Complete API reference
+- **[docs/DEPLOYMENT.md](docs/DEPLOYMENT.md)** - Deployment guide (local, Docker, VPS, K8s)
+- **[docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md)** - Common issues & fixes
+
+### Deep Dive
+- **[docs/whitepaper.md](docs/whitepaper.md)** - STAR algorithm whitepaper
+- **[specs/spec.md](specs/spec.md)** - System specification with diagrams
+- **[specs/current-standards/](specs/current-standards/)** - Active standards (001-010)
+- **[engine/src/README.md](engine/src/README.md)** - Source code overview
+
+### Integration
+- **[mcp-server/README.md](mcp-server/README.md)** - MCP integration (Claude, Cursor, Qwen)
+- **[tests/README.md](tests/README.md)** - Testing guide
+- **[benchmarks/README.md](benchmarks/README.md)** - Performance benchmarks
+
+---
+
+## 🤝 Contributing
+
+We're building in the open and welcome your input!
+
+- **Star the repo** – Helps others find it
+- **Open an issue** – Bugs, features, questions
+- **Start a discussion** – Share use cases
+- **Contribute** – PRs welcome!
+
+See [`CONTRIBUTING.md`](CONTRIBUTING.md) for guidelines.
+
+**Community Health:**
+- [`CODEOWNERS`](.github/CODEOWNERS) - Automatic reviewer assignment
+- [`CODE_OF_CONDUCT.md`](CODE_OF_CONDUCT.md) - Community standards
+- [`CONTRIBUTING.md`](CONTRIBUTING.md) - Contribution guide
+
+---
+
+## 📜 License
+
+AGPL-3.0 – see [LICENSE](LICENSE).
+
+---
+
+## 🙏 Acknowledgments
+
+Built with ❤️ by Robert Balch II and contributors.  
+Inspired by the r/LocalLLaMA and r/aiagents communities.
+
+**Citation:**
+```bibtex
+@software{anchor_engine,
+  title = {STAR: Semantic Temporal Associative Retrieval},
+  author = {Balch II, R.S.},
+  version = {4.8.0},
+  date = {2026-03-18},
+  url = {https://github.com/RSBalchII/anchor-engine-node},
+  doi = {10.5281/zenodo.18841399}
+}
+```
+
+---
+
+*Your AI's anchor to reality.* ⚓

package/anchor.bat

@@ -0,0 +1,5 @@
+@echo off
+echo Launching Anchor Console...
+cd engine
+npx ts-node tools/anchor.ts
+pause

package/docs/API.md

@@ -0,0 +1,314 @@
+# Anchor Engine - API Reference
+
+**Version:** 4.8.0 | **Base URL:** `http://localhost:3160`
+
+---
+
+## Quick Reference
+
+| Category | Endpoints |
+|----------|-----------|
+| **🔍 Search** | `POST /v1/memory/search`, `POST /v1/memory/explore` |
+| **📦 Distill** | `POST /v1/memory/distill` |
+| **📥 Ingest** | `POST /v1/ingest`, `POST /v1/ingest/streaming` |
+| **📂 Files** | `GET /v1/files/read` |
+| **⚙️ System** | `GET /v1/system/status`, `GET /v1/stats` |
+| **🔧 Admin** | `POST /v1/admin/*` (requires auth) |
+
+Full API map: [`specs/API-ROUTE-MAP.md`](../specs/API-ROUTE-MAP.md)
+
+---
+
+## 🔍 Search Endpoints
+
+### POST `/v1/memory/search`
+
+Search the knowledge graph using STAR algorithm.
+
+**Request:**
+```json
+{
+  "query": "OAuth authentication setup",
+  "token_budget": 2048,
+  "max_chars": 8192,
+  "provenance": "all",
+  "buckets": ["inbox"],
+  "tags": ["authentication"],
+  "strategy": "standard"
+}
+```
+
+**Response:**
+```json
+{
+  "results": [
+    {
+      "id": "atom_123",
+      "content": "We set up GitHub OAuth...",
+      "source": "inbox/chat-2026-03-09.jsonl",
+      "score": 0.95,
+      "timestamp": "2026-03-09T21:41:20Z"
+    }
+  ],
+  "metadata": {
+    "atomCount": 5,
+    "filledPercent": 45
+  }
+}
+```
+
+---
+
+### POST `/v1/memory/explore`
+
+Graph exploration (BFS traversal).
+
+**Request:**
+```json
+{
+  "seed": {
+    "query": "agent frameworks",
+    "limit_seeds": 8
+  },
+  "max_depth": 3,
+  "max_nodes": 50
+}
+```
+
+**Response:**
+```json
+{
+  "results": [
+    {
+      "id": "atom_456",
+      "content": "...",
+      "score": 0.87,
+      "hop_distance": 1
+    }
+  ]
+}
+```
+
+---
+
+## 📦 Distill Endpoints
+
+### POST `/v1/memory/distill`
+
+Run radial distillation to compress knowledge.
+
+**Request:**
+```json
+{
+  "seed": {
+    "query": "career planning"
+  },
+  "radius": 3,
+  "max_nodes": 500,
+  "output_format": "json"
+}
+```
+
+**Response:**
+```json
+{
+  "status": "success",
+  "stats": {
+    "compounds_processed": 15,
+    "blocks_total": 234,
+    "blocks_unique": 89,
+    "compression_ratio": "2.6:1"
+  },
+  "output": {
+    "path": "distills/distilled_2026-03-18.json"
+  }
+}
+```
+
+---
+
+## 📥 Ingest Endpoints
+
+### POST `/v1/ingest`
+
+Ingest text content.
+
+**Request:**
+```json
+{
+  "content": "Meeting notes...",
+  "filename": "meeting-2026-03-18.md",
+  "bucket": "inbox"
+}
+```
+
+### POST `/v1/ingest/streaming`
+
+Stream large file ingestion with progress tracking.
+
+---
+
+## 📂 Files Endpoints
+
+### GET `/v1/files/read`
+
+Read file with optional line range.
+
+**Request:**
+```
+GET /v1/files/read?path=inbox%2Fmeeting-notes.md&start_line=0&end_line=100
+```
+
+**Response:**
+```json
+{
+  "content": "# Meeting Notes\n\n...",
+  "line_count": 45
+}
+```
+
+---
+
+## ⚙️ System Endpoints
+
+### GET `/v1/system/status`
+
+Get system health status.
+
+**Response:**
+```json
+{
+  "status": "ok",
+  "uptime": "2d 4h 15m",
+  "memory_mb": 512
+}
+```
+
+### GET `/v1/stats`
+
+Get knowledge graph statistics.
+
+**Response:**
+```json
+{
+  "atoms": 314000,
+  "molecules": 280000,
+  "compounds": 183,
+  "tags": 842
+}
+```
+
+---
+
+## 🔧 MCP Tools
+
+For MCP clients (Claude, Cursor, Qwen Code):
+
+| Tool | Description |
+|------|-------------|
+| `anchor_query` | Search memory |
+| `anchor_distill` | Run distillation |
+| `anchor_illuminate` | BFS traversal |
+| `anchor_read_file` | Read files |
+| `anchor_list_compounds` | List compounds |
+| `anchor_get_stats` | Get statistics |
+| `anchor_ingest_text` | Ingest text (opt-in) |
+| `anchor_ingest_file` | Ingest file (opt-in) |
+
+Full MCP docs: [`mcp-server/README.md`](../mcp-server/README.md)
+
+---
+
+## Authentication
+
+Most endpoints are open for local use. Admin endpoints require API key:
+
+```bash
+curl -H "Authorization: Bearer YOUR_API_KEY" \
+  http://localhost:3160/v1/admin/...
+```
+
+Configure in `user_settings.json`:
+```json
+{
+  "server": {
+    "api_key": "your-secret-key"
+  }
+}
+```
+
+---
+
+## Error Responses
+
+**400 Bad Request:**
+```json
+{
+  "status": "error",
+  "error": "Invalid token_budget: must be positive integer"
+}
+```
+
+**404 Not Found:**
+```json
+{
+  "status": "error",
+  "error": "File not found: inbox/missing.md"
+}
+```
+
+**500 Internal Error:**
+```json
+{
+  "status": "error",
+  "error": "Database not initialized"
+}
+```
+
+---
+
+## Rate Limiting
+
+Default: 60 requests/minute. Configure in `user_settings.json`:
+```json
+{
+  "rate_limit": {
+    "requests_per_minute": 100
+  }
+}
+```
+
+---
+
+## Examples
+
+### Search with Context
+```bash
+curl -X POST http://localhost:3160/v1/memory/search \
+  -H "Content-Type: application/json" \
+  -d '{
+    "query": "What did we discuss about OAuth?",
+    "token_budget": 4096
+  }'
+```
+
+### Distill Knowledge
+```bash
+curl -X POST http://localhost:3160/v1/memory/distill \
+  -H "Content-Type: application/json" \
+  -d '{
+    "seed": {"query": "AI agents"},
+    "radius": 3
+  }'
+```
+
+### Ingest Text
+```bash
+curl -X POST http://localhost:3160/v1/research/upload-raw \
+  -H "Content-Type: application/json" \
+  -d '{
+    "content": "Meeting notes...",
+    "filename": "notes.md",
+    "bucket": "inbox"
+  }'
+```