engram-sdk 0.1.0

package/CONTRIBUTING.md

@@ -0,0 +1,65 @@
+# Contributing to Engram
+
+Thanks for wanting to contribute! Engram is early-stage and moving fast — contributions are welcome.
+
+## Getting Started
+
+```bash
+git clone https://github.com/tstockham96/engram.git
+cd engram
+npm install
+npm test        # Run all tests (59 and counting)
+npm run lint    # Type check
+```
+
+## Development
+
+```bash
+npm run cli -- remember "test memory"   # Test CLI
+npm run cli -- recall "test"            # Test recall
+npm run serve                           # Start REST API on :3800
+npm test                                # Run tests
+```
+
+## What We Need Help With
+
+- **Python SDK** — thin client wrapping the REST API
+- **Framework integrations** — OpenClaw, LangChain, CrewAI, AutoGen adapters
+- **Consolidation improvements** — smarter rule-based consolidation, better LLM prompts
+- **Embedding providers** — Voyage, Cohere, local models
+- **Documentation** — examples, tutorials, guides
+- **Testing** — edge cases, stress tests, benchmarks
+
+## Pull Request Process
+
+1. Fork the repo and create a branch from `main`
+2. Add tests for any new functionality
+3. Make sure all tests pass (`npm test`)
+4. Make sure types check (`npm run lint`)
+5. Keep PRs focused — one feature or fix per PR
+6. Write a clear description of what and why
+
+## Code Style
+
+- TypeScript, strict mode
+- Use Zod for input validation
+- Tests in `src/__tests__/` using Vitest
+- No external runtime dependencies without discussion (we're at 4 deps total — let's keep it lean)
+
+## Architecture Decisions
+
+- **Zero-dependency server**: The REST API uses Node's built-in `http` module, not Express/Hono/Fastify. This is intentional — fewer deps = fewer supply chain risks.
+- **SQLite**: Single-file database, no server setup. sqlite-vec for vector search.
+- **Local-first**: Everything runs on your machine by default. Cloud is opt-in.
+
+## Reporting Issues
+
+Open an issue on GitHub. Include:
+- What you expected
+- What happened instead
+- Steps to reproduce
+- Node version, OS
+
+## License
+
+By contributing, you agree that your contributions will be licensed under the MIT License.

package/Dockerfile

@@ -0,0 +1,21 @@
+FROM node:20-slim
+
+WORKDIR /app
+
+# Install build dependencies for better-sqlite3 native addon
+RUN apt-get update && apt-get install -y python3 make g++ && rm -rf /var/lib/apt/lists/*
+
+COPY package.json package-lock.json* ./
+RUN npm ci
+
+COPY . .
+RUN npm run build
+RUN npm prune --production
+
+# Persistent data volume mount point
+RUN mkdir -p /data
+
+EXPOSE 3800
+
+# Use hosted multi-tenant server
+CMD ["node", "dist/hosted.js"]

package/EVAL-FRAMEWORK.md

@@ -0,0 +1,70 @@
+# Engram Eval Framework
+
+## The Question
+Does Engram make an AI agent meaningfully better at its job compared to flat-file memory (MEMORY.md + daily notes + vector search)?
+
+## What "Better" Means
+1. **Fewer repeated questions** — Agent doesn't ask things it should already know
+2. **Surprise recall** — Agent surfaces relevant context the human didn't prompt for
+3. **Contradiction prevention** — Agent catches itself before giving conflicting info
+4. **Reduced curation toil** — Human spends less time maintaining memory files
+5. **Token efficiency** — Same or better context quality with fewer tokens
+
+## Daily Automated Eval (runs via cron)
+
+### Recall Accuracy Test
+Run a fixed set of 20 questions with known answers. Score: % of correct top-3 results.
+Track over time as memories accumulate and consolidation runs.
+
+Questions should cover:
+- Factual (What is Thomas's job? → Senior PM at BambooHR)
+- Procedural (How do I deploy the site? → cd engram-site && npx vercel --prod)
+- Relational (Who are Engram's competitors? → Mem0, Zep, Letta, LangMem)
+- Temporal (What did we build yesterday? → depends on date)
+- Personal preference (How does Thomas like communication? → direct, no fluff)
+
+### Token Comparison
+- Measure: bytes of MEMORY.md + loaded daily files vs Engram briefing() output
+- Track both as they grow over time
+- Calculate: tokens saved per request if using Engram briefing instead of file dump
+
+### Consolidation Quality
+After each consolidation run, evaluate:
+- Are new semantic memories factually accurate?
+- Are they non-redundant with existing memories?
+- Do they capture something the raw episodes didn't explicitly state? (insight generation)
+
+### Memory Freshness
+- Count: memories with status "pending" that are actually fulfilled
+- Count: memories that contradict each other
+- Track whether lifecycle management is happening
+
+## Weekly Human Eval
+
+Thomas rates (1-5) on a weekly basis:
+1. Did the agent remember things it should have? 
+2. Did the agent forget things it shouldn't have?
+3. Did the agent surprise you with relevant context?
+4. Was there less repetitive Q&A this week vs last?
+
+## Success Criteria
+
+### Engram is WORKING when:
+- Recall accuracy consistently >80% on the fixed question set
+- Agent surfaces relevant context Thomas didn't ask for at least 2x/week
+- Agent catches a contradiction or stale commitment at least 1x/week
+- MEMORY.md stops growing because Engram captures what it used to
+
+### Engram is NOT WORKING when:
+- MEMORY.md remains the primary useful memory source
+- Recall accuracy stays flat or degrades as memories accumulate
+- Consolidation produces redundant summaries instead of insights
+- Token cost increases without quality improvement
+
+## Scale Milestones
+Track value at each milestone:
+- 220 memories (current) — baseline
+- 500 memories — first consolidation cycle quality check
+- 1,000 memories — graph density threshold
+- 5,000 memories — flat-file memory should be breaking down by here
+- 10,000+ memories — Engram must be clearly better or it's not working

package/EVAL.md

@@ -0,0 +1,127 @@
+# Engram Self-Eval: Agent Performance With vs Without Memory
+
+*Designed Feb 16, 2026. Running over 72 hours.*
+
+## Hypothesis
+
+An AI agent with Engram memory performs measurably better at tasks requiring context from prior conversations than the same agent without it.
+
+## Design
+
+**Subject:** Jarvis (Claude Opus via OpenClaw) — the same agent, tested in two conditions:
+
+1. **Baseline (no Engram):** Agent uses only OpenClaw's default memory system (MEMORY.md flat files + vector search over markdown)
+2. **Engram-enhanced:** Agent uses Engram vault (auto-ingest, semantic recall, consolidation, knowledge graph)
+
+**Interface:** REST API + auto-ingest pipeline — the same way any OpenClaw user would run it. No MCP needed. Just the Engram server running alongside the agent, auto-ingesting conversations, and serving recall queries via HTTP.
+
+**Method:** Over 72 hours of normal use with Thomas, I'll log specific moments where memory matters. At each moment, I'll:
+1. Query Engram's REST API for relevant context
+2. Note what my flat-file memory (MEMORY.md + vector search) would have returned
+3. Compare the quality of recall
+
+This isn't a synthetic benchmark — it's real-world usage on the same setup any OpenClaw user would have. More ecologically valid than a lab test.
+
+## Metrics
+
+### 1. Recall Accuracy (quantitative)
+At each memory-dependent moment, score:
+- **Hit**: Correct, relevant memory surfaced → 1 point
+- **Partial**: Related but incomplete or slightly wrong → 0.5 points
+- **Miss**: Relevant memory existed but wasn't recalled → 0 points
+- **Hallucination**: Confidently wrong memory → -1 point
+
+Score both systems independently. Compare hit rates.
+
+### 2. Context Richness (qualitative, 1-5 scale)
+When recalling context, rate the depth:
+- 1: No useful context
+- 2: Basic facts only ("Thomas works at BambooHR")
+- 3: Facts + preferences ("Thomas works at BambooHR, prefers direct communication")
+- 4: Facts + preferences + history ("...and he pivoted from Kin to Engram because thin-margin hosting has no moat")
+- 5: Full situational awareness including decisions, reasoning, and emotional context
+
+### 3. Consolidation Quality (qualitative)
+After each consolidation run:
+- How many genuinely useful semantic memories were created?
+- Were any contradictions correctly identified?
+- Were any stale memories correctly superseded?
+- Did the knowledge graph capture meaningful relationships?
+
+### 4. Time-to-Context (qualitative)
+How quickly does the agent reach useful context?
+- Engram: single API call, structured results
+- Flat files: grep through markdown, parse manually, hope the right section is there
+
+### 5. "Surprise" Moments
+Log any moment where Engram surfaced context the agent wouldn't have thought to look for. These are the moments that demonstrate proactive value.
+
+## Test Scenarios
+
+Run these naturally over the 72-hour period:
+
+### Day 1 (Feb 16-17): Foundation
+- [x] Seed vault from existing conversations
+- [x] Run auto-ingest on current session
+- [x] First consolidation run
+- [ ] Evening: Ask about decisions made in morning (test same-day recall)
+- [ ] Evening: Run consolidation, note what it produces
+
+### Day 2 (Feb 17-18): Cross-Session Recall
+- [ ] Morning: Can Engram recall yesterday's context without being told?
+- [ ] Ask about pricing decisions (tests semantic memory from consolidation)
+- [ ] Ask about Thomas's preferences (tests across multiple conversations)
+- [ ] Work on a task that references past decisions — compare with/without
+- [ ] Run consolidation, note cross-day patterns
+
+### Day 3 (Feb 18-19): Stress Tests
+- [ ] Ask a question that requires connecting 3+ separate memories
+- [ ] Ask about something mentioned once 3 days ago (tests decay/persistence)
+- [ ] Introduce contradictory information — does consolidation catch it?
+- [ ] Ask an ambiguous question — does the knowledge graph help disambiguate?
+- [ ] Final consolidation + full analysis
+
+## Logging Format
+
+Each test moment gets logged as:
+
+```markdown
+### Test #N: [Description]
+**Time:** YYYY-MM-DD HH:MM
+**Query:** "What I asked / needed to recall"
+**Engram result:** [what Engram returned]
+**Flat-file result:** [what MEMORY.md search would return]
+**Recall accuracy:** Hit / Partial / Miss / Hallucination (for each system)
+**Context richness:** X/5 (for each system)
+**Notes:** [observations]
+```
+
+## Success Criteria
+
+Engram is **better** if:
+- Recall accuracy is ≥20% higher than flat files
+- Context richness averages ≥1 point higher
+- At least 3 "surprise" moments where Engram surfaced unexpected relevant context
+- Consolidation produces ≥10 genuinely useful semantic memories over 72 hours
+- Zero hallucinated memories (confidence should prevent false recalls)
+
+Engram is **equivalent** if scores are within 10%. Even equivalence is a win if Engram requires less manual effort (no hand-editing MEMORY.md).
+
+Engram **fails** if:
+- Recall accuracy is lower than flat files
+- Consolidation produces mostly noise
+- The agent performs worse on tasks due to bad memory surfacing
+
+## Results
+
+*Will be filled in as the eval progresses.*
+
+### Summary (to be completed Feb 19)
+- Total test moments: 
+- Engram recall accuracy: 
+- Flat-file recall accuracy: 
+- Engram context richness avg: 
+- Flat-file context richness avg: 
+- Surprise moments: 
+- Semantic memories from consolidation: 
+- Verdict: 

package/LICENSE

@@ -0,0 +1,17 @@
+                    GNU AFFERO GENERAL PUBLIC LICENSE
+                       Version 3, 19 November 2007
+
+ Copyright (C) 2026 Thomas Stockham
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU Affero General Public License as published
+ by the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ GNU Affero General Public License for more details.
+
+ You should have received a copy of the GNU Affero General Public License
+ along with this program.  If not, see <https://www.gnu.org/licenses/>.

package/README.md

@@ -0,0 +1,309 @@
+# 🧠 Engram
+
+**Universal memory protocol for AI agents.**
+
+AI agents have amnesia. Every session starts blank. Engram fixes this — a memory protocol with a REST API, knowledge graph, and consolidation engine that turns raw episodes into structured knowledge. Think of it as giving your agent a hippocampus.
+
+## Why
+
+Every AI agent framework bolts on memory as an afterthought — a flat file, a vector DB, maybe a conversation log. None of them solve the real problem:
+
+- **Amnesia**: Agents forget everything between sessions
+- **No consolidation**: Raw episodes pile up, never distilled into knowledge
+- **No relationships**: Memories exist in isolation, no graph of connections
+- **No decay**: Everything is equally important forever
+- **No portability**: Memory locked into one framework
+
+Engram is a protocol, not a plugin. It works with any agent, any framework, any language.
+
+## Quick Start (60 seconds)
+
+### 1. Install & run
+
+```bash
+npm install engram-sdk
+export GEMINI_API_KEY=your-key-here  # Get one free at ai.google.dev
+npx engram-serve
+# → Engram listening on http://localhost:3800
+```
+
+That's it. Engram is running with vector search, auto-extraction, and consolidation.
+
+> **No Gemini key?** Engram works without one — you just won't get embeddings or LLM-powered features. Or use our [hosted API](https://engram-site.vercel.app) and skip setup entirely.
+
+### 2. Remember & recall
+
+```bash
+# Store a memory
+curl -X POST http://localhost:3800/v1/memories \
+  -H 'Content-Type: application/json' \
+  -d '{"content": "User prefers dark mode and concise answers"}'
+
+# Recall relevant memories
+curl -X POST http://localhost:3800/v1/memories/recall \
+  -H 'Content-Type: application/json' \
+  -d '{"query": "user preferences", "context": "settings page"}'
+
+# Consolidate (distill episodes into knowledge)
+curl -X POST http://localhost:3800/v1/consolidate
+```
+
+### TypeScript SDK
+
+```typescript
+import { Vault } from 'engram-sdk';
+
+const vault = new Vault();
+
+// Remember
+await vault.remember('User prefers dark mode and concise answers');
+await vault.remember('User is training for a marathon in April');
+
+// Recall
+const memories = await vault.recall('What are the user preferences?');
+console.log(memories);
+
+// Consolidate
+await vault.consolidate();
+vault.close();
+```
+
+### Python (via REST API)
+
+```python
+import requests
+
+API = "http://localhost:3800/v1"
+
+# Store a memory
+requests.post(f"{API}/memories", json={
+    "content": "User prefers dark mode and concise answers",
+    "entities": ["User"],
+    "topics": ["preferences"],
+    "salience": 0.8,
+})
+
+# Recall
+memories = requests.get(f"{API}/memories/recall", params={"context": "user preferences"}).json()
+```
+
+Python SDK coming soon. The REST API works with any language — no SDK required.
+
+## REST API Reference
+
+Start the server:
+
+```bash
+# Environment variables
+ENGRAM_OWNER=my-agent        # Vault owner ID (required)
+ENGRAM_DB_PATH=./my.db       # Database path (optional)
+ENGRAM_PORT=3800             # Port (default: 3800)
+ENGRAM_HOST=127.0.0.1        # Host (default: 127.0.0.1)
+ENGRAM_LLM_PROVIDER=anthropic # LLM for consolidation (optional)
+ENGRAM_LLM_API_KEY=sk-...    # LLM API key (optional)
+
+npx engram serve
+# or
+npm run serve
+```
+
+### Endpoints
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `POST` | `/v1/memories` | Store a memory |
+| `GET` | `/v1/memories/recall?context=...` | Recall memories (simple) |
+| `POST` | `/v1/memories/recall` | Recall memories (complex query) |
+| `DELETE` | `/v1/memories/:id` | Forget a memory |
+| `GET` | `/v1/memories/:id/neighbors` | Graph traversal |
+| `POST` | `/v1/connections` | Connect two memories |
+| `POST` | `/v1/consolidate` | Run consolidation engine |
+| `GET` | `/v1/entities` | List tracked entities |
+| `GET` | `/v1/stats` | Vault statistics |
+| `POST` | `/v1/export` | Export full vault as JSON |
+| `GET` | `/health` | Health check |
+
+### POST /v1/memories
+
+```json
+{
+  "content": "User prefers TypeScript over JavaScript",
+  "type": "episodic",           // episodic | semantic | procedural
+  "entities": ["User", "TypeScript", "JavaScript"],
+  "topics": ["preferences", "engineering"],
+  "salience": 0.8,              // 0-1, importance
+  "confidence": 0.9,            // 0-1, certainty
+  "visibility": "owner_agents"  // private | owner_agents | shared | public
+}
+```
+
+### GET /v1/memories/recall
+
+Query params: `context` (required), `entities`, `topics`, `types`, `limit`
+
+### POST /v1/memories/recall
+
+```json
+{
+  "context": "project status",
+  "entities": ["Engram"],
+  "topics": ["engineering"],
+  "types": ["semantic"],
+  "minSalience": 0.5,
+  "limit": 10
+}
+```
+
+### POST /v1/connections
+
+```json
+{
+  "sourceId": "memory-uuid-1",
+  "targetId": "memory-uuid-2",
+  "type": "supports",           // supports | contradicts | elaborates | supersedes | causes | caused_by | ...
+  "strength": 0.7
+}
+```
+
+## Core Concepts
+
+### Memory Types
+
+| Type | What | Example |
+|------|------|---------|
+| **Episodic** | Events, conversations, observations | "User asked about React performance" |
+| **Semantic** | Facts, knowledge, patterns | "User prefers TypeScript over JavaScript" |
+| **Procedural** | How-to knowledge, workflows | "To deploy: run tests → build → push to main" |
+
+### The Memory Lifecycle
+
+```
+Episode → Remember → Store → Recall → Consolidate → Knowledge
+                                           ↓
+                                    Decay / Archive
+```
+
+1. **Remember**: Raw episodes go in with metadata (entities, topics, salience)
+2. **Recall**: Hybrid retrieval — entity matching, topic matching, semantic search, recency
+3. **Consolidate**: The "sleep cycle" — episodes get distilled into semantic memories, entities get discovered, connections form, contradictions surface
+4. **Decay**: Memories naturally fade unless reinforced by access. High-salience memories resist decay.
+
+### Memory Graph
+
+Memories aren't flat — they're a graph. Edges connect related memories:
+
+- `supports` / `contradicts` — agreement or conflict
+- `elaborates` — adds detail
+- `supersedes` — replaces outdated info
+- `causes` / `caused_by` — causal chains
+- `temporal_next` — sequential events
+- `derived_from` — consolidation lineage
+
+### Entities
+
+Engram automatically tracks entities (people, places, projects, concepts) across memories. Entity frequency and co-occurrence drive importance scores and recall relevance.
+
+## LLM-Powered Consolidation
+
+For the full consolidation experience, configure an LLM:
+
+```typescript
+const vault = new Vault({
+  owner: 'my-agent',
+  llm: {
+    provider: 'anthropic',
+    apiKey: process.env.ANTHROPIC_API_KEY!,
+    model: 'claude-3-5-haiku-20241022',
+  },
+});
+```
+
+Or via environment variables with the REST API:
+
+```bash
+ENGRAM_LLM_PROVIDER=anthropic ENGRAM_LLM_API_KEY=sk-... npx engram serve
+```
+
+The LLM analyzes episodes and extracts:
+- **Semantic memories**: General facts and patterns
+- **Entities**: People, places, projects with properties
+- **Contradictions**: Conflicting information
+- **Connections**: How episodes relate to each other
+
+## CLI
+
+```bash
+npx engram remember "User prefers React over Vue"
+npx engram recall "frontend preferences"
+npx engram stats
+npx engram entities
+npx engram consolidate
+npx engram export > backup.json
+npx engram repl          # Interactive mode
+npx engram serve         # Start REST API server
+```
+
+## Architecture
+
+```
+┌─────────────────────────────────────────────────┐
+│                  REST API Server                  │
+│  POST /v1/memories · GET /v1/memories/recall · …  │
+├─────────────────────────────────────────────────┤
+│                    Vault API                      │
+│  remember() · recall() · consolidate() · forget() │
+├─────────────────────────────────────────────────┤
+│               Retrieval Engine                    │
+│  Entity match · Topic match · Vector search · FTS │
+├─────────────────────────────────────────────────┤
+│            Consolidation Engine                   │
+│  Rule-based │ LLM-powered (Anthropic/OpenAI)      │
+├─────────────────────────────────────────────────┤
+│              SQLite Storage Layer                  │
+│  Memories · Edges · Entities · Embeddings          │
+└─────────────────────────────────────────────────┘
+```
+
+**Open source**: Fully MIT licensed. Memory is sensitive data — you should be able to see exactly what happens with it. Self-host, fork, contribute, or use the hosted API for production.
+
+**Portable**: Export your entire vault as JSON. Import it elsewhere. Your agent's memory belongs to you.
+
+## Comparison
+
+| Feature | Engram | Mem0 | Zep | Letta/MemGPT |
+|---------|--------|------|-----|-------------|
+| Spreading activation | ✅ | ❌ | ❌ | ❌ |
+| LLM consolidation | ✅ | ❌ | ❌ | Partial |
+| Knowledge graph | ✅ | ✅ (graph memory) | ✅ | ❌ |
+| Entity tracking | ✅ | ✅ | ✅ | ❌ |
+| Memory decay | ✅ | ❌ | ❌ | ❌ |
+| REST API | ✅ | ✅ | ✅ | ✅ |
+| Local-first | ✅ | Cloud-default | Cloud-default | ✅ |
+| Language-agnostic | ✅ | Python-first | Python-first | Python-first |
+| Source-available | ✅ | Partial | Partial | ✅ |
+
+## Roadmap
+
+- [x] TypeScript SDK
+- [x] REST API server
+- [x] sqlite-vec vector search
+- [x] LLM-powered consolidation
+- [x] CLI with REPL
+- [ ] Hosted service (api.engram.ai)
+- [ ] Python SDK
+- [ ] Framework integrations (OpenClaw, LangChain, CrewAI)
+- [ ] Protocol spec (open standard)
+- [ ] Multi-agent vault sharing
+- [ ] Conflict resolution across agents
+
+## Contributing
+
+See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
+
+## License
+
+MIT. See [LICENSE](LICENSE) for details.
+
+---
+
+*Built by [Thomas Stockham](https://tstockham.com). Engram is the memory layer the AI agent ecosystem is missing.*