@shadowforge0/aquifer-memory 0.8.0 → 0.9.1

package/README.md

@@ -32,7 +32,7 @@ Sessions, summaries, turn-level embeddings, entity graph — all live in one dat
 | **Ranking** | 3-way RRF: FTS + session embedding + turn embedding | Single vector similarity |
 | **Knowledge graph** | Built-in entity extraction & co-occurrence | Usually separate system |
 | **Multi-tenant** | `tenant_id` on every table, day-1 | Often an afterthought |
-| **Dependencies** | Just `pg` | Multiple SDKs |
+| **Dependencies** | `pg` + MCP SDK | Multiple SDKs |
 
 ### Before and after
 
@@ -48,80 +48,150 @@ Sessions, summaries, turn-level embeddings, entity graph — all live in one dat
 
 ---
 
-## Quick Start
+## Requirements
 
-### Prerequisites
+| Component | Required? | Purpose | Example |
+|-----------|-----------|---------|---------|
+| Node.js >= 18 | Yes | Runtime | — |
+| PostgreSQL 15+ | Yes | Storage for sessions, summaries, entities | Local, Docker, or managed |
+| pgvector extension | Yes | Vector similarity search | `CREATE EXTENSION vector;` (included in `pgvector/pgvector` Docker image) |
+| Embedding endpoint | Yes (for recall) | Turn + session embedding | Ollama `bge-m3`, OpenAI `text-embedding-3-small`, any OpenAI-compatible API |
+| LLM endpoint | Optional | Built-in summarization during `enrich` | Ollama, OpenRouter, OpenAI — or provide your own `summaryFn` |
+| `@modelcontextprotocol/sdk` + `zod` | Yes (for MCP server) | MCP protocol runtime | Included in dependencies — installed automatically |
 
-- Node.js >= 18
-- PostgreSQL 15+ with [pgvector](https://github.com/pgvector/pgvector) extension
-- An embedding API (OpenAI, Ollama, or any OpenAI-compatible endpoint)
+---
+
+## Quick Start (MCP Server)
+
+This gets you from zero to a working MCP memory server. For library API usage, see [API Reference](#api-reference) below.
+
+### 1. Start the stack
+
+```bash
+docker compose up -d
+# Starts PostgreSQL 16 + pgvector and Ollama with bge-m3 (auto-pulled).
+# First run takes a few minutes while Ollama downloads the model.
+```
+
+Already have PostgreSQL + pgvector and an embedding endpoint? Skip this step.
 
-### Install
+### 2. Install
 
 ```bash
 npm install @shadowforge0/aquifer-memory
 ```
 
-### Initialize
+### 3. Configure + verify
 
-```javascript
-const { createAquifer } = require('@shadowforge0/aquifer-memory');
+```bash
+export DATABASE_URL="postgresql://aquifer:aquifer@localhost:5432/aquifer"
+export AQUIFER_EMBED_BASE_URL="http://localhost:11434/v1"
+export AQUIFER_EMBED_MODEL="bge-m3"
 
-const aquifer = createAquifer({
-  db: 'postgresql://user:pass@localhost:5432/mydb',  // connection string or pg.Pool
-  schema: 'memory',                    // PG schema name (default: 'aquifer')
-  tenantId: 'default',                 // multi-tenant isolation
-  embed: {
-    fn: async (texts) => embeddings,   // your embedding function
-    dim: 1024,                         // optional dimension hint
-  },
-  llm: {
-    fn: async (prompt) => text,        // your LLM function (for built-in summarize)
-  },
-  entities: {
-    enabled: true,
-    scope: 'my-app',                   // entity namespace (default: 'default')
-  },
-});
+npx aquifer quickstart
+```
 
-// Run migrations (safe to call multiple times)
-await aquifer.migrate();
+`quickstart` runs migrations, commits a test session, embeds it, recalls it, and cleans up. If it prints `✓ Aquifer is working`, you're done.
+
+### 4. Start the MCP server
+
+```bash
+npx aquifer mcp
 ```
 
-### Write path: commit + enrich
+See [.env.example](.env.example) for all env vars, or [docs/setup.md](docs/setup.md) for the full setup guide.
 
-```javascript
-// 1. Store the session
-await aquifer.commit('conv-001', [
-  { role: 'user', content: 'Let me tell you about our new auth approach...' },
-  { role: 'assistant', content: 'Got it. So the plan is...' },
-], { agentId: 'main' });
-
-// 2. Enrich: summarize + embed turns + extract entities
-const result = await aquifer.enrich('conv-001', {
-  agentId: 'main',
-  // Optional: bring your own summarize pipeline
-  summaryFn: async (msgs) => ({ summaryText, structuredSummary, entityRaw }),
-  entityParseFn: (text) => [{ name, normalizedName, type, aliases }],
-  // Optional: post-commit hook for downstream processing
-  postProcess: async (ctx) => {
-    // ctx contains session, summary, embedding, parsedEntities, etc.
-  },
-});
+---
+
+## Environment Variables
+
+| Variable | Required? | Purpose | Example |
+|----------|-----------|---------|---------|
+| `DATABASE_URL` | Yes | PostgreSQL connection string | `postgresql://user:pass@localhost:5432/mydb` |
+| `AQUIFER_SCHEMA` | No | PG schema name (default: `aquifer`) | `memory` |
+| `AQUIFER_TENANT_ID` | No | Multi-tenant key (default: `default`) | `my-app` |
+| `AQUIFER_EMBED_BASE_URL` | Yes (for recall) | Embedding API base URL | `http://localhost:11434/v1` |
+| `AQUIFER_EMBED_MODEL` | Yes (for recall) | Embedding model name | `bge-m3` |
+| `AQUIFER_EMBED_API_KEY` | Provider-dependent | API key for hosted embedding providers | `sk-...` |
+| `AQUIFER_EMBED_DIM` | No | Embedding dimension override (auto-detected) | `1024` |
+| `AQUIFER_LLM_BASE_URL` | No | LLM API base URL (for built-in summarization) | `http://localhost:11434/v1` |
+| `AQUIFER_LLM_MODEL` | No | LLM model name | `llama3.1` |
+| `AQUIFER_LLM_API_KEY` | Provider-dependent | API key for hosted LLM providers | `sk-...` |
+| `AQUIFER_ENTITIES_ENABLED` | No | Enable knowledge graph (default: `false`) | `true` |
+| `AQUIFER_ENTITY_SCOPE` | No | Entity namespace (default: `default`) | `my-app` |
+| `AQUIFER_RERANK_ENABLED` | No | Enable cross-encoder reranking | `true` |
+| `AQUIFER_RERANK_PROVIDER` | No | Reranker provider: `tei`, `jina`, `openrouter` | `tei` |
+| `AQUIFER_RERANK_BASE_URL` | No | Reranker endpoint | `http://localhost:8080` |
+| `AQUIFER_AGENT_ID` | No | Default agent ID | `main` |
+
+Full env-to-config mapping is in [consumers/shared/config.js](consumers/shared/config.js).
+
+---
+
+## Host Integration
+
+MCP is the primary integration surface. Agent hosts connect to the Aquifer MCP server, which exposes four tools: `session_recall`, `session_feedback`, `memory_stats`, `memory_pending`.
+
+| Integration | Route | Status | When to use |
+|-------------|-------|--------|-------------|
+| MCP server | `consumers/mcp.js` | Primary | Claude Code, OpenClaw, Codex, any MCP-capable host |
+| Library API | `createAquifer()` | Primary | Backend apps, custom pipelines, direct Node.js usage |
+| CLI | `consumers/cli.js` | Secondary | Operations, debugging, manual recall/backfill |
+| OpenClaw plugin | `consumers/openclaw-plugin.js` | Compatibility only | Session capture via `before_reset` — not for tool delivery |
+
+### Claude Code
+
+Add to your project's `.claude.json` or user-level MCP config:
+
+```json
+{
+  "mcpServers": {
+    "aquifer": {
+      "type": "stdio",
+      "command": "node",
+      "args": ["/path/to/aquifer/consumers/mcp.js"],
+      "env": {
+        "DATABASE_URL": "postgresql://...",
+        "AQUIFER_EMBED_BASE_URL": "http://localhost:11434/v1",
+        "AQUIFER_EMBED_MODEL": "bge-m3"
+      }
+    }
+  }
+}
 ```
 
-### Read path: recall
+Tools appear as `mcp__aquifer__session_recall`, `mcp__aquifer__session_feedback`, etc.
 
-```javascript
-const results = await aquifer.recall('auth middleware decision', {
-  agentId: 'main',
-  limit: 5,
-  entities: ['auth-middleware'],       // optional: entity-aware search
-  entityMode: 'all',                   // 'any' (boost) or 'all' (hard filter)
-});
-// Returns ranked sessions with scores, using 3-way RRF fusion
+### OpenClaw
+
+Add to `openclaw.json` under `mcp.servers`:
+
+```json
+{
+  "mcp": {
+    "servers": {
+      "aquifer": {
+        "command": "node",
+        "args": ["/path/to/aquifer/consumers/mcp.js"],
+        "env": {
+          "DATABASE_URL": "postgresql://...",
+          "AQUIFER_EMBED_BASE_URL": "http://localhost:11434/v1",
+          "AQUIFER_EMBED_MODEL": "bge-m3"
+        }
+      }
+    }
+  }
+}
 ```
 
+Tools materialize as `aquifer__session_recall`, `aquifer__session_feedback`, `aquifer__memory_stats`, `aquifer__memory_pending` (server name prefix added by the host).
+
+The OpenClaw plugin (`consumers/openclaw-plugin.js`) is retained for session capture via `before_reset` but is **not** the recommended tool delivery path. Use MCP.
+
+### Other MCP-capable hosts
+
+Any host that supports MCP stdio can connect the same way — point it at `node consumers/mcp.js` with the required env vars. The MCP server is the canonical external contract.
+
 ---
 
 ## Architecture
@@ -161,8 +231,6 @@ const results = await aquifer.recall('auth middleware decision', {
     └──────────────────────────────────┘
 ```
 
-**Integration model:** MCP is the primary integration surface. Agent hosts connect to Aquifer through the MCP server (`consumers/mcp.js`), which exposes `session_recall`, `session_feedback`, `memory_stats`, and `memory_pending`. The CLI wraps the same engine for command-line use. The OpenClaw plugin (`consumers/openclaw-plugin.js`) is retained as a compatibility adapter for session capture but is not the primary tool delivery path.
-
 ### File Reference
 
 | File | Purpose |
@@ -375,7 +443,9 @@ Closes the PostgreSQL connection pool (only if Aquifer created it).
 
 ## Configuration
 
-Aquifer takes a `db` connection (string or `pg.Pool`), plus optional `embed` and `llm` functions:
+Aquifer resolves config from three sources in priority order: config file → environment variables → programmatic overrides. See [consumers/shared/config.js](consumers/shared/config.js) for the full env-to-config mapping.
+
+Config file is auto-discovered at `aquifer.config.json` in the working directory, or set `AQUIFER_CONFIG=/path/to/config.json`.
 
 ```javascript
 createAquifer({
@@ -409,61 +479,6 @@ createAquifer({
 
 Fallback chain: `config.entities.scope` → `'default'`.
 
-### MCP Server (primary integration)
-
-Agent hosts should connect through the Aquifer MCP server. For OpenClaw, add to `openclaw.json`:
-
-```json
-{
-  "mcp": {
-    "servers": {
-      "aquifer": {
-        "command": "node",
-        "args": ["/path/to/aquifer/consumers/mcp.js"],
-        "env": {
-          "DATABASE_URL": "postgresql://...",
-          "AQUIFER_SCHEMA": "aquifer",
-          "AQUIFER_EMBED_BASE_URL": "http://localhost:11434/v1",
-          "AQUIFER_EMBED_MODEL": "bge-m3"
-        }
-      }
-    }
-  }
-}
-```
-
-Tools are exposed as `aquifer__session_recall`, `aquifer__session_feedback`, `aquifer__memory_stats`, `aquifer__memory_pending` (server name prefix is added by the host).
-
-For Claude Code, add to `.claude.json`:
-
-```json
-{
-  "mcpServers": {
-    "aquifer": {
-      "type": "stdio",
-      "command": "node",
-      "args": ["/path/to/aquifer/consumers/mcp.js"]
-    }
-  }
-}
-```
-
-### CLI (secondary)
-
-For command-line use with environment variables:
-
-```bash
-export DATABASE_URL="postgresql://..."
-export AQUIFER_EMBED_BASE_URL="http://localhost:11434/v1"
-export AQUIFER_EMBED_MODEL="bge-m3"
-export AQUIFER_ENTITIES_ENABLED=true
-
-aquifer migrate
-aquifer recall "search query" --limit 5
-aquifer backfill --concurrency 3
-aquifer stats --json
-```
-
 ---
 
 ## Database Schema
@@ -478,6 +493,8 @@ aquifer stats --json
 
 Key indexes: GIN on messages, GiST on `tsvector`, ivfflat on embeddings, B-tree on tenant/agent/timestamps.
 
+Note: the schema uses basic ivfflat indexes suitable for development and moderate-scale use. For large deployments (100k+ embeddings), consider adding HNSW indexes — this is a future optimization area, not included out of the box.
+
 ### 002-entities.sql
 
 | Table | Purpose |
@@ -499,15 +516,29 @@ Also adds `trust_score` column to `session_summaries` (default 0.5, range 0–1)
 
 ---
 
+## Troubleshooting
+
+**`error: type "vector" does not exist`** — pgvector extension is not installed. Run `CREATE EXTENSION IF NOT EXISTS vector;` as a superuser, or use the `pgvector/pgvector` Docker image which includes it.
+
+**`aquifer mcp requires @modelcontextprotocol/sdk and zod`** — These are now regular dependencies and should be installed automatically. If you see this error, run `npm install` again to ensure all deps are present.
+
+**Recall returns no results** — Make sure you've run `enrich` after `commit`. Raw sessions are not searchable until enriched (summarized + embedded). Check `aquifer stats` to see if summaries and turn embeddings exist.
+
+**OpenClaw tools not visible** — Use `mcp.servers.aquifer` in `openclaw.json`, not the plugin. Tools appear as `aquifer__session_recall` etc. The plugin (`consumers/openclaw-plugin.js`) is for session capture only.
+
+**Embedding provider connection refused** — Verify your `AQUIFER_EMBED_BASE_URL` is reachable. For local Ollama, make sure the server is running and the model is pulled (`ollama pull bge-m3`).
+
+---
+
 ## Dependencies
 
 | Package | Purpose |
 |---------|---------|
 | `pg` ≥ 8.13 | PostgreSQL client |
+| `@modelcontextprotocol/sdk` ≥ 1.29 | MCP server protocol |
+| `zod` ≥ 3.25 | Schema validation (MCP tools) |
 
-That's it. Aquifer has **one runtime dependency**.
-
-LLM and embedding calls use raw HTTP — no SDK required.
+LLM and embedding calls use raw HTTP — no additional SDK required.
 
 ---
 

package/consumers/cli.js

@@ -5,6 +5,7 @@
  * Aquifer CLI
  *
  * Usage:
+ *   aquifer quickstart                  Verify end-to-end setup
  *   aquifer migrate                     Run database migrations
  *   aquifer recall <query> [options]    Search sessions
  *   aquifer backfill [options]          Enrich pending sessions
@@ -15,6 +16,26 @@
 
 const { createAquiferFromConfig } = require('./shared/factory');
 
+function formatDate(value, fallback) {
+  if (!value) return fallback;
+  const parsed = new Date(value);
+  return isNaN(parsed.getTime()) ? fallback : parsed.toISOString().slice(0, 10);
+}
+
+function quoteIdentifier(identifier) {
+  if (!/^[a-zA-Z_]\w{0,62}$/.test(identifier)) {
+    throw new Error(`Invalid schema name: "${identifier}"`);
+  }
+  return `"${identifier}"`;
+}
+
+function parsePositiveInt(value, fallback) {
+  if (value === undefined || value === null || value === true) return fallback;
+  const parsed = parseInt(value, 10);
+  if (!Number.isFinite(parsed)) return fallback;
+  return Math.max(1, parsed);
+}
+
 // ---------------------------------------------------------------------------
 // Argument parser (minimal, no deps)
 // ---------------------------------------------------------------------------
@@ -56,7 +77,7 @@ async function cmdRecall(aquifer, args) {
   }
 
   const recallOpts = {
-    limit: parseInt(args.flags.limit || '5', 10),
+    limit: parsePositiveInt(args.flags.limit, 5),
     agentId: args.flags['agent-id'] || undefined,
     source: args.flags.source || undefined,
     dateFrom: args.flags['date-from'] || undefined,
@@ -82,7 +103,7 @@ async function cmdRecall(aquifer, args) {
     const r = results[i];
     const ss = r.structuredSummary || {};
     const title = ss.title || r.summaryText?.slice(0, 60) || '(untitled)';
-    const date = r.startedAt ? new Date(r.startedAt).toISOString().slice(0, 10) : '?';
+    const date = formatDate(r.startedAt, '?');
     console.log(`${i + 1}. [${r.score?.toFixed(3)}] ${title} (${date}, ${r.agentId})`);
     if (ss.overview) console.log(`   ${ss.overview.slice(0, 200)}`);
     if (r.matchedTurnText) console.log(`   > ${r.matchedTurnText.slice(0, 150)}`);
@@ -113,7 +134,7 @@ async function cmdFeedback(aquifer, args) {
 }
 
 async function cmdBackfill(aquifer, args) {
-  const limit = parseInt(args.flags.limit || '100', 10);
+  const limit = parsePositiveInt(args.flags.limit, 100);
   const dryRun = !!args.flags['dry-run'];
   const skipSummary = !!args.flags['skip-summary'];
   const skipTurnEmbed = !!args.flags['skip-turn-embed'];
@@ -159,13 +180,81 @@ async function cmdStats(aquifer, args) {
     console.log(`Summaries: ${stats.summaries}`);
     console.log(`Turn embeddings: ${stats.turnEmbeddings}`);
     console.log(`Entities: ${stats.entities}`);
-    if (stats.earliest) console.log(`Range: ${new Date(stats.earliest).toISOString().slice(0, 10)} — ${new Date(stats.latest).toISOString().slice(0, 10)}`);
+    if (stats.earliest) console.log(`Range: ${formatDate(stats.earliest, '?')} — ${formatDate(stats.latest, '?')}`);
+  }
+}
+
+async function cmdQuickstart(aquifer) {
+  console.log('Aquifer quickstart — verifying end-to-end setup.\n');
+
+  // 1. Migrate
+  console.log('1/5  Running migrations...');
+  await aquifer.migrate();
+  console.log('     OK\n');
+
+  // 2. Commit
+  const sessionId = `quickstart-${Date.now()}`;
+  console.log('2/5  Committing test session...');
+  await aquifer.commit(sessionId, [
+    { role: 'user', content: 'We decided to use PostgreSQL with pgvector for the AI memory store instead of a separate vector database.' },
+    { role: 'assistant', content: 'Good choice. PG gives us ACID transactions, full-text search, and vector similarity all in one place.' },
+    { role: 'user', content: 'The main advantage is turn-level embedding — we can find the exact moment a decision was made.' },
+  ], { agentId: 'quickstart', source: 'quickstart' });
+  console.log('     OK\n');
+
+  // 3. Enrich (skip summary — LLM may not be configured)
+  console.log('3/5  Enriching (turn embeddings)...');
+  const enrichResult = await aquifer.enrich(sessionId, {
+    agentId: 'quickstart',
+    skipSummary: true,
+    skipEntities: true,
+  });
+  console.log(`     OK — ${enrichResult.turnsEmbedded} turns embedded\n`);
+
+  // 4. Recall
+  console.log('4/5  Recalling "PostgreSQL memory store"...');
+  const results = await aquifer.recall('PostgreSQL memory store', { limit: 3 });
+  if (results.length === 0) {
+    console.error('     FAIL — no results returned. Check your embedding config.');
+    process.exitCode = 1;
+    return;
+  }
+  console.log(`     OK — ${results.length} result(s), top score: ${results[0].score?.toFixed(3)}`);
+  if (results[0].matchedTurnText) {
+    console.log(`     Matched: "${results[0].matchedTurnText.slice(0, 100)}..."`);
   }
+  console.log();
+
+  // 5. Cleanup
+  console.log('5/5  Cleaning up test data...');
+  const { Pool } = require('pg');
+  const { loadConfig } = require('./shared/config');
+  const config = loadConfig();
+  const pool = new Pool({ connectionString: config.db.url });
+  const schema = quoteIdentifier(config.schema || 'aquifer');
+  const tenantId = config.tenantId || 'default';
+  try {
+    await pool.query('BEGIN');
+    await pool.query(
+      `DELETE FROM ${schema}.sessions WHERE tenant_id = $1 AND agent_id = $2 AND session_id = $3`,
+      [tenantId, 'quickstart', sessionId]
+    );
+    await pool.query('COMMIT');
+  } catch (err) {
+    await pool.query('ROLLBACK').catch(() => {});
+    throw err;
+  } finally {
+    await pool.end();
+  }
+  console.log('     OK\n');
+
+  console.log('✓ Aquifer is working. You can now start the MCP server:');
+  console.log('  npx aquifer mcp');
 }
 
 async function cmdExport(aquifer, args) {
   const output = args.flags.output || null;
-  const limit = parseInt(args.flags.limit || '1000', 10);
+  const limit = parsePositiveInt(args.flags.limit, 1000);
 
   const rows = await aquifer.exportSessions({
     agentId: args.flags['agent-id'],
@@ -201,6 +290,7 @@ async function main() {
     console.log(`Usage: aquifer <command> [options]
 
 Commands:
+  quickstart                  Verify end-to-end setup (migrate → commit → enrich → recall)
   migrate                     Run database migrations
   recall <query>              Search sessions (requires embed config)
   feedback                    Record trust feedback on a session
@@ -250,6 +340,9 @@ Options:
 
   try {
     switch (command) {
+      case 'quickstart':
+        await cmdQuickstart(aquifer);
+        break;
       case 'migrate':
         await cmdMigrate(aquifer);
         break;

package/consumers/mcp.js

@@ -38,9 +38,11 @@ function formatResults(results, query) {
     const r = results[i];
     const ss = r.structuredSummary || {};
     const title = ss.title || r.summaryText?.slice(0, 60) || '(untitled)';
-    const date = r.startedAt
-      ? new Date(r.startedAt).toISOString().slice(0, 10)
-      : 'unknown';
+    let date = 'unknown';
+    if (r.startedAt) {
+      const parsed = new Date(r.startedAt);
+      if (!isNaN(parsed.getTime())) date = parsed.toISOString().slice(0, 10);
+    }
 
     lines.push(`### ${i + 1}. ${title} (${date}, ${r.agentId || 'default'})`);
     if (ss.overview || r.summaryText) {
@@ -65,6 +67,8 @@ async function main() {
     ({ StdioServerTransport } = require('@modelcontextprotocol/sdk/server/stdio.js'));
     ({ z } = require('zod'));
   } catch (e) {
+    const missingDep = e && (e.code === 'MODULE_NOT_FOUND' || /Cannot find module|^missing\b/i.test(e.message || ''));
+    if (!missingDep) throw e;
     process.stderr.write(
       'aquifer mcp requires @modelcontextprotocol/sdk and zod.\n' +
       'Install: npm install @modelcontextprotocol/sdk zod\n'
@@ -74,7 +78,7 @@ async function main() {
 
   const server = new McpServer({
     name: 'aquifer-memory',
-    version: '0.8.0',
+    version: '0.9.0',
   });
 
   server.tool(

package/consumers/openclaw-plugin.js

@@ -79,15 +79,19 @@ function normalizeEntries(rawEntries) {
   };
 }
 
+function formatDate(value) {
+  if (!value) return 'unknown';
+  const parsed = new Date(value);
+  return isNaN(parsed.getTime()) ? 'unknown' : parsed.toISOString().slice(0, 10);
+}
+
 function formatRecallResults(results) {
   if (results.length === 0) return 'No matching sessions found.';
 
   return results.map((r, i) => {
     const ss = r.structuredSummary || {};
     const title = ss.title || r.summaryText?.slice(0, 60) || '(untitled)';
-    const date = r.startedAt
-      ? new Date(r.startedAt).toISOString().slice(0, 10)
-      : 'unknown';
+    const date = formatDate(r.startedAt);
 
     const lines = [`### ${i + 1}. ${title} (${date}, ${r.agentId || 'default'})`];
     if (ss.overview || r.summaryText) {

package/core/aquifer.js

@@ -583,7 +583,22 @@ function createAquifer(config) {
         weights: overrideWeights,
         entities: explicitEntities,
         entityMode = 'any',
+        strictSearchErrors = false,
       } = opts;
+      const searchErrors = [];
+
+      function recordSearchError(pathName, err) {
+        searchErrors.push({
+          path: pathName,
+          message: err && err.message ? err.message : String(err),
+        });
+      }
+
+      function maybeThrowSearchErrors() {
+        if (!strictSearchErrors || searchErrors.length === 0) return;
+        const details = searchErrors.map(e => `${e.path}: ${e.message}`).join('; ');
+        throw new Error(`Recall search failed: ${details}`);
+      }
 
       // Normalize agentId/agentIds into a single resolved value
       // agentIds takes precedence; agentId is sugar for agentIds: [agentId]
@@ -692,17 +707,26 @@ function createAquifer(config) {
         runFts
           ? storage.searchSessions(pool, query, {
               schema, tenantId, agentIds: resolvedAgentIds, source, dateFrom, dateTo, limit: fetchLimit, ftsConfig,
-            }).catch(() => [])
+            }).catch((err) => {
+              recordSearchError('fts', err);
+              return [];
+            })
           : Promise.resolve([]),
         runVector
           ? embeddingSearchSummaries(queryVec, {
               agentIds: resolvedAgentIds, source, dateFrom, dateTo, limit: fetchLimit,
-            }).catch(() => [])
+            }).catch((err) => {
+              recordSearchError('summary-vector', err);
+              return [];
+            })
           : Promise.resolve([]),
         runVector
           ? storage.searchTurnEmbeddings(pool, {
               schema, tenantId, queryVec, dateFrom, dateTo, agentIds: resolvedAgentIds, source, limit: fetchLimit,
-            }).catch(() => ({ rows: [] }))
+            }).catch((err) => {
+              recordSearchError('turn-vector', err);
+              return { rows: [] };
+            })
           : Promise.resolve({ rows: [] }),
       ]);
 
@@ -718,6 +742,7 @@ function createAquifer(config) {
       const filteredTurn = filterFn(turnRows);
 
       if (filteredFts.length === 0 && filteredEmb.length === 0 && filteredTurn.length === 0) {
+        maybeThrowSearchErrors();
         return [];
       }
 
@@ -737,7 +762,7 @@ function createAquifer(config) {
       const EXTERNAL_TIMEOUT = 10000;
       const externalRows = [];
       const externalPromises = [];
-      for (const [, sourceConfig] of sources) {
+      for (const [name, sourceConfig] of sources) {
         if (typeof sourceConfig.search === 'function') {
           const w = sourceConfig.weight !== null && sourceConfig.weight !== undefined ? sourceConfig.weight : 1.0;
           externalPromises.push(
@@ -750,7 +775,9 @@ function createAquifer(config) {
                   if (r && r.session_id) externalRows.push({ ...r, _externalWeight: w });
                 }
               }
-            }).catch(() => { /* external source failure/timeout non-fatal */ })
+            }).catch((err) => {
+              recordSearchError(`external:${name}`, err);
+            })
           );
         }
       }
@@ -835,6 +862,7 @@ function createAquifer(config) {
           hybridScore: r._hybridScore ?? r._score,
           rerankScore: r._rerankScore ?? null,
           rerankFallback: r._rerankFallback || false,
+          searchErrors: searchErrors.slice(),
         },
       }));
     },

package/core/hybrid-rank.js

@@ -36,12 +36,12 @@ function rrfFusion(ftsResults = [], embResults = [], turnResults = [], K = 60) {
 // timeDecay — sigmoid decay based on age in days
 // ---------------------------------------------------------------------------
 
-function timeDecay(startedAt, midpointDays = 45, steepness = 0.05) {
+function timeDecay(startedAt, midpointDays = 45, steepness = 0.05, nowMs = Date.now()) {
   if (!startedAt) return 0.5;
   const dt = typeof startedAt === 'string' ? new Date(startedAt) : startedAt;
   if (isNaN(dt.getTime())) return 0.5;
 
-  const ageDays = (Date.now() - dt.getTime()) / (1000 * 60 * 60 * 24);
+  const ageDays = (nowMs - dt.getTime()) / (1000 * 60 * 60 * 24);
   return 1 / (1 + Math.exp(steepness * (ageDays - midpointDays)));
 }
 
@@ -49,14 +49,14 @@ function timeDecay(startedAt, midpointDays = 45, steepness = 0.05) {
 // accessScore — exponential decay on access recency (30-day half-life)
 // ---------------------------------------------------------------------------
 
-function accessScore(accessCount, lastAccessedAt) {
+function accessScore(accessCount, lastAccessedAt, nowMs = Date.now()) {
   if (!accessCount || accessCount <= 0) return 0;
   if (!lastAccessedAt) return 0;
 
   const dt = typeof lastAccessedAt === 'string' ? new Date(lastAccessedAt) : lastAccessedAt;
   if (isNaN(dt.getTime())) return 0;
 
-  const daysSince = (Date.now() - dt.getTime()) / (1000 * 60 * 60 * 24);
+  const daysSince = (nowMs - dt.getTime()) / (1000 * 60 * 60 * 24);
   return accessCount * Math.exp(-0.693 * daysSince / 30);
 }
 
@@ -89,6 +89,7 @@ function hybridRank(ftsResults, embResults, turnResults, opts = {}) {
   } = opts;
 
   const w = { ...DEFAULT_WEIGHTS, ...weights };
+  const nowMs = opts.nowMs ?? Date.now();
 
   // Build allResults map: session_id → result object
   const allResults = new Map();
@@ -140,11 +141,12 @@ function hybridRank(ftsResults, embResults, turnResults, opts = {}) {
     const rawRrf = rrfScores.get(sessionId) || 0;
     const normRrf = maxRrf > 0 ? rawRrf / maxRrf : 0;
 
-    const td = timeDecay(result.started_at);
+    const td = timeDecay(result.started_at, 45, 0.05, nowMs);
 
     const accessEff = accessScore(
       result.access_count || 0,
       result.last_accessed_at,
+      nowMs,
     );
     const as = 1 - Math.exp(-accessEff / 5);
 

package/docs/setup.md

@@ -0,0 +1,194 @@
+# Aquifer Setup Guide
+
+This guide walks you through installing Aquifer and verifying a complete write → enrich → recall cycle. By the end, you will have a working MCP memory server that an agent host can connect to.
+
+## Prerequisites
+
+You need three things running before Aquifer can work:
+
+1. **PostgreSQL 15+** with the **pgvector** extension installed
+2. **Node.js 18+**
+3. **An embedding endpoint** — Ollama (local), OpenAI, or any OpenAI-compatible API
+
+## Step 1: Database
+
+### Option A: Docker (recommended for local dev)
+
+The repo includes a `docker-compose.yml` that starts PostgreSQL 16 with pgvector and Ollama with bge-m3 auto-pulled:
+
+```bash
+cd /path/to/aquifer
+docker compose up -d
+```
+
+This gives you a database at `postgresql://aquifer:aquifer@localhost:5432/aquifer` with pgvector ready, plus an Ollama server with bge-m3 for embeddings. First run takes a few minutes while the model downloads.
+
+### Option B: Existing PostgreSQL
+
+Make sure pgvector is installed. Connect as a superuser and run:
+
+```sql
+CREATE EXTENSION IF NOT EXISTS vector;
+```
+
+If your PostgreSQL was installed from a package manager, you may need to install the pgvector package separately. See [pgvector installation](https://github.com/pgvector/pgvector#installation).
+
+## Step 2: Install Aquifer
+
+```bash
+npm install @shadowforge0/aquifer-memory
+```
+
+All dependencies including MCP SDK and zod are installed automatically.
+
+## Step 3: Configure
+
+Aquifer reads configuration from three sources (in priority order):
+
+1. Config file: `aquifer.config.json` in the working directory, or set `AQUIFER_CONFIG=/path/to/config.json`
+2. Environment variables (see below)
+3. Programmatic overrides via `createAquifer()`
+
+### Minimum env vars for MCP recall
+
+```bash
+export DATABASE_URL="postgresql://aquifer:aquifer@localhost:5432/aquifer"
+export AQUIFER_EMBED_BASE_URL="http://localhost:11434/v1"
+export AQUIFER_EMBED_MODEL="bge-m3"
+```
+
+### Optional but common
+
+```bash
+# PG schema (default: aquifer) — useful for running multiple instances in one database
+export AQUIFER_SCHEMA="aquifer"
+
+# LLM for built-in summarization — without this, enrich requires a custom summaryFn
+export AQUIFER_LLM_BASE_URL="http://localhost:11434/v1"
+export AQUIFER_LLM_MODEL="llama3.1"
+
+# Knowledge graph
+export AQUIFER_ENTITIES_ENABLED="true"
+```
+
+Copy `.env.example` from the repo root for a full annotated list.
+
+## Step 4: Verify everything works
+
+```bash
+npx aquifer quickstart
+```
+
+This single command runs migrations, commits a test session, embeds it, recalls it, and cleans up. If it prints `✓ Aquifer is working`, your setup is correct.
+
+You can also run individual steps manually: `npx aquifer migrate`, `npx aquifer stats`, etc.
+
+## Step 5: Start the MCP server
+
+```bash
+npx aquifer mcp
+```
+
+The server starts on stdio and waits for MCP client connections. There is no visible output on success — the server is ready when the process stays running without error.
+
+### Verify with the library API (optional)
+
+If you want to test the library directly instead of the CLI:
+
+```javascript
+const { createAquifer, createEmbedder } = require('@shadowforge0/aquifer-memory');
+
+const embedder = createEmbedder({
+  provider: 'ollama',
+  ollamaUrl: 'http://localhost:11434',
+  model: 'bge-m3',
+});
+
+const aquifer = createAquifer({
+  db: process.env.DATABASE_URL,
+  schema: 'aquifer',
+  embed: { fn: (texts) => embedder.embedBatch(texts) },
+});
+
+await aquifer.migrate();
+
+// Commit a test session
+await aquifer.commit('test-001', [
+  { role: 'user', content: 'We decided to use PostgreSQL for the memory store.' },
+  { role: 'assistant', content: 'Good choice — PG gives us ACID, FTS, and pgvector in one place.' },
+], { agentId: 'test' });
+
+// Enrich (embed turns — summarization needs LLM config)
+await aquifer.enrich('test-001', { agentId: 'test', skipSummary: true });
+
+// Recall
+const results = await aquifer.recall('PostgreSQL memory', { limit: 3 });
+console.log('Results:', results.length); // Should be >= 1
+
+await aquifer.close();
+```
+
+## Connecting a host
+
+Once the MCP server is verified, connect your agent host:
+
+### Claude Code
+
+Add to `.claude.json` (project-level) or user-level MCP config:
+
+```json
+{
+  "mcpServers": {
+    "aquifer": {
+      "type": "stdio",
+      "command": "node",
+      "args": ["/absolute/path/to/aquifer/consumers/mcp.js"],
+      "env": {
+        "DATABASE_URL": "postgresql://aquifer:aquifer@localhost:5432/aquifer",
+        "AQUIFER_EMBED_BASE_URL": "http://localhost:11434/v1",
+        "AQUIFER_EMBED_MODEL": "bge-m3"
+      }
+    }
+  }
+}
+```
+
+Tools appear as `mcp__aquifer__session_recall`, `mcp__aquifer__session_feedback`, `mcp__aquifer__memory_stats`, `mcp__aquifer__memory_pending`.
+
+### OpenClaw
+
+Add to `openclaw.json`:
+
+```json
+{
+  "mcp": {
+    "servers": {
+      "aquifer": {
+        "command": "node",
+        "args": ["/absolute/path/to/aquifer/consumers/mcp.js"],
+        "env": {
+          "DATABASE_URL": "postgresql://...",
+          "AQUIFER_EMBED_BASE_URL": "http://localhost:11434/v1",
+          "AQUIFER_EMBED_MODEL": "bge-m3"
+        }
+      }
+    }
+  }
+}
+```
+
+Tools materialize as `aquifer__session_recall`, `aquifer__session_feedback`, `aquifer__memory_stats`, `aquifer__memory_pending`.
+
+Do **not** use the OpenClaw plugin (`consumers/openclaw-plugin.js`) for tool delivery. The plugin is retained for session capture via `before_reset` only.
+
+## Troubleshooting
+
+**`error: type "vector" does not exist`** — pgvector is not installed. Use the `pgvector/pgvector` Docker image, or install the extension manually: `CREATE EXTENSION IF NOT EXISTS vector;` (requires superuser).
+
+**`aquifer mcp requires @modelcontextprotocol/sdk and zod`** — These are regular dependencies and should be installed automatically. Run `npm install` again to ensure all deps are present.
+
+**Recall returns empty results** — Sessions must be enriched before they are searchable. Run `npx aquifer stats` and check that summaries and/or turn embeddings exist. If not, run `npx aquifer backfill` to enrich pending sessions.
+
+**`ECONNREFUSED` on embed calls** — Your embedding endpoint is not reachable. For Ollama: make sure it is running (`ollama serve`) and the model is pulled (`ollama pull bge-m3`).
+
+**Enrich fails with "no LLM configured"** — The built-in summarizer needs `AQUIFER_LLM_BASE_URL` + `AQUIFER_LLM_MODEL`. Alternatively, pass `skipSummary: true` to enrich without summarization (turn embeddings still work), or provide your own `summaryFn`.

package/package.json

@@ -1,14 +1,16 @@
 {
   "name": "@shadowforge0/aquifer-memory",
-  "version": "0.8.0",
-  "description": "PG-native long-term memory for AI agents. Turn-level embedding, hybrid RRF ranking, optional knowledge graph. Includes CLI, MCP server, and OpenClaw plugin.",
+  "version": "0.9.1",
+  "description": "PG-native long-term memory for AI agents. Turn-level embedding, hybrid RRF ranking, optional knowledge graph. MCP server, CLI, and library API.",
   "main": "index.js",
   "files": [
     "index.js",
     "core/",
     "pipeline/",
     "schema/",
-    "consumers/"
+    "consumers/",
+    "docs/",
+    "scripts/"
   ],
   "bin": {
     "aquifer": "./consumers/cli.js"
@@ -32,10 +34,8 @@
   },
   "author": "shadowforge0",
   "dependencies": {
-    "pg": "^8.13.0"
-  },
-  "optionalDependencies": {
     "@modelcontextprotocol/sdk": "^1.29.0",
+    "pg": "^8.13.0",
     "zod": "^3.25.76"
   },
   "engines": {

package/scripts/smoke.mjs

@@ -0,0 +1,115 @@
+#!/usr/bin/env node
+
+/**
+ * Aquifer smoke test — validates the full write → enrich → recall cycle.
+ *
+ * Prerequisites:
+ *   - DATABASE_URL set to a PostgreSQL database with pgvector
+ *   - AQUIFER_EMBED_BASE_URL + AQUIFER_EMBED_MODEL set (e.g., Ollama bge-m3)
+ *
+ * Usage:
+ *   node scripts/smoke.mjs
+ */
+
+import { createRequire } from 'module';
+const require = createRequire(import.meta.url);
+
+const { createAquifer, createEmbedder } = require('../index.js');
+const { loadConfig } = require('../consumers/shared/config.js');
+
+const config = loadConfig();
+
+if (!config.db.url) {
+  console.error('ERROR: DATABASE_URL is not set.');
+  process.exit(1);
+}
+
+if (!config.embed.baseUrl || !config.embed.model) {
+  console.error('ERROR: AQUIFER_EMBED_BASE_URL and AQUIFER_EMBED_MODEL must be set.');
+  process.exit(1);
+}
+
+// Build embedder
+const isOllama = config.embed.baseUrl.includes('11434') || config.embed.baseUrl.includes('ollama');
+const embedder = isOllama
+  ? createEmbedder({
+      provider: 'ollama',
+      ollamaUrl: config.embed.baseUrl.replace(/\/v1\/?$/, ''),
+      model: config.embed.model,
+    })
+  : createEmbedder({
+      provider: 'openai',
+      openaiApiKey: config.embed.apiKey || '',
+      openaiModel: config.embed.model,
+    });
+
+const aquifer = createAquifer({
+  db: config.db.url,
+  schema: config.schema || 'aquifer',
+  tenantId: config.tenantId || 'default',
+  embed: { fn: (texts) => embedder.embedBatch(texts), dim: config.embed.dim || null },
+  entities: { enabled: false },
+});
+
+const SESSION_ID = `smoke-test-${Date.now()}`;
+
+try {
+  // 1. Migrate
+  console.log('1. Running migrations...');
+  await aquifer.migrate();
+  console.log('   OK');
+
+  // 2. Commit a test session
+  console.log('2. Committing test session...');
+  const commitResult = await aquifer.commit(SESSION_ID, [
+    { role: 'user', content: 'We decided to use PostgreSQL with pgvector for the AI memory store instead of a separate vector database.' },
+    { role: 'assistant', content: 'Good choice. PG gives us ACID transactions, full-text search, and vector similarity all in one place.' },
+    { role: 'user', content: 'The main advantage is turn-level embedding — we can find the exact moment a decision was made.' },
+  ], { agentId: 'smoke-test', source: 'smoke' });
+  console.log(`   OK — session ${commitResult.isNew ? 'created' : 'updated'}`);
+
+  // 3. Enrich (skip summary since LLM may not be configured)
+  console.log('3. Enriching (turn embeddings, skip summary)...');
+  const enrichResult = await aquifer.enrich(SESSION_ID, {
+    agentId: 'smoke-test',
+    skipSummary: true,
+    skipEntities: true,
+  });
+  console.log(`   OK — ${enrichResult.turnsEmbedded} turns embedded`);
+
+  // 4. Recall
+  console.log('4. Recalling "PostgreSQL memory store"...');
+  const results = await aquifer.recall('PostgreSQL memory store', { limit: 3 });
+  if (results.length === 0) {
+    console.error('   FAIL — no results returned');
+    process.exit(1);
+  }
+  console.log(`   OK — ${results.length} result(s), top score: ${results[0].score?.toFixed(3)}`);
+  if (results[0].matchedTurnText) {
+    console.log(`   Matched turn: "${results[0].matchedTurnText.slice(0, 100)}..."`);
+  }
+
+  // 5. Stats
+  console.log('5. Checking stats...');
+  const stats = await aquifer.getStats();
+  console.log(`   Sessions: ${stats.sessionTotal}, Turn embeddings: ${stats.turnEmbeddings}`);
+
+  // 6. Cleanup — remove smoke test session
+  console.log('6. Cleaning up...');
+  const { Pool } = require('pg');
+  const pool = new Pool({ connectionString: config.db.url });
+  const schema = config.schema || 'aquifer';
+  await pool.query(`DELETE FROM ${schema}.turn_embeddings WHERE session_id IN (SELECT id FROM ${schema}.sessions WHERE session_id = $1)`, [SESSION_ID]);
+  await pool.query(`DELETE FROM ${schema}.session_summaries WHERE session_id IN (SELECT id FROM ${schema}.sessions WHERE session_id = $1)`, [SESSION_ID]);
+  await pool.query(`DELETE FROM ${schema}.sessions WHERE session_id = $1`, [SESSION_ID]);
+  await pool.end();
+  console.log('   OK');
+
+  console.log('\n✓ smoke test passed');
+} catch (err) {
+  console.error(`\n✗ smoke test failed: ${err.message}`);
+  if (err.stack) console.error(err.stack);
+  process.exit(1);
+} finally {
+  await aquifer.close();
+}