claude-recall 0.36.1 → 0.37.0

package/README.md

@@ -191,6 +191,7 @@ Once installed, Claude Recall works in the background (CC = Claude Code):
 | **Sub-agent spawned** | Rules are injected into the sub-agent; its outcome is captured | ✓ |  |  |
 | **Session exit** | An auto-checkpoint (`{completed, remaining, blockers}`) is saved for next time | ✓ | ✓ |  |
 | **End of session** | Failure patterns become candidate lessons; validated ones are promoted to rules | ✓ | ✓ |  |
+| **Hard-won success** | A goal that failed repeatedly then finally worked is captured as a reusable `solution` ([details](#success-capture)) | ✓ | ✓ |  |
 | **Once a day** | The memory janitor reviews stored rules with the runtime's LLM: demotes noise, merges duplicates, rewrites vague rules ([details](#memory-janitor)) | ✓ |  | ✓ |
 
 Classification runs on each runtime's **own** LLM — Claude Code via headless `claude -p` on your subscription; Kiro via `kiro-cli chat --no-interactive` on Kiro credits — with regex as the fallback. **An exported `ANTHROPIC_API_KEY` is never touched** unless you explicitly opt in with `CLAUDE_RECALL_PREFER_API_KEY=1`. No API key is ever required; no configuration needed.
@@ -277,7 +278,7 @@ claude-recall stats                      # Memory statistics (--global for all p
 claude-recall list                       # List memories, newest first (-t <type>, --all, --json, --global)
 claude-recall search "query"             # Search memories (--global, --json, --project <id>)
 claude-recall store "content"            # Store memory directly
-claude-recall store "content" -t <type>  # Type: preference, correction, failure, devops, project-knowledge
+claude-recall store "content" -t <type>  # Type: preference, correction, failure, devops, project-knowledge, solution
 claude-recall export backup.json         # Export current project (--global for all)
 claude-recall import backup.json         # Import memories from JSON
 claude-recall delete <key>               # Delete one memory by key (get keys from `search`)
@@ -353,6 +354,15 @@ action → outcome event → episode → candidate lesson → promotion → acti
 
 Failures become candidate lessons (deduplicated by similarity); lessons seen 2+ times (or once, if severe) are promoted to active rules; every just-in-time injection (Claude Code, Pi) is recorded and resolved against the tool's outcome, building per-rule effectiveness data over time.
 
+### Success capture
+
+Auto-capture is failure-biased by design — it learns from what breaks. But a hard-won *success* is just as reusable: the command, flag, or sequence you finally landed after several dead ends. Claude Recall captures those as a first-class `solution` memory, two ways:
+
+- **Automatically**, when a session shows a goal that failed **repeatedly** (≥2 distinct failed attempts) and then finally worked — it stores the reusable technique, generalized away from the one-off task. The multi-failure gate is deliberate: a first-try success or an unresolved struggle captures nothing, so routine wins don't become noise.
+- **Deliberately**, when you (or the agent) call `store_memory` with `type: "solution"` — the intended home for "I cracked this, don't make me re-derive it."
+
+A solution is active immediately (no wait for a second occurrence — you rarely crack the same hard thing twice), injected at every surface alongside your other rules, and ranked just below explicit corrections. List them with `claude-recall list --type solution`. Unlike other rules, solutions are exempt from the never-cited auto-demote sweep, so a rarely-needed-but-valuable win isn't retired.
+
 ### Memory janitor
 
 Automatic capture inevitably stores some noise — a conversational fragment misfiled as a preference, the same lesson in five wordings, a rule too vague to act on. Counters can flag *unused* rules (`CLAUDE_RECALL_AUTO_DEMOTE`), but they can't tell a rarely-cited gem from junk. Once a day, the **memory janitor** has the runtime's own LLM (same backend policy as capture — your Claude subscription or Kiro credits, never an API key unless you opted in) review the stored rules and:

package/dist/cli/claude-recall-cli.js

@@ -795,22 +795,44 @@ class ClaudeRecallCLI {
     }
     /**
      * Pull the human-readable text out of a memory value, whether it's a raw
-     * string, a JSON string, or a structured object with a content/value field.
+     * string, a JSON string, or a structured object. Failure memories nest their
+     * payload as an OBJECT under `content` ({ what_failed, why_failed, ... }), so
+     * a naive `String(value.content)` renders "[object Object]" — unwrap nested
+     * content/value/text wrappers and surface the failure gist instead.
      */
     extractContent(value) {
         let v = value;
-        if (typeof v === 'string') {
-            try {
-                v = JSON.parse(v);
+        // Descend through nested wrappers. The payload can be an object, a JSON
+        // string, or an object whose content/value/text is itself a JSON string
+        // (failure memories come in both shapes) — so parse-as-we-go, bounded.
+        for (let depth = 0; depth < 6; depth++) {
+            if (typeof v === 'string') {
+                const t = v.trim();
+                if (t.startsWith('{') || t.startsWith('[')) {
+                    try {
+                        v = JSON.parse(t);
+                        continue;
+                    }
+                    catch {
+                        return v;
+                    }
+                }
+                return v;
             }
-            catch {
-                return value;
+            if (v && typeof v === 'object' && !Array.isArray(v)) {
+                // Failure-shaped object: show what broke → what to do instead.
+                if (typeof v.what_failed === 'string') {
+                    return v.what_should_do ? `${v.what_failed} → ${v.what_should_do}` : v.what_failed;
+                }
+                const next = v.content ?? v.value ?? v.text;
+                if (next === undefined)
+                    return JSON.stringify(v);
+                v = next;
+                continue;
             }
+            break;
         }
-        if (v && typeof v === 'object') {
-            return String(v.content ?? v.value ?? v.text ?? JSON.stringify(v));
-        }
-        return String(v);
+        return typeof v === 'string' ? v : JSON.stringify(v);
     }
     /**
      * Export memories to a file

package/dist/cli/commands/kiro-commands.js

@@ -450,7 +450,7 @@ class KiroCommands {
                 : `project: ${projectId} (from working directory)`);
             line('•', `total memories (all projects): ${stats.total}`);
             const rules = ms.loadActiveRules(projectId);
-            const ruleCount = rules.preferences.length + rules.corrections.length + rules.failures.length + rules.devops.length;
+            const ruleCount = rules.preferences.length + rules.corrections.length + rules.failures.length + rules.devops.length + (rules.solutions ?? []).length;
             line(ruleCount > 0 ? '✓' : '•', `active rules for this project: ${ruleCount}`);
         }
         catch (err) {

package/dist/core/retrieval.js

@@ -270,6 +270,7 @@ class MemoryRetrieval {
 exports.MemoryRetrieval = MemoryRetrieval;
 MemoryRetrieval.TYPE_PRIORITY = {
     'correction': 6,
+    'solution': 5.5, // hard-won reusable solutions — high signal, rank just below corrections
     'project-knowledge': 5,
     'preference': 4,
     'devops': 3,

package/dist/hooks/kiro-hooks.js

@@ -125,11 +125,12 @@ function formatRulesForContext() {
     const sections = [
         section('Preferences', rules.preferences),
         section('Corrections', rules.corrections),
+        section('Solutions (hard-won — reuse these)', rules.solutions ?? []),
         section('Failures', rules.failures),
         section('DevOps Rules', rules.devops),
     ].filter((s) => s !== null);
     const total = rules.preferences.length + rules.corrections.length
-        + rules.failures.length + rules.devops.length;
+        + rules.failures.length + rules.devops.length + (rules.solutions ?? []).length;
     return { body: sections.join('\n\n'), total };
 }
 /**

package/dist/hooks/llm-classifier.js

@@ -257,6 +257,11 @@ const SESSION_EXTRACTION_PROMPT = `You are analyzing a coding session transcript
 
 The transcript shows tool calls (Bash, Edit, Read, Grep, etc.) and their results, plus user and assistant messages. Your primary job is to identify CAUSE-AND-EFFECT patterns — what failed, why, and what fixed it.
 
+PRIORITY 0 — Hard-won solutions (type "solution"):
+The MOST valuable thing to capture. Look for a goal the agent attempted, FAILED at REPEATEDLY (two or more distinct failed attempts, errors, wrong approaches, or retries on the same objective), and then FINALLY got working. Capture the reusable technique that cracked it — the command, config, flag, sequence, or approach that worked — phrased so a FUTURE session facing the same goal can apply it directly.
+STRICT gate: only emit a "solution" when there is clear evidence of (a) multiple failed attempts AND (b) an eventual success on that same goal. A task that worked first try, or that never resolved, is NOT a solution — skip it.
+Generalize away the one-off specifics (this repo's file names, this task's data) but KEEP the reusable mechanism. Example: after several failed attempts, "To submit to a Kaggle competition programmatically, use \`kaggle competitions submit -c <slug> -f <file> -m <msg>\` after \`kaggle config set -n competition -v <slug>\` — the web-form flow can't be scripted."
+
 PRIORITY 1 — Failure → Fix sequences:
 Look for tool calls that failed (errors, timeouts, non-zero exits) followed by a different approach that succeeded. Extract the lesson as an imperative rule.
 Examples:
@@ -280,7 +285,7 @@ Do NOT extract:
 - Anything in the EXISTING MEMORIES list below
 
 Respond with ONLY valid JSON (no markdown fences):
-[{"type":"project-knowledge|preference|devops|failure","content":"<imperative statement>","confidence":0.0-1.0}]
+[{"type":"solution|project-knowledge|preference|devops|failure","content":"<imperative statement>","confidence":0.0-1.0}]
 
 Return [] if nothing durable was learned. Max 10 items. Each content should be a concise, actionable rule (e.g. "Pipe 'y' to scripts/upgrade-sandbox.sh — it has an interactive confirmation prompt").`;
 /**
@@ -299,7 +304,7 @@ async function extractSessionLearningsWithLLM(summary, existingMemories) {
         const results = parseJSON(text);
         if (!Array.isArray(results))
             return null;
-        const validTypes = ['project-knowledge', 'preference', 'devops', 'failure'];
+        const validTypes = ['project-knowledge', 'preference', 'devops', 'failure', 'solution'];
         return results
             .filter((r) => r && validTypes.includes(r.type) && typeof r.content === 'string' && r.content.length > 5)
             .map((r) => ({

package/dist/hooks/post-compact-reload.js

@@ -31,6 +31,9 @@ function formatRules(rules) {
     if (rules.corrections.length > 0) {
         sections.push('## Corrections\n' + rules.corrections.map(m => `- ${extractVal(m.value)}`).join('\n'));
     }
+    if ((rules.solutions ?? []).length > 0) {
+        sections.push('## Solutions (hard-won — reuse these)\n' + rules.solutions.map(m => `- ${extractVal(m.value)}`).join('\n'));
+    }
     if (rules.failures.length > 0) {
         sections.push('## Failures\n' + rules.failures.map(m => `- ${extractVal(m.value)}`).join('\n'));
     }
@@ -44,7 +47,7 @@ async function handlePostCompactReload(_input) {
         const projectId = config_1.ConfigService.getInstance().getProjectId();
         const rules = memory_1.MemoryService.getInstance().loadActiveRules(projectId);
         const totalRules = rules.preferences.length + rules.corrections.length +
-            rules.failures.length + rules.devops.length;
+            rules.failures.length + rules.devops.length + (rules.solutions ?? []).length;
         if (totalRules === 0)
             return;
         const body = formatRules(rules);

package/dist/hooks/rule-injector.js

@@ -108,6 +108,7 @@ async function computeInjection(toolName, toolInput, toolUseId) {
         ...activeRules.corrections,
         ...activeRules.failures,
         ...activeRules.devops,
+        ...(activeRules.solutions ?? []),
     ].map(m => ({
         key: m.key,
         type: m.type,

package/dist/hooks/subagent-hooks.js

@@ -34,6 +34,9 @@ function formatRulesCompact(rules) {
     if (rules.corrections.length > 0) {
         sections.push('Corrections:\n' + rules.corrections.map(m => `- ${extractVal(m.value)}`).join('\n'));
     }
+    if ((rules.solutions ?? []).length > 0) {
+        sections.push('Solutions (hard-won — reuse these):\n' + rules.solutions.map(m => `- ${extractVal(m.value)}`).join('\n'));
+    }
     if (rules.failures.length > 0) {
         sections.push('Failures:\n' + rules.failures.map(m => `- ${extractVal(m.value)}`).join('\n'));
     }
@@ -52,7 +55,7 @@ async function handleSubagentStart(input) {
         const projectId = config_1.ConfigService.getInstance().getProjectId();
         const rules = memory_1.MemoryService.getInstance().loadActiveRules(projectId);
         const totalRules = rules.preferences.length + rules.corrections.length +
-            rules.failures.length + rules.devops.length;
+            rules.failures.length + rules.devops.length + (rules.solutions ?? []).length;
         if (totalRules === 0)
             return;
         const body = formatRulesCompact(rules);

package/dist/mcp/tools/memory-tools.js

@@ -108,7 +108,7 @@ class MemoryTools {
             },
             {
                 name: 'store_memory',
-                description: 'Store a rule or learning. Use for: corrections, preferences, devops rules, failures. The stored rule is immediately active in this conversation.',
+                description: 'Store a rule or learning. Use for: corrections, preferences, devops rules, failures, and solutions. IMPORTANT: when you crack a hard problem after real trial-and-error (a working command, config, or sequence you had to discover), store it with type "solution" — auto-capture only learns from failures, never from your wins, so a hard-won success is lost unless you save it here. The stored rule is immediately active in this conversation.',
                 inputSchema: {
                     type: 'object',
                     properties: {
@@ -118,7 +118,7 @@ class MemoryTools {
                         },
                         metadata: {
                             type: 'object',
-                            description: 'Optional metadata. Set "type" to one of: preference, correction, devops, failure'
+                            description: 'Optional metadata. Set "type" to one of: preference, correction, devops, failure, solution (use "solution" for a hard-won working fix you discovered through trial-and-error)'
                         },
                         scope: {
                             type: 'string',
@@ -146,7 +146,7 @@ class MemoryTools {
                         },
                         type: {
                             type: 'string',
-                            description: 'Filter by memory type: preference, correction, devops, failure, project-knowledge'
+                            description: 'Filter by memory type: preference, correction, devops, failure, project-knowledge, solution'
                         },
                         projectId: {
                             type: 'string',
@@ -221,7 +221,7 @@ class MemoryTools {
                 throw new Error('Content is required and must be a string');
             }
             // Use metadata.type as the memory type so it appears in future load_rules calls
-            const validTypes = ['preference', 'correction', 'devops', 'failure', 'project-knowledge', 'tool-use'];
+            const validTypes = ['preference', 'correction', 'devops', 'failure', 'project-knowledge', 'solution', 'tool-use'];
             const detectedType = (metadata?.type && validTypes.includes(metadata.type))
                 ? metadata.type
                 : 'preference';
@@ -336,8 +336,9 @@ class MemoryTools {
                 }
                 return kept;
             };
-            // Allocate in priority order (corrections first to protect high-signal items).
+            // Allocate in priority order (corrections + solutions first to protect high-signal items).
             const keptCorrections = takeBounded(rules.corrections);
+            const keptSolutions = takeBounded([...(rules.solutions ?? [])].sort(byCiteThenFresh));
             const keptPreferences = takeBounded([...rules.preferences].sort(byCiteThenFresh));
             const keptDevops = takeBounded([...rules.devops].sort(byCiteThenFresh));
             const keptFailures = takeBounded(rules.failures, 3);
@@ -376,6 +377,12 @@ class MemoryTools {
                     }).join('\n'));
                 }
             }
+            if (keptSolutions.length > 0) {
+                sections.push('## Solutions (hard-won — reuse these before re-deriving)\n' + keptSolutions.map(m => {
+                    const val = formatRuleValue(m.value) + precisionNudge(m.value);
+                    return `- ${val}`;
+                }).join('\n'));
+            }
             if (keptDevops.length > 0) {
                 sections.push('## DevOps Rules\n' + keptDevops.map(m => {
                     const val = formatRuleValue(m.value) + precisionNudge(m.value);
@@ -388,8 +395,8 @@ class MemoryTools {
                 sections.push(`*${droppedCount} more rules available via \`search_memory\`. Run \`npx claude-recall outcomes\` for full stats.*`);
             }
             const totalRules = keptPreferences.length + keptCorrections.length +
-                keptFailures.length + keptDevops.length;
-            const keptAll = [...keptPreferences, ...keptCorrections, ...keptFailures, ...keptDevops];
+                keptFailures.length + keptDevops.length + keptSolutions.length;
+            const keptAll = [...keptPreferences, ...keptCorrections, ...keptFailures, ...keptDevops, ...keptSolutions];
             const resultTokens = this.estimateTokens(keptAll);
             // Record to SearchMonitor so monitoring/stats still work
             this.searchMonitor.recordSearch('load_rules', totalRules, context.sessionId, 'mcp', { tool: 'load_rules', tokenMetrics: { resultTokens, tokensSaved: totalRules > 0 ? totalRules * 200 : 0 } });
@@ -436,6 +443,7 @@ class MemoryTools {
                     corrections: keptCorrections.length,
                     failures: keptFailures.length,
                     devops: keptDevops.length,
+                    solutions: keptSolutions.length,
                     total: totalRules,
                     dropped: droppedCount,
                 },

package/dist/memory/storage.js

@@ -1060,7 +1060,7 @@ class MemoryStorage {
 }
 exports.MemoryStorage = MemoryStorage;
 /** Rule-type memories: the only types subject to fuzzy dedup and retro-dedup. */
-MemoryStorage.RULE_TYPES = ['preference', 'correction', 'failure', 'devops', 'project-knowledge'];
+MemoryStorage.RULE_TYPES = ['preference', 'correction', 'failure', 'devops', 'project-knowledge', 'solution'];
 /**
  * Supersession sentinels written by automatic hygiene passes (as opposed to
  * a USER preference override, where superseded_by names the winning key).

package/dist/pi/extension.js

@@ -71,6 +71,9 @@ function formatRules(rules) {
     if (rules.corrections.length > 0) {
         sections.push('## Corrections\n' + rules.corrections.map(m => `- ${extractVal(m.value)}`).join('\n'));
     }
+    if ((rules.solutions ?? []).length > 0) {
+        sections.push('## Solutions (hard-won — reuse these)\n' + rules.solutions.map(m => `- ${extractVal(m.value)}`).join('\n'));
+    }
     if (rules.failures.length > 0) {
         sections.push('## Failures\n' + rules.failures.map(m => `- ${extractVal(m.value)}`).join('\n'));
     }

package/dist/services/memory.js

@@ -456,11 +456,18 @@ class MemoryService {
                 .slice(0, 5);
             // DevOps: all active rules
             const devops = this.storage.searchByContext({ ...searchContext, type: 'devops' }).filter(isActive);
+            // Solutions: hard-won reusable resolutions, top 5 by timestamp
+            const allSolutions = this.storage.searchByContext({ ...searchContext, type: 'solution' });
+            const solutions = allSolutions
+                .filter(isActive)
+                .sort((a, b) => (b.timestamp || 0) - (a.timestamp || 0))
+                .slice(0, 5);
             const counts = [
                 preferences.length && `${preferences.length} preferences`,
                 corrections.length && `${corrections.length} corrections`,
                 failures.length && `${failures.length} failures`,
                 devops.length && `${devops.length} devops rules`,
+                solutions.length && `${solutions.length} solutions`,
             ].filter(Boolean);
             const summary = counts.length > 0
                 ? `Loaded ${counts.join(', ')}`
@@ -468,16 +475,16 @@ class MemoryService {
             this.logger.info('MemoryService', summary, { projectId: pid });
             // Increment load_count for all returned rules
             const allIds = [
-                ...preferences, ...corrections, ...failures, ...devops
+                ...preferences, ...corrections, ...failures, ...devops, ...solutions
             ].map(m => m.id).filter((id) => id !== undefined);
             if (allIds.length > 0) {
                 this.storage.incrementLoadCounts(allIds);
             }
-            return { preferences, corrections, failures, devops, summary };
+            return { preferences, corrections, failures, devops, solutions, summary };
         }
         catch (error) {
             this.logger.logServiceError('MemoryService', 'loadActiveRules', error);
-            return { preferences: [], corrections: [], failures: [], devops: [], summary: 'Error loading rules' };
+            return { preferences: [], corrections: [], failures: [], devops: [], solutions: [], summary: 'Error loading rules' };
         }
     }
     /**

package/dist/shared/event-processors.js

@@ -440,7 +440,7 @@ async function extractSessionLearnings(entries, sessionId, projectId, maxStore =
         try {
             const ms = memory_1.MemoryService.getInstance();
             const rules = ms.loadActiveRules(projectId);
-            const all = [...rules.preferences, ...rules.corrections, ...rules.failures, ...rules.devops];
+            const all = [...rules.preferences, ...rules.corrections, ...rules.failures, ...rules.devops, ...(rules.solutions ?? [])];
             for (const m of all.slice(0, 20)) {
                 const val = typeof m.value === 'object' ? (m.value?.content || JSON.stringify(m.value)) : String(m.value);
                 existingMemories.push(truncate(val, 80));

package/docs/design-hybrid-retrieval-fts5.md

@@ -0,0 +1,185 @@
+# Design note: FTS5 / BM25 hybrid retrieval
+
+**Status:** proposal — not yet approved for implementation
+**Date:** 2026-07-22 · **Baseline:** v0.36.2
+**Scope of this note:** lexical BM25 ranking via SQLite FTS5. Local embeddings /
+semantic similarity are a deliberately-separate later phase (§9).
+
+---
+
+## 1. Goal
+
+Replace the crude `LIKE '%kw%'` candidate filter + `includes()` keyword boost
+with real **BM25 ranking** from SQLite's built-in FTS5 — improving recall for
+paraphrase and ranking quality as the corpus grows, **without adding a single
+dependency and without giving up the local-only / offline promise.**
+
+Non-goals: embeddings, vector search, entity graphs (see §9).
+
+## 2. Current retrieval, precisely
+
+Two stages, both in-process:
+
+1. **Candidate fetch** — `MemoryStorage.searchByContext()` (`src/memory/storage.ts:666`).
+   When keywords are present it hard-filters rows with `value LIKE ?` per keyword
+   (`storage.ts:698-713`): ≥3 keywords → require ≥2 matches; <3 → match any. The
+   match is against the **raw JSON string of the `value` column** — so it matches
+   field names as well as content, and misses any paraphrase.
+2. **Re-rank** — `MemoryRetrieval.calculateRelevance()` (`src/core/retrieval.ts:141`).
+   A multiplicative pipeline: base `relevance_score` × keyword boost
+   (`retrieval.ts:150-174`, up to ~6× via `String.includes`) × time-decay
+   forgetting curve (`:176-181`) × project/file boosts (`:184-189`) × strength
+   (`:191-193`) × evidence (`:196-198`) × helpfulness prior (`:201-204`) ×
+   staleness penalty (`:206-212`). Then sorted by `TYPE_PRIORITY` then score,
+   sliced to top 5 (`:105-128`).
+
+The weakest link is the **lexical layer** — steps 1 and the boost in step 2.
+Everything else (decay, strength, evidence, project scoping) is worth keeping
+exactly as-is; this change is surgical to the lexical component.
+
+## 3. Proposed design
+
+### 3.1 An external-content FTS5 table, kept in sync by triggers
+
+Add one virtual table plus three triggers. **No change to the `memories` table
+and no change to the TypeScript write path** — the triggers do the syncing in
+SQL, so every writer (upsert, import, janitor, migrations) stays covered
+automatically. That decoupling is the main reason to prefer this over a
+`search_text` column maintained in `save()`.
+
+```sql
+-- external-content FTS mirror of memories.value, keyed by memories.id
+CREATE VIRTUAL TABLE IF NOT EXISTS memories_fts USING fts5(
+  value,
+  content='memories',
+  content_rowid='id'
+);
+
+CREATE TRIGGER IF NOT EXISTS memories_ai AFTER INSERT ON memories BEGIN
+  INSERT INTO memories_fts(rowid, value) VALUES (new.id, new.value);
+END;
+CREATE TRIGGER IF NOT EXISTS memories_ad AFTER DELETE ON memories BEGIN
+  INSERT INTO memories_fts(memories_fts, rowid, value) VALUES('delete', old.id, old.value);
+END;
+CREATE TRIGGER IF NOT EXISTS memories_au AFTER UPDATE ON memories BEGIN
+  INSERT INTO memories_fts(memories_fts, rowid, value) VALUES('delete', old.id, old.value);
+  INSERT INTO memories_fts(rowid, value) VALUES (new.id, new.value);
+END;
+```
+
+**v1 indexes the raw `value` JSON.** The default `unicode61` tokenizer splits on
+braces/quotes/punctuation, so JSON structure mostly falls away; field-name tokens
+(`what_failed`, `content`) are minor noise and can be dropped later by indexing a
+derived plaintext projection (a refinement, not a v1 requirement).
+
+### 3.2 Candidate fetch via MATCH
+
+Replace the `LIKE` branch in `searchByContext` with an FTS `MATCH` that returns
+candidate ids **and** their BM25 rank, while preserving the existing scope
+predicate (`project_id = ? OR scope = 'universal' OR project_id IS NULL`,
+`storage.ts:680-684`) and type filter:
+
+```sql
+SELECT m.*, bm25(memories_fts) AS bm25_rank
+FROM memories_fts
+JOIN memories m ON m.id = memories_fts.rowid
+WHERE memories_fts MATCH ?
+  AND (m.project_id = ? OR m.scope = 'universal' OR m.project_id IS NULL)
+ORDER BY bm25_rank;      -- SQLite bm25 is negative; more-negative = better
+```
+
+Empty/purely-stopword queries keep today's behavior (return all scoped rows, no
+lexical filter) so `load_rules`-style "give me everything" calls are unaffected.
+
+### 3.3 Query sanitization (must-have, not optional)
+
+Raw user keywords fed to `MATCH` are a **syntax hazard** — bare `AND`/`OR`/`NEAR`,
+hyphens, quotes, and `*` are FTS5 operators and throw `SQLITE_ERROR` on malformed
+input. Every term must be wrapped as a quoted phrase and OR-joined:
+
+```
+"kaggle" OR "submission" OR "api"
+```
+
+A bad query must **fall back to the LIKE path**, never crash retrieval.
+
+### 3.4 Score fusion
+
+Compute a normalized `bm25Score ∈ [0,1]` per candidate (min-max across the
+candidate set, Mem0-style) and **replace** the `includes()` boost at
+`retrieval.ts:150-174` with it — leaving every other multiplicative term
+untouched:
+
+```ts
+// was: score *= 1 + matchRatio * 3.0 (+1.5 all-match / ×0.3 no-overlap)
+score *= 1 + wLexical * bm25Score;   // wLexical ~ 3.0 to preserve current dynamic range
+```
+
+Keeping the fusion multiplicative means decay/strength/evidence/project boosts
+behave identically — the only thing that changes is *how the lexical signal is
+measured*. This bounds the blast radius to one term.
+
+## 4. Migration & backfill
+
+In `migrateSchema()` (`storage.ts`, alongside the existing `CREATE INDEX IF NOT
+EXISTS` migrations ~`:119-149`):
+
+1. Feature-detect FTS5 (see §5). If absent → skip everything, leave a flag off.
+2. Create the vtable + triggers (idempotent).
+3. Backfill once: `INSERT INTO memories_fts(rowid, value) SELECT id, value FROM memories`
+   guarded by "only if the FTS table is empty."
+
+Backfill cost is a single scan — trivial at the 10k-row cap.
+
+## 5. Feature detection, fallback, rollback
+
+- **Detect** at init by attempting `CREATE VIRTUAL TABLE … USING fts5` in a
+  try/catch (verified working in the bundled `better-sqlite3`, but self-built or
+  exotic SQLite may lack it). On failure, set `ftsAvailable = false` and use the
+  existing LIKE path everywhere. **Retrieval must work identically with FTS off.**
+- **Rollback is safe and non-destructive:** the FTS table + triggers are derived,
+  redundant data. `DROP TRIGGER … ; DROP TABLE memories_fts;` reverts to LIKE with
+  zero risk to `memories`.
+
+## 6. Config / opt-in
+
+`CLAUDE_RECALL_RETRIEVAL = fts | like`. Ship **`like` as default** first so the
+change is inert on upgrade, flip to `fts` as default only after the benchmark
+(§7) shows a win. This mirrors how AUTO_DEMOTE / the janitor shipped off-by-default.
+
+## 7. Benchmark tie-in (finding #4)
+
+This change is the reason to stand up the LongMemEval-subset harness **first**:
+without a before/after number we can't tell whether BM25 actually helps or just
+adds surface area. Track both **recall/accuracy** and **tokens-per-query** (the
+2000-token `load_rules` budget is the natural denominator). The harness doubles as
+the regression guard for the fusion weights.
+
+## 8. Risks / open questions
+
+- **Tokenizing raw JSON** dilutes ranking with field-name tokens. Acceptable for
+  v1; the clean fix (derived plaintext column) means a write-path change.
+- **`is_active` / superseded rows**: `searchByContext` currently returns inactive
+  rows too (no `is_active` predicate at `storage.ts:677`). Confirm FTS candidate
+  fetch matches whatever the intended active-set semantics are — don't silently
+  change them in this PR.
+- **Fusion weight `wLexical`** needs tuning against the benchmark; the ×0.3
+  no-overlap penalty has no direct BM25 equivalent (non-matches simply aren't
+  returned by MATCH) — verify that doesn't over-promote weak matches.
+- **WAL + triggers**: triggers run inside the same transaction as the write; the
+  existing `wal_checkpoint(TRUNCATE)` after writes is unaffected.
+
+## 9. Later phase (out of scope here)
+
+Optional local embeddings (`sqlite-vec` or a small local model) for semantic
+similarity, gated behind an opt-in flag so the default stays dependency-free —
+fused as a third signal exactly like Mem0 (semantic + BM25 + entity). BM25 first;
+it captures most of the paraphrase win at none of the cost.
+
+## 10. Suggested PR breakdown
+
+1. FTS vtable + triggers + migration + feature-detect + backfill (no retrieval
+   wiring yet — pure additive, inert).
+2. LongMemEval-subset benchmark harness (measure the `LIKE` baseline).
+3. Candidate fetch via MATCH + sanitization + fusion, behind `CLAUDE_RECALL_RETRIEVAL=fts`.
+4. Flip default to `fts` once the benchmark confirms the win.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "claude-recall",
-  "version": "0.36.1",
+  "version": "0.37.0",
   "description": "Persistent memory for Claude Code and Pi with native Skills integration, automatic capture, failure learning, and project scoping",
   "main": "dist/index.js",
   "bin": {