recallnest 2.2.0 → 2.3.0

package/docs/superpowers/plans/2026-04-11-constructive-retrieval-emotion.md

@@ -1,1195 +0,0 @@
-# Constructive Retrieval + Emotional Valence Implementation Plan
-
-> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
-
-**Goal:** Add simulation-theory-inspired context reconstruction and emotional valence scoring to RecallNest's retrieval and decay pipelines.
-
-**Architecture:** Two independent features behind feature flags. Emotion valence adds a metadata field + heuristic detector + decay/retrieval scoring stages. Constructive retrieval adds a post-pipeline LLM reconstruction layer with triple-fallback grounding. Both degrade gracefully to current behavior when disabled.
-
-**Tech Stack:** Bun, TypeScript strict, LanceDB, Jina v5 embeddings, OpenAI-compatible LLM API, Zod validation.
-
-**Spec:** `docs/superpowers/specs/2026-04-11-constructive-retrieval-emotion-design.md`
-
----
-
-## File Map
-
-| File | Action | Responsibility |
-|------|--------|----------------|
-| `src/emotion-detector.ts` | **CREATE** | Heuristic + LLM emotion detection |
-| `src/context-reconstructor.ts` | **CREATE** | Reconstruction engine + grounding validator |
-| `src/memory-schema.ts` | MODIFY | Add EmotionMetadata type |
-| `src/decay-engine.ts` | MODIFY | Add emotion-adjusted half-life |
-| `src/retriever.ts` | MODIFY | Add emotion scoring stage + reconstruction gate |
-| `src/store.ts` | MODIFY | Call emotion detector at store time |
-| `src/context-composer.ts` | MODIFY | Call reconstructor in resume flow |
-| `src/mcp-server.ts` | MODIFY | Add `reconstruct` param to search_memory |
-| `src/llm-client.ts` | MODIFY | Extend smartExtract prompt for emotion |
-| `src/__tests__/emotion-detector.test.ts` | **CREATE** | Tests for emotion detection |
-| `src/__tests__/emotion-decay.test.ts` | **CREATE** | Tests for emotion-adjusted decay |
-| `src/__tests__/emotion-retrieval.test.ts` | **CREATE** | Tests for emotion scoring stage |
-| `src/__tests__/context-reconstructor.test.ts` | **CREATE** | Tests for reconstruction + grounding |
-
----
-
-## Phase 1: Emotional Valence (Tasks 1-6)
-
-### Task 1: EmotionMetadata Type + Feature Flag
-
-**Files:**
-- Modify: `src/memory-schema.ts` (add type after line 37)
-
-- [ ] **Step 1: Add EmotionMetadata type to memory-schema.ts**
-
-Open `src/memory-schema.ts`. After the existing type exports (line 37), add:
-
-```typescript
-// --- Emotional Valence (Philosophy of Memory: Affective Memory) ---
-export interface EmotionMetadata {
-  /** Negative (-1) to Positive (+1) */
-  valence: number;
-  /** Calm (0) to Excited (1) */
-  arousal: number;
-  /** Human-readable label */
-  label?: string;
-}
-
-export const EmotionMetadataSchema = z.object({
-  valence: z.number().min(-1).max(1),
-  arousal: z.number().min(0).max(1),
-  label: z.string().max(30).optional(),
-});
-
-/** Parse emotion from metadata JSON string, returns null if absent */
-export function parseEmotion(metadata: string | undefined): EmotionMetadata | null {
-  if (!metadata) return null;
-  try {
-    const parsed = JSON.parse(metadata);
-    if (parsed.emotion && typeof parsed.emotion.valence === "number") {
-      return EmotionMetadataSchema.parse(parsed.emotion);
-    }
-  } catch { /* malformed metadata - safe to ignore */ }
-  return null;
-}
-
-/** Default neutral emotion for memories without emotion data */
-export const NEUTRAL_EMOTION: EmotionMetadata = { valence: 0, arousal: 0, label: "neutral" };
-```
-
-- [ ] **Step 2: Add feature flag function**
-
-Add to the bottom of `src/memory-schema.ts`, following the pattern in `src/multi-vector.ts:24-26`:
-
-```typescript
-export function isEmotionScoringEnabled(): boolean {
-  return process.env.RECALLNEST_EMOTION_SCORING === "true";
-}
-```
-
-- [ ] **Step 3: Verify TypeScript compiles**
-
-Run: `cd /Users/anxianjingya/Projects/recallnest && bun run build 2>&1 | head -20`
-Expected: No errors related to emotion types.
-
-- [ ] **Step 4: Commit**
-
-```bash
-cd /Users/anxianjingya/Projects/recallnest
-git add src/memory-schema.ts
-git commit -m "feat: add EmotionMetadata type and feature flag for emotion scoring"
-```
-
----
-
-### Task 2: Emotion Detector Module
-
-**Files:**
-- Create: `src/emotion-detector.ts`
-- Create: `src/__tests__/emotion-detector.test.ts`
-
-- [ ] **Step 1: Write the failing tests**
-
-Create `src/__tests__/emotion-detector.test.ts`:
-
-```typescript
-import { describe, test, expect } from "bun:test";
-import { detectEmotion } from "../emotion-detector.js";
-
-describe("emotion-detector: heuristic", () => {
-  test("detects strong negative text", () => {
-    const result = detectEmotion("This bug is driving me crazy, everything is broken and wrong");
-    expect(result.valence).toBeLessThan(-0.3);
-    expect(result.label).toBe("negative");
-  });
-
-  test("detects strong positive text", () => {
-    const result = detectEmotion("Finally solved it! Works perfectly, great progress today");
-    expect(result.valence).toBeGreaterThan(0.3);
-    expect(result.label).toBe("positive");
-  });
-
-  test("detects neutral text", () => {
-    const result = detectEmotion("The user prefers dark mode and uses VS Code");
-    expect(Math.abs(result.valence)).toBeLessThan(0.3);
-    expect(result.label).toBe("neutral");
-  });
-
-  test("detects high arousal", () => {
-    const result = detectEmotion("URGENT! Critical production issue, fix immediately!");
-    expect(result.arousal).toBeGreaterThan(0.3);
-  });
-
-  test("detects low arousal for calm text", () => {
-    const result = detectEmotion("The default configuration uses port 3000");
-    expect(result.arousal).toBeLessThan(0.3);
-  });
-
-  test("handles mixed signals with net positive", () => {
-    const result = detectEmotion("Had a frustrating bug but finally solved it perfectly");
-    expect(result.valence).toBeGreaterThan(0);
-  });
-
-  test("handles Chinese text", () => {
-    const result = detectEmotion("Too painful, keeps failing");
-    expect(result.valence).toBeLessThan(-0.3);
-  });
-
-  test("handles empty text gracefully", () => {
-    const result = detectEmotion("");
-    expect(result.valence).toBe(0);
-    expect(result.arousal).toBe(0);
-    expect(result.label).toBe("neutral");
-  });
-
-  test("clamps valence to [-1, 1]", () => {
-    const result = detectEmotion("broken error wrong bug failure hate broken error wrong");
-    expect(result.valence).toBeGreaterThanOrEqual(-1);
-    expect(result.valence).toBeLessThanOrEqual(1);
-  });
-
-  test("clamps arousal to [0, 1]", () => {
-    const result = detectEmotion("URGENT! CRITICAL! IMMEDIATELY! ASAP!");
-    expect(result.arousal).toBeGreaterThanOrEqual(0);
-    expect(result.arousal).toBeLessThanOrEqual(1);
-  });
-});
-```
-
-- [ ] **Step 2: Run tests to verify they fail**
-
-Run: `cd /Users/anxianjingya/Projects/recallnest && bun test src/__tests__/emotion-detector.test.ts`
-Expected: FAIL with module not found.
-
-- [ ] **Step 3: Implement emotion-detector.ts**
-
-Create `src/emotion-detector.ts`:
-
-```typescript
-import type { EmotionMetadata } from "./memory-schema.js";
-import { isEmotionScoringEnabled } from "./memory-schema.js";
-
-// --- Signal Dictionaries ---
-
-const NEGATIVE_SIGNALS: string[] = [
-  "fail", "failed", "failure", "broken", "bug", "error", "wrong", "crash",
-  "frustrat", "hate", "terrible", "awful", "annoying", "pain", "stuck",
-  "problem", "issue", "mess", "ugly", "worst",
-];
-
-const POSITIVE_SIGNALS: string[] = [
-  "solved", "fixed", "works", "perfect", "great", "love", "excellent",
-  "success", "breakthrough", "finally", "awesome", "beautiful", "clean",
-  "elegant", "smooth", "done", "shipped",
-];
-
-const HIGH_AROUSAL_SIGNALS: string[] = [
-  "!", "!!", "urgent", "critical", "immediately", "ASAP", "emergency",
-];
-
-function clamp(value: number, min: number, max: number): number {
-  return Math.max(min, Math.min(max, value));
-}
-
-/**
- * Detect emotional valence and arousal from text using keyword heuristics.
- * Zero LLM cost. Returns neutral emotion for empty text.
- */
-export function detectEmotion(text: string): EmotionMetadata {
-  if (!text || text.length === 0) {
-    return { valence: 0, arousal: 0, label: "neutral" };
-  }
-
-  const lower = text.toLowerCase();
-
-  const negCount = NEGATIVE_SIGNALS.filter(s => lower.includes(s.toLowerCase())).length;
-  const posCount = POSITIVE_SIGNALS.filter(s => lower.includes(s.toLowerCase())).length;
-  const arousalCount = HIGH_AROUSAL_SIGNALS.filter(s => text.includes(s)).length;
-
-  const valence = clamp((posCount - negCount) * 0.25, -1, 1);
-  const arousal = clamp(arousalCount * 0.25, 0, 1);
-  const label = valence > 0.3 ? "positive" : valence < -0.3 ? "negative" : "neutral";
-
-  return { valence, arousal, label };
-}
-
-/**
- * Conditionally detect emotion. Returns null when feature flag is off.
- */
-export function detectEmotionIfEnabled(text: string): EmotionMetadata | null {
-  if (!isEmotionScoringEnabled()) return null;
-  return detectEmotion(text);
-}
-```
-
-- [ ] **Step 4: Run tests to verify they pass**
-
-Run: `cd /Users/anxianjingya/Projects/recallnest && bun test src/__tests__/emotion-detector.test.ts`
-Expected: All 10 tests PASS.
-
-- [ ] **Step 5: Run full test suite**
-
-Run: `cd /Users/anxianjingya/Projects/recallnest && bun test`
-Expected: All existing tests pass + 10 new.
-
-- [ ] **Step 6: Commit**
-
-```bash
-cd /Users/anxianjingya/Projects/recallnest
-git add src/emotion-detector.ts src/__tests__/emotion-detector.test.ts
-git commit -m "feat: add heuristic emotion detector"
-```
-
----
-
-### Task 3: Emotion-Adjusted Decay
-
-**Files:**
-- Modify: `src/decay-engine.ts` (after line 87)
-- Create: `src/__tests__/emotion-decay.test.ts`
-
-- [ ] **Step 1: Write the failing tests**
-
-Create `src/__tests__/emotion-decay.test.ts`:
-
-```typescript
-import { describe, test, expect } from "bun:test";
-import { weibullDecay, adjustHalfLifeForEmotion, computeArousalBoost } from "../decay-engine.js";
-
-describe("emotion-adjusted decay", () => {
-  test("returns base half-life when no emotion", () => {
-    expect(adjustHalfLifeForEmotion(60, undefined)).toBe(60);
-    expect(adjustHalfLifeForEmotion(60, null)).toBe(60);
-  });
-
-  test("strong negative emotion extends half-life ~27%", () => {
-    const adjusted = adjustHalfLifeForEmotion(60, { valence: -0.9, arousal: 0.5, label: "frustration" });
-    expect(adjusted).toBeCloseTo(60 * 1.27, 0);
-  });
-
-  test("strong positive emotion extends half-life ~24%", () => {
-    const adjusted = adjustHalfLifeForEmotion(60, { valence: 0.8, arousal: 0.3, label: "excitement" });
-    expect(adjusted).toBeCloseTo(60 * 1.24, 0);
-  });
-
-  test("neutral emotion has negligible effect", () => {
-    const adjusted = adjustHalfLifeForEmotion(60, { valence: 0.05, arousal: 0.1, label: "neutral" });
-    expect(adjusted).toBeCloseTo(60, 0);
-  });
-
-  test("emotional memory decays slower at 30 days", () => {
-    const baseHL = 60;
-    const emotionalHL = adjustHalfLifeForEmotion(baseHL, { valence: -0.8, arousal: 0.7, label: "frustration" });
-    const neutralDecay = weibullDecay(30, baseHL, "working");
-    const emotionalDecay = weibullDecay(30, emotionalHL, "working");
-    expect(emotionalDecay).toBeGreaterThan(neutralDecay);
-  });
-
-  test("arousal boost is 1.0 for zero arousal", () => {
-    expect(computeArousalBoost({ valence: 0.5, arousal: 0, label: "positive" })).toBe(1.0);
-  });
-
-  test("arousal boost max ~1.1 for high arousal", () => {
-    const boost = computeArousalBoost({ valence: 0.5, arousal: 0.9, label: "excitement" });
-    expect(boost).toBeCloseTo(1.09, 1);
-  });
-
-  test("arousal boost is 1.0 for null emotion", () => {
-    expect(computeArousalBoost(undefined)).toBe(1.0);
-    expect(computeArousalBoost(null)).toBe(1.0);
-  });
-});
-```
-
-- [ ] **Step 2: Run tests to verify they fail**
-
-Run: `cd /Users/anxianjingya/Projects/recallnest && bun test src/__tests__/emotion-decay.test.ts`
-Expected: FAIL.
-
-- [ ] **Step 3: Implement in decay-engine.ts**
-
-Open `src/decay-engine.ts`. Add import at top:
-
-```typescript
-import type { EmotionMetadata } from "./memory-schema.js";
-```
-
-After the `weibullDecay` function (after line 87), add:
-
-```typescript
-/**
- * Adjust half-life based on emotional intensity.
- * Strong emotion extends half-life by up to 30%.
- */
-export function adjustHalfLifeForEmotion(
-  baseHalfLife: number,
-  emotion: EmotionMetadata | null | undefined,
-): number {
-  if (!emotion) return baseHalfLife;
-  const intensity = Math.abs(emotion.valence);
-  return baseHalfLife * (1 + 0.3 * intensity);
-}
-
-/**
- * Compute initial strength boost from arousal (flashbulb memory effect).
- * Returns multiplier in [1.0, 1.1].
- */
-export function computeArousalBoost(
-  emotion: EmotionMetadata | null | undefined,
-): number {
-  if (!emotion) return 1.0;
-  return 1 + 0.1 * emotion.arousal;
-}
-```
-
-- [ ] **Step 4: Run tests to verify they pass**
-
-Run: `cd /Users/anxianjingya/Projects/recallnest && bun test src/__tests__/emotion-decay.test.ts`
-Expected: All 8 tests PASS.
-
-- [ ] **Step 5: Run full test suite**
-
-Run: `cd /Users/anxianjingya/Projects/recallnest && bun test`
-Expected: All pass.
-
-- [ ] **Step 6: Commit**
-
-```bash
-cd /Users/anxianjingya/Projects/recallnest
-git add src/decay-engine.ts src/__tests__/emotion-decay.test.ts
-git commit -m "feat: add emotion-adjusted half-life and arousal boost to decay engine"
-```
-
----
-
-### Task 4: Emotion Scoring Stage in Retriever
-
-**Files:**
-- Modify: `src/retriever.ts`
-- Create: `src/__tests__/emotion-retrieval.test.ts`
-
-- [ ] **Step 1: Write the failing tests**
-
-Create `src/__tests__/emotion-retrieval.test.ts`:
-
-```typescript
-import { describe, test, expect } from "bun:test";
-import { applyEmotionWeight } from "../retriever.js";
-
-describe("applyEmotionWeight", () => {
-  const makeResult = (id: string, score: number, emotionJson?: object) => ({
-    id,
-    text: "test",
-    vector: [],
-    category: "events" as const,
-    scope: "test",
-    importance: 0.5,
-    timestamp: Date.now(),
-    metadata: emotionJson ? JSON.stringify({ emotion: emotionJson }) : "{}",
-    score,
-    sources: {},
-  });
-
-  test("null query emotion returns unchanged", () => {
-    const results = [makeResult("a", 0.8, { valence: 0.9, arousal: 0.5 })];
-    const output = applyEmotionWeight(results, null);
-    expect(output[0].score).toBe(0.8);
-  });
-
-  test("low-valence query returns unchanged", () => {
-    const queryEmotion = { valence: 0.1, arousal: 0, label: "neutral" };
-    const results = [makeResult("a", 0.8, { valence: 0.9, arousal: 0.5 })];
-    const output = applyEmotionWeight(results, queryEmotion);
-    expect(output[0].score).toBe(0.8);
-  });
-
-  test("negative query boosts negative memories", () => {
-    const queryEmotion = { valence: -0.8, arousal: 0.5, label: "frustration" };
-    const results = [
-      makeResult("neg", 0.7, { valence: -0.7, arousal: 0.5 }),
-      makeResult("pos", 0.7, { valence: 0.8, arousal: 0.3 }),
-    ];
-    const output = applyEmotionWeight(results, queryEmotion);
-    expect(output.find(r => r.id === "neg")!.score).toBeGreaterThan(
-      output.find(r => r.id === "pos")!.score
-    );
-  });
-
-  test("no emotion data leaves score unchanged", () => {
-    const queryEmotion = { valence: -0.8, arousal: 0.5, label: "negative" };
-    const results = [makeResult("no-emo", 0.7)];
-    const output = applyEmotionWeight(results, queryEmotion);
-    expect(output[0].score).toBe(0.7);
-  });
-
-  test("max boost is 15%", () => {
-    const queryEmotion = { valence: 1.0, arousal: 1.0, label: "positive" };
-    const results = [makeResult("a", 1.0, { valence: 1.0, arousal: 1.0 })];
-    const output = applyEmotionWeight(results, queryEmotion);
-    expect(output[0].score).toBeCloseTo(1.15, 2);
-  });
-});
-```
-
-- [ ] **Step 2: Run tests to verify they fail**
-
-Run: `cd /Users/anxianjingya/Projects/recallnest && bun test src/__tests__/emotion-retrieval.test.ts`
-Expected: FAIL.
-
-- [ ] **Step 3: Add applyEmotionWeight to retriever.ts**
-
-Add imports at top of `src/retriever.ts`:
-
-```typescript
-import { parseEmotion, isEmotionScoringEnabled } from "./memory-schema.js";
-import type { EmotionMetadata } from "./memory-schema.js";
-import { detectEmotion } from "./emotion-detector.js";
-```
-
-Add the exported function near other `apply*Weight` functions:
-
-```typescript
-/**
- * Emotion scoring stage: boost memories matching query emotional tone.
- * No-op when query is neutral or memory lacks emotion data.
- */
-export function applyEmotionWeight(
-  results: Array<{ score: number; metadata: string; [k: string]: unknown }>,
-  queryEmotion: EmotionMetadata | null,
-): typeof results {
-  if (!queryEmotion || Math.abs(queryEmotion.valence) < 0.2) {
-    return results;
-  }
-  return results.map(r => {
-    const memEmotion = parseEmotion(r.metadata);
-    if (!memEmotion) return r;
-    const alignment = 1 - Math.abs(queryEmotion.valence - memEmotion.valence) / 2;
-    const boost = 1.0 + 0.15 * alignment;
-    return { ...r, score: r.score * boost };
-  });
-}
-```
-
-Integrate into `retrieve()` pipeline after importance weight:
-
-```typescript
-if (isEmotionScoringEnabled()) {
-  const queryEmotion = detectEmotion(context.query);
-  scored = applyEmotionWeight(scored, queryEmotion);
-}
-```
-
-- [ ] **Step 4: Run tests**
-
-Run: `cd /Users/anxianjingya/Projects/recallnest && bun test src/__tests__/emotion-retrieval.test.ts`
-Expected: All 5 PASS.
-
-- [ ] **Step 5: Full suite**
-
-Run: `cd /Users/anxianjingya/Projects/recallnest && bun test`
-Expected: All pass.
-
-- [ ] **Step 6: Commit**
-
-```bash
-cd /Users/anxianjingya/Projects/recallnest
-git add src/retriever.ts src/__tests__/emotion-retrieval.test.ts
-git commit -m "feat: add emotion scoring stage to retrieval pipeline"
-```
-
----
-
-### Task 5: Store Path Integration
-
-**Files:**
-- Modify: `src/store.ts` (store method ~line 343, storeBatch ~line 369)
-
-- [ ] **Step 1: Add import**
-
-At top of `src/store.ts`:
-
-```typescript
-import { detectEmotionIfEnabled } from "./emotion-detector.js";
-```
-
-- [ ] **Step 2: Inject emotion in store() method**
-
-In `store()` (line 343), after metadata is set (line 350), add:
-
-```typescript
-const emotionResult = detectEmotionIfEnabled(entry.text);
-if (emotionResult) {
-  const existingMeta = JSON.parse(fullEntry.metadata || "{}");
-  existingMeta.emotion = emotionResult;
-  fullEntry.metadata = JSON.stringify(existingMeta);
-}
-```
-
-- [ ] **Step 3: Inject emotion in storeBatch() method**
-
-In `storeBatch()` (line 369), before the table add call, map entries:
-
-```typescript
-const enrichedEntries = fullEntries.map(e => {
-  const emotionResult = detectEmotionIfEnabled(e.text);
-  if (emotionResult) {
-    const meta = JSON.parse(e.metadata || "{}");
-    meta.emotion = emotionResult;
-    return { ...e, metadata: JSON.stringify(meta) };
-  }
-  return e;
-});
-```
-
-Use `enrichedEntries` in the table add.
-
-- [ ] **Step 4: Run full test suite**
-
-Run: `cd /Users/anxianjingya/Projects/recallnest && bun test`
-Expected: All pass (emotion is no-op when flag off).
-
-- [ ] **Step 5: Verify with flag ON**
-
-Run: `cd /Users/anxianjingya/Projects/recallnest && RECALLNEST_EMOTION_SCORING=true bun test`
-Expected: All pass.
-
-- [ ] **Step 6: Commit**
-
-```bash
-cd /Users/anxianjingya/Projects/recallnest
-git add src/store.ts
-git commit -m "feat: inject emotion metadata at store time"
-```
-
----
-
-### Task 6: LLM Emotion Piggyback (Optional)
-
-**Files:**
-- Modify: `src/llm-client.ts` (SmartExtraction interface ~line 128, smartExtract ~line 222)
-
-- [ ] **Step 1: Extend SmartExtraction interface**
-
-Add to `SmartExtraction` interface (line 128-138):
-
-```typescript
-emotion?: {
-  valence: number;
-  arousal: number;
-  label?: string;
-};
-```
-
-- [ ] **Step 2: Extend smartExtract system prompt**
-
-Append to the system prompt in smartExtract:
-
-```
-Also rate emotional tone:
-- emotion.valence: [-1, 1] (negative to positive)
-- emotion.arousal: [0, 1] (calm to excited)
-- emotion.label: one word (e.g., "frustration", "neutral")
-```
-
-- [ ] **Step 3: Parse emotion from response**
-
-In the JSON parsing section of smartExtract, add:
-
-```typescript
-if (isEmotionScoringEnabled() && parsed.emotion) {
-  result.emotion = {
-    valence: Math.max(-1, Math.min(1, parsed.emotion.valence ?? 0)),
-    arousal: Math.max(0, Math.min(1, parsed.emotion.arousal ?? 0)),
-    label: parsed.emotion.label,
-  };
-}
-```
-
-Add import: `import { isEmotionScoringEnabled } from "./memory-schema.js";`
-
-- [ ] **Step 4: Full test suite**
-
-Run: `cd /Users/anxianjingya/Projects/recallnest && bun test`
-Expected: All pass.
-
-- [ ] **Step 5: Commit**
-
-```bash
-cd /Users/anxianjingya/Projects/recallnest
-git add src/llm-client.ts
-git commit -m "feat: piggyback emotion detection on LLM smartExtract"
-```
-
----
-
-## Phase 2: Constructive Retrieval (Tasks 7-10)
-
-### Task 7: Context Reconstructor Core
-
-**Files:**
-- Create: `src/context-reconstructor.ts`
-- Create: `src/__tests__/context-reconstructor.test.ts`
-
-- [ ] **Step 1: Write the failing tests**
-
-Create `src/__tests__/context-reconstructor.test.ts`:
-
-```typescript
-import { describe, test, expect } from "bun:test";
-import {
-  shouldReconstruct,
-  extractCitedIds,
-  removeSentencesWithId,
-  computeCoverage,
-} from "../context-reconstructor.js";
-
-describe("reconstruction gate", () => {
-  test("false when flag off", () => {
-    expect(shouldReconstruct({ flagEnabled: false, callerOptIn: true, resultCount: 5, llmAvailable: true })).toBe(false);
-  });
-  test("false when no opt-in", () => {
-    expect(shouldReconstruct({ flagEnabled: true, callerOptIn: false, resultCount: 5, llmAvailable: true })).toBe(false);
-  });
-  test("false when too few results", () => {
-    expect(shouldReconstruct({ flagEnabled: true, callerOptIn: true, resultCount: 2, llmAvailable: true })).toBe(false);
-  });
-  test("false when LLM down", () => {
-    expect(shouldReconstruct({ flagEnabled: true, callerOptIn: true, resultCount: 5, llmAvailable: false })).toBe(false);
-  });
-  test("true when all conditions met", () => {
-    expect(shouldReconstruct({ flagEnabled: true, callerOptIn: true, resultCount: 5, llmAvailable: true })).toBe(true);
-  });
-});
-
-describe("extractCitedIds", () => {
-  test("extracts [src:ID] references", () => {
-    expect(extractCitedIds("A [src:abc]. B [src:def].")).toEqual(["abc", "def"]);
-  });
-  test("empty for no citations", () => {
-    expect(extractCitedIds("No refs.")).toEqual([]);
-  });
-  test("deduplicates", () => {
-    expect(extractCitedIds("A [src:abc]. B [src:abc].")).toEqual(["abc"]);
-  });
-});
-
-describe("removeSentencesWithId", () => {
-  test("removes sentence with invalid ID", () => {
-    const text = "Valid [src:real]. Fake [src:bad]. Also valid [src:ok].";
-    const result = removeSentencesWithId(text, "bad");
-    expect(result).toContain("[src:real]");
-    expect(result).toContain("[src:ok]");
-    expect(result).not.toContain("[src:bad]");
-  });
-  test("unchanged if ID not found", () => {
-    const text = "All valid [src:a].";
-    expect(removeSentencesWithId(text, "nope")).toBe(text);
-  });
-});
-
-describe("computeCoverage", () => {
-  test("high for identical text", () => {
-    expect(computeCoverage("user prefers dark mode", ["user prefers dark mode"])).toBeGreaterThan(0.8);
-  });
-  test("low for unrelated text", () => {
-    expect(computeCoverage("quantum physics wave function", ["user prefers dark mode"])).toBeLessThan(0.3);
-  });
-  test("partial for mixed", () => {
-    const cov = computeCoverage("User likes TypeScript. Weather is sunny.", ["User likes TypeScript", "User uses Bun"]);
-    expect(cov).toBeGreaterThan(0.3);
-    expect(cov).toBeLessThan(0.9);
-  });
-});
-```
-
-- [ ] **Step 2: Run tests to fail**
-
-Run: `cd /Users/anxianjingya/Projects/recallnest && bun test src/__tests__/context-reconstructor.test.ts`
-Expected: FAIL.
-
-- [ ] **Step 3: Implement context-reconstructor.ts**
-
-Create `src/context-reconstructor.ts`:
-
-```typescript
-import type { RetrievalResult } from "./retriever.js";
-import type { LLMClient } from "./llm-client.js";
-
-// --- Types ---
-
-export interface ReconstructionInput {
-  query: string;
-  results: RetrievalResult[];
-  mode: "resume" | "search";
-  maxTokens?: number;
-}
-
-export interface ReconstructionOutput {
-  reconstructed: string | null;
-  sources: string[];
-  confidence: number;
-  fallbackReason?: string;
-  raw: RetrievalResult[];
-}
-
-export interface GateConditions {
-  flagEnabled: boolean;
-  callerOptIn: boolean;
-  resultCount: number;
-  llmAvailable: boolean;
-}
-
-// --- Gate ---
-
-export function shouldReconstruct(c: GateConditions): boolean {
-  return c.flagEnabled && c.callerOptIn && c.resultCount >= 3 && c.llmAvailable;
-}
-
-// --- Grounding Utilities ---
-
-export function extractCitedIds(text: string): string[] {
-  const ids = new Set<string>();
-  const regex = /\[src:([^\]]+)\]/g;
-  let match: RegExpExecArray | null;
-  while ((match = regex.exec(text)) !== null) {
-    ids.add(match[1]);
-  }
-  return [...ids];
-}
-
-export function removeSentencesWithId(text: string, id: string): string {
-  const sentences = text.split(/(?<=[.!?])\s+/);
-  return sentences.filter(s => !s.includes(`[src:${id}]`)).join(" ");
-}
-
-/**
- * Word-overlap coverage: fraction of reconstructed sentences grounded in sources.
- * Lightweight heuristic, no embedding calls.
- */
-export function computeCoverage(reconstructed: string, sourceTexts: string[]): number {
-  const sentences = reconstructed.split(/(?<=[.!?])\s+/).filter(s => s.length > 5);
-  if (sentences.length === 0) return 0;
-
-  const sourceWords = new Set<string>();
-  for (const src of sourceTexts) {
-    for (const w of src.toLowerCase().split(/\s+/)) {
-      if (w.length > 2) sourceWords.add(w);
-    }
-  }
-
-  let covered = 0;
-  for (const sent of sentences) {
-    const words = sent.toLowerCase().split(/\s+/).filter(w => w.length > 2);
-    if (words.length === 0) continue;
-    const overlap = words.filter(w => sourceWords.has(w)).length / words.length;
-    if (overlap > 0.3) covered++;
-  }
-  return covered / sentences.length;
-}
-
-// --- Prompt ---
-
-function buildPrompt(input: ReconstructionInput): { system: string; user: string } {
-  const modeHint = input.mode === "resume"
-    ? "Focus on what the user was working on, key decisions, and pending actions."
-    : "Focus on the most relevant facts for the query.";
-
-  const system = `You are a memory reconstruction engine. Synthesize a coherent summary from stored memories.
-
-${modeHint}
-
-Rules:
-1. Every claim MUST cite [src:MEMORY_ID]
-2. Do NOT invent facts not in source memories
-3. Contradictions: present both with [conflict]
-4. Keep under ${input.maxTokens ?? 500} tokens`;
-
-  const block = input.results.slice(0, 10).map(r =>
-    `[ID: ${r.id}] ${r.text} (importance: ${r.importance})`
-  ).join("\n\n");
-
-  return { system, user: `Context: ${input.query}\n\nMemories:\n${block}\n\nReconstruct:` };
-}
-
-// --- Main Pipeline ---
-
-const TIMEOUT_MS = 3000;
-const COVERAGE_FLOOR = 0.6;
-
-export async function reconstruct(
-  input: ReconstructionInput,
-  llmClient: LLMClient,
-): Promise<ReconstructionOutput> {
-  const raw = input.results;
-
-  const timeout = new Promise<ReconstructionOutput>(resolve =>
-    setTimeout(() => resolve({
-      reconstructed: null, sources: [], confidence: 0, fallbackReason: "timeout", raw,
-    }), TIMEOUT_MS)
-  );
-
-  const work = (async (): Promise<ReconstructionOutput> => {
-    const { system, user } = buildPrompt(input);
-    const response = await llmClient.generateReconstruction(system, user);
-
-    if (!response) {
-      return { reconstructed: null, sources: [], confidence: 0, fallbackReason: "llm_empty", raw };
-    }
-
-    let text = response;
-    let confidence = 1.0;
-
-    // Layer 1: ID verification
-    const validIds = new Set(raw.map(r => r.id));
-    for (const id of extractCitedIds(text)) {
-      if (!validIds.has(id)) {
-        text = removeSentencesWithId(text, id);
-        confidence -= 0.2;
-      }
-    }
-    confidence = Math.max(0, confidence);
-
-    // Layer 2: Coverage
-    const coverage = computeCoverage(text, raw.map(r => r.text));
-    if (coverage < COVERAGE_FLOOR) {
-      return { reconstructed: null, sources: [], confidence: 0, fallbackReason: "low_grounding", raw };
-    }
-
-    const validSources = extractCitedIds(text).filter(id => validIds.has(id));
-    return {
-      reconstructed: text,
-      sources: validSources,
-      confidence: Math.min(confidence, coverage),
-      raw,
-    };
-  })();
-
-  return Promise.race([work, timeout]);
-}
-```
-
-- [ ] **Step 4: Run tests**
-
-Run: `cd /Users/anxianjingya/Projects/recallnest && bun test src/__tests__/context-reconstructor.test.ts`
-Expected: All 12 PASS.
-
-- [ ] **Step 5: Full suite**
-
-Run: `cd /Users/anxianjingya/Projects/recallnest && bun test`
-Expected: All pass.
-
-- [ ] **Step 6: Commit**
-
-```bash
-cd /Users/anxianjingya/Projects/recallnest
-git add src/context-reconstructor.ts src/__tests__/context-reconstructor.test.ts
-git commit -m "feat: context reconstructor with grounding validator and triple fallback"
-```
-
----
-
-### Task 8: LLM Client Extension
-
-**Files:**
-- Modify: `src/llm-client.ts` (add public methods)
-
-- [ ] **Step 1: Add generateReconstruction method**
-
-In `src/llm-client.ts`, after the existing public methods, add:
-
-```typescript
-/** Public wrapper for reconstruction pipeline */
-async generateReconstruction(system: string, user: string): Promise<string | null> {
-  return this.chat(system, user);
-}
-
-/** Check if LLM circuit breaker allows requests */
-isAvailable(): boolean {
-  return !this.circuitBreaker.isOpen();
-}
-```
-
-- [ ] **Step 2: Full test suite**
-
-Run: `cd /Users/anxianjingya/Projects/recallnest && bun test`
-Expected: All pass.
-
-- [ ] **Step 3: Commit**
-
-```bash
-cd /Users/anxianjingya/Projects/recallnest
-git add src/llm-client.ts
-git commit -m "feat: add generateReconstruction and isAvailable to LLMClient"
-```
-
----
-
-### Task 9: Retriever + Composer + MCP Integration
-
-**Files:**
-- Modify: `src/retriever.ts` (RetrievalContext + output, ~line 136-155, ~line 670)
-- Modify: `src/context-composer.ts` (resume flow, ~line 445)
-- Modify: `src/mcp-server.ts` (search_memory schema, ~line 889)
-
-- [ ] **Step 1: Extend RetrievalContext in retriever.ts**
-
-Add to `RetrievalContext` interface (line 136-155):
-
-```typescript
-/** Request constructive retrieval reconstruction */
-reconstruct?: boolean;
-```
-
-- [ ] **Step 2: Add reconstruction gate to retrieve() output**
-
-At end of `retrieve()` (around line 670), before final return, add:
-
-```typescript
-import { shouldReconstruct, reconstruct as runReconstruction, type ReconstructionOutput } from "./context-reconstructor.js";
-
-// In retrieve(), before return:
-const constructiveFlag = process.env.RECALLNEST_CONSTRUCTIVE_RETRIEVAL === "true";
-if (shouldReconstruct({
-  flagEnabled: constructiveFlag,
-  callerOptIn: context.reconstruct === true,
-  resultCount: finalResults.length,
-  llmAvailable: this.llmClient?.isAvailable() ?? false,
-})) {
-  try {
-    const reconstruction = await runReconstruction(
-      { query: context.query, results: finalResults, mode: "search" },
-      this.llmClient!,
-    );
-    // Attach to first result's metadata as side-channel
-    // (preserves return type compatibility)
-    if (finalResults.length > 0) {
-      const meta = JSON.parse(finalResults[0].metadata || "{}");
-      meta._reconstruction = {
-        text: reconstruction.reconstructed,
-        sources: reconstruction.sources,
-        confidence: reconstruction.confidence,
-        fallbackReason: reconstruction.fallbackReason,
-      };
-      finalResults[0] = { ...finalResults[0], metadata: JSON.stringify(meta) };
-    }
-  } catch {
-    // Silent degradation
-  }
-}
-```
-
-- [ ] **Step 3: Add reconstruct param to search_memory in mcp-server.ts**
-
-In `src/mcp-server.ts`, find search_memory input schema (line 889-906). Add:
-
-```typescript
-reconstruct: z.boolean().default(false).describe(
-  "Return LLM-synthesized reconstruction alongside raw results. Requires RECALLNEST_CONSTRUCTIVE_RETRIEVAL=true."
-),
-```
-
-Pass it through in the handler:
-
-```typescript
-reconstruct: input.reconstruct,
-```
-
-In the response formatting, check for reconstruction data:
-
-```typescript
-// After building the results sections
-const firstMeta = results[0] ? JSON.parse(results[0].metadata || "{}") : {};
-if (firstMeta._reconstruction?.text) {
-  const r = firstMeta._reconstruction;
-  sections.unshift(
-    `## Reconstructed Context (confidence: ${r.confidence.toFixed(2)})\n${r.text}\n\nSources: ${r.sources.join(", ")}`
-  );
-}
-```
-
-- [ ] **Step 4: Integrate into context-composer.ts**
-
-In `src/context-composer.ts`, in `composeResumeContext` (around line 445), add:
-
-```typescript
-import { shouldReconstruct, reconstruct as runReconstruction } from "./context-reconstructor.js";
-
-// After stable + patterns + cases assembly:
-const constructiveFlag = process.env.RECALLNEST_CONSTRUCTIVE_RETRIEVAL === "true";
-if (constructiveFlag && deps.llmClient?.isAvailable?.()) {
-  const allResults = [...(stableResults || []), ...(patternResults || []), ...(caseResults || [])];
-  if (allResults.length >= 3) {
-    try {
-      const taskQuery = latestCheckpoint?.summary ?? "general context";
-      const recon = await runReconstruction(
-        { query: taskQuery, results: allResults, mode: "resume", maxTokens: 600 },
-        deps.llmClient,
-      );
-      if (recon.reconstructed) {
-        response.reconstructedContext = recon.reconstructed;
-        response.reconstructionConfidence = recon.confidence;
-      }
-    } catch { /* silent fallback */ }
-  }
-}
-```
-
-Add `reconstructedContext?: string` and `reconstructionConfidence?: number` to the response schema.
-
-- [ ] **Step 5: Full test suite**
-
-Run: `cd /Users/anxianjingya/Projects/recallnest && bun test`
-Expected: All pass (feature flag off = no-op).
-
-- [ ] **Step 6: Commit**
-
-```bash
-cd /Users/anxianjingya/Projects/recallnest
-git add src/retriever.ts src/mcp-server.ts src/context-composer.ts
-git commit -m "feat: integrate constructive retrieval into search_memory and resume_context"
-```
-
----
-
-### Task 10: Integration Tests + Docs + Ship
-
-**Files:**
-- Create: `src/__tests__/feature-flags-integration.test.ts`
-- Modify: `CLAUDE.md` (feature flags section)
-
-- [ ] **Step 1: Write integration tests**
-
-Create `src/__tests__/feature-flags-integration.test.ts`:
-
-```typescript
-import { describe, test, expect, afterAll } from "bun:test";
-
-describe("feature flag isolation", () => {
-  const saved = {
-    emo: process.env.RECALLNEST_EMOTION_SCORING,
-    con: process.env.RECALLNEST_CONSTRUCTIVE_RETRIEVAL,
-  };
-
-  afterAll(() => {
-    if (saved.emo !== undefined) process.env.RECALLNEST_EMOTION_SCORING = saved.emo;
-    else delete process.env.RECALLNEST_EMOTION_SCORING;
-    if (saved.con !== undefined) process.env.RECALLNEST_CONSTRUCTIVE_RETRIEVAL = saved.con;
-    else delete process.env.RECALLNEST_CONSTRUCTIVE_RETRIEVAL;
-  });
-
-  test("emotion off -> detectEmotionIfEnabled returns null", () => {
-    delete process.env.RECALLNEST_EMOTION_SCORING;
-    const { detectEmotionIfEnabled } = require("../emotion-detector.js");
-    expect(detectEmotionIfEnabled("terrible bug")).toBeNull();
-  });
-
-  test("constructive off -> shouldReconstruct false", () => {
-    const { shouldReconstruct } = require("../context-reconstructor.js");
-    expect(shouldReconstruct({
-      flagEnabled: false, callerOptIn: true, resultCount: 10, llmAvailable: true,
-    })).toBe(false);
-  });
-});
-
-describe("emotion + decay end-to-end", () => {
-  test("emotional memory retains more score at 30 days", () => {
-    const { adjustHalfLifeForEmotion } = require("../decay-engine.js");
-    const { weibullDecay } = require("../decay-engine.js");
-
-    const baseHL = 60;
-    const emoHL = adjustHalfLifeForEmotion(baseHL, { valence: -0.8, arousal: 0.7, label: "frustration" });
-    expect(weibullDecay(30, emoHL, "working")).toBeGreaterThan(weibullDecay(30, baseHL, "working"));
-  });
-});
-
-describe("grounding end-to-end", () => {
-  test("removes phantom citations and lowers confidence", () => {
-    const { extractCitedIds, removeSentencesWithId } = require("../context-reconstructor.js");
-
-    const text = "Valid [src:r1]. Fake [src:phantom]. Also valid [src:r2].";
-    const validIds = new Set(["r1", "r2"]);
-    let result = text;
-    let confidence = 1.0;
-
-    for (const id of extractCitedIds(result)) {
-      if (!validIds.has(id)) {
-        result = removeSentencesWithId(result, id);
-        confidence -= 0.2;
-      }
-    }
-
-    expect(result).not.toContain("phantom");
-    expect(confidence).toBe(0.8);
-  });
-});
-```
-
-- [ ] **Step 2: Run integration tests**
-
-Run: `cd /Users/anxianjingya/Projects/recallnest && bun test src/__tests__/feature-flags-integration.test.ts`
-Expected: All PASS.
-
-- [ ] **Step 3: Full suite flags OFF**
-
-Run: `cd /Users/anxianjingya/Projects/recallnest && bun test`
-Expected: ALL pass.
-
-- [ ] **Step 4: Full suite flags ON**
-
-Run: `cd /Users/anxianjingya/Projects/recallnest && RECALLNEST_EMOTION_SCORING=true RECALLNEST_CONSTRUCTIVE_RETRIEVAL=true bun test`
-Expected: All pass.
-
-- [ ] **Step 5: Update CLAUDE.md feature flags**
-
-Add to section 5 in `CLAUDE.md`:
-
-```markdown
-- `RECALLNEST_EMOTION_SCORING=true` — Emotion detection + adjusted decay + retrieval scoring
-- `RECALLNEST_CONSTRUCTIVE_RETRIEVAL=true` — LLM context reconstruction with grounding (resume default, search opt-in)
-```
-
-- [ ] **Step 6: Commit + push**
-
-```bash
-cd /Users/anxianjingya/Projects/recallnest
-git add src/__tests__/feature-flags-integration.test.ts CLAUDE.md
-git commit -m "feat: integration tests and feature flag documentation for v1.5"
-git push
-```
-
----
-
-## Task Summary
-
-| # | Phase | Component | Risk | Est. LOC |
-|---|-------|-----------|------|----------|
-| 1 | Emotion | Types + flag | None | ~40 |
-| 2 | Emotion | Heuristic detector | None | ~70+80 |
-| 3 | Emotion | Decay adjustment | Low | ~25+50 |
-| 4 | Emotion | Retrieval scoring | Low | ~25+50 |
-| 5 | Emotion | Store integration | Low | ~15 |
-| 6 | Emotion | LLM piggyback | Low | ~20 |
-| 7 | Constructive | Reconstructor core | Medium | ~180+80 |
-| 8 | Constructive | LLM client extension | Low | ~15 |
-| 9 | Constructive | Pipeline integration | Medium | ~60 |
-| 10 | Both | Integration tests + docs | None | ~80 |
-| | | **Total** | | **~790** |