@inetafrica/open-claudia 2.6.57 → 2.6.58

package/CHANGELOG.md

@@ -1,5 +1,13 @@
 # Changelog
 
+## v2.6.58
+- **Tiered recall gate — stop paying 15 seconds for "thanks".** Measured over 700+ live turns, the old discoverer pre-gate returned true whenever *any* seed matched, so it gated just 0.4% of turns — a bare "ok" with an incidental keyword hit still paid the full seed → graph → walker pass (p50 ~15s). A new `recallTier()` classifies each turn into three tiers: **skip** (pleasantries/acks up to 4 words *regardless of seeds*, emoji-only, or ≤2 words with no seeds — no recall at all), **seeds** (short command-like turns ≤4 words — user-origin keyword seeds inject directly with no graph expansion, no walker, no excerpts; the classic-engine baseline at ~ms cost), and **full** (substantive turns — the whole pipeline). The tier is logged per turn and the metrics summary now tracks the seeds-only share alongside the gated share.
+- **Walker candidate cap.** The latency deep-dive showed the walk itself dominates, not process spawn: 15–40 candidates × 600-char excerpts = 10–25KB prompts to the haiku judge on every substantive turn. Full-tier candidates are now priority-ordered — user-origin seeds by match score, then context seeds, then graph-activated nodes by activation — and capped at `RECALL_WALKER_MAX_CANDIDATES` (default 14), so the judge reads the strongest few instead of everything the graph touched.
+- **Episodic memory — recall now remembers how it did things last time.** The discoverer gains a transcript hop: on full-tier turns it queries the project-transcript FTS index (up to `RECALL_EPISODE_LIMIT`, default 3; `RECALL_EPISODES=off` disables) and offers the hits to the walker as `episode:` candidates with their matched snippets. The walker keeps one only if that past exchange holds a decision, method, or outcome directly reusable for the current message — and the fail-open path (walker off/failed) *drops* episodes rather than injecting unjudged snippets. Kept episodes render as a "Past work that looks related" block with the snippet plus a ready-to-run `open-claudia transcript-search … --all` pointer for the full context, and surface in the `/recall` banner as 📓 lines.
+- **Real cost + phase telemetry for recall.** The warm walker's stream-json `total_cost_usd` is cumulative per session; it now tracks the running total and reports the per-walk *delta*, which flows into `recall-metrics.jsonl` per turn (`costUsd`) instead of the flat 0 logged before. Each turn also records `seedMs` / `expandMs` / `walkerMs` and which walker path ran (`warm`/`cold`/`off`/`failed`), so the p50 can be decomposed instead of guessed at.
+- **Weak-use signal from the post-turn reviewer.** Precision metrics only counted explicit 📖 opens as "used". Now, when the reviewer decides a turn's work updated pack X or entity Y, that adjudication is logged as use (`source: "reviewer"`) and co-updated nodes get a Hebbian co-work reinforcement — so packs that quietly shape work without being re-opened stop looking like noise.
+- **Journal dedupe.** The reviewer could append near-identical Journal lines turn after turn (observed: 28 same-day duplicates on one busy pack). `updatePack` now token-Jaccard-compares a new journal line (dates stripped) against the last 5 entries and skips appends at ≥0.75 similarity, flagging `journalDeduped` instead. Adds `test-journal-dedupe.js`.
+
 ## v2.6.57
 - **Move the inactivity watchdog into the live runner.** Deep review found v2.6.56 put the watchdog in the legacy `bot-agent.js` monolith, but the LaunchAgent runs `bot.js`, which delegates heavy turns through `core/runner.js`. That meant the installed live path still only had the 6-hour hard timeout and could still wedge on a silent child. This release ports the watchdog to `core/runner.js`, updates activity on child stdout/stderr, terminates the process tree after `OC_TURN_IDLE_MS` of silence, suppresses the confusing partial/no-output final message, clears stuck busy state with a failsafe, and drains any queued follow-up messages instead of stranding them. Adds a static regression test that fails if the watchdog only exists in `bot-agent.js` again.
 

package/README.md

@@ -436,6 +436,8 @@ All stored in `~/.open-claudia/`:
 | `MEMORY_RECALL_MAX_CHARS` | No | Hard cap for auto-injected pack/entity memory per turn (default `9000`, `off` disables auto recall injection) |
 | `RECALL_ENGINE` | No | Default recall engine when a chat hasn't set one via `/engine` (`classic` or `discoverer`, default `classic`) |
 | `RECALL_GRAPH_DB` / `RECALL_METRICS` | No | Override the discoverer graph DB path; `off` on metrics disables per-turn recall logging |
+| `RECALL_WALKER_MAX_CANDIDATES` | No | Cap on candidates sent to the discoverer's walker judge per turn (default `14`) |
+| `RECALL_EPISODES` / `RECALL_EPISODE_LIMIT` | No | `off` disables episodic transcript candidates in recall; limit caps transcript hits per turn (default `3`) |
 | `PROJECT_TRANSCRIPTS` | No | Enable redacted project transcripts (default `true`) |
 | `TRANSCRIPT_MAX_ENTRY_CHARS` | No | Max chars per transcript entry (default `12000`) |
 | `TRANSCRIPTS_DIR` / `PACKS_DIR` / `ENTITIES_DIR` | No | Override storage directories |

package/core/pack-review.js

@@ -312,9 +312,11 @@ function reviewTurn({ userText, assistantText, channelId, announce }) {
     const decision = parseDecision(text);
     if (!decision || (decision.actions.length === 0 && decision.entities.length === 0 && (decision.lessons || []).length === 0)) return;
     const lines = [];
+    const usedIds = []; // nodes this turn's work demonstrably touched
     for (const a of decision.actions) {
       try {
         const r = applyAction(a);
+        if (r && (r.kind === "update" || r.kind === "create") && r.dir) usedIds.push(`pack:${r.dir}`);
         if (r && r.kind === "skipped") {
           lines.push(`🛡️ Skipped a memory write that looked like an injected instruction (${r.reason}).`);
         } else if (r && r.kind === "create" && r.ability) {
@@ -332,6 +334,7 @@ function reviewTurn({ userText, assistantText, channelId, announce }) {
       try {
         const r = applyEntityAction(ea);
         if (r) {
+          if (r.slug) usedIds.push(`entity:${r.slug}`);
           const emoji = ENTITY_EMOJI[r.type] || ENTITY_EMOJI.thing;
           lines.push(r.kind === "create"
             ? `${emoji} Now tracking ${r.name} (${r.type}) — ${clipWords(r.note, 160)}`
@@ -341,6 +344,16 @@ function reviewTurn({ userText, assistantText, channelId, announce }) {
         console.warn(`[pack-review] entity apply failed: ${e.message}`);
       }
     }
+    // Weak-use signal: the reviewer deciding a pack/entity needed updating is
+    // evidence the turn genuinely worked on that node — use, observed after
+    // the fact, even when the agent never re-opened the note (it may already
+    // have been injected). Feeds the same precision metric as 📖 opens and
+    // reinforces co-use edges. This is co-WORK, never co-recall — surfacing
+    // alone still reinforces nothing.
+    if (usedIds.length) {
+      try { require("./recall/metrics").logUse(usedIds, "reviewer"); } catch (e) {}
+      try { if (usedIds.length > 1) require("./recall/graph").reinforceSet(usedIds); } catch (e) {}
+    }
     for (const la of (decision.lessons || [])) {
       try {
         const r = applyLessonAction(la);

package/core/packs.js

@@ -343,8 +343,35 @@ function recordForegroundWrite(dir, { tool, oldString, content } = {}) {
   } catch (e) {}
 }
 
+// Near-duplicate check for journal appends. The reviewer runs after EVERY
+// substantial turn, so a long working session on one topic used to stack
+// dozens of same-day entries that only differ in phrasing (observed: 28 in a
+// day on one pack) — crowding real history out of the capped Journal. Token
+// Jaccard against the most recent entries; dates stripped before comparing.
+function journalTokens(s) {
+  return new Set(
+    String(s || "").toLowerCase()
+      .replace(/^-\s*\[[^\]]*\]\s*/, "")
+      .replace(/[^a-z0-9\s]/g, " ")
+      .split(/\s+/).filter((w) => w.length > 1)
+  );
+}
+
+function isNearDupJournal(line, recentEntries) {
+  const a = journalTokens(line);
+  if (!a.size) return false;
+  for (const e of recentEntries) {
+    const b = journalTokens(e);
+    if (!b.size) continue;
+    let inter = 0;
+    for (const w of a) if (b.has(w)) inter++;
+    if (inter / (a.size + b.size - inter) >= 0.75) return true;
+  }
+  return false;
+}
+
 // Apply a reviewer mutation. Only supplied fields change; journal entries
-// append (capped); state replaces (it represents "current truth").
+// append (capped + deduped); state replaces (it represents "current truth").
 function updatePack(dir, { description, tags, stance, procedure, state, journal, skill, kind, learned_on } = {}, origin = "user") {
   const pack = readPack(dir);
   if (!pack) throw new Error(`no pack: ${dir}`);
@@ -370,9 +397,14 @@ function updatePack(dir, { description, tags, stance, procedure, state, journal,
   if (typeof state === "string" && state.trim()) { pack.sections.State = state.trim(); applied.push("State"); }
   if (typeof journal === "string" && journal.trim()) {
     const entries = pack.sections.Journal.split("\n").filter((l) => l.trim());
-    entries.push(`- [${today()}] ${journal.trim().replace(/\n+/g, " ")}`);
-    pack.sections.Journal = entries.slice(-MAX_JOURNAL_ENTRIES).join("\n");
-    applied.push("Journal");
+    const line = journal.trim().replace(/\n+/g, " ");
+    if (isNearDupJournal(line, entries.slice(-5))) {
+      pack.journalDeduped = true;
+    } else {
+      entries.push(`- [${today()}] ${line}`);
+      pack.sections.Journal = entries.slice(-MAX_JOURNAL_ENTRIES).join("\n");
+      applied.push("Journal");
+    }
   }
   writePack(pack);
   if (applied.length) setProvenance(dir, applied, origin);
@@ -590,7 +622,7 @@ function matchPacks(text, { limit = 3, threshold = null } = {}) {
 
 module.exports = {
   PACKS_DIR, SECTIONS, slugify,
-  listPacks, findPack, readPack, writePack, createPack, updatePack, removePack,
+  listPacks, findPack, readPack, writePack, createPack, updatePack, removePack, isNearDupJournal,
   listSkillPacks, setSkill, listAbilities, setKind, parentByPrefix, setParent, recordApplied, recordCoUse,
   touchUsed, packNameFromPath, matchPacks, reindex, markIndexDirty,
   archivePack, restorePack, listArchived,

package/core/recall/discoverer.js

@@ -26,10 +26,21 @@ const toolsLib = require("../tools");
 // entity-only). Optional — absent on old node where node:sqlite is missing.
 let toolGraph = null;
 try { toolGraph = require("../tool-graph"); } catch (e) { /* old node */ }
+// Episodic memory: FTS over past transcripts (same optionality as the graphs).
+let transcriptIndex = null;
+try { transcriptIndex = require("../transcript-index"); } catch (e) { /* old node */ }
 
 const WALKER_MODEL = process.env.RECALL_DISCOVERER_MODEL || "haiku";
 const WALKER_TIMEOUT_MS = Number(process.env.RECALL_DISCOVERER_TIMEOUT_MS || 25000);
 const WALKER_ENABLED = String(process.env.RECALL_DISCOVERER_WALKER || "on").toLowerCase() !== "off";
+// Cap on pack/entity candidates handed to the walker. Uncapped, a wordy turn
+// seeds 15-40 nodes × 600-char excerpts — the walker then reads a 10-25KB
+// prompt, which measured as THE dominant recall latency (p50 ~13.5s even with
+// the warm process). Capped, the prompt stays small; priority order below keeps
+// the strongest evidence.
+const MAX_WALKER_CANDIDATES = Number(process.env.RECALL_WALKER_MAX_CANDIDATES || 14);
+const EPISODES_ENABLED = String(process.env.RECALL_EPISODES || "on").toLowerCase() !== "off";
+const EPISODE_LIMIT = Number(process.env.RECALL_EPISODE_LIMIT || 3);
 const EXCERPT_CHARS = 600;
 const CONTEXT_CLIP = 3000;
 
@@ -45,16 +56,30 @@ function maybeSync(packsLib, entitiesLib) {
   try { graph.syncFromCorpus(packsLib, entitiesLib); } catch (e) {}
 }
 
-// Cheap pre-gate: terse acknowledgements / pure pleasantries don't need recall.
-function needsRecall(userText, seedCount) {
+// Tiered pre-gate. The old boolean gate returned true whenever FTS found ANY
+// seed — with a 147-pack corpus that is virtually every turn (measured: gated
+// 0.4% of 500 turns), so "thanks!" still paid a full 13s walk. Three tiers:
+//   skip  — pure pleasantry/ack/emoji: no recall at all, regardless of seeds.
+//   seeds — short command-like turns (≤4 words): keyword seeds are precise
+//           enough; inject user-origin seeds directly and skip graph + walker
+//           (the classic baseline, ~ms instead of ~13s).
+//   full  — substantive turns: graph expansion + walker judgement.
+const PLEASANTRY_RE = /^(ok(ay)?|k|kk|thanks?|thank you|thx|ty|yes|yep|yeah|no|nope|nah|cool|nice|great|perfect|awesome|lovely|done|sure|got it|sounds good|will do|lol|haha|hi|hey|hello|yo|morning|evening|night|good (morning|afternoon|evening|night)|bye|goodnight|cheers|please do|go ahead)\b/;
+
+function recallTier(userText, seedCount) {
   const t = String(userText || "").trim().toLowerCase();
-  if (!t) return false;
-  if (seedCount > 0) return true; // FTS already found something — never gate it out
-  // No seeds: only bother with graph/walker work for a substantive message.
+  if (!t) return "skip";
   const words = t.split(/\s+/).filter(Boolean);
-  if (words.length <= 2) return false;
-  if (/^(ok|okay|thanks|thank you|yes|yep|no|nope|cool|nice|great|done|sure|got it|k)\b/.test(t)) return false;
-  return true;
+  if (!/[a-z0-9]/.test(t)) return "skip"; // emoji/punctuation only
+  if (words.length <= 4 && PLEASANTRY_RE.test(t)) return "skip";
+  if (words.length <= 2 && seedCount === 0) return "skip";
+  if (words.length <= 4) return "seeds";
+  return "full";
+}
+
+// Back-compat boolean view of the tier (exported/tested API).
+function needsRecall(userText, seedCount) {
+  return recallTier(userText, seedCount) !== "skip";
 }
 
 function idFor(m) { return m.dir ? `pack:${m.dir}` : `entity:${m.slug}`; }
@@ -95,6 +120,7 @@ const WALKER_SYSTEM = [
   "Decide which nodes are GENUINELY relevant to what the user is doing now.",
   "Keep graph-linked nodes ONLY if they actually bear on the task (e.g. a shared theme/commit-style that governs the thing being worked on). Drop incidental keyword overlaps.",
   "Some candidates are reusable tools (id starts `tool:`). Keep a tool ONLY if running it would actually help with THIS task; drop tools merely related by keyword.",
+  "Some candidates are past-conversation episodes (id starts `episode:`). Keep one ONLY if that past exchange holds a decision, method, or outcome directly reusable for the current message — not mere topic overlap.",
   "For each kept node write a terse why (≤12 words). Quote concrete facts verbatim; never invent.",
   'Reply with ONLY a JSON array: [{"id":"pack:foo","why":"shared lime theme governs this app"}]. Use [] if none.',
 ].join("\n");
@@ -102,20 +128,25 @@ const WALKER_SYSTEM = [
 // Run the walker model on the prompt. Prefer the warm (reused) process for
 // low latency; on any warm-path error fall back to a cold spawn — identical
 // model/prompt/contract, so quality is unchanged and recall never silently
-// degrades to the classic engine.
+// degrades to the classic engine. Returns { text, costUsd, source }.
 async function runWalker(prompt) {
   const opts = { model: WALKER_MODEL, systemPrompt: WALKER_SYSTEM, timeoutMs: WALKER_TIMEOUT_MS };
   if (warmWalker.isEnabled()) {
     try {
-      return await warmWalker.walkWarm(prompt, opts);
+      const r = await warmWalker.walkWarm(prompt, opts);
+      return { text: r.text, costUsd: r.costUsd, source: "warm" };
     } catch (e) { /* fall back to cold spawn below */ }
   }
   const { text } = await spawnSubagent(prompt, opts);
-  return text;
+  return { text, costUsd: null, source: "cold" };
 }
 
+// Judge candidates. Returns { kept: Map|null, costUsd, source, walkerMs } —
+// kept null means fail-open (walker off/failed/unparseable).
 async function walk(userText, contextText, candidates) {
-  if (!WALKER_ENABLED || candidates.length === 0) return null;
+  if (!WALKER_ENABLED || candidates.length === 0) {
+    return { kept: null, costUsd: null, source: "off", walkerMs: 0 };
+  }
   const lines = candidates.map((c) => {
     const tag = c.activated ? ` [linked via ${c.via || "graph"}]` : "";
     return `- ${c.id}: ${c.name}${tag}${c.description ? `\n    ${c.description}` : ""}${c.excerpt ? `\n    excerpt: ${c.excerpt.replace(/\n/g, " ")}` : ""}`;
@@ -130,12 +161,14 @@ async function walk(userText, contextText, candidates) {
     "",
     'Reply ONLY with the JSON array of kept nodes and their why, e.g. [{"id":"' + candidates[0].id + '","why":"..."}].',
   ].join("\n");
+  const t0 = Date.now();
   try {
-    const text = await runWalker(prompt);
+    const { text, costUsd, source } = await runWalker(prompt);
+    const walkerMs = Date.now() - t0;
     const match = String(text || "").match(/\[[\s\S]*\]/);
-    if (!match) return null;
+    if (!match) return { kept: null, costUsd, source, walkerMs };
     const arr = JSON.parse(match[0]);
-    if (!Array.isArray(arr)) return null;
+    if (!Array.isArray(arr)) return { kept: null, costUsd, source, walkerMs };
     const known = new Set(candidates.map((c) => c.id));
     const out = new Map();
     for (const item of arr) {
@@ -143,9 +176,9 @@ async function walk(userText, contextText, candidates) {
         out.set(item.id, String(item.why || "").slice(0, 120));
       }
     }
-    return out;
+    return { kept: out, costUsd, source, walkerMs };
   } catch (e) {
-    return null;
+    return { kept: null, costUsd: null, source: "failed", walkerMs: Date.now() - t0 };
   }
 }
 
@@ -154,8 +187,6 @@ async function run(ctx) {
   const { packsLib, entitiesLib, mergeMatches, buildPackBlock, buildEntityBlock } = helpers;
   const started = Date.now();
 
-  maybeSync(packsLib, entitiesLib);
-
   // 1+2: seed via FTS (user words + context), same as classic.
   let packSeeds = [];
   let entSeeds = [];
@@ -185,32 +216,55 @@ async function run(ctx) {
   } catch (e) {}
 
   const seedCount = packSeeds.length + entSeeds.length + toolSeedByName.size;
+  const seedMs = Date.now() - started;
 
-  // 1: pre-gate.
-  if (!needsRecall(userText, seedCount)) {
-    metrics.logTurn({ engine: "discoverer", query: userText, gated: true, latencyMs: Date.now() - started });
-    return { packBlock: "", entityBlock: "", toolBlock: "", packMatches: [], entityMatches: [], toolMatches: [], why: {}, gated: true };
+  // 1: tiered pre-gate.
+  const tier = recallTier(userText, seedCount);
+  if (tier === "skip") {
+    metrics.logTurn({ engine: "discoverer", query: userText, gated: true, tier, latencyMs: Date.now() - started });
+    return { packBlock: "", entityBlock: "", toolBlock: "", episodeBlock: "", packMatches: [], entityMatches: [], toolMatches: [], episodeMatches: [], why: {}, gated: true };
   }
 
-  // 3: spreading activation from seeds across the graph.
+  // 3: spreading activation from seeds across the graph (full tier only — the
+  // seeds tier injects keyword seeds directly, no graph, no walker).
   const seedNodes = [
     ...packSeeds.map((m) => ({ id: `pack:${m.dir}`, score: m.score || 2 })),
     ...entSeeds.map((m) => ({ id: `entity:${m.slug}`, score: m.score || 2 })),
   ];
   const seedIds = new Set(seedNodes.map((s) => s.id));
   let activated = new Map();
-  try { activated = graph.expand(seedNodes, {}); } catch (e) {}
+  const expandStarted = Date.now();
+  if (tier === "full") {
+    maybeSync(packsLib, entitiesLib);
+    try { activated = graph.expand(seedNodes, {}); } catch (e) {}
+  }
+  const expandMs = Date.now() - expandStarted;
 
-  // 4: assemble candidates (seeds + activated), build excerpts, run the walker.
-  const candIds = new Set(seedIds);
-  for (const id of activated.keys()) candIds.add(id);
+  // 4: assemble candidates (seeds + activated) in priority order — user seeds
+  // by score, then context seeds, then graph-activated by activation — capped
+  // at MAX_WALKER_CANDIDATES so the walker prompt stays small and fast. Only
+  // the full tier pays for excerpts; the seeds tier never reads them.
+  const seedRank = new Map();
+  for (const m of packSeeds) seedRank.set(`pack:${m.dir}`, { score: m.score || 2, origin: m.origin });
+  for (const m of entSeeds) seedRank.set(`entity:${m.slug}`, { score: m.score || 2, origin: m.origin });
+  const orderedIds = [...seedIds].sort((a, b) => {
+    const ra = seedRank.get(a) || {}, rb = seedRank.get(b) || {};
+    const ua = ra.origin === "user" ? 0 : 1, ub = rb.origin === "user" ? 0 : 1;
+    return ua - ub || (rb.score || 0) - (ra.score || 0);
+  });
+  for (const [id] of [...activated.entries()].sort((x, y) => (y[1].activation || 0) - (x[1].activation || 0))) {
+    if (!seedIds.has(id)) orderedIds.push(id);
+  }
   const candidates = [];
-  for (const id of candIds) {
-    const base = excerptFor(id, packsLib, entitiesLib);
-    if (!base) continue;
-    const act = activated.get(id);
-    candidates.push({ ...base, activated: !seedIds.has(id), via: act ? act.via : null });
+  if (tier === "full") {
+    for (const id of orderedIds.slice(0, MAX_WALKER_CANDIDATES)) {
+      const base = excerptFor(id, packsLib, entitiesLib);
+      if (!base) continue;
+      const act = activated.get(id);
+      candidates.push({ ...base, activated: !seedIds.has(id), via: act ? act.via : null });
+    }
   }
+  const candIds = new Set(tier === "full" ? candidates.map((c) => c.id) : seedIds);
 
   // Tool candidates = direct tool seeds + tools that belong to any candidate
   // pack (declared via the tool's `pack:` header, or used-in-topic per the
@@ -221,7 +275,7 @@ async function run(ctx) {
   for (const id of candIds) if (id.startsWith("pack:")) candPackDirs.add(id.slice(5));
   const toolCand = new Map();
   for (const [name, t] of toolSeedByName) toolCand.set(name, { name, description: t.description, via: null });
-  if (candPackDirs.size) {
+  if (tier === "full" && candPackDirs.size) {
     try {
       const byPack = new Map();
       for (const t of toolsLib.listTools()) {
@@ -247,13 +301,40 @@ async function run(ctx) {
       } catch (e) {}
     }
   }
-  for (const c of toolCand.values()) {
-    const base = excerptFor(`tool:${c.name}`, packsLib, entitiesLib);
-    if (!base) continue;
-    candidates.push({ ...base, activated: !!c.via, via: c.via });
+  if (tier === "full") {
+    for (const c of toolCand.values()) {
+      const base = excerptFor(`tool:${c.name}`, packsLib, entitiesLib);
+      if (!base) continue;
+      candidates.push({ ...base, activated: !!c.via, via: c.via });
+    }
   }
 
-  const whyById = (await walk(userText, contextText || fullContext, candidates)) || null;
+  // Episodic hop: FTS over past transcripts turns up prior sessions on this
+  // topic; the walker judges them like any node. This is the bridge from
+  // knowledge (packs) to episodes (what we actually did last time) — kept ones
+  // inject as a snippet plus a pointer to reopen the conversation window.
+  let episodeCands = [];
+  if (tier === "full" && EPISODES_ENABLED && transcriptIndex && transcriptIndex.available()) {
+    try {
+      const { hits } = transcriptIndex.search(userText, { limit: EPISODE_LIMIT });
+      episodeCands = (hits || []).map((h) => {
+        const terms = [...String(h.snip || "").matchAll(/>>(.+?)<</g)].map((m) => m[1]).slice(0, 4);
+        return {
+          id: `episode:${require("path").basename(String(h.file || ""), ".jsonl")}:${h.line}`,
+          name: `Past conversation${h.project ? ` (${h.project})` : ""}${h.ts ? ` ${String(h.ts).slice(0, 10)}` : ""}`,
+          description: "",
+          excerpt: clip(String(h.snip || "").replace(/>>|<</g, ""), EXCERPT_CHARS),
+          activated: true,
+          via: "transcript",
+          _terms: terms,
+        };
+      });
+      candidates.push(...episodeCands);
+    } catch (e) { /* episodic recall is best-effort */ }
+  }
+
+  const walkRes = await walk(userText, contextText || fullContext, candidates);
+  const whyById = walkRes.kept;
 
   // Decide the kept set. Walker result wins; on fail-open keep the user-origin
   // seeds only (classic baseline) and drop graph-expanded/context guesses.
@@ -320,20 +401,40 @@ async function run(ctx) {
     toolBlock = `\n\n## Tools that may help here\nSurfaced for this turn — run with \`open-claudia tool run <name> [args]\`; read docs/source with \`open-claudia tool show <name>\`:\n${lines.join("\n")}\n`;
   }
 
+  // Past-conversation episodes the walker kept (fail-open drops them — an
+  // unjudged transcript snippet is a guess, not a memory).
+  const episodeMatches = episodeCands
+    .filter((c) => keptIds.has(c.id))
+    .map((c) => ({ id: c.id, name: c.name, excerpt: c.excerpt, terms: c._terms, why: whyById ? whyById.get(c.id) || "" : "" }));
+  let episodeBlock = "";
+  if (episodeMatches.length) {
+    const lines = episodeMatches.map((m) => {
+      const q = (m.terms && m.terms.length ? m.terms.join(" ") : String(m.excerpt).split(/\s+/).slice(0, 4).join(" ")).replace(/"/g, "");
+      return `- ${m.name}: "${m.excerpt}"${m.why ? ` — ${m.why}` : ""}\n  ↳ full context: open-claudia transcript-search "${q}" --all`;
+    });
+    episodeBlock = `\n\n## Past work that looks related\nFrom earlier conversations — verify before relying on it:\n${lines.join("\n")}\n`;
+  }
+
   metrics.logTurn({
     engine: "discoverer",
     query: userText,
+    tier,
     seeds: seedNodes,
     activated: [...activated.entries()].map(([id, a]) => ({ id, activation: a.activation, hop: a.hop })),
     kept: [...keptIds].map((id) => ({ id, why: whyById ? whyById.get(id) : "" })),
     gated: false,
     latencyMs: Date.now() - started,
+    seedMs,
+    expandMs,
+    walkerMs: walkRes.walkerMs,
+    walker: walkRes.source,
+    costUsd: walkRes.costUsd || 0,
   });
 
   return {
-    packBlock, entityBlock, toolBlock, packMatches: finalPacks, entityMatches: finalEnts,
-    toolMatches, why: whyById ? Object.fromEntries(whyById) : {}, gated: false,
+    packBlock, entityBlock, toolBlock, episodeBlock, packMatches: finalPacks, entityMatches: finalEnts,
+    toolMatches, episodeMatches, why: whyById ? Object.fromEntries(whyById) : {}, gated: false,
   };
 }
 
-module.exports = { name: "discoverer", run, needsRecall, walk };
+module.exports = { name: "discoverer", run, needsRecall, recallTier, walk };

package/core/recall/metrics.js

@@ -17,8 +17,9 @@ function enabled() {
 }
 
 // Record one recall turn. `entry`:
-//   { engine, query, seeds:[{id,score}], activated:[{id,activation,hop}],
-//     kept:[{id,why}], gated (bool), latencyMs, costUsd }
+//   { engine, query, tier, seeds:[{id,score}], activated:[{id,activation,hop}],
+//     kept:[{id,why}], gated (bool), latencyMs, seedMs, expandMs, walkerMs,
+//     walker ("warm"|"cold"|"off"|"failed"), costUsd }
 function logTurn(entry) {
   if (!enabled()) return;
   try {
@@ -26,11 +27,16 @@ function logTurn(entry) {
       ts: new Date().toISOString(),
       engine: entry.engine || "discoverer",
       query: String(entry.query || "").slice(0, 200),
+      tier: entry.tier || "",
       seeds: (entry.seeds || []).map((s) => ({ id: s.id, score: s.score })),
       activated: (entry.activated || []).map((a) => ({ id: a.id, activation: round(a.activation), hop: a.hop })),
       kept: (entry.kept || []).map((k) => ({ id: k.id, why: String(k.why || "").slice(0, 200) })),
       gated: !!entry.gated,
       latencyMs: entry.latencyMs || 0,
+      seedMs: entry.seedMs || 0,
+      expandMs: entry.expandMs || 0,
+      walkerMs: entry.walkerMs || 0,
+      walker: entry.walker || "",
       costUsd: entry.costUsd || 0,
     };
     fs.appendFileSync(LOG_FILE, JSON.stringify(rec) + "\n");
@@ -38,15 +44,19 @@ function logTurn(entry) {
   } catch (e) {}
 }
 
-// Record which nodes were actually opened (📖) this turn. Matched against the
-// most recent surfaced sets to derive precision/noise without a second pass
-// over the whole log.
-function logUse(openedIds) {
+// Record which nodes were actually used this turn. Matched against the most
+// recent surfaced sets to derive precision/noise without a second pass over
+// the whole log. `source` distinguishes how use was observed: "" / undefined
+// for 📖 opens (agent explicitly read the note), "reviewer" for post-turn
+// reviewer adjudication (the turn's work updated that pack/entity).
+function logUse(openedIds, source) {
   if (!enabled()) return;
   const ids = [...new Set((openedIds || []).filter(Boolean))];
   if (!ids.length) return;
   try {
-    fs.appendFileSync(LOG_FILE, JSON.stringify({ ts: new Date().toISOString(), use: ids }) + "\n");
+    const rec = { ts: new Date().toISOString(), use: ids };
+    if (source) rec.source = String(source);
+    fs.appendFileSync(LOG_FILE, JSON.stringify(rec) + "\n");
     const s = readSummary();
     s.opens = (s.opens || 0) + ids.length;
     writeSummary(s);
@@ -66,6 +76,7 @@ function bumpSummary(rec) {
   const s = readSummary();
   s.turns = (s.turns || 0) + 1;
   if (rec.gated) s.gatedTurns = (s.gatedTurns || 0) + 1;
+  if (rec.tier === "seeds") s.seedsOnlyTurns = (s.seedsOnlyTurns || 0) + 1;
   s.seedsTotal = (s.seedsTotal || 0) + rec.seeds.length;
   s.activatedTotal = (s.activatedTotal || 0) + rec.activated.length;
   s.keptTotal = (s.keptTotal || 0) + rec.kept.length;
@@ -88,6 +99,8 @@ function summary() {
     turns,
     gatedTurns: s.gatedTurns || 0,
     gatedPct: fmtPct(s.gatedTurns || 0, turns),
+    seedsOnlyTurns: s.seedsOnlyTurns || 0,
+    seedsOnlyPct: fmtPct(s.seedsOnlyTurns || 0, turns),
     avgSeeds: turns ? round((s.seedsTotal || 0) / turns) : 0,
     avgActivated: turns ? round((s.activatedTotal || 0) / turns) : 0,
     avgKept: turns ? round((s.keptTotal || 0) / turns) : 0,

package/core/recall/warm-walker.js

@@ -30,6 +30,7 @@ let pending = null;        // { resolve, reject, timer } for the in-flight walk
 let chain = Promise.resolve(); // serialises walks (one message in flight)
 let msgCount = 0;          // messages sent to the current process
 let charCount = 0;         // prompt chars sent to the current process
+let costTotal = 0;         // cumulative total_cost_usd reported by the session
 
 function isEnabled() {
   return String(process.env.RECALL_WARM_WALKER || "on").toLowerCase() !== "off";
@@ -60,6 +61,7 @@ function spawnChild(cfg) {
   child = proc;
   msgCount = 0;
   charCount = 0;
+  costTotal = 0;
   let buf = "";
   let asstText = "";
 
@@ -88,7 +90,14 @@ function spawnChild(cfg) {
       } else if (evt.type === "result") {
         const text = (typeof evt.result === "string" && evt.result) ? evt.result : asstText;
         asstText = "";
-        if (text && !evt.is_error) settle("resolve", redactSensitive(String(text).trim()));
+        // total_cost_usd is cumulative for the stream-json session; the delta
+        // from the previous result is this walk's cost.
+        let costUsd = null;
+        if (typeof evt.total_cost_usd === "number") {
+          costUsd = Math.max(0, evt.total_cost_usd - costTotal);
+          costTotal = evt.total_cost_usd;
+        }
+        if (text && !evt.is_error) settle("resolve", { text: redactSensitive(String(text).trim()), costUsd });
         else settle("reject", new Error("warm walker: empty/error result"));
       }
     }
@@ -136,6 +145,8 @@ function doWalk(promptText, opts) {
 
 // Serialise: one message in flight at a time. A failed walk must not poison the
 // queue, so the chain swallows outcomes while callers still see their result.
+// Resolves { text, costUsd } — costUsd is the per-walk delta of the session's
+// cumulative total_cost_usd (null when the CLI doesn't report it).
 function walkWarm(promptText, opts = {}) {
   const run = () => doWalk(promptText, opts);
   const p = chain.then(run, run);

package/core/runner.js

@@ -1035,7 +1035,7 @@ async function runClaude(prompt, cwd, replyToMsgId, opts = {}) {
       const fmt = (arr, icon) => arr.map((x) => (x.why ? `${icon} <b>${esc(x.name)}</b> — ${esc(x.why)}` : `${icon} <b>${esc(x.name)}</b>`));
       // 🧠 packs/entities recall — gated by /recall.
       if (settings.showRecall) {
-        const lines = [...fmt(r.packs || [], "📦"), ...fmt(r.entities || [], "👤")];
+        const lines = [...fmt(r.packs || [], "📦"), ...fmt(r.entities || [], "👤"), ...fmt(r.episodes || [], "📓")];
         if (lines.length) send(`🧠 <b>Recall this turn</b> (${esc(r.engine)})\n${lines.join("\n")}`, telegramHtmlOpts()).catch(() => {});
         else if (r.gated) send(`🧠 <b>Recall</b> (${esc(r.engine)}): skipped by pre-gate — trivial turn.`, telegramHtmlOpts()).catch(() => {});
       }

package/core/system-prompt.js

@@ -743,6 +743,7 @@ async function promptWithDynamicContext(prompt, opts = {}) {
     });
     const { packBlock, entityBlock } = result;
     const toolBlock = result.toolBlock || "";
+    const episodeBlock = result.episodeBlock || "";
     const why = result.why || {};
     lastInjected.recall = {
       engine: engine.name || recall.activeEngineName(settings),
@@ -753,12 +754,14 @@ async function promptWithDynamicContext(prompt, opts = {}) {
       // Drives the /tooltrace "surfaced" banner; the toolBlock below is the
       // context the agent actually sees.
       tools: (result.toolMatches || []).map((m) => ({ name: m.name, why: m.why || why[`tool:${m.name}`] || "" })),
+      // Past-conversation episodes the walker kept (drives the /recall banner).
+      episodes: (result.episodeMatches || []).map((m) => ({ name: m.name, why: m.why || "" })),
     };
     const budgetNote = budget.omitted > 0
       ? `\n\n## Memory budget\n${budget.omitted} matched memory item${budget.omitted === 1 ? " was" : "s were"} omitted to keep this turn under the recall budget (${budget.maxChars} chars). Use \`open-claudia pack show <dir>\`, \`entity show <slug>\`, or transcript search if deeper context is needed.`
       : "";
     const transcriptPointer = opts.includeTranscriptPointer ? `${transcriptPointerNote(state)}\n\n` : "";
-    return `${transcriptPointer}${buildDynamicContextBlock()}${packBlock}${entityBlock}${toolBlock}${budgetNote}${voiceReplyGuidance()}\n\nCurrent user request:\n${prompt}`;
+    return `${transcriptPointer}${buildDynamicContextBlock()}${packBlock}${entityBlock}${toolBlock}${episodeBlock}${budgetNote}${voiceReplyGuidance()}\n\nCurrent user request:\n${prompt}`;
   } catch (e) {
     return prompt;
   }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@inetafrica/open-claudia",
-  "version": "2.6.57",
+  "version": "2.6.58",
   "description": "Your always-on AI coding assistant — Claude Code, Cursor Agent, and OpenAI Codex via Telegram or Kazee Chat",
   "main": "bot.js",
   "bin": {
@@ -9,7 +9,7 @@
   "scripts": {
     "setup": "node setup.js",
     "start": "node bot.js",
-    "test": "OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node -e \"require('./vault'); console.log('OK')\" && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-runner-watchdog-static.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-usage-accounting.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-recall-engine.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-recall-graph.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-recall-discoverer.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-project-transcripts-smoke.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-abilities.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-ability-extraction.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-ability-couse.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-ability-transfer.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-ability-tiers.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-ability-merge-guard.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-learning-e2e.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-pack-nesting.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-read-signal.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-tools.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-tool-graph.js"
+    "test": "OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node -e \"require('./vault'); console.log('OK')\" && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-runner-watchdog-static.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-usage-accounting.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-recall-engine.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-recall-graph.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-recall-discoverer.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-project-transcripts-smoke.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-abilities.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-ability-extraction.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-ability-couse.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-ability-transfer.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-ability-tiers.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-ability-merge-guard.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-learning-e2e.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-pack-nesting.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-journal-dedupe.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-read-signal.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-tools.js && OPEN_CLAUDIA_TEST=1 WORKSPACE=/tmp/open-claudia-test CLAUDE_PATH=node node test-tool-graph.js"
   },
   "files": [
     "bot.js",

package/test-recall-discoverer.js

@@ -24,13 +24,21 @@ toolsLib.addTool((() => {
   return s;
 })(), { name: "mikrotik-audit", description: "audit the mikrotik router fleet inventory" });
 
-// --- pre-gate ---
+// --- tiered pre-gate ---
 assert.strictEqual(disc.needsRecall("", 0), false, "empty → no recall");
 assert.strictEqual(disc.needsRecall("thanks", 0), false, "pleasantry → no recall");
 assert.strictEqual(disc.needsRecall("ok", 0), false, "terse ack → no recall");
 assert.strictEqual(disc.needsRecall("yo", 0), false, "two words, no seeds → no recall");
 assert.strictEqual(disc.needsRecall("how do I deploy the mobile app", 0), true, "substantive → recall");
-assert.strictEqual(disc.needsRecall("k", 5), true, "seeds present always recall");
+// The old gate kept ANY turn with seeds (measured: gated 0.4% of turns, so
+// "thanks!" paid a full walk). New contract: pleasantries gate regardless.
+assert.strictEqual(disc.needsRecall("k", 5), false, "terse ack gates even when seeds match");
+assert.strictEqual(disc.recallTier("thanks!", 8), "skip", "pleasantry + seeds → skip");
+assert.strictEqual(disc.recallTier("👍", 3), "skip", "emoji-only → skip");
+assert.strictEqual(disc.recallTier("restart bot", 3), "seeds", "short command → seeds-only tier");
+assert.strictEqual(disc.recallTier("show fup logs", 2), "seeds", "≤4 words → seeds-only tier");
+assert.strictEqual(disc.recallTier("yes we need to see the 100", 2), "full", "substantive yes-prefixed turn → full");
+assert.strictEqual(disc.recallTier("how do I deploy the mobile app", 4), "full", "substantive → full");
 
 // --- shared stub corpus + helpers ---
 const packs = {
@@ -110,5 +118,26 @@ const helpers = { packsLib, entitiesLib, mergeMatches, buildPackBlock, buildEnti
   assert.strictEqual(outNoTool.toolBlock, "", "no tool match → empty toolBlock");
   assert.strictEqual(outNoTool.toolMatches.length, 0, "no tool match → no toolMatches");
 
+  // seeds tier: a short command-like turn skips graph+walker but still injects
+  // keyword seeds directly (classic baseline, ~ms path).
+  builtPacks = null;
+  const seedsTier = await disc.run({
+    userText: "mobile app status", contextText: "", fullContext: "mobile app status",
+    packLimit: 6, budget: {}, helpers,
+  });
+  assert.strictEqual(seedsTier.gated, false, "seeds tier is not gated");
+  assert.strictEqual(seedsTier.packBlock, "PACKBLOCK", "seeds tier injects keyword seeds");
+  assert.ok(seedsTier.packMatches.some((m) => m.dir === "kazee-mobile"), "seeds tier keeps the user-origin seed");
+  assert.strictEqual(seedsTier.episodeBlock, "", "seeds tier never injects episodes");
+
+  // episodes: with the walker off, transcript hits are candidates but the
+  // fail-open path must DROP them — an unjudged snippet never injects.
+  const epi = await disc.run({
+    userText: "help me deploy the mobile app updater again", contextText: "",
+    fullContext: "help me deploy the mobile app updater again", packLimit: 6, budget: {}, helpers,
+  });
+  assert.strictEqual(epi.episodeBlock, "", "fail-open drops unjudged episodes");
+  assert.deepStrictEqual(epi.episodeMatches, [], "fail-open keeps no episodeMatches");
+
   console.log("recall discoverer OK");
 })().catch((e) => { console.error(e); process.exit(1); });