@inetafrica/open-claudia 2.6.53 → 2.6.54

package/CHANGELOG.md

@@ -1,5 +1,12 @@
 # Changelog
 
+## v2.6.54
+- **Tools as first-class discoverer nodes.** Reusable tools now go through the same recall pipeline as packs and entities — seed → activate → judge → render — instead of being dumped wholesale into every prompt. Until now `buildToolIndexBlock` listed *all* ~40 tools on every turn; that flat block is gone. In its place, the discoverer (`core/recall/discoverer.js`) seeds tool candidates from a new in-memory `matchTools()` (lexical scoring over name/description = strong, requires/pack/source = weak; `core/tools.js`), activates the tools belonging to any pack already in the candidate set (read from the pack↔tool graph + tool `pack:` headers — the latency-sensitive recall `expand()` stays pack/entity-only, zero regression risk), and lets the haiku walker judge them with a "keep a tool only if running it would actually help" instruction. Survivors render into a focused **"Tools that may help here"** block; the system prompt now just states the count and points to `tool search`/`tool list` for the rest. `matchTools` is in-memory (works on every node version, unlike the SQLite graphs).
+- **`/tooltrace` toggle (default off).** Mirrors `/recall`. When on, posts short 🔧 lines around each reply: which tools were **surfaced** for the turn, which were **run**, and any **created/updated** — so you can watch the tool layer work. All three signals are now gated behind this one toggle (`settings.showToolTrace`); the run/created/updated banners were removed from the always-on path and the surfaced list was split out of the 🧠 `/recall` banner, so the two toggles are cleanly independent.
+- **Tool usage telemetry.** A JSON sidecar (`.usage.json` in the tools dir: `runCount` + `lastUsed` per tool, recorded at end-of-turn for each successful run) works on every node version. `matchTools` breaks score ties by `runCount` so well-worn tools surface first.
+- **`tool search` + add-time duplicate guard.** New `open-claudia tool search <query>` for finding a tool on demand; `tool add` now warns (non-blocking) when an existing tool looks similar, nudging toward extending it with a subcommand over spawning a near-duplicate.
+- **Dream tool hygiene.** The nightly pass now also flags tools created-but-never-run for 30+ days and near-duplicate tool pairs (alongside the existing dangling-`--pack` flag). Flag-only — a tool is never auto-deleted. Extends `test-tools.js` (matchTools/telemetry) and `test-recall-discoverer.js` (tool seeding). Degrades to a graceful no-op where `node:sqlite` is unavailable.
+
 ## v2.6.53
 - **Pack↔tool contextual edges — surface how a tool is used in a topic.** Completes the v2.6.49 tool story. The directed tool-graph answers "what runs *next*"; this adds the orthogonal question "how is a tool *used in this context*". A new `pack_tool_edges` table (`core/tool-graph.js`, sharing the existing SQLite store, decay and pruning) records, for each turn, the reusable tools that ran while a context pack was open — keyed by `(pack, tool, command-shape)` so distinct invocations of the same tool coexist and the strongest surface. Capture is automatic and deterministic (no hand-authoring): `runner.js` already tracks the tools run and the packs opened each turn, so at end-of-turn it records a pack↔tool edge per (opened-pack × run-tool), Hebbian-reinforced. Commands are stored as **shapes, not literals** — a new `shapeCommand()` strips ids, paths, numbers and quoted text to placeholders (`spaces docs --task-id <id> <path>`) so the memory is a reusable pattern, not a stale one-off; `parseToolRuns()` extracts `{name, shape}` from a shell command (split across chained invocations) and feeds both graphs. Surfacing: when a pack is injected into context, `formatPackForContext` appends a **"Tools used in this pack"** block with the 1–2 strongest invocations, so the agent sees the concrete command instead of re-deriving it; `tool show` gains the inverse "Used in packs" view; an optional `intent` column lets a one-line *why* be enriched later without a migration. The nightly dream tends these edges alongside the follows-edges (decay + prune orphaned tools/packs). Degrades to a graceful no-op where `node:sqlite` is unavailable, exactly like the existing tool-graph. Extends `test-tool-graph.js`.
 

package/bin/cli.js

@@ -370,7 +370,7 @@ Memory tools:
   open-claudia transcript-window <pattern>   Search project transcript, show hits with context
                                              (alias: tw; --help for options)
   open-claudia pack list|show|match|archive|restore|archived  Context packs: living topic docs (skills + memory)
-  open-claudia tool list|show|add|run|remove Reusable tools: executable scripts the agent saves & re-runs (keyring preauth)
+  open-claudia tool list|search|show|add|run|remove Reusable tools: executable scripts the agent saves & re-runs (keyring preauth)
   open-claudia entity list|show|match|note   Entity notes: people/places/projects memory
   open-claudia lessons list|add|remove|show  Always-loaded learned rules (cross-cutting, promoted after a miss)
   open-claudia ideas list|add|remove|show    Self-improvement backlog (captured by the nightly dream)

package/bin/tool.js

@@ -3,6 +3,7 @@
 // heredoc. The executable sibling of context packs (`open-claudia pack`).
 //
 //   open-claudia tool list                       — index (name — description [skill])
+//   open-claudia tool search <query>             — lexical match (find before adding a dup)
 //   open-claudia tool show <name>                — full docs, path, keyring status
 //   open-claudia tool add <path> [--name n]      — register a script as a tool
 //       [--pack <dir>] [--desc "..."] [--requires "k1,k2"] [--usage "..."]
@@ -53,6 +54,25 @@ function run(args) {
       break;
     }
 
+    case "search":
+    case "find": {
+      const query = rest.join(" ").trim();
+      if (!query) { console.error("Usage: tool search <query>"); process.exitCode = 1; return; }
+      const matches = tools.matchTools(query, { limit: 10 });
+      if (!matches.length) {
+        console.log(`No tools match "${query}". See all: open-claudia tool list`);
+        return;
+      }
+      console.log(`${matches.length} tool(s) matching "${query}":\n`);
+      for (const m of matches) {
+        const t = tools.findTool(m.name);
+        const needs = t && t.requires && t.requires.length ? `  (keyring: ${t.requires.join(", ")})` : "";
+        console.log(`• ${m.name} — ${m.description || "(no description)"}${needs}`);
+      }
+      console.log(`\nRun: open-claudia tool run <name> [args]   ·   Docs: open-claudia tool show <name>`);
+      break;
+    }
+
     case "show":
     case "cat": {
       const name = rest[0];
@@ -112,6 +132,17 @@ function run(args) {
           const missing = tools.missingRequires(t);
           if (missing.length) console.log(`Note: missing keyring keys it needs: ${missing.join(", ")}`);
         }
+        // Similarity guard (warn, don't block): if existing tools already cover
+        // this, prefer extending one with a subcommand over a near-duplicate.
+        try {
+          const near = tools.matchTools([t.name, t.description].filter(Boolean).join(" "), { limit: 3 })
+            .filter((m) => m.name !== t.name);
+          if (near.length) {
+            console.log(`\nHeads up — ${near.length} existing tool(s) look similar:`);
+            for (const m of near) console.log(`  • ${m.name} — ${m.description || "(no description)"}`);
+            console.log("Consider extending one with a new subcommand instead of keeping a near-duplicate (open-claudia tool show <name>).");
+          }
+        } catch (e) { /* best-effort */ }
       } catch (e) {
         console.error(`Could not add tool: ${e.message}`); process.exitCode = 1;
       }
@@ -149,7 +180,7 @@ function run(args) {
       break;
 
     default:
-      console.log('Usage: open-claudia tool [list | show <name> | add <path> [flags] | run <name> [args] | remove <name> | path]');
+      console.log('Usage: open-claudia tool [list | search <query> | show <name> | add <path> [flags] | run <name> [args] | remove <name> | path]');
   }
 }
 

package/core/actions.js

@@ -290,6 +290,12 @@ async function handleAction(envelope) {
     await send(`Recall debug: ${state.settings.showRecall ? "on" : "off"}`);
     return;
   }
+  if (d.startsWith("tt:")) {
+    state.settings.showToolTrace = d.slice(3) === "on";
+    saveState();
+    await send(`Tool trace: ${state.settings.showToolTrace ? "on" : "off"}`);
+    return;
+  }
   if (d.startsWith("cw:")) {
     const v = d.slice(3);
     if (v === "default") state.settings.compactWindow = null;

package/core/dream.js

@@ -838,10 +838,35 @@ async function runDream({ trigger = "manual" } = {}) {
       if (tg.edges || tg.decayed || tg.pruned) {
         bits.push(`🔧 Tool graph: ${tg.edges} edges / ${tg.nodes} nodes (decayed ${tg.decayed}, pruned ${tg.pruned}).`);
       }
-      const dangling = toolsLib.listTools().filter((t) => t.pack && !packs.readPack(t.pack));
+      const allTools = toolsLib.listTools();
+      const dangling = allTools.filter((t) => t.pack && !packs.readPack(t.pack));
       if (dangling.length) {
         bits.push(`🔗 ${dangling.length} tool(s) link a missing skill pack: ${dangling.map((t) => `${t.name}→${t.pack}`).join(", ")} — re-link with --pack or remove.`);
       }
+      // Zero-run tools: created but never run and gone cold. Telemetry runCount
+      // is authoritative; file mtime stands in for "how long ago". Flag only —
+      // a never-used tool may still be worth keeping, so the user decides.
+      const STALE_DAYS = Number(process.env.TOOL_STALE_DAYS || 30);
+      const cutoff = Date.now() - STALE_DAYS * 86400000;
+      const coldUnused = allTools.filter((t) => !(t.runCount > 0) && !t.lastUsed).filter((t) => {
+        try { return fs.statSync(t.file).mtimeMs < cutoff; } catch (e) { return false; }
+      });
+      if (coldUnused.length) {
+        bits.push(`🗃️ ${coldUnused.length} tool(s) created but never run in ${STALE_DAYS}+ days: ${coldUnused.map((t) => t.name).join(", ")} — keep, fold into another, or remove.`);
+      }
+      // Near-duplicate pairs: two tools whose name+description overlap strongly,
+      // a sign one should have been a subcommand of the other. Dedupe by sorted
+      // pair key; flag only (consolidating is a judgement call for the user).
+      const DUP_THRESHOLD = Number(process.env.TOOL_DUP_THRESHOLD || 4);
+      const dupPairs = new Set();
+      for (const t of allTools) {
+        const near = toolsLib.matchTools([t.name, t.description].filter(Boolean).join(" "), { threshold: DUP_THRESHOLD, limit: 5 })
+          .filter((m) => m.name !== t.name);
+        for (const m of near) dupPairs.add([t.name, m.name].sort().join(" ↔ "));
+      }
+      if (dupPairs.size) {
+        bits.push(`👯 ${dupPairs.size} near-duplicate tool pair(s): ${[...dupPairs].join(", ")} — consider merging into one tool with subcommands.`);
+      }
       toolNote = bits.join("\n");
     } catch (e) { /* tool graph is best-effort (old node) */ }
 

package/core/handlers.js

@@ -123,7 +123,7 @@ register({
     if (!authorized(env)) return;
     send([
       "Session: /session /sessions /projects /continue /status /stop /end",
-      "Settings: /model /effort /budget /plan /compact /compactwindow /worktree /mode /engine /recall",
+      "Settings: /model /effort /budget /plan /compact /compactwindow /worktree /mode /engine /recall /tooltrace",
       "Identity: /whoami /link",
       "Team: /people /intros /auth (owner)",
       "Automation: /cron /vault /soul /dreamsummary",
@@ -743,6 +743,29 @@ register({
   },
 });
 
+register({
+  name: "tooltrace", description: "Show tool activity each turn (surfaced/ran/updated)", args: "[on|off]",
+  handler: async (env, { tail }) => {
+    if (!authorized(env)) return;
+    const { settings } = currentState();
+    if (tail) {
+      const v = tail.trim().toLowerCase();
+      if (v === "on" || v === "true") settings.showToolTrace = true;
+      else if (v === "off" || v === "false") settings.showToolTrace = false;
+      else return send(`Usage: /tooltrace [on|off]. Currently ${settings.showToolTrace ? "on" : "off"}.`);
+      saveState();
+      return send(`Tool trace: ${settings.showToolTrace ? "on" : "off"}`);
+    }
+    send(
+      `Tool trace: ${settings.showToolTrace ? "on" : "off"}\n\n` +
+      "When on, I post short 🔧 lines around each reply: which reusable tools were surfaced for the turn, which I actually ran, and any I created or updated. Lets you watch the tool layer work. Off by default.",
+      { keyboard: { inline_keyboard: [
+        [{ text: "On", callback_data: "tt:on" }, { text: "Off", callback_data: "tt:off" }],
+      ] } },
+    );
+  },
+});
+
 register({
   name: "budget", description: "Set max spend for next task", args: "[$N]",
   handler: async (env, { tail }) => {

package/core/recall/discoverer.js

@@ -19,6 +19,13 @@ const graph = require("./graph");
 const metrics = require("./metrics");
 const { spawnSubagent } = require("../subagent");
 const warmWalker = require("./warm-walker");
+const toolsLib = require("../tools");
+// Tool-graph is the directed-usage source of truth (pack↔tool edges). The
+// discoverer READS it to activate tools that belong to a surfaced pack; it never
+// writes the recall graph with tool nodes (keeps the hot expand() path pack/
+// entity-only). Optional — absent on old node where node:sqlite is missing.
+let toolGraph = null;
+try { toolGraph = require("../tool-graph"); } catch (e) { /* old node */ }
 
 const WALKER_MODEL = process.env.RECALL_DISCOVERER_MODEL || "haiku";
 const WALKER_TIMEOUT_MS = Number(process.env.RECALL_DISCOVERER_TIMEOUT_MS || 25000);
@@ -64,6 +71,18 @@ function excerptFor(id, packsLib, entitiesLib) {
       const body = [p.sections.Stance, p.sections.State].filter(Boolean).join("\n").trim();
       return { id, name: p.name, description: p.description, excerpt: clip(body, EXCERPT_CHARS) };
     }
+    if (id.startsWith("tool:")) {
+      const name = id.slice(5);
+      const t = toolsLib.findTool(name);
+      if (!t) return null;
+      // Excerpt the walker judges a tool against: what it does (usage) plus the
+      // concrete command shapes it's actually been run as in a topic, so "would
+      // this help here?" is answerable without reading the source.
+      let shapes = [];
+      if (toolGraph) { try { shapes = toolGraph.toolPacks(name, 2).map((p) => p.command).filter(Boolean); } catch (e) {} }
+      const excerpt = [t.usage, shapes.length ? `e.g. ${shapes.join(" | ")}` : ""].filter(Boolean).join(" — ");
+      return { id, name, description: t.description, excerpt: clip(excerpt, EXCERPT_CHARS) };
+    }
     const e = entitiesLib.readEntity(id.slice(7));
     if (!e) return null;
     return { id, name: e.name, description: e.description, excerpt: clip((e.sections.Notes || "").trim(), EXCERPT_CHARS) };
@@ -75,6 +94,7 @@ const WALKER_SYSTEM = [
   "You are given the user's message, recent conversation, and candidate memory nodes (some matched by keyword, some pulled in by graph links).",
   "Decide which nodes are GENUINELY relevant to what the user is doing now.",
   "Keep graph-linked nodes ONLY if they actually bear on the task (e.g. a shared theme/commit-style that governs the thing being worked on). Drop incidental keyword overlaps.",
+  "Some candidates are reusable tools (id starts `tool:`). Keep a tool ONLY if running it would actually help with THIS task; drop tools merely related by keyword.",
   "For each kept node write a terse why (≤12 words). Quote concrete facts verbatim; never invent.",
   'Reply with ONLY a JSON array: [{"id":"pack:foo","why":"shared lime theme governs this app"}]. Use [] if none.',
 ].join("\n");
@@ -154,12 +174,22 @@ async function run(ctx) {
     );
   } catch (e) {}
 
-  const seedCount = packSeeds.length + entSeeds.length;
+  // Tool seeds: direct FTS-style match of the user's words (and context) against
+  // the tool corpus — the SEED stage for tools, peer to pack/entity seeding.
+  const toolSeedByName = new Map();
+  try {
+    for (const t of toolsLib.matchTools(userText, { limit: 5 })) toolSeedByName.set(t.name, { ...t, origin: "user" });
+    if (fullContext) for (const t of toolsLib.matchTools(fullContext, { limit: 5 })) {
+      if (!toolSeedByName.has(t.name)) toolSeedByName.set(t.name, { ...t, origin: "context" });
+    }
+  } catch (e) {}
+
+  const seedCount = packSeeds.length + entSeeds.length + toolSeedByName.size;
 
   // 1: pre-gate.
   if (!needsRecall(userText, seedCount)) {
     metrics.logTurn({ engine: "discoverer", query: userText, gated: true, latencyMs: Date.now() - started });
-    return { packBlock: "", entityBlock: "", packMatches: [], entityMatches: [], why: {}, gated: true };
+    return { packBlock: "", entityBlock: "", toolBlock: "", packMatches: [], entityMatches: [], toolMatches: [], why: {}, gated: true };
   }
 
   // 3: spreading activation from seeds across the graph.
@@ -182,6 +212,47 @@ async function run(ctx) {
     candidates.push({ ...base, activated: !seedIds.has(id), via: act ? act.via : null });
   }
 
+  // Tool candidates = direct tool seeds + tools that belong to any candidate
+  // pack (declared via the tool's `pack:` header, or used-in-topic per the
+  // tool-graph). This is the tool "activation": tools ride on whatever packs the
+  // FTS+graph already surfaced, so a pack seed pulls in its tools the user never
+  // named — then the walker decides which actually help. via != null => activated.
+  const candPackDirs = new Set();
+  for (const id of candIds) if (id.startsWith("pack:")) candPackDirs.add(id.slice(5));
+  const toolCand = new Map();
+  for (const [name, t] of toolSeedByName) toolCand.set(name, { name, description: t.description, via: null });
+  if (candPackDirs.size) {
+    try {
+      const byPack = new Map();
+      for (const t of toolsLib.listTools()) {
+        if (!t.pack) continue;
+        if (!byPack.has(t.pack)) byPack.set(t.pack, []);
+        byPack.get(t.pack).push(t);
+      }
+      for (const dir of candPackDirs) {
+        for (const t of (byPack.get(dir) || [])) {
+          if (!toolCand.has(t.name)) toolCand.set(t.name, { name: t.name, description: t.description, via: `pack:${dir}` });
+        }
+      }
+    } catch (e) {}
+    if (toolGraph) {
+      try {
+        for (const dir of candPackDirs) {
+          for (const u of toolGraph.packTools(dir, 3)) {
+            if (toolCand.has(u.tool)) continue;
+            const t = toolsLib.findTool(u.tool);
+            if (t) toolCand.set(u.tool, { name: u.tool, description: t.description, via: `pack:${dir}` });
+          }
+        }
+      } catch (e) {}
+    }
+  }
+  for (const c of toolCand.values()) {
+    const base = excerptFor(`tool:${c.name}`, packsLib, entitiesLib);
+    if (!base) continue;
+    candidates.push({ ...base, activated: !!c.via, via: c.via });
+  }
+
   const whyById = (await walk(userText, contextText || fullContext, candidates)) || null;
 
   // Decide the kept set. Walker result wins; on fail-open keep the user-origin
@@ -192,7 +263,8 @@ async function run(ctx) {
   } else {
     keptIds = new Set(
       [...packSeeds.filter((m) => m.origin === "user").map((m) => `pack:${m.dir}`),
-       ...entSeeds.filter((m) => m.origin === "user").map((m) => `entity:${m.slug}`)],
+       ...entSeeds.filter((m) => m.origin === "user").map((m) => `entity:${m.slug}`),
+       ...[...toolSeedByName.values()].filter((t) => t.origin === "user").map((t) => `tool:${t.name}`)],
     );
   }
 
@@ -229,6 +301,25 @@ async function run(ctx) {
     if (bullets.length) packBlock = `\n\n### Why these surfaced (discoverer)\n${bullets.join("\n")}${packBlock}`;
   }
 
+  // Tools the walker kept (or, on fail-open, user-origin tool seeds). Rendered as
+  // their own contextual block — this replaces the old flat "all 40 tools every
+  // turn" dump: tools surface only when they bear on the task, with a why.
+  const toolMatches = [];
+  for (const c of toolCand.values()) {
+    if (keptIds.has(`tool:${c.name}`)) {
+      toolMatches.push({ name: c.name, description: c.description, why: whyById ? whyById.get(`tool:${c.name}`) || "" : "" });
+    }
+  }
+  let toolBlock = "";
+  if (toolMatches.length) {
+    const lines = toolMatches.map((m) => {
+      const t = toolsLib.findTool(m.name);
+      const needs = t && t.requires && t.requires.length ? ` (keyring: ${t.requires.join(", ")})` : "";
+      return `- ${m.name} — ${m.description || "(no description)"}${needs}${m.why ? ` — ${m.why}` : ""}`;
+    });
+    toolBlock = `\n\n## Tools that may help here\nSurfaced for this turn — run with \`open-claudia tool run <name> [args]\`; read docs/source with \`open-claudia tool show <name>\`:\n${lines.join("\n")}\n`;
+  }
+
   metrics.logTurn({
     engine: "discoverer",
     query: userText,
@@ -240,8 +331,8 @@ async function run(ctx) {
   });
 
   return {
-    packBlock, entityBlock, packMatches: finalPacks, entityMatches: finalEnts,
-    why: whyById ? Object.fromEntries(whyById) : {}, gated: false,
+    packBlock, entityBlock, toolBlock, packMatches: finalPacks, entityMatches: finalEnts,
+    toolMatches, why: whyById ? Object.fromEntries(whyById) : {}, gated: false,
   };
 }
 

package/core/runner.js

@@ -889,6 +889,13 @@ async function runClaude(prompt, cwd, replyToMsgId, opts = {}) {
     skillNotified.add(key);
     chatContext.run(store, () => send(text).catch(() => {}));
   };
+  // Tool-activity visibility (surfaced / ran / created-updated). Off by default,
+  // toggled per-channel with /tooltrace — mirrors /recall's showRecall. Dedup +
+  // delivery reuse notifySkill so a tool touched twice a turn announces once.
+  const notifyToolTrace = (key, text) => {
+    if (!settings.showToolTrace) return;
+    notifySkill(key, text);
+  };
   const noteSkillToolUse = (toolName, input) => {
     try {
       if (toolName === "Skill" && input?.skill) {
@@ -918,8 +925,8 @@ async function runClaude(prompt, cwd, replyToMsgId, opts = {}) {
         }
         const toolName2 = toolsLib.toolNameFromPath(filePath);
         if (toolName2) {
-          if (toolsLib.findTool(toolName2)) notifySkill(`tool:${toolName2}`, `🔧 Updating tool: ${toolName2} — open-claudia tool show ${toolName2} to inspect.`);
-          else notifySkill(`tool:${toolName2}`, `🔧 New tool: ${toolName2} — open-claudia tool run ${toolName2} to use it.`);
+          if (toolsLib.findTool(toolName2)) notifyToolTrace(`tool:${toolName2}`, `🔧 Updating tool: ${toolName2} — open-claudia tool show ${toolName2} to inspect.`);
+          else notifyToolTrace(`tool:${toolName2}`, `🔧 New tool: ${toolName2} — open-claudia tool run ${toolName2} to use it.`);
           return;
         }
         const dir = skillsLib.skillNameFromPath(filePath);
@@ -971,8 +978,23 @@ async function runClaude(prompt, cwd, replyToMsgId, opts = {}) {
       // next); the sanitized command shape feeds pack↔tool capture (how a tool
       // is used in a topic). Parsing lives in tool-graph so it stays testable.
       try {
-        for (const run of require("./tool-graph").parseToolRuns(cmd)) toolRunsThisTurn.push(run);
+        for (const run of require("./tool-graph").parseToolRuns(cmd)) {
+          toolRunsThisTurn.push(run);
+          // Visibility (gated by /tooltrace): one 🔧 line per distinct tool run
+          // this turn. Telemetry (recordRun) fires at end-of-turn on success.
+          notifyToolTrace(`ran:${run.name}`, `🔧 Ran tool: ${run.name}`);
+        }
       } catch (e) { /* graph optional (old node) */ }
+      // `open-claudia tool add <src> [--name X]` — a tool being saved/updated.
+      // The Write/Edit path above only catches direct edits to TOOLS_DIR; `tool
+      // add` copies an external file in, so detect it here. Gated by /tooltrace.
+      const addRe = /\btool\s+add\s+(\S+)([^\n;|&]*)/gi;
+      while ((m = addRe.exec(cmd))) {
+        let name = (m[2].match(/--name\s+["']?([^\s"']+)/i) || [])[1];
+        if (!name) name = (String(m[1]).split(/[\\/]/).pop() || "").replace(/\.[^.]+$/, "");
+        name = toolsLib.sanitizeName(name);
+        if (name) notifyToolTrace(`added:${name}`, `🛠️ Saved tool: ${name} — open-claudia tool show ${name} to inspect.`);
+      }
     } catch (e) { /* announcements are best-effort */ }
   };
 
@@ -1007,13 +1029,21 @@ async function runClaude(prompt, cwd, replyToMsgId, opts = {}) {
   // drained here to keep the per-turn buffer from leaking into the next turn.)
   try {
     const injected = require("./system-prompt").consumeLastInjected();
-    if (settings.showRecall && injected && injected.recall) {
+    if (injected && injected.recall) {
       const r = injected.recall;
       const esc = (s) => String(s).replace(/&/g, "&amp;").replace(/</g, "&lt;").replace(/>/g, "&gt;");
       const fmt = (arr, icon) => arr.map((x) => (x.why ? `${icon} <b>${esc(x.name)}</b> — ${esc(x.why)}` : `${icon} <b>${esc(x.name)}</b>`));
-      const lines = [...fmt(r.packs || [], "📦"), ...fmt(r.entities || [], "👤")];
-      if (lines.length) send(`🧠 <b>Recall this turn</b> (${esc(r.engine)})\n${lines.join("\n")}`, telegramHtmlOpts()).catch(() => {});
-      else if (r.gated) send(`🧠 <b>Recall</b> (${esc(r.engine)}): skipped by pre-gate — trivial turn.`, telegramHtmlOpts()).catch(() => {});
+      // 🧠 packs/entities recall — gated by /recall.
+      if (settings.showRecall) {
+        const lines = [...fmt(r.packs || [], "📦"), ...fmt(r.entities || [], "👤")];
+        if (lines.length) send(`🧠 <b>Recall this turn</b> (${esc(r.engine)})\n${lines.join("\n")}`, telegramHtmlOpts()).catch(() => {});
+        else if (r.gated) send(`🧠 <b>Recall</b> (${esc(r.engine)}): skipped by pre-gate — trivial turn.`, telegramHtmlOpts()).catch(() => {});
+      }
+      // 🔧 tools surfaced this turn (first-class discoverer nodes) — gated by /tooltrace.
+      if (settings.showToolTrace) {
+        const toolLines = fmt(r.tools || [], "🔧");
+        if (toolLines.length) send(`🔧 <b>Tools surfaced this turn</b>\n${toolLines.join("\n")}`, telegramHtmlOpts()).catch(() => {});
+      }
     }
   } catch (e) { /* best-effort */ }
   // /stop landed during the pre-spawn window (recall/compaction): bail before
@@ -1369,6 +1399,15 @@ async function runClaude(prompt, cwd, replyToMsgId, opts = {}) {
       catch (e) { /* best-effort */ }
     }
 
+    // Tool usage telemetry: a JSON sidecar (runCount + lastUsed per tool) that
+    // works on every node version (unlike the SQLite graphs above). matchTools
+    // breaks score ties by runCount so well-worn tools surface first, and dream
+    // hygiene flags tools created-but-never-run. Fires for any successful run.
+    if (turnSucceeded && toolRunsThisTurn.length) {
+      try { for (const t of toolRunsThisTurn) toolsLib.recordRun(t.name); }
+      catch (e) { /* best-effort */ }
+    }
+
     // Pack↔tool contextual edges: a reusable tool run while a context pack was
     // open this turn records "this tool was used in this topic, as `<shape>`",
     // so next time the pack is active the discoverer can surface the concrete

package/core/system-prompt.js

@@ -66,37 +66,18 @@ function buildSkillIndexBlock() {
   }
 }
 
-// Always-on tool index: the executable sibling of the skill index. Each entry
-// is a real script the agent crystallised from a working procedure, surfaced by
-// name+description every turn (progressive disclosure — full docs/source load on
-// demand via `open-claudia tool show <name>`). Tools run pre-authed: the
-// operational keyring is merged into their env, so a tool references a credential
-// as $NAME and never hardcodes a secret.
+// Always-on tool guidance: teaches the BEHAVIOUR (crystallise procedures into
+// tools; how to find and run them). It does NOT dump the tool list — that used to
+// inject all ~40 tools every turn regardless of relevance. Tools now surface
+// contextually as first-class discoverer nodes (see discoverer.js → toolBlock),
+// the same seed→activate→judge→render path as packs/entities; this block only
+// names the count and the lookup commands (`tool list` / `tool search`).
 function buildToolIndexBlock() {
   let listing = "";
   try {
-    const all = require("./tools").listTools();
-    if (all.length) {
-      // Directed tool-graph: surface the strongest "runs next" follower per tool
-      // so a known chain (auth → list → download) is visible inline. Optional —
-      // absent on old node where node:sqlite is missing.
-      let graph = null;
-      try { graph = require("./tool-graph"); } catch (e) {}
-      listing = "\nTools you already have (run with `open-claudia tool run <name> [args]`; read docs/source with `open-claudia tool show <name>`):\n\n" +
-        all.slice(0, 40)
-          .map((t) => {
-            const skill = t.pack ? ` [skill: ${t.pack}]` : "";
-            const needs = t.requires && t.requires.length ? ` (keyring: ${t.requires.join(", ")})` : "";
-            let next = "";
-            if (graph) {
-              try {
-                const after = graph.followers(t.name, 2).map((r) => r.name);
-                if (after.length) next = ` → usually followed by: ${after.join(", ")}`;
-              } catch (e) {}
-            }
-            return `- ${t.name} — ${t.description || "(no description)"}${needs}${skill}${next}`;
-          })
-          .join("\n") + "\n";
+    const count = require("./tools").listTools().length;
+    if (count) {
+      listing = `\nYou have ${count} reusable tool${count === 1 ? "" : "s"}. The ones relevant to a turn are surfaced automatically below (\"Tools that may help here\"). To find others on demand: \`open-claudia tool search <query>\` (lexical match) or \`open-claudia tool list\`; read docs/source with \`open-claudia tool show <name>\`; run with \`open-claudia tool run <name> [args]\`.\n`;
     } else {
       listing = "\nNo reusable tools saved yet — the first time you work out how to hit an API or drive an interface, save it as one.\n";
     }
@@ -761,18 +742,23 @@ async function promptWithDynamicContext(prompt, opts = {}) {
       userText, contextText, fullContext, packLimit, budget, helpers,
     });
     const { packBlock, entityBlock } = result;
+    const toolBlock = result.toolBlock || "";
     const why = result.why || {};
     lastInjected.recall = {
       engine: engine.name || recall.activeEngineName(settings),
       gated: !!result.gated,
       packs: (result.packMatches || []).map((m) => ({ name: m.name || m.dir, why: why[`pack:${m.dir}`] || "" })),
       entities: (result.entityMatches || []).map((m) => ({ name: m.name || m.slug, why: why[`entity:${m.slug}`] || "" })),
+      // Tools surfaced as first-class discoverer nodes this turn (name + why).
+      // Drives the /tooltrace "surfaced" banner; the toolBlock below is the
+      // context the agent actually sees.
+      tools: (result.toolMatches || []).map((m) => ({ name: m.name, why: m.why || why[`tool:${m.name}`] || "" })),
     };
     const budgetNote = budget.omitted > 0
       ? `\n\n## Memory budget\n${budget.omitted} matched memory item${budget.omitted === 1 ? " was" : "s were"} omitted to keep this turn under the recall budget (${budget.maxChars} chars). Use \`open-claudia pack show <dir>\`, \`entity show <slug>\`, or transcript search if deeper context is needed.`
       : "";
     const transcriptPointer = opts.includeTranscriptPointer ? `${transcriptPointerNote(state)}\n\n` : "";
-    return `${transcriptPointer}${buildDynamicContextBlock()}${packBlock}${entityBlock}${budgetNote}${voiceReplyGuidance()}\n\nCurrent user request:\n${prompt}`;
+    return `${transcriptPointer}${buildDynamicContextBlock()}${packBlock}${entityBlock}${toolBlock}${budgetNote}${voiceReplyGuidance()}\n\nCurrent user request:\n${prompt}`;
   } catch (e) {
     return prompt;
   }

package/core/tool-graph.js

@@ -5,9 +5,18 @@
 // Why separate: the recall graph spreads activation over packs+entities to pick
 // which MEMORY to surface. Tools are a different corpus with different physics —
 // the useful signal is the ORDER a chain runs in (auth → list → download), not
-// symmetric "these are related". Mixing tool nodes into the recall graph would
-// let a tool fire a pack headline (and vice-versa), which is noise. So tools get
-// their own tiny graph.
+// symmetric "these are related". So tools get their own tiny graph and the recall
+// graph's hot expand() path stays pack/entity-only (no tuning, no crowding of
+// structural pack edges, works unchanged on every node version).
+//
+// How tools still reach the discoverer: this graph is the directed-usage source
+// of truth, and the discoverer READS it (recordPackTool → packTools/toolPacks) to
+// activate the tools belonging to a pack it already surfaced — then its haiku
+// walker judges each tool with a why and drops incidental ones. That walker is
+// the noise filter an earlier "mixing would be noise" worry assumed didn't exist;
+// because the discoverer judges candidates, tools participate as first-class
+// nodes (seed via matchTools → activate via this graph → judge → render) WITHOUT
+// ever being written into the recall graph as nodes.
 //
 // Why directed: A→B ("B follows A") carries real information that B→A does not.
 // auth-then-fetch is a pipeline; fetch-then-auth is nonsense. Edges are stored

package/core/tools.js

@@ -35,6 +35,97 @@ function sanitizeName(name) {
     .replace(/[^a-z0-9._-]+/g, "-").replace(/^[-.]+|[-.]+$/g, "").slice(0, 60);
 }
 
+// ── Run telemetry ────────────────────────────────────────────────────────────
+// A tiny JSON sidecar (name -> { runCount, lastUsed }) kept beside the tools.
+// Deliberately NOT a SQLite graph: telemetry must work on the node:20 pods where
+// node:sqlite is absent, and it's a flat counter, not a graph. Dotfile so
+// listTools (which skips ".") never mistakes it for a tool.
+const USAGE_FILE = path.join(TOOLS_DIR, ".usage.json");
+
+function readUsage() {
+  try { return JSON.parse(fs.readFileSync(USAGE_FILE, "utf-8")) || {}; }
+  catch (e) { return {}; }
+}
+
+function writeUsage(u) {
+  try { ensureDir(); fs.writeFileSync(USAGE_FILE, JSON.stringify(u), { mode: 0o600 }); }
+  catch (e) { /* best-effort */ }
+}
+
+// Bump a tool's run counter. Called at end of turn from the runner when it parses
+// an `open-claudia tool run <name>` out of the shell. Feeds matchTools' tiebreak
+// (popular tools rank up) and the dream's zero-run hygiene flag.
+function recordRun(name) {
+  const n = sanitizeName(name);
+  if (!n) return;
+  const u = readUsage();
+  const cur = u[n] || { runCount: 0, lastUsed: "" };
+  cur.runCount = (cur.runCount || 0) + 1;
+  cur.lastUsed = new Date().toISOString();
+  u[n] = cur;
+  writeUsage(u);
+}
+
+// ── Lexical matching (in-memory) ─────────────────────────────────────────────
+// Tools are few and short, so we score in memory rather than standing up a third
+// FTS index — and unlike the recall/tool graphs this then works on every node
+// version. Mirrors matchPacks' semantics: a hit in a strong field (name /
+// description / usage) counts 2, a weak-field hit (requires / pack / source) 1,
+// tools at/above the threshold win. This is the SEED stage of the discoverer for
+// the tool corpus — keyword feeds the agentic walker, it does not replace it.
+const TOOL_STOPWORDS = new Set(("a an and are as at be but by can did do for from had has have how i if in is it its " +
+  "me my no not of on or our so that the their then there this to up us was we what when where which who why " +
+  "will with you your yes yeah ok okay please thanks just like dont im its run tool use").split(" "));
+
+function toolQueryTerms(text) {
+  const seen = new Set();
+  const terms = [];
+  for (const raw of String(text || "").toLowerCase().split(/[^a-z0-9_.-]+/)) {
+    const t = raw.replace(/^[.-]+|[.-]+$/g, "");
+    if (t.length < 2 || TOOL_STOPWORDS.has(t) || seen.has(t)) continue;
+    seen.add(t);
+    terms.push(t);
+    if (terms.length >= 40) break;
+  }
+  return terms;
+}
+
+function wordsOf(s) {
+  return new Set(String(s || "").toLowerCase().split(/[^a-z0-9_.-]+/).filter(Boolean));
+}
+
+// A term hits a word set on exact match, or a light prefix stem (deploy~deploys,
+// upload~uploads) for terms long enough that a prefix isn't noise.
+function termHits(term, wordSet) {
+  if (wordSet.has(term)) return true;
+  if (term.length >= 4) {
+    for (const w of wordSet) if (w.startsWith(term) || term.startsWith(w)) return true;
+  }
+  return false;
+}
+
+// Score registered tools against free text. Returns [{ name, description, score,
+// runCount }] over the threshold, strongest first (runCount breaks ties so a
+// well-worn tool outranks a never-run namesake).
+function matchTools(text, { limit = 5, threshold = null } = {}) {
+  const terms = toolQueryTerms(text);
+  if (!terms.length) return [];
+  const min = threshold ?? Number(process.env.TOOL_MATCH_THRESHOLD || 2);
+  const out = [];
+  for (const t of listTools()) {
+    const strong = wordsOf([t.name, t.description, t.usage].filter(Boolean).join(" "));
+    const weak = wordsOf([(t.requires || []).join(" "), t.pack, t.content].filter(Boolean).join(" "));
+    let score = 0;
+    for (const term of terms) {
+      if (termHits(term, strong)) score += 2;
+      else if (termHits(term, weak)) score += 1;
+    }
+    if (score >= min) out.push({ name: t.name, description: t.description, score, runCount: t.runCount || 0 });
+  }
+  out.sort((a, b) => b.score - a.score || (b.runCount || 0) - (a.runCount || 0) || a.name.localeCompare(b.name));
+  return out.slice(0, Math.max(1, Math.min(10, limit)));
+}
+
 // Strip a leading comment prefix ("# " / "// ") from a header line. Returns the
 // remainder, or null if the line is not a comment (header block has ended).
 function uncomment(line) {
@@ -75,7 +166,7 @@ function toolFile(name) {
   return path.join(TOOLS_DIR, sanitizeName(name));
 }
 
-function readTool(name) {
+function readTool(name, usageMap) {
   const file = toolFile(name);
   let content;
   try { content = fs.readFileSync(file, "utf-8"); } catch (e) { return null; }
@@ -83,6 +174,7 @@ function readTool(name) {
   if (!header) return null;
   let stat = null;
   try { stat = fs.statSync(file); } catch (e) {}
+  const usage = (usageMap || readUsage())[header.name || sanitizeName(name)] || {};
   return {
     name: header.name || sanitizeName(name),
     description: header.description || "",
@@ -92,6 +184,8 @@ function readTool(name) {
     file,
     executable: stat ? !!(stat.mode & 0o111) : false,
     updatedAt: stat ? stat.mtime.toISOString() : "",
+    runCount: usage.runCount || 0,
+    lastUsed: usage.lastUsed || "",
     content,
   };
 }
@@ -99,12 +193,13 @@ function readTool(name) {
 function listTools() {
   let entries;
   try { entries = fs.readdirSync(TOOLS_DIR); } catch (e) { return []; }
+  const usage = readUsage();
   const tools = [];
   for (const name of entries) {
     if (name.startsWith(".")) continue;
     try { if (!fs.statSync(path.join(TOOLS_DIR, name)).isFile()) continue; }
     catch (e) { continue; }
-    const t = readTool(name);
+    const t = readTool(name, usage);
     if (t) tools.push(t);
   }
   tools.sort((a, b) => a.name.localeCompare(b.name));
@@ -221,7 +316,8 @@ function toolNameFromPath(filePath) {
 }
 
 module.exports = {
-  TOOLS_DIR, MARKER,
+  TOOLS_DIR, MARKER, USAGE_FILE,
   parseHeader, readTool, listTools, findTool, addTool, removeTool,
   runEnv, missingRequires, toolNameFromPath, sanitizeName, ensureDir,
+  matchTools, recordRun, readUsage,
 };

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@inetafrica/open-claudia",
-  "version": "2.6.53",
+  "version": "2.6.54",
   "description": "Your always-on AI coding assistant — Claude Code, Cursor Agent, and OpenAI Codex via Telegram or Kazee Chat",
   "main": "bot.js",
   "bin": {

package/test-recall-discoverer.js

@@ -10,8 +10,19 @@ const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "recall-disc-"));
 process.env.RECALL_GRAPH_DB = path.join(tmp, "graph.db");
 process.env.RECALL_DISCOVERER_WALKER = "off";
 process.env.RECALL_METRICS = "off";
+process.env.TOOLS_DIR = path.join(tmp, "tools");
 
 const disc = require("./core/recall/discoverer");
+const toolsLib = require("./core/tools");
+
+// A reusable tool whose description matches a query should seed the discoverer
+// as a first-class candidate (origin user), survive the walker-off fail-open,
+// and render into the toolBlock — exactly like a pack/entity seed.
+toolsLib.addTool((() => {
+  const s = path.join(tmp, "mikrotik-audit.sh");
+  fs.writeFileSync(s, "#!/usr/bin/env bash\necho audit\n");
+  return s;
+})(), { name: "mikrotik-audit", description: "audit the mikrotik router fleet inventory" });
 
 // --- pre-gate ---
 assert.strictEqual(disc.needsRecall("", 0), false, "empty → no recall");
@@ -82,5 +93,22 @@ const helpers = { packsLib, entitiesLib, mergeMatches, buildPackBlock, buildEnti
   });
   assert.strictEqual(typeof out2.packBlock, "string");
 
+  // tool seeding: a tool matching the user text surfaces as a kept tool match and
+  // renders into toolBlock; a turn that matches no tool yields an empty toolBlock
+  const outTool = await disc.run({
+    userText: "audit the mikrotik router fleet", contextText: "",
+    fullContext: "audit the mikrotik router fleet", packLimit: 6, budget: {}, helpers,
+  });
+  assert.ok(outTool.toolMatches.some((m) => m.name === "mikrotik-audit"), "matching tool is kept (fail-open user seed)");
+  assert.ok(/mikrotik-audit/.test(outTool.toolBlock), "kept tool renders into the toolBlock");
+  assert.ok(/Tools that may help here/.test(outTool.toolBlock), "toolBlock carries its heading");
+
+  const outNoTool = await disc.run({
+    userText: "help with the mobile app", contextText: "", fullContext: "help with the mobile app",
+    packLimit: 6, budget: {}, helpers,
+  });
+  assert.strictEqual(outNoTool.toolBlock, "", "no tool match → empty toolBlock");
+  assert.strictEqual(outNoTool.toolMatches.length, 0, "no tool match → no toolMatches");
+
   console.log("recall discoverer OK");
 })().catch((e) => { console.error(e); process.exit(1); });

package/test-tools.js

@@ -85,6 +85,36 @@ assert.strictEqual(
 // ── parseHeader returns null for a script with no marker line ──
 assert.strictEqual(tools.parseHeader("#!/bin/sh\necho hi\n"), null, "no marker → not a tool");
 
+// ── matchTools: lexical scoring over name/description (strong) vs requires/pack
+//    /source (weak); a non-matching query returns nothing ──
+const mHello = tools.matchTools("say hi");
+assert.ok(mHello.some((m) => m.name === "hello"), "matchTools finds 'hello' by its description terms");
+const mFetch = tools.matchTools("fetch a thing");
+assert.ok(mFetch.some((m) => m.name === "fetch"), "matchTools finds 'fetch'");
+assert.ok(!mFetch.some((m) => m.name === "hello"), "an unrelated tool does not match");
+assert.deepStrictEqual(tools.matchTools("zzzznomatchatall"), [], "no terms hit → empty match");
+assert.deepStrictEqual(tools.matchTools(""), [], "empty query → empty match");
+
+// ── recordRun telemetry: bumps runCount + stamps lastUsed, visible via readUsage
+//    and surfaced on the tool record itself ──
+assert.strictEqual(tools.findTool("hello").runCount, 0, "fresh tool has runCount 0");
+tools.recordRun("hello");
+assert.strictEqual(tools.readUsage().hello.runCount, 1, "recordRun bumps the usage sidecar");
+assert.ok(tools.readUsage().hello.lastUsed, "recordRun stamps lastUsed");
+assert.strictEqual(tools.findTool("hello").runCount, 1, "runCount surfaces on the tool record");
+tools.recordRun("hello");
+assert.strictEqual(tools.findTool("hello").runCount, 2, "a second run increments again");
+
+// ── matchTools tie-break: equal score → more-run tool ranks first ──
+for (const n of ["alpha", "beta"]) {
+  const s = path.join(tmp, `${n}.sh`);
+  fs.writeFileSync(s, "#!/usr/bin/env bash\necho x\n");
+  tools.addTool(s, { name: n, description: "common widget gadget" });
+}
+tools.recordRun("beta");
+const ranked = tools.matchTools("common widget gadget").map((m) => m.name);
+assert.ok(ranked.indexOf("beta") < ranked.indexOf("alpha"), "on equal score, the more-run tool ranks first");
+
 // ── remove ──
 assert.ok(tools.removeTool("hello"), "removeTool returns the removed tool");
 assert.strictEqual(tools.findTool("hello"), null, "removed tool is gone");