@inetafrica/open-claudia 2.6.52 → 2.6.54
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +10 -0
- package/bin/cli.js +1 -1
- package/bin/tool.js +40 -1
- package/core/actions.js +6 -0
- package/core/dream.js +27 -2
- package/core/handlers.js +24 -1
- package/core/recall/discoverer.js +96 -5
- package/core/runner.js +74 -16
- package/core/system-prompt.js +29 -29
- package/core/tool-graph.js +193 -19
- package/core/tools.js +99 -3
- package/package.json +1 -1
- package/test-recall-discoverer.js +28 -0
- package/test-tool-graph.js +59 -0
- package/test-tools.js +30 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,15 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## v2.6.54
|
|
4
|
+
- **Tools as first-class discoverer nodes.** Reusable tools now go through the same recall pipeline as packs and entities — seed → activate → judge → render — instead of being dumped wholesale into every prompt. Until now `buildToolIndexBlock` listed *all* ~40 tools on every turn; that flat block is gone. In its place, the discoverer (`core/recall/discoverer.js`) seeds tool candidates from a new in-memory `matchTools()` (lexical scoring over name/description = strong, requires/pack/source = weak; `core/tools.js`), activates the tools belonging to any pack already in the candidate set (read from the pack↔tool graph + tool `pack:` headers — the latency-sensitive recall `expand()` stays pack/entity-only, zero regression risk), and lets the haiku walker judge them with a "keep a tool only if running it would actually help" instruction. Survivors render into a focused **"Tools that may help here"** block; the system prompt now just states the count and points to `tool search`/`tool list` for the rest. `matchTools` is in-memory (works on every node version, unlike the SQLite graphs).
|
|
5
|
+
- **`/tooltrace` toggle (default off).** Mirrors `/recall`. When on, posts short 🔧 lines around each reply: which tools were **surfaced** for the turn, which were **run**, and any **created/updated** — so you can watch the tool layer work. All three signals are now gated behind this one toggle (`settings.showToolTrace`); the run/created/updated banners were removed from the always-on path and the surfaced list was split out of the 🧠 `/recall` banner, so the two toggles are cleanly independent.
|
|
6
|
+
- **Tool usage telemetry.** A JSON sidecar (`.usage.json` in the tools dir: `runCount` + `lastUsed` per tool, recorded at end-of-turn for each successful run) works on every node version. `matchTools` breaks score ties by `runCount` so well-worn tools surface first.
|
|
7
|
+
- **`tool search` + add-time duplicate guard.** New `open-claudia tool search <query>` for finding a tool on demand; `tool add` now warns (non-blocking) when an existing tool looks similar, nudging toward extending it with a subcommand over spawning a near-duplicate.
|
|
8
|
+
- **Dream tool hygiene.** The nightly pass now also flags tools created-but-never-run for 30+ days and near-duplicate tool pairs (alongside the existing dangling-`--pack` flag). Flag-only — a tool is never auto-deleted. Extends `test-tools.js` (matchTools/telemetry) and `test-recall-discoverer.js` (tool seeding). Degrades to a graceful no-op where `node:sqlite` is unavailable.
|
|
9
|
+
|
|
10
|
+
## v2.6.53
|
|
11
|
+
- **Pack↔tool contextual edges — surface how a tool is used in a topic.** Completes the v2.6.49 tool story. The directed tool-graph answers "what runs *next*"; this adds the orthogonal question "how is a tool *used in this context*". A new `pack_tool_edges` table (`core/tool-graph.js`, sharing the existing SQLite store, decay and pruning) records, for each turn, the reusable tools that ran while a context pack was open — keyed by `(pack, tool, command-shape)` so distinct invocations of the same tool coexist and the strongest surface. Capture is automatic and deterministic (no hand-authoring): `runner.js` already tracks the tools run and the packs opened each turn, so at end-of-turn it records a pack↔tool edge per (opened-pack × run-tool), Hebbian-reinforced. Commands are stored as **shapes, not literals** — a new `shapeCommand()` strips ids, paths, numbers and quoted text to placeholders (`spaces docs --task-id <id> <path>`) so the memory is a reusable pattern, not a stale one-off; `parseToolRuns()` extracts `{name, shape}` from a shell command (split across chained invocations) and feeds both graphs. Surfacing: when a pack is injected into context, `formatPackForContext` appends a **"Tools used in this pack"** block with the 1–2 strongest invocations, so the agent sees the concrete command instead of re-deriving it; `tool show` gains the inverse "Used in packs" view; an optional `intent` column lets a one-line *why* be enriched later without a migration. The nightly dream tends these edges alongside the follows-edges (decay + prune orphaned tools/packs). Degrades to a graceful no-op where `node:sqlite` is unavailable, exactly like the existing tool-graph. Extends `test-tool-graph.js`.
|
|
12
|
+
|
|
3
13
|
## v2.6.49
|
|
4
14
|
- **Reusable tools + a directed tool-usage graph.** Two linked features. First, **reusable tools** (`core/tools.js`, `bin/tool.js`): the executable sibling of context packs — when the agent works out an operational procedure (hit an API, drive an interface, transform a file) it crystallises the script into a re-runnable command instead of a throwaway heredoc. Tools are one executable file with a parseable comment header, run pre-authed (the operational keyring is merged into their env so they reference `$NAME` and never hardcode a secret), can link to an owning skill pack, and are surfaced every turn as an always-on index (full docs/source on demand via `open-claudia tool show <name>`). Full CLI: `open-claudia tool list|show|add|run|remove`; `/learn` now crystallises the executable part too. Second, a **directed tool-graph** (`core/tool-graph.js`): a "tool B follows tool A" graph kept deliberately separate from the recall graph — for tools the useful signal is the *order* a chain runs in (auth → list → download), so edges are stored and traversed directionally. Each run-sequence reinforces consecutive pairs (Hebbian, decayed nightly, no structural floor so unused chains prune away). `tool show` and the system-prompt index surface "usually followed by …"; `pack show` gains the reverse view (a pack's linked tools); the nightly dream tends the graph (decay + orphan prune) and flags tools whose `--pack` link dangles. Adds `test-tools.js` + `test-tool-graph.js`.
|
|
5
15
|
|
package/bin/cli.js
CHANGED
|
@@ -370,7 +370,7 @@ Memory tools:
|
|
|
370
370
|
open-claudia transcript-window <pattern> Search project transcript, show hits with context
|
|
371
371
|
(alias: tw; --help for options)
|
|
372
372
|
open-claudia pack list|show|match|archive|restore|archived Context packs: living topic docs (skills + memory)
|
|
373
|
-
open-claudia tool list|show|add|run|remove Reusable tools: executable scripts the agent saves & re-runs (keyring preauth)
|
|
373
|
+
open-claudia tool list|search|show|add|run|remove Reusable tools: executable scripts the agent saves & re-runs (keyring preauth)
|
|
374
374
|
open-claudia entity list|show|match|note Entity notes: people/places/projects memory
|
|
375
375
|
open-claudia lessons list|add|remove|show Always-loaded learned rules (cross-cutting, promoted after a miss)
|
|
376
376
|
open-claudia ideas list|add|remove|show Self-improvement backlog (captured by the nightly dream)
|
package/bin/tool.js
CHANGED
|
@@ -3,6 +3,7 @@
|
|
|
3
3
|
// heredoc. The executable sibling of context packs (`open-claudia pack`).
|
|
4
4
|
//
|
|
5
5
|
// open-claudia tool list — index (name — description [skill])
|
|
6
|
+
// open-claudia tool search <query> — lexical match (find before adding a dup)
|
|
6
7
|
// open-claudia tool show <name> — full docs, path, keyring status
|
|
7
8
|
// open-claudia tool add <path> [--name n] — register a script as a tool
|
|
8
9
|
// [--pack <dir>] [--desc "..."] [--requires "k1,k2"] [--usage "..."]
|
|
@@ -53,6 +54,25 @@ function run(args) {
|
|
|
53
54
|
break;
|
|
54
55
|
}
|
|
55
56
|
|
|
57
|
+
case "search":
|
|
58
|
+
case "find": {
|
|
59
|
+
const query = rest.join(" ").trim();
|
|
60
|
+
if (!query) { console.error("Usage: tool search <query>"); process.exitCode = 1; return; }
|
|
61
|
+
const matches = tools.matchTools(query, { limit: 10 });
|
|
62
|
+
if (!matches.length) {
|
|
63
|
+
console.log(`No tools match "${query}". See all: open-claudia tool list`);
|
|
64
|
+
return;
|
|
65
|
+
}
|
|
66
|
+
console.log(`${matches.length} tool(s) matching "${query}":\n`);
|
|
67
|
+
for (const m of matches) {
|
|
68
|
+
const t = tools.findTool(m.name);
|
|
69
|
+
const needs = t && t.requires && t.requires.length ? ` (keyring: ${t.requires.join(", ")})` : "";
|
|
70
|
+
console.log(`• ${m.name} — ${m.description || "(no description)"}${needs}`);
|
|
71
|
+
}
|
|
72
|
+
console.log(`\nRun: open-claudia tool run <name> [args] · Docs: open-claudia tool show <name>`);
|
|
73
|
+
break;
|
|
74
|
+
}
|
|
75
|
+
|
|
56
76
|
case "show":
|
|
57
77
|
case "cat": {
|
|
58
78
|
const name = rest[0];
|
|
@@ -77,6 +97,14 @@ function run(args) {
|
|
|
77
97
|
const before = graph.predecessors(t.name);
|
|
78
98
|
if (after.length) console.log(`Often followed by: ${fmt(after)}`);
|
|
79
99
|
if (before.length) console.log(`Often preceded by: ${fmt(before)}`);
|
|
100
|
+
// Pack↔tool: contexts this tool has been used in, with the shape used.
|
|
101
|
+
const inPacks = graph.toolPacks(t.name, 5);
|
|
102
|
+
if (inPacks.length) {
|
|
103
|
+
console.log("Used in packs:");
|
|
104
|
+
for (const u of inPacks) {
|
|
105
|
+
console.log(` • ${u.pack}: open-claudia tool run ${u.command || t.name}${u.intent ? ` — ${u.intent}` : ""}`);
|
|
106
|
+
}
|
|
107
|
+
}
|
|
80
108
|
} catch (e) { /* graph optional (old node) */ }
|
|
81
109
|
console.log(`\n----- source -----\n${t.content}`);
|
|
82
110
|
break;
|
|
@@ -104,6 +132,17 @@ function run(args) {
|
|
|
104
132
|
const missing = tools.missingRequires(t);
|
|
105
133
|
if (missing.length) console.log(`Note: missing keyring keys it needs: ${missing.join(", ")}`);
|
|
106
134
|
}
|
|
135
|
+
// Similarity guard (warn, don't block): if existing tools already cover
|
|
136
|
+
// this, prefer extending one with a subcommand over a near-duplicate.
|
|
137
|
+
try {
|
|
138
|
+
const near = tools.matchTools([t.name, t.description].filter(Boolean).join(" "), { limit: 3 })
|
|
139
|
+
.filter((m) => m.name !== t.name);
|
|
140
|
+
if (near.length) {
|
|
141
|
+
console.log(`\nHeads up — ${near.length} existing tool(s) look similar:`);
|
|
142
|
+
for (const m of near) console.log(` • ${m.name} — ${m.description || "(no description)"}`);
|
|
143
|
+
console.log("Consider extending one with a new subcommand instead of keeping a near-duplicate (open-claudia tool show <name>).");
|
|
144
|
+
}
|
|
145
|
+
} catch (e) { /* best-effort */ }
|
|
107
146
|
} catch (e) {
|
|
108
147
|
console.error(`Could not add tool: ${e.message}`); process.exitCode = 1;
|
|
109
148
|
}
|
|
@@ -141,7 +180,7 @@ function run(args) {
|
|
|
141
180
|
break;
|
|
142
181
|
|
|
143
182
|
default:
|
|
144
|
-
console.log('Usage: open-claudia tool [list | show <name> | add <path> [flags] | run <name> [args] | remove <name> | path]');
|
|
183
|
+
console.log('Usage: open-claudia tool [list | search <query> | show <name> | add <path> [flags] | run <name> [args] | remove <name> | path]');
|
|
145
184
|
}
|
|
146
185
|
}
|
|
147
186
|
|
package/core/actions.js
CHANGED
|
@@ -290,6 +290,12 @@ async function handleAction(envelope) {
|
|
|
290
290
|
await send(`Recall debug: ${state.settings.showRecall ? "on" : "off"}`);
|
|
291
291
|
return;
|
|
292
292
|
}
|
|
293
|
+
if (d.startsWith("tt:")) {
|
|
294
|
+
state.settings.showToolTrace = d.slice(3) === "on";
|
|
295
|
+
saveState();
|
|
296
|
+
await send(`Tool trace: ${state.settings.showToolTrace ? "on" : "off"}`);
|
|
297
|
+
return;
|
|
298
|
+
}
|
|
293
299
|
if (d.startsWith("cw:")) {
|
|
294
300
|
const v = d.slice(3);
|
|
295
301
|
if (v === "default") state.settings.compactWindow = null;
|
package/core/dream.js
CHANGED
|
@@ -833,15 +833,40 @@ async function runDream({ trigger = "manual" } = {}) {
|
|
|
833
833
|
let toolNote = "";
|
|
834
834
|
try {
|
|
835
835
|
const toolsLib = require("./tools");
|
|
836
|
-
const tg = require("./tool-graph").tend(toolsLib);
|
|
836
|
+
const tg = require("./tool-graph").tend(toolsLib, { packsLib: packs });
|
|
837
837
|
const bits = [];
|
|
838
838
|
if (tg.edges || tg.decayed || tg.pruned) {
|
|
839
839
|
bits.push(`🔧 Tool graph: ${tg.edges} edges / ${tg.nodes} nodes (decayed ${tg.decayed}, pruned ${tg.pruned}).`);
|
|
840
840
|
}
|
|
841
|
-
const
|
|
841
|
+
const allTools = toolsLib.listTools();
|
|
842
|
+
const dangling = allTools.filter((t) => t.pack && !packs.readPack(t.pack));
|
|
842
843
|
if (dangling.length) {
|
|
843
844
|
bits.push(`🔗 ${dangling.length} tool(s) link a missing skill pack: ${dangling.map((t) => `${t.name}→${t.pack}`).join(", ")} — re-link with --pack or remove.`);
|
|
844
845
|
}
|
|
846
|
+
// Zero-run tools: created but never run and gone cold. Telemetry runCount
|
|
847
|
+
// is authoritative; file mtime stands in for "how long ago". Flag only —
|
|
848
|
+
// a never-used tool may still be worth keeping, so the user decides.
|
|
849
|
+
const STALE_DAYS = Number(process.env.TOOL_STALE_DAYS || 30);
|
|
850
|
+
const cutoff = Date.now() - STALE_DAYS * 86400000;
|
|
851
|
+
const coldUnused = allTools.filter((t) => !(t.runCount > 0) && !t.lastUsed).filter((t) => {
|
|
852
|
+
try { return fs.statSync(t.file).mtimeMs < cutoff; } catch (e) { return false; }
|
|
853
|
+
});
|
|
854
|
+
if (coldUnused.length) {
|
|
855
|
+
bits.push(`🗃️ ${coldUnused.length} tool(s) created but never run in ${STALE_DAYS}+ days: ${coldUnused.map((t) => t.name).join(", ")} — keep, fold into another, or remove.`);
|
|
856
|
+
}
|
|
857
|
+
// Near-duplicate pairs: two tools whose name+description overlap strongly,
|
|
858
|
+
// a sign one should have been a subcommand of the other. Dedupe by sorted
|
|
859
|
+
// pair key; flag only (consolidating is a judgement call for the user).
|
|
860
|
+
const DUP_THRESHOLD = Number(process.env.TOOL_DUP_THRESHOLD || 4);
|
|
861
|
+
const dupPairs = new Set();
|
|
862
|
+
for (const t of allTools) {
|
|
863
|
+
const near = toolsLib.matchTools([t.name, t.description].filter(Boolean).join(" "), { threshold: DUP_THRESHOLD, limit: 5 })
|
|
864
|
+
.filter((m) => m.name !== t.name);
|
|
865
|
+
for (const m of near) dupPairs.add([t.name, m.name].sort().join(" ↔ "));
|
|
866
|
+
}
|
|
867
|
+
if (dupPairs.size) {
|
|
868
|
+
bits.push(`👯 ${dupPairs.size} near-duplicate tool pair(s): ${[...dupPairs].join(", ")} — consider merging into one tool with subcommands.`);
|
|
869
|
+
}
|
|
845
870
|
toolNote = bits.join("\n");
|
|
846
871
|
} catch (e) { /* tool graph is best-effort (old node) */ }
|
|
847
872
|
|
package/core/handlers.js
CHANGED
|
@@ -123,7 +123,7 @@ register({
|
|
|
123
123
|
if (!authorized(env)) return;
|
|
124
124
|
send([
|
|
125
125
|
"Session: /session /sessions /projects /continue /status /stop /end",
|
|
126
|
-
"Settings: /model /effort /budget /plan /compact /compactwindow /worktree /mode /engine /recall",
|
|
126
|
+
"Settings: /model /effort /budget /plan /compact /compactwindow /worktree /mode /engine /recall /tooltrace",
|
|
127
127
|
"Identity: /whoami /link",
|
|
128
128
|
"Team: /people /intros /auth (owner)",
|
|
129
129
|
"Automation: /cron /vault /soul /dreamsummary",
|
|
@@ -743,6 +743,29 @@ register({
|
|
|
743
743
|
},
|
|
744
744
|
});
|
|
745
745
|
|
|
746
|
+
register({
|
|
747
|
+
name: "tooltrace", description: "Show tool activity each turn (surfaced/ran/updated)", args: "[on|off]",
|
|
748
|
+
handler: async (env, { tail }) => {
|
|
749
|
+
if (!authorized(env)) return;
|
|
750
|
+
const { settings } = currentState();
|
|
751
|
+
if (tail) {
|
|
752
|
+
const v = tail.trim().toLowerCase();
|
|
753
|
+
if (v === "on" || v === "true") settings.showToolTrace = true;
|
|
754
|
+
else if (v === "off" || v === "false") settings.showToolTrace = false;
|
|
755
|
+
else return send(`Usage: /tooltrace [on|off]. Currently ${settings.showToolTrace ? "on" : "off"}.`);
|
|
756
|
+
saveState();
|
|
757
|
+
return send(`Tool trace: ${settings.showToolTrace ? "on" : "off"}`);
|
|
758
|
+
}
|
|
759
|
+
send(
|
|
760
|
+
`Tool trace: ${settings.showToolTrace ? "on" : "off"}\n\n` +
|
|
761
|
+
"When on, I post short 🔧 lines around each reply: which reusable tools were surfaced for the turn, which I actually ran, and any I created or updated. Lets you watch the tool layer work. Off by default.",
|
|
762
|
+
{ keyboard: { inline_keyboard: [
|
|
763
|
+
[{ text: "On", callback_data: "tt:on" }, { text: "Off", callback_data: "tt:off" }],
|
|
764
|
+
] } },
|
|
765
|
+
);
|
|
766
|
+
},
|
|
767
|
+
});
|
|
768
|
+
|
|
746
769
|
register({
|
|
747
770
|
name: "budget", description: "Set max spend for next task", args: "[$N]",
|
|
748
771
|
handler: async (env, { tail }) => {
|
|
@@ -19,6 +19,13 @@ const graph = require("./graph");
|
|
|
19
19
|
const metrics = require("./metrics");
|
|
20
20
|
const { spawnSubagent } = require("../subagent");
|
|
21
21
|
const warmWalker = require("./warm-walker");
|
|
22
|
+
const toolsLib = require("../tools");
|
|
23
|
+
// Tool-graph is the directed-usage source of truth (pack↔tool edges). The
|
|
24
|
+
// discoverer READS it to activate tools that belong to a surfaced pack; it never
|
|
25
|
+
// writes the recall graph with tool nodes (keeps the hot expand() path pack/
|
|
26
|
+
// entity-only). Optional — absent on old node where node:sqlite is missing.
|
|
27
|
+
let toolGraph = null;
|
|
28
|
+
try { toolGraph = require("../tool-graph"); } catch (e) { /* old node */ }
|
|
22
29
|
|
|
23
30
|
const WALKER_MODEL = process.env.RECALL_DISCOVERER_MODEL || "haiku";
|
|
24
31
|
const WALKER_TIMEOUT_MS = Number(process.env.RECALL_DISCOVERER_TIMEOUT_MS || 25000);
|
|
@@ -64,6 +71,18 @@ function excerptFor(id, packsLib, entitiesLib) {
|
|
|
64
71
|
const body = [p.sections.Stance, p.sections.State].filter(Boolean).join("\n").trim();
|
|
65
72
|
return { id, name: p.name, description: p.description, excerpt: clip(body, EXCERPT_CHARS) };
|
|
66
73
|
}
|
|
74
|
+
if (id.startsWith("tool:")) {
|
|
75
|
+
const name = id.slice(5);
|
|
76
|
+
const t = toolsLib.findTool(name);
|
|
77
|
+
if (!t) return null;
|
|
78
|
+
// Excerpt the walker judges a tool against: what it does (usage) plus the
|
|
79
|
+
// concrete command shapes it's actually been run as in a topic, so "would
|
|
80
|
+
// this help here?" is answerable without reading the source.
|
|
81
|
+
let shapes = [];
|
|
82
|
+
if (toolGraph) { try { shapes = toolGraph.toolPacks(name, 2).map((p) => p.command).filter(Boolean); } catch (e) {} }
|
|
83
|
+
const excerpt = [t.usage, shapes.length ? `e.g. ${shapes.join(" | ")}` : ""].filter(Boolean).join(" — ");
|
|
84
|
+
return { id, name, description: t.description, excerpt: clip(excerpt, EXCERPT_CHARS) };
|
|
85
|
+
}
|
|
67
86
|
const e = entitiesLib.readEntity(id.slice(7));
|
|
68
87
|
if (!e) return null;
|
|
69
88
|
return { id, name: e.name, description: e.description, excerpt: clip((e.sections.Notes || "").trim(), EXCERPT_CHARS) };
|
|
@@ -75,6 +94,7 @@ const WALKER_SYSTEM = [
|
|
|
75
94
|
"You are given the user's message, recent conversation, and candidate memory nodes (some matched by keyword, some pulled in by graph links).",
|
|
76
95
|
"Decide which nodes are GENUINELY relevant to what the user is doing now.",
|
|
77
96
|
"Keep graph-linked nodes ONLY if they actually bear on the task (e.g. a shared theme/commit-style that governs the thing being worked on). Drop incidental keyword overlaps.",
|
|
97
|
+
"Some candidates are reusable tools (id starts `tool:`). Keep a tool ONLY if running it would actually help with THIS task; drop tools merely related by keyword.",
|
|
78
98
|
"For each kept node write a terse why (≤12 words). Quote concrete facts verbatim; never invent.",
|
|
79
99
|
'Reply with ONLY a JSON array: [{"id":"pack:foo","why":"shared lime theme governs this app"}]. Use [] if none.',
|
|
80
100
|
].join("\n");
|
|
@@ -154,12 +174,22 @@ async function run(ctx) {
|
|
|
154
174
|
);
|
|
155
175
|
} catch (e) {}
|
|
156
176
|
|
|
157
|
-
|
|
177
|
+
// Tool seeds: direct FTS-style match of the user's words (and context) against
|
|
178
|
+
// the tool corpus — the SEED stage for tools, peer to pack/entity seeding.
|
|
179
|
+
const toolSeedByName = new Map();
|
|
180
|
+
try {
|
|
181
|
+
for (const t of toolsLib.matchTools(userText, { limit: 5 })) toolSeedByName.set(t.name, { ...t, origin: "user" });
|
|
182
|
+
if (fullContext) for (const t of toolsLib.matchTools(fullContext, { limit: 5 })) {
|
|
183
|
+
if (!toolSeedByName.has(t.name)) toolSeedByName.set(t.name, { ...t, origin: "context" });
|
|
184
|
+
}
|
|
185
|
+
} catch (e) {}
|
|
186
|
+
|
|
187
|
+
const seedCount = packSeeds.length + entSeeds.length + toolSeedByName.size;
|
|
158
188
|
|
|
159
189
|
// 1: pre-gate.
|
|
160
190
|
if (!needsRecall(userText, seedCount)) {
|
|
161
191
|
metrics.logTurn({ engine: "discoverer", query: userText, gated: true, latencyMs: Date.now() - started });
|
|
162
|
-
return { packBlock: "", entityBlock: "", packMatches: [], entityMatches: [], why: {}, gated: true };
|
|
192
|
+
return { packBlock: "", entityBlock: "", toolBlock: "", packMatches: [], entityMatches: [], toolMatches: [], why: {}, gated: true };
|
|
163
193
|
}
|
|
164
194
|
|
|
165
195
|
// 3: spreading activation from seeds across the graph.
|
|
@@ -182,6 +212,47 @@ async function run(ctx) {
|
|
|
182
212
|
candidates.push({ ...base, activated: !seedIds.has(id), via: act ? act.via : null });
|
|
183
213
|
}
|
|
184
214
|
|
|
215
|
+
// Tool candidates = direct tool seeds + tools that belong to any candidate
|
|
216
|
+
// pack (declared via the tool's `pack:` header, or used-in-topic per the
|
|
217
|
+
// tool-graph). This is the tool "activation": tools ride on whatever packs the
|
|
218
|
+
// FTS+graph already surfaced, so a pack seed pulls in its tools the user never
|
|
219
|
+
// named — then the walker decides which actually help. via != null => activated.
|
|
220
|
+
const candPackDirs = new Set();
|
|
221
|
+
for (const id of candIds) if (id.startsWith("pack:")) candPackDirs.add(id.slice(5));
|
|
222
|
+
const toolCand = new Map();
|
|
223
|
+
for (const [name, t] of toolSeedByName) toolCand.set(name, { name, description: t.description, via: null });
|
|
224
|
+
if (candPackDirs.size) {
|
|
225
|
+
try {
|
|
226
|
+
const byPack = new Map();
|
|
227
|
+
for (const t of toolsLib.listTools()) {
|
|
228
|
+
if (!t.pack) continue;
|
|
229
|
+
if (!byPack.has(t.pack)) byPack.set(t.pack, []);
|
|
230
|
+
byPack.get(t.pack).push(t);
|
|
231
|
+
}
|
|
232
|
+
for (const dir of candPackDirs) {
|
|
233
|
+
for (const t of (byPack.get(dir) || [])) {
|
|
234
|
+
if (!toolCand.has(t.name)) toolCand.set(t.name, { name: t.name, description: t.description, via: `pack:${dir}` });
|
|
235
|
+
}
|
|
236
|
+
}
|
|
237
|
+
} catch (e) {}
|
|
238
|
+
if (toolGraph) {
|
|
239
|
+
try {
|
|
240
|
+
for (const dir of candPackDirs) {
|
|
241
|
+
for (const u of toolGraph.packTools(dir, 3)) {
|
|
242
|
+
if (toolCand.has(u.tool)) continue;
|
|
243
|
+
const t = toolsLib.findTool(u.tool);
|
|
244
|
+
if (t) toolCand.set(u.tool, { name: u.tool, description: t.description, via: `pack:${dir}` });
|
|
245
|
+
}
|
|
246
|
+
}
|
|
247
|
+
} catch (e) {}
|
|
248
|
+
}
|
|
249
|
+
}
|
|
250
|
+
for (const c of toolCand.values()) {
|
|
251
|
+
const base = excerptFor(`tool:${c.name}`, packsLib, entitiesLib);
|
|
252
|
+
if (!base) continue;
|
|
253
|
+
candidates.push({ ...base, activated: !!c.via, via: c.via });
|
|
254
|
+
}
|
|
255
|
+
|
|
185
256
|
const whyById = (await walk(userText, contextText || fullContext, candidates)) || null;
|
|
186
257
|
|
|
187
258
|
// Decide the kept set. Walker result wins; on fail-open keep the user-origin
|
|
@@ -192,7 +263,8 @@ async function run(ctx) {
|
|
|
192
263
|
} else {
|
|
193
264
|
keptIds = new Set(
|
|
194
265
|
[...packSeeds.filter((m) => m.origin === "user").map((m) => `pack:${m.dir}`),
|
|
195
|
-
...entSeeds.filter((m) => m.origin === "user").map((m) => `entity:${m.slug}`)
|
|
266
|
+
...entSeeds.filter((m) => m.origin === "user").map((m) => `entity:${m.slug}`),
|
|
267
|
+
...[...toolSeedByName.values()].filter((t) => t.origin === "user").map((t) => `tool:${t.name}`)],
|
|
196
268
|
);
|
|
197
269
|
}
|
|
198
270
|
|
|
@@ -229,6 +301,25 @@ async function run(ctx) {
|
|
|
229
301
|
if (bullets.length) packBlock = `\n\n### Why these surfaced (discoverer)\n${bullets.join("\n")}${packBlock}`;
|
|
230
302
|
}
|
|
231
303
|
|
|
304
|
+
// Tools the walker kept (or, on fail-open, user-origin tool seeds). Rendered as
|
|
305
|
+
// their own contextual block — this replaces the old flat "all 40 tools every
|
|
306
|
+
// turn" dump: tools surface only when they bear on the task, with a why.
|
|
307
|
+
const toolMatches = [];
|
|
308
|
+
for (const c of toolCand.values()) {
|
|
309
|
+
if (keptIds.has(`tool:${c.name}`)) {
|
|
310
|
+
toolMatches.push({ name: c.name, description: c.description, why: whyById ? whyById.get(`tool:${c.name}`) || "" : "" });
|
|
311
|
+
}
|
|
312
|
+
}
|
|
313
|
+
let toolBlock = "";
|
|
314
|
+
if (toolMatches.length) {
|
|
315
|
+
const lines = toolMatches.map((m) => {
|
|
316
|
+
const t = toolsLib.findTool(m.name);
|
|
317
|
+
const needs = t && t.requires && t.requires.length ? ` (keyring: ${t.requires.join(", ")})` : "";
|
|
318
|
+
return `- ${m.name} — ${m.description || "(no description)"}${needs}${m.why ? ` — ${m.why}` : ""}`;
|
|
319
|
+
});
|
|
320
|
+
toolBlock = `\n\n## Tools that may help here\nSurfaced for this turn — run with \`open-claudia tool run <name> [args]\`; read docs/source with \`open-claudia tool show <name>\`:\n${lines.join("\n")}\n`;
|
|
321
|
+
}
|
|
322
|
+
|
|
232
323
|
metrics.logTurn({
|
|
233
324
|
engine: "discoverer",
|
|
234
325
|
query: userText,
|
|
@@ -240,8 +331,8 @@ async function run(ctx) {
|
|
|
240
331
|
});
|
|
241
332
|
|
|
242
333
|
return {
|
|
243
|
-
packBlock, entityBlock, packMatches: finalPacks, entityMatches: finalEnts,
|
|
244
|
-
why: whyById ? Object.fromEntries(whyById) : {}, gated: false,
|
|
334
|
+
packBlock, entityBlock, toolBlock, packMatches: finalPacks, entityMatches: finalEnts,
|
|
335
|
+
toolMatches, why: whyById ? Object.fromEntries(whyById) : {}, gated: false,
|
|
245
336
|
};
|
|
246
337
|
}
|
|
247
338
|
|
package/core/runner.js
CHANGED
|
@@ -889,6 +889,13 @@ async function runClaude(prompt, cwd, replyToMsgId, opts = {}) {
|
|
|
889
889
|
skillNotified.add(key);
|
|
890
890
|
chatContext.run(store, () => send(text).catch(() => {}));
|
|
891
891
|
};
|
|
892
|
+
// Tool-activity visibility (surfaced / ran / created-updated). Off by default,
|
|
893
|
+
// toggled per-channel with /tooltrace — mirrors /recall's showRecall. Dedup +
|
|
894
|
+
// delivery reuse notifySkill so a tool touched twice a turn announces once.
|
|
895
|
+
const notifyToolTrace = (key, text) => {
|
|
896
|
+
if (!settings.showToolTrace) return;
|
|
897
|
+
notifySkill(key, text);
|
|
898
|
+
};
|
|
892
899
|
const noteSkillToolUse = (toolName, input) => {
|
|
893
900
|
try {
|
|
894
901
|
if (toolName === "Skill" && input?.skill) {
|
|
@@ -918,8 +925,8 @@ async function runClaude(prompt, cwd, replyToMsgId, opts = {}) {
|
|
|
918
925
|
}
|
|
919
926
|
const toolName2 = toolsLib.toolNameFromPath(filePath);
|
|
920
927
|
if (toolName2) {
|
|
921
|
-
if (toolsLib.findTool(toolName2))
|
|
922
|
-
else
|
|
928
|
+
if (toolsLib.findTool(toolName2)) notifyToolTrace(`tool:${toolName2}`, `🔧 Updating tool: ${toolName2} — open-claudia tool show ${toolName2} to inspect.`);
|
|
929
|
+
else notifyToolTrace(`tool:${toolName2}`, `🔧 New tool: ${toolName2} — open-claudia tool run ${toolName2} to use it.`);
|
|
923
930
|
return;
|
|
924
931
|
}
|
|
925
932
|
const dir = skillsLib.skillNameFromPath(filePath);
|
|
@@ -939,10 +946,11 @@ async function runClaude(prompt, cwd, replyToMsgId, opts = {}) {
|
|
|
939
946
|
// Nodes the agent actually OPENED this turn (📖). This is the co-use signal
|
|
940
947
|
// the recall graph reinforces on — actually-read, not merely surfaced.
|
|
941
948
|
const openedThisTurn = new Set();
|
|
942
|
-
// Reusable tools the agent RAN this turn, in order. Feeds
|
|
943
|
-
//
|
|
944
|
-
//
|
|
945
|
-
// (unlike openedThisTurn, a Set), because
|
|
949
|
+
// Reusable tools the agent RAN this turn, in order, as { name, shape }. Feeds
|
|
950
|
+
// two graphs (core/tool-graph.js): names → directed follows-edges ("you ran X
|
|
951
|
+
// — you usually run Y next"); shapes → pack↔tool edges ("in this topic, tool X
|
|
952
|
+
// is used as `…`"). Order matters here (unlike openedThisTurn, a Set), because
|
|
953
|
+
// a tool chain is a pipeline.
|
|
946
954
|
const toolRunsThisTurn = [];
|
|
947
955
|
const noteRecallFromShell = (command) => {
|
|
948
956
|
try {
|
|
@@ -965,11 +973,28 @@ async function runClaude(prompt, cwd, replyToMsgId, opts = {}) {
|
|
|
965
973
|
openedThisTurn.add(`entity:${slug}`);
|
|
966
974
|
notifySkill(`recall:entity:${slug}`, `📖 Recalled my notes on: ${name}`);
|
|
967
975
|
}
|
|
968
|
-
// `open-claudia tool run|exec <name>` — record
|
|
969
|
-
//
|
|
970
|
-
// next
|
|
971
|
-
|
|
972
|
-
|
|
976
|
+
// `open-claudia tool run|exec <name> <args>` — record each run as
|
|
977
|
+
// { name, shape }. The name feeds the directed follows-graph (what runs
|
|
978
|
+
// next); the sanitized command shape feeds pack↔tool capture (how a tool
|
|
979
|
+
// is used in a topic). Parsing lives in tool-graph so it stays testable.
|
|
980
|
+
try {
|
|
981
|
+
for (const run of require("./tool-graph").parseToolRuns(cmd)) {
|
|
982
|
+
toolRunsThisTurn.push(run);
|
|
983
|
+
// Visibility (gated by /tooltrace): one 🔧 line per distinct tool run
|
|
984
|
+
// this turn. Telemetry (recordRun) fires at end-of-turn on success.
|
|
985
|
+
notifyToolTrace(`ran:${run.name}`, `🔧 Ran tool: ${run.name}`);
|
|
986
|
+
}
|
|
987
|
+
} catch (e) { /* graph optional (old node) */ }
|
|
988
|
+
// `open-claudia tool add <src> [--name X]` — a tool being saved/updated.
|
|
989
|
+
// The Write/Edit path above only catches direct edits to TOOLS_DIR; `tool
|
|
990
|
+
// add` copies an external file in, so detect it here. Gated by /tooltrace.
|
|
991
|
+
const addRe = /\btool\s+add\s+(\S+)([^\n;|&]*)/gi;
|
|
992
|
+
while ((m = addRe.exec(cmd))) {
|
|
993
|
+
let name = (m[2].match(/--name\s+["']?([^\s"']+)/i) || [])[1];
|
|
994
|
+
if (!name) name = (String(m[1]).split(/[\\/]/).pop() || "").replace(/\.[^.]+$/, "");
|
|
995
|
+
name = toolsLib.sanitizeName(name);
|
|
996
|
+
if (name) notifyToolTrace(`added:${name}`, `🛠️ Saved tool: ${name} — open-claudia tool show ${name} to inspect.`);
|
|
997
|
+
}
|
|
973
998
|
} catch (e) { /* announcements are best-effort */ }
|
|
974
999
|
};
|
|
975
1000
|
|
|
@@ -1004,13 +1029,21 @@ async function runClaude(prompt, cwd, replyToMsgId, opts = {}) {
|
|
|
1004
1029
|
// drained here to keep the per-turn buffer from leaking into the next turn.)
|
|
1005
1030
|
try {
|
|
1006
1031
|
const injected = require("./system-prompt").consumeLastInjected();
|
|
1007
|
-
if (
|
|
1032
|
+
if (injected && injected.recall) {
|
|
1008
1033
|
const r = injected.recall;
|
|
1009
1034
|
const esc = (s) => String(s).replace(/&/g, "&").replace(/</g, "<").replace(/>/g, ">");
|
|
1010
1035
|
const fmt = (arr, icon) => arr.map((x) => (x.why ? `${icon} <b>${esc(x.name)}</b> — ${esc(x.why)}` : `${icon} <b>${esc(x.name)}</b>`));
|
|
1011
|
-
|
|
1012
|
-
if (
|
|
1013
|
-
|
|
1036
|
+
// 🧠 packs/entities recall — gated by /recall.
|
|
1037
|
+
if (settings.showRecall) {
|
|
1038
|
+
const lines = [...fmt(r.packs || [], "📦"), ...fmt(r.entities || [], "👤")];
|
|
1039
|
+
if (lines.length) send(`🧠 <b>Recall this turn</b> (${esc(r.engine)})\n${lines.join("\n")}`, telegramHtmlOpts()).catch(() => {});
|
|
1040
|
+
else if (r.gated) send(`🧠 <b>Recall</b> (${esc(r.engine)}): skipped by pre-gate — trivial turn.`, telegramHtmlOpts()).catch(() => {});
|
|
1041
|
+
}
|
|
1042
|
+
// 🔧 tools surfaced this turn (first-class discoverer nodes) — gated by /tooltrace.
|
|
1043
|
+
if (settings.showToolTrace) {
|
|
1044
|
+
const toolLines = fmt(r.tools || [], "🔧");
|
|
1045
|
+
if (toolLines.length) send(`🔧 <b>Tools surfaced this turn</b>\n${toolLines.join("\n")}`, telegramHtmlOpts()).catch(() => {});
|
|
1046
|
+
}
|
|
1014
1047
|
}
|
|
1015
1048
|
} catch (e) { /* best-effort */ }
|
|
1016
1049
|
// /stop landed during the pre-spawn window (recall/compaction): bail before
|
|
@@ -1362,10 +1395,35 @@ async function runClaude(prompt, cwd, replyToMsgId, opts = {}) {
|
|
|
1362
1395
|
// so it never bleeds into recall's spreading activation. Reinforce only on a
|
|
1363
1396
|
// successful turn, and only when ≥2 ran (a single run has no succession).
|
|
1364
1397
|
if (turnSucceeded && toolRunsThisTurn.length > 1) {
|
|
1365
|
-
try { require("./tool-graph").reinforceSequence(toolRunsThisTurn); }
|
|
1398
|
+
try { require("./tool-graph").reinforceSequence(toolRunsThisTurn.map((t) => t.name)); }
|
|
1399
|
+
catch (e) { /* best-effort */ }
|
|
1400
|
+
}
|
|
1401
|
+
|
|
1402
|
+
// Tool usage telemetry: a JSON sidecar (runCount + lastUsed per tool) that
|
|
1403
|
+
// works on every node version (unlike the SQLite graphs above). matchTools
|
|
1404
|
+
// breaks score ties by runCount so well-worn tools surface first, and dream
|
|
1405
|
+
// hygiene flags tools created-but-never-run. Fires for any successful run.
|
|
1406
|
+
if (turnSucceeded && toolRunsThisTurn.length) {
|
|
1407
|
+
try { for (const t of toolRunsThisTurn) toolsLib.recordRun(t.name); }
|
|
1366
1408
|
catch (e) { /* best-effort */ }
|
|
1367
1409
|
}
|
|
1368
1410
|
|
|
1411
|
+
// Pack↔tool contextual edges: a reusable tool run while a context pack was
|
|
1412
|
+
// open this turn records "this tool was used in this topic, as `<shape>`",
|
|
1413
|
+
// so next time the pack is active the discoverer can surface the concrete
|
|
1414
|
+
// invocation instead of leaving the agent to re-derive it. Fires even for a
|
|
1415
|
+
// single run (one tool-in-a-pack is still a usage) — unlike the follows-graph
|
|
1416
|
+
// above which needs a succession. Best-effort; never blocks the turn.
|
|
1417
|
+
if (turnSucceeded && toolRunsThisTurn.length && openedThisTurn.size) {
|
|
1418
|
+
try {
|
|
1419
|
+
const tg = require("./tool-graph");
|
|
1420
|
+
const packDirs = [...openedThisTurn].filter((n) => n.startsWith("pack:")).map((n) => n.slice(5));
|
|
1421
|
+
for (const dir of packDirs) {
|
|
1422
|
+
for (const run of toolRunsThisTurn) tg.recordPackTool(dir, run.name, { command: run.shape, bump: 1 });
|
|
1423
|
+
}
|
|
1424
|
+
} catch (e) { /* best-effort */ }
|
|
1425
|
+
}
|
|
1426
|
+
|
|
1369
1427
|
// Close the learning loop: when an ABILITY pack is opened in the same turn
|
|
1370
1428
|
// as a project (context) pack, the ability was demonstrably applied while
|
|
1371
1429
|
// working on that project. recordCoUse grows the ability's applied_on, which
|
package/core/system-prompt.js
CHANGED
|
@@ -66,37 +66,18 @@ function buildSkillIndexBlock() {
|
|
|
66
66
|
}
|
|
67
67
|
}
|
|
68
68
|
|
|
69
|
-
// Always-on tool
|
|
70
|
-
//
|
|
71
|
-
//
|
|
72
|
-
//
|
|
73
|
-
//
|
|
74
|
-
//
|
|
69
|
+
// Always-on tool guidance: teaches the BEHAVIOUR (crystallise procedures into
|
|
70
|
+
// tools; how to find and run them). It does NOT dump the tool list — that used to
|
|
71
|
+
// inject all ~40 tools every turn regardless of relevance. Tools now surface
|
|
72
|
+
// contextually as first-class discoverer nodes (see discoverer.js → toolBlock),
|
|
73
|
+
// the same seed→activate→judge→render path as packs/entities; this block only
|
|
74
|
+
// names the count and the lookup commands (`tool list` / `tool search`).
|
|
75
75
|
function buildToolIndexBlock() {
|
|
76
76
|
let listing = "";
|
|
77
77
|
try {
|
|
78
|
-
const
|
|
79
|
-
if (
|
|
80
|
-
|
|
81
|
-
// so a known chain (auth → list → download) is visible inline. Optional —
|
|
82
|
-
// absent on old node where node:sqlite is missing.
|
|
83
|
-
let graph = null;
|
|
84
|
-
try { graph = require("./tool-graph"); } catch (e) {}
|
|
85
|
-
listing = "\nTools you already have (run with `open-claudia tool run <name> [args]`; read docs/source with `open-claudia tool show <name>`):\n\n" +
|
|
86
|
-
all.slice(0, 40)
|
|
87
|
-
.map((t) => {
|
|
88
|
-
const skill = t.pack ? ` [skill: ${t.pack}]` : "";
|
|
89
|
-
const needs = t.requires && t.requires.length ? ` (keyring: ${t.requires.join(", ")})` : "";
|
|
90
|
-
let next = "";
|
|
91
|
-
if (graph) {
|
|
92
|
-
try {
|
|
93
|
-
const after = graph.followers(t.name, 2).map((r) => r.name);
|
|
94
|
-
if (after.length) next = ` → usually followed by: ${after.join(", ")}`;
|
|
95
|
-
} catch (e) {}
|
|
96
|
-
}
|
|
97
|
-
return `- ${t.name} — ${t.description || "(no description)"}${needs}${skill}${next}`;
|
|
98
|
-
})
|
|
99
|
-
.join("\n") + "\n";
|
|
78
|
+
const count = require("./tools").listTools().length;
|
|
79
|
+
if (count) {
|
|
80
|
+
listing = `\nYou have ${count} reusable tool${count === 1 ? "" : "s"}. The ones relevant to a turn are surfaced automatically below (\"Tools that may help here\"). To find others on demand: \`open-claudia tool search <query>\` (lexical match) or \`open-claudia tool list\`; read docs/source with \`open-claudia tool show <name>\`; run with \`open-claudia tool run <name> [args]\`.\n`;
|
|
100
81
|
} else {
|
|
101
82
|
listing = "\nNo reusable tools saved yet — the first time you work out how to hit an API or drive an interface, save it as one.\n";
|
|
102
83
|
}
|
|
@@ -483,6 +464,20 @@ function formatPackForContext(pack, packsLib, opts = {}) {
|
|
|
483
464
|
const journal = (pack.sections.Journal || "").trim().split("\n").filter(Boolean).slice(-10).join("\n");
|
|
484
465
|
if (journal) parts.push(`#### Journal (recent)\n${journal}`);
|
|
485
466
|
}
|
|
467
|
+
// Pack↔tool usages: concrete reusable-tool invocations recorded while this
|
|
468
|
+
// pack was active, so the agent sees HOW a tool was used in this topic (not
|
|
469
|
+
// just that it exists). Strongest 2; best-effort + optional (absent on old
|
|
470
|
+
// node without node:sqlite). Compact enough to show in every mode.
|
|
471
|
+
try {
|
|
472
|
+
const usages = require("./tool-graph").packTools(pack.dir, 2);
|
|
473
|
+
if (usages && usages.length) {
|
|
474
|
+
const lines = usages.map((u) => {
|
|
475
|
+
const cmd = u.command ? `\`open-claudia tool run ${u.command}\`` : `\`open-claudia tool run ${u.tool}\``;
|
|
476
|
+
return `- ${cmd}${u.intent ? ` — ${u.intent}` : ""}`;
|
|
477
|
+
}).join("\n");
|
|
478
|
+
parts.push(`#### Tools used in this pack\n${lines}`);
|
|
479
|
+
}
|
|
480
|
+
} catch (e) { /* tool-graph optional */ }
|
|
486
481
|
return clip(parts.join("\n\n"), PACK_INJECT_MAX_CHARS);
|
|
487
482
|
}
|
|
488
483
|
|
|
@@ -747,18 +742,23 @@ async function promptWithDynamicContext(prompt, opts = {}) {
|
|
|
747
742
|
userText, contextText, fullContext, packLimit, budget, helpers,
|
|
748
743
|
});
|
|
749
744
|
const { packBlock, entityBlock } = result;
|
|
745
|
+
const toolBlock = result.toolBlock || "";
|
|
750
746
|
const why = result.why || {};
|
|
751
747
|
lastInjected.recall = {
|
|
752
748
|
engine: engine.name || recall.activeEngineName(settings),
|
|
753
749
|
gated: !!result.gated,
|
|
754
750
|
packs: (result.packMatches || []).map((m) => ({ name: m.name || m.dir, why: why[`pack:${m.dir}`] || "" })),
|
|
755
751
|
entities: (result.entityMatches || []).map((m) => ({ name: m.name || m.slug, why: why[`entity:${m.slug}`] || "" })),
|
|
752
|
+
// Tools surfaced as first-class discoverer nodes this turn (name + why).
|
|
753
|
+
// Drives the /tooltrace "surfaced" banner; the toolBlock below is the
|
|
754
|
+
// context the agent actually sees.
|
|
755
|
+
tools: (result.toolMatches || []).map((m) => ({ name: m.name, why: m.why || why[`tool:${m.name}`] || "" })),
|
|
756
756
|
};
|
|
757
757
|
const budgetNote = budget.omitted > 0
|
|
758
758
|
? `\n\n## Memory budget\n${budget.omitted} matched memory item${budget.omitted === 1 ? " was" : "s were"} omitted to keep this turn under the recall budget (${budget.maxChars} chars). Use \`open-claudia pack show <dir>\`, \`entity show <slug>\`, or transcript search if deeper context is needed.`
|
|
759
759
|
: "";
|
|
760
760
|
const transcriptPointer = opts.includeTranscriptPointer ? `${transcriptPointerNote(state)}\n\n` : "";
|
|
761
|
-
return `${transcriptPointer}${buildDynamicContextBlock()}${packBlock}${entityBlock}${budgetNote}${voiceReplyGuidance()}\n\nCurrent user request:\n${prompt}`;
|
|
761
|
+
return `${transcriptPointer}${buildDynamicContextBlock()}${packBlock}${entityBlock}${toolBlock}${budgetNote}${voiceReplyGuidance()}\n\nCurrent user request:\n${prompt}`;
|
|
762
762
|
} catch (e) {
|
|
763
763
|
return prompt;
|
|
764
764
|
}
|
package/core/tool-graph.js
CHANGED
|
@@ -5,9 +5,18 @@
|
|
|
5
5
|
// Why separate: the recall graph spreads activation over packs+entities to pick
|
|
6
6
|
// which MEMORY to surface. Tools are a different corpus with different physics —
|
|
7
7
|
// the useful signal is the ORDER a chain runs in (auth → list → download), not
|
|
8
|
-
// symmetric "these are related".
|
|
9
|
-
//
|
|
10
|
-
//
|
|
8
|
+
// symmetric "these are related". So tools get their own tiny graph and the recall
|
|
9
|
+
// graph's hot expand() path stays pack/entity-only (no tuning, no crowding of
|
|
10
|
+
// structural pack edges, works unchanged on every node version).
|
|
11
|
+
//
|
|
12
|
+
// How tools still reach the discoverer: this graph is the directed-usage source
|
|
13
|
+
// of truth, and the discoverer READS it (recordPackTool → packTools/toolPacks) to
|
|
14
|
+
// activate the tools belonging to a pack it already surfaced — then its haiku
|
|
15
|
+
// walker judges each tool with a why and drops incidental ones. That walker is
|
|
16
|
+
// the noise filter an earlier "mixing would be noise" worry assumed didn't exist;
|
|
17
|
+
// because the discoverer judges candidates, tools participate as first-class
|
|
18
|
+
// nodes (seed via matchTools → activate via this graph → judge → render) WITHOUT
|
|
19
|
+
// ever being written into the recall graph as nodes.
|
|
11
20
|
//
|
|
12
21
|
// Why directed: A→B ("B follows A") carries real information that B→A does not.
|
|
13
22
|
// auth-then-fetch is a pipeline; fetch-then-auth is nonsense. Edges are stored
|
|
@@ -55,6 +64,24 @@ function openDb() {
|
|
|
55
64
|
)`);
|
|
56
65
|
db.exec("CREATE INDEX IF NOT EXISTS tool_edges_src ON tool_edges (src)");
|
|
57
66
|
db.exec("CREATE INDEX IF NOT EXISTS tool_edges_dst ON tool_edges (dst)");
|
|
67
|
+
// Pack↔tool contextual edges: "while pack P was open, tool T was run as
|
|
68
|
+
// <command shape>". A different physics from the tool→tool follows-edges
|
|
69
|
+
// above (this is "how a tool is used in a topic", not "what runs next"), but
|
|
70
|
+
// it shares the same DB, decay and pruning machinery. One row per distinct
|
|
71
|
+
// (pack, tool, command-shape) so several ways of using a tool in a topic can
|
|
72
|
+
// coexist and the strongest surface.
|
|
73
|
+
db.exec(`CREATE TABLE IF NOT EXISTS pack_tool_edges (
|
|
74
|
+
pack TEXT NOT NULL,
|
|
75
|
+
tool TEXT NOT NULL,
|
|
76
|
+
command TEXT NOT NULL DEFAULT '',
|
|
77
|
+
intent TEXT NOT NULL DEFAULT '',
|
|
78
|
+
weight REAL NOT NULL DEFAULT 1,
|
|
79
|
+
last_reinforced TEXT,
|
|
80
|
+
created TEXT,
|
|
81
|
+
PRIMARY KEY (pack, tool, command)
|
|
82
|
+
)`);
|
|
83
|
+
db.exec("CREATE INDEX IF NOT EXISTS pack_tool_edges_pack ON pack_tool_edges (pack)");
|
|
84
|
+
db.exec("CREATE INDEX IF NOT EXISTS pack_tool_edges_tool ON pack_tool_edges (tool)");
|
|
58
85
|
_db = db;
|
|
59
86
|
return db;
|
|
60
87
|
} catch (e) {
|
|
@@ -142,21 +169,136 @@ function predecessors(name, limit = 5) {
|
|
|
142
169
|
} catch (e) { return []; }
|
|
143
170
|
}
|
|
144
171
|
|
|
172
|
+
// ── Pack↔tool contextual edges ──────────────────────────────────────────────
|
|
173
|
+
// "While context pack P was open, reusable tool T was run as `<command shape>`."
|
|
174
|
+
// Captured automatically at end of turn (see runner.js) so the discoverer can
|
|
175
|
+
// later surface the concrete invocation for a pack — the agent sees HOW a tool
|
|
176
|
+
// is used in a topic, not merely that the tool exists. Reinforced Hebbian-style
|
|
177
|
+
// and decayed/pruned by tend() exactly like the follows-edges.
|
|
178
|
+
|
|
179
|
+
// Reduce a literal invocation to a reusable SHAPE: strip ids, paths, numbers and
|
|
180
|
+
// quoted text to placeholders so the stored command is a pattern, not a one-off
|
|
181
|
+
// with stale literals. Keeps subcommands and flag names — the structure worth
|
|
182
|
+
// remembering. Input may include a leading `[open-claudia ]tool run|exec`
|
|
183
|
+
// wrapper, which is dropped so the shape starts at the tool name.
|
|
184
|
+
function shapeCommand(raw) {
|
|
185
|
+
let s = String(raw || "").trim();
|
|
186
|
+
if (!s) return "";
|
|
187
|
+
s = s.replace(/^\s*(?:open-claudia\s+)?tool\s+(?:run|exec)\s+/i, "");
|
|
188
|
+
s = s.replace(/(["']).*?\1/g, "<text>"); // quoted strings
|
|
189
|
+
const toks = s.split(/\s+/).filter(Boolean).map((t) => {
|
|
190
|
+
if (/^--?[a-z][\w-]*$/i.test(t)) return t; // keep flag names
|
|
191
|
+
if (/^[0-9a-f]{12,}$/i.test(t)) return "<id>"; // long hex ids
|
|
192
|
+
if (/^[0-9a-f]{8}-[0-9a-f]{4}-/i.test(t)) return "<id>"; // uuid
|
|
193
|
+
if (t.includes("/") || t.includes("\\")) return "<path>";
|
|
194
|
+
if (/^\d[\d.,:t-]*$/i.test(t)) return "<n>"; // numbers / dates
|
|
195
|
+
if (t.length >= 24) return "<arg>"; // long opaque token
|
|
196
|
+
return t;
|
|
197
|
+
});
|
|
198
|
+
return toks.join(" ").slice(0, 80).trim();
|
|
199
|
+
}
|
|
200
|
+
|
|
201
|
+
// Parse `tool run|exec <name> <args...>` invocations out of a shell command,
|
|
202
|
+
// returning [{ name, shape }] in order. Splits on shell separators so one
|
|
203
|
+
// invocation's args don't bleed into the next. Feeds both the follows-graph
|
|
204
|
+
// (names) and pack↔tool capture (shapes).
|
|
205
|
+
function parseToolRuns(command) {
|
|
206
|
+
const out = [];
|
|
207
|
+
const cmd = String(command || "");
|
|
208
|
+
if (!/tool\s+(?:run|exec)\s+/i.test(cmd)) return out;
|
|
209
|
+
for (const seg of cmd.split(/&&|\|\||[;\n]/)) {
|
|
210
|
+
const m = seg.match(/\btool\s+(?:run|exec)\s+["']?([a-z0-9][\w.-]*)["']?\s*(.*)$/i);
|
|
211
|
+
if (!m) continue;
|
|
212
|
+
out.push({ name: m[1], shape: shapeCommand(`${m[1]} ${m[2] || ""}`) });
|
|
213
|
+
}
|
|
214
|
+
return out;
|
|
215
|
+
}
|
|
216
|
+
|
|
217
|
+
// Upsert/reinforce a pack↔tool contextual usage. `command` is the (shaped)
|
|
218
|
+
// invocation, `intent` an optional one-line why. `bump` adds Hebbian weight;
|
|
219
|
+
// absent it just ensures the edge exists. A later non-empty intent overwrites an
|
|
220
|
+
// earlier blank one so the reviewer can enrich a deterministically-captured edge.
|
|
221
|
+
function recordPackTool(pack, tool, opts = {}) {
|
|
222
|
+
const db = openDb();
|
|
223
|
+
if (!db || !pack || !tool) return false;
|
|
224
|
+
const command = String(opts.command || "").slice(0, 120);
|
|
225
|
+
const bump = Number.isFinite(opts.bump) ? opts.bump : 0;
|
|
226
|
+
const weight = Number.isFinite(opts.weight) ? opts.weight : 1;
|
|
227
|
+
try {
|
|
228
|
+
const row = db.prepare("SELECT weight, intent, last_reinforced FROM pack_tool_edges WHERE pack=? AND tool=? AND command=?").get(pack, tool, command);
|
|
229
|
+
if (row) {
|
|
230
|
+
const next = bump ? row.weight + bump : Math.max(row.weight, weight);
|
|
231
|
+
const intent = (opts.intent != null && opts.intent !== "") ? String(opts.intent).slice(0, 200) : row.intent;
|
|
232
|
+
db.prepare("UPDATE pack_tool_edges SET weight=?, intent=?, last_reinforced=? WHERE pack=? AND tool=? AND command=?")
|
|
233
|
+
.run(next, intent, bump ? now() : row.last_reinforced, pack, tool, command);
|
|
234
|
+
} else {
|
|
235
|
+
const initial = bump ? bump : weight;
|
|
236
|
+
db.prepare("INSERT INTO pack_tool_edges (pack, tool, command, intent, weight, last_reinforced, created) VALUES (?,?,?,?,?,?,?)")
|
|
237
|
+
.run(pack, tool, command, String(opts.intent || "").slice(0, 200), initial, bump ? now() : null, now());
|
|
238
|
+
}
|
|
239
|
+
return true;
|
|
240
|
+
} catch (e) {
|
|
241
|
+
return false;
|
|
242
|
+
}
|
|
243
|
+
}
|
|
244
|
+
|
|
245
|
+
// Strongest contextual tool usages recorded under a pack, for surfacing in that
|
|
246
|
+
// pack's injected context. Ordered by reinforced weight, newest first as a
|
|
247
|
+
// tiebreak. One row per distinct command shape.
|
|
248
|
+
function packTools(pack, limit = 2) {
|
|
249
|
+
const db = openDb();
|
|
250
|
+
if (!db || !pack) return [];
|
|
251
|
+
try {
|
|
252
|
+
return db.prepare(
|
|
253
|
+
"SELECT tool, command, intent, weight FROM pack_tool_edges WHERE pack=? ORDER BY weight DESC, last_reinforced DESC LIMIT ?"
|
|
254
|
+
).all(pack, limit);
|
|
255
|
+
} catch (e) { return []; }
|
|
256
|
+
}
|
|
257
|
+
|
|
258
|
+
// Inverse view: which packs a tool has been used under (for `tool show`).
|
|
259
|
+
function toolPacks(tool, limit = 5) {
|
|
260
|
+
const db = openDb();
|
|
261
|
+
if (!db || !tool) return [];
|
|
262
|
+
try {
|
|
263
|
+
return db.prepare(
|
|
264
|
+
"SELECT pack, command, intent, weight FROM pack_tool_edges WHERE tool=? ORDER BY weight DESC, last_reinforced DESC LIMIT ?"
|
|
265
|
+
).all(tool, limit);
|
|
266
|
+
} catch (e) { return []; }
|
|
267
|
+
}
|
|
268
|
+
|
|
269
|
+
function removePackTool(pack, tool, command) {
|
|
270
|
+
const db = openDb();
|
|
271
|
+
if (!db) return false;
|
|
272
|
+
try { db.prepare("DELETE FROM pack_tool_edges WHERE pack=? AND tool=? AND command=?").run(pack, tool, command || ""); return true; }
|
|
273
|
+
catch (e) { return false; }
|
|
274
|
+
}
|
|
275
|
+
|
|
276
|
+
function allPackToolEdges() {
|
|
277
|
+
const db = openDb();
|
|
278
|
+
if (!db) return [];
|
|
279
|
+
try { return db.prepare("SELECT pack, tool, command, intent, weight, last_reinforced FROM pack_tool_edges").all(); }
|
|
280
|
+
catch (e) { return []; }
|
|
281
|
+
}
|
|
282
|
+
|
|
145
283
|
// Exponential time decay. Tools have no structural floor, so edges decay toward
|
|
146
284
|
// zero and are pruned once under `pruneBelow` — a chain that fell out of use
|
|
147
|
-
// disappears instead of lingering as stale advice.
|
|
285
|
+
// disappears instead of lingering as stale advice. Applies to both the tool→tool
|
|
286
|
+
// follows-edges and the pack↔tool contextual edges (same physics).
|
|
148
287
|
function decay({ halfLifeDays = DEFAULTS.halfLifeDays, pruneBelow = DEFAULTS.pruneBelow } = {}) {
|
|
149
288
|
const db = openDb();
|
|
150
289
|
if (!db) return { decayed: 0, pruned: 0 };
|
|
151
|
-
const rows = allEdges();
|
|
152
290
|
const nowMs = Date.now();
|
|
153
291
|
const ln2 = Math.log(2);
|
|
154
292
|
let decayed = 0, pruned = 0;
|
|
155
|
-
|
|
156
|
-
if (!
|
|
157
|
-
const ageDays = (nowMs - Date.parse(
|
|
158
|
-
if (!(ageDays > 0))
|
|
159
|
-
|
|
293
|
+
const decayFactor = (lastReinforced) => {
|
|
294
|
+
if (!lastReinforced) return null;
|
|
295
|
+
const ageDays = (nowMs - Date.parse(lastReinforced)) / 86400000;
|
|
296
|
+
if (!(ageDays > 0)) return null;
|
|
297
|
+
return Math.exp(-ln2 * ageDays / Math.max(1, halfLifeDays));
|
|
298
|
+
};
|
|
299
|
+
for (const e of allEdges()) {
|
|
300
|
+
const factor = decayFactor(e.last_reinforced);
|
|
301
|
+
if (factor === null) continue;
|
|
160
302
|
const next = e.weight * factor;
|
|
161
303
|
if (next < pruneBelow) {
|
|
162
304
|
if (removeEdge(e.src, e.dst)) pruned++;
|
|
@@ -169,32 +311,62 @@ function decay({ halfLifeDays = DEFAULTS.halfLifeDays, pruneBelow = DEFAULTS.pru
|
|
|
169
311
|
} catch (e2) {}
|
|
170
312
|
}
|
|
171
313
|
}
|
|
314
|
+
for (const e of allPackToolEdges()) {
|
|
315
|
+
const factor = decayFactor(e.last_reinforced);
|
|
316
|
+
if (factor === null) continue;
|
|
317
|
+
const next = e.weight * factor;
|
|
318
|
+
if (next < pruneBelow) {
|
|
319
|
+
if (removePackTool(e.pack, e.tool, e.command)) pruned++;
|
|
320
|
+
continue;
|
|
321
|
+
}
|
|
322
|
+
if (Math.abs(next - e.weight) > 1e-6) {
|
|
323
|
+
try {
|
|
324
|
+
db.prepare("UPDATE pack_tool_edges SET weight=? WHERE pack=? AND tool=? AND command=?").run(next, e.pack, e.tool, e.command);
|
|
325
|
+
decayed++;
|
|
326
|
+
} catch (e2) {}
|
|
327
|
+
}
|
|
328
|
+
}
|
|
172
329
|
return { decayed, pruned };
|
|
173
330
|
}
|
|
174
331
|
|
|
175
332
|
// Drop edges whose endpoints are no longer registered tools (a tool was removed
|
|
176
|
-
// or renamed). Keeps the graph from pointing at things that can't be run.
|
|
177
|
-
|
|
333
|
+
// or renamed). Keeps the graph from pointing at things that can't be run. Also
|
|
334
|
+
// drops pack↔tool edges whose tool is gone, and — when a packs lib is supplied —
|
|
335
|
+
// whose pack is gone, so a removed pack doesn't leave dangling usages.
|
|
336
|
+
function pruneOrphans(toolsLib, packsLib) {
|
|
178
337
|
const db = openDb();
|
|
179
338
|
if (!db) return 0;
|
|
180
|
-
let
|
|
181
|
-
try {
|
|
339
|
+
let liveTools;
|
|
340
|
+
try { liveTools = new Set(toolsLib.listTools().map((t) => t.name)); }
|
|
182
341
|
catch (e) { return 0; }
|
|
183
|
-
if (!
|
|
342
|
+
if (!liveTools.size) return 0;
|
|
184
343
|
let removed = 0;
|
|
185
344
|
for (const e of allEdges()) {
|
|
186
|
-
if (!
|
|
345
|
+
if (!liveTools.has(e.src) || !liveTools.has(e.dst)) {
|
|
187
346
|
if (removeEdge(e.src, e.dst)) removed++;
|
|
188
347
|
}
|
|
189
348
|
}
|
|
349
|
+
let livePacks = null;
|
|
350
|
+
if (packsLib) {
|
|
351
|
+
try { livePacks = new Set(packsLib.listPacks().map((p) => p.dir)); }
|
|
352
|
+
catch (e) { livePacks = null; }
|
|
353
|
+
}
|
|
354
|
+
for (const e of allPackToolEdges()) {
|
|
355
|
+
const toolGone = !liveTools.has(e.tool);
|
|
356
|
+
const packGone = livePacks ? !livePacks.has(e.pack) : false;
|
|
357
|
+
if (toolGone || packGone) {
|
|
358
|
+
if (removePackTool(e.pack, e.tool, e.command)) removed++;
|
|
359
|
+
}
|
|
360
|
+
}
|
|
190
361
|
return removed;
|
|
191
362
|
}
|
|
192
363
|
|
|
193
364
|
// Nightly maintenance (called by the dream): decay reinforced weights and drop
|
|
194
|
-
// orphaned/faded edges. Deterministic + safe — no model needed.
|
|
365
|
+
// orphaned/faded edges. Deterministic + safe — no model needed. Pass
|
|
366
|
+
// { packsLib } to also prune pack↔tool edges whose pack was removed.
|
|
195
367
|
function tend(toolsLib, opts = {}) {
|
|
196
368
|
const { decayed, pruned } = decay(opts);
|
|
197
|
-
const orphaned = toolsLib ? pruneOrphans(toolsLib) : 0;
|
|
369
|
+
const orphaned = toolsLib ? pruneOrphans(toolsLib, opts.packsLib) : 0;
|
|
198
370
|
return { decayed, pruned: pruned + orphaned, ...stats() };
|
|
199
371
|
}
|
|
200
372
|
|
|
@@ -202,7 +374,7 @@ function stats() {
|
|
|
202
374
|
const rows = allEdges();
|
|
203
375
|
const nodes = new Set();
|
|
204
376
|
for (const e of rows) { nodes.add(e.src); nodes.add(e.dst); }
|
|
205
|
-
return { edges: rows.length, nodes: nodes.size };
|
|
377
|
+
return { edges: rows.length, nodes: nodes.size, packToolEdges: allPackToolEdges().length };
|
|
206
378
|
}
|
|
207
379
|
|
|
208
380
|
// Test seam.
|
|
@@ -213,5 +385,7 @@ module.exports = {
|
|
|
213
385
|
available, openDb,
|
|
214
386
|
addFollow, reinforceSequence, removeEdge, allEdges,
|
|
215
387
|
followers, predecessors, decay, pruneOrphans, tend, stats,
|
|
388
|
+
shapeCommand, parseToolRuns, recordPackTool, packTools, toolPacks,
|
|
389
|
+
removePackTool, allPackToolEdges,
|
|
216
390
|
_resetForTest,
|
|
217
391
|
};
|
package/core/tools.js
CHANGED
|
@@ -35,6 +35,97 @@ function sanitizeName(name) {
|
|
|
35
35
|
.replace(/[^a-z0-9._-]+/g, "-").replace(/^[-.]+|[-.]+$/g, "").slice(0, 60);
|
|
36
36
|
}
|
|
37
37
|
|
|
38
|
+
// ── Run telemetry ────────────────────────────────────────────────────────────
|
|
39
|
+
// A tiny JSON sidecar (name -> { runCount, lastUsed }) kept beside the tools.
|
|
40
|
+
// Deliberately NOT a SQLite graph: telemetry must work on the node:20 pods where
|
|
41
|
+
// node:sqlite is absent, and it's a flat counter, not a graph. Dotfile so
|
|
42
|
+
// listTools (which skips ".") never mistakes it for a tool.
|
|
43
|
+
const USAGE_FILE = path.join(TOOLS_DIR, ".usage.json");
|
|
44
|
+
|
|
45
|
+
function readUsage() {
|
|
46
|
+
try { return JSON.parse(fs.readFileSync(USAGE_FILE, "utf-8")) || {}; }
|
|
47
|
+
catch (e) { return {}; }
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
function writeUsage(u) {
|
|
51
|
+
try { ensureDir(); fs.writeFileSync(USAGE_FILE, JSON.stringify(u), { mode: 0o600 }); }
|
|
52
|
+
catch (e) { /* best-effort */ }
|
|
53
|
+
}
|
|
54
|
+
|
|
55
|
+
// Bump a tool's run counter. Called at end of turn from the runner when it parses
|
|
56
|
+
// an `open-claudia tool run <name>` out of the shell. Feeds matchTools' tiebreak
|
|
57
|
+
// (popular tools rank up) and the dream's zero-run hygiene flag.
|
|
58
|
+
function recordRun(name) {
|
|
59
|
+
const n = sanitizeName(name);
|
|
60
|
+
if (!n) return;
|
|
61
|
+
const u = readUsage();
|
|
62
|
+
const cur = u[n] || { runCount: 0, lastUsed: "" };
|
|
63
|
+
cur.runCount = (cur.runCount || 0) + 1;
|
|
64
|
+
cur.lastUsed = new Date().toISOString();
|
|
65
|
+
u[n] = cur;
|
|
66
|
+
writeUsage(u);
|
|
67
|
+
}
|
|
68
|
+
|
|
69
|
+
// ── Lexical matching (in-memory) ─────────────────────────────────────────────
|
|
70
|
+
// Tools are few and short, so we score in memory rather than standing up a third
|
|
71
|
+
// FTS index — and unlike the recall/tool graphs this then works on every node
|
|
72
|
+
// version. Mirrors matchPacks' semantics: a hit in a strong field (name /
|
|
73
|
+
// description / usage) counts 2, a weak-field hit (requires / pack / source) 1,
|
|
74
|
+
// tools at/above the threshold win. This is the SEED stage of the discoverer for
|
|
75
|
+
// the tool corpus — keyword feeds the agentic walker, it does not replace it.
|
|
76
|
+
const TOOL_STOPWORDS = new Set(("a an and are as at be but by can did do for from had has have how i if in is it its " +
|
|
77
|
+
"me my no not of on or our so that the their then there this to up us was we what when where which who why " +
|
|
78
|
+
"will with you your yes yeah ok okay please thanks just like dont im its run tool use").split(" "));
|
|
79
|
+
|
|
80
|
+
function toolQueryTerms(text) {
|
|
81
|
+
const seen = new Set();
|
|
82
|
+
const terms = [];
|
|
83
|
+
for (const raw of String(text || "").toLowerCase().split(/[^a-z0-9_.-]+/)) {
|
|
84
|
+
const t = raw.replace(/^[.-]+|[.-]+$/g, "");
|
|
85
|
+
if (t.length < 2 || TOOL_STOPWORDS.has(t) || seen.has(t)) continue;
|
|
86
|
+
seen.add(t);
|
|
87
|
+
terms.push(t);
|
|
88
|
+
if (terms.length >= 40) break;
|
|
89
|
+
}
|
|
90
|
+
return terms;
|
|
91
|
+
}
|
|
92
|
+
|
|
93
|
+
function wordsOf(s) {
|
|
94
|
+
return new Set(String(s || "").toLowerCase().split(/[^a-z0-9_.-]+/).filter(Boolean));
|
|
95
|
+
}
|
|
96
|
+
|
|
97
|
+
// A term hits a word set on exact match, or a light prefix stem (deploy~deploys,
|
|
98
|
+
// upload~uploads) for terms long enough that a prefix isn't noise.
|
|
99
|
+
function termHits(term, wordSet) {
|
|
100
|
+
if (wordSet.has(term)) return true;
|
|
101
|
+
if (term.length >= 4) {
|
|
102
|
+
for (const w of wordSet) if (w.startsWith(term) || term.startsWith(w)) return true;
|
|
103
|
+
}
|
|
104
|
+
return false;
|
|
105
|
+
}
|
|
106
|
+
|
|
107
|
+
// Score registered tools against free text. Returns [{ name, description, score,
|
|
108
|
+
// runCount }] over the threshold, strongest first (runCount breaks ties so a
|
|
109
|
+
// well-worn tool outranks a never-run namesake).
|
|
110
|
+
function matchTools(text, { limit = 5, threshold = null } = {}) {
|
|
111
|
+
const terms = toolQueryTerms(text);
|
|
112
|
+
if (!terms.length) return [];
|
|
113
|
+
const min = threshold ?? Number(process.env.TOOL_MATCH_THRESHOLD || 2);
|
|
114
|
+
const out = [];
|
|
115
|
+
for (const t of listTools()) {
|
|
116
|
+
const strong = wordsOf([t.name, t.description, t.usage].filter(Boolean).join(" "));
|
|
117
|
+
const weak = wordsOf([(t.requires || []).join(" "), t.pack, t.content].filter(Boolean).join(" "));
|
|
118
|
+
let score = 0;
|
|
119
|
+
for (const term of terms) {
|
|
120
|
+
if (termHits(term, strong)) score += 2;
|
|
121
|
+
else if (termHits(term, weak)) score += 1;
|
|
122
|
+
}
|
|
123
|
+
if (score >= min) out.push({ name: t.name, description: t.description, score, runCount: t.runCount || 0 });
|
|
124
|
+
}
|
|
125
|
+
out.sort((a, b) => b.score - a.score || (b.runCount || 0) - (a.runCount || 0) || a.name.localeCompare(b.name));
|
|
126
|
+
return out.slice(0, Math.max(1, Math.min(10, limit)));
|
|
127
|
+
}
|
|
128
|
+
|
|
38
129
|
// Strip a leading comment prefix ("# " / "// ") from a header line. Returns the
|
|
39
130
|
// remainder, or null if the line is not a comment (header block has ended).
|
|
40
131
|
function uncomment(line) {
|
|
@@ -75,7 +166,7 @@ function toolFile(name) {
|
|
|
75
166
|
return path.join(TOOLS_DIR, sanitizeName(name));
|
|
76
167
|
}
|
|
77
168
|
|
|
78
|
-
function readTool(name) {
|
|
169
|
+
function readTool(name, usageMap) {
|
|
79
170
|
const file = toolFile(name);
|
|
80
171
|
let content;
|
|
81
172
|
try { content = fs.readFileSync(file, "utf-8"); } catch (e) { return null; }
|
|
@@ -83,6 +174,7 @@ function readTool(name) {
|
|
|
83
174
|
if (!header) return null;
|
|
84
175
|
let stat = null;
|
|
85
176
|
try { stat = fs.statSync(file); } catch (e) {}
|
|
177
|
+
const usage = (usageMap || readUsage())[header.name || sanitizeName(name)] || {};
|
|
86
178
|
return {
|
|
87
179
|
name: header.name || sanitizeName(name),
|
|
88
180
|
description: header.description || "",
|
|
@@ -92,6 +184,8 @@ function readTool(name) {
|
|
|
92
184
|
file,
|
|
93
185
|
executable: stat ? !!(stat.mode & 0o111) : false,
|
|
94
186
|
updatedAt: stat ? stat.mtime.toISOString() : "",
|
|
187
|
+
runCount: usage.runCount || 0,
|
|
188
|
+
lastUsed: usage.lastUsed || "",
|
|
95
189
|
content,
|
|
96
190
|
};
|
|
97
191
|
}
|
|
@@ -99,12 +193,13 @@ function readTool(name) {
|
|
|
99
193
|
function listTools() {
|
|
100
194
|
let entries;
|
|
101
195
|
try { entries = fs.readdirSync(TOOLS_DIR); } catch (e) { return []; }
|
|
196
|
+
const usage = readUsage();
|
|
102
197
|
const tools = [];
|
|
103
198
|
for (const name of entries) {
|
|
104
199
|
if (name.startsWith(".")) continue;
|
|
105
200
|
try { if (!fs.statSync(path.join(TOOLS_DIR, name)).isFile()) continue; }
|
|
106
201
|
catch (e) { continue; }
|
|
107
|
-
const t = readTool(name);
|
|
202
|
+
const t = readTool(name, usage);
|
|
108
203
|
if (t) tools.push(t);
|
|
109
204
|
}
|
|
110
205
|
tools.sort((a, b) => a.name.localeCompare(b.name));
|
|
@@ -221,7 +316,8 @@ function toolNameFromPath(filePath) {
|
|
|
221
316
|
}
|
|
222
317
|
|
|
223
318
|
module.exports = {
|
|
224
|
-
TOOLS_DIR, MARKER,
|
|
319
|
+
TOOLS_DIR, MARKER, USAGE_FILE,
|
|
225
320
|
parseHeader, readTool, listTools, findTool, addTool, removeTool,
|
|
226
321
|
runEnv, missingRequires, toolNameFromPath, sanitizeName, ensureDir,
|
|
322
|
+
matchTools, recordRun, readUsage,
|
|
227
323
|
};
|
package/package.json
CHANGED
|
@@ -10,8 +10,19 @@ const tmp = fs.mkdtempSync(path.join(os.tmpdir(), "recall-disc-"));
|
|
|
10
10
|
process.env.RECALL_GRAPH_DB = path.join(tmp, "graph.db");
|
|
11
11
|
process.env.RECALL_DISCOVERER_WALKER = "off";
|
|
12
12
|
process.env.RECALL_METRICS = "off";
|
|
13
|
+
process.env.TOOLS_DIR = path.join(tmp, "tools");
|
|
13
14
|
|
|
14
15
|
const disc = require("./core/recall/discoverer");
|
|
16
|
+
const toolsLib = require("./core/tools");
|
|
17
|
+
|
|
18
|
+
// A reusable tool whose description matches a query should seed the discoverer
|
|
19
|
+
// as a first-class candidate (origin user), survive the walker-off fail-open,
|
|
20
|
+
// and render into the toolBlock — exactly like a pack/entity seed.
|
|
21
|
+
toolsLib.addTool((() => {
|
|
22
|
+
const s = path.join(tmp, "mikrotik-audit.sh");
|
|
23
|
+
fs.writeFileSync(s, "#!/usr/bin/env bash\necho audit\n");
|
|
24
|
+
return s;
|
|
25
|
+
})(), { name: "mikrotik-audit", description: "audit the mikrotik router fleet inventory" });
|
|
15
26
|
|
|
16
27
|
// --- pre-gate ---
|
|
17
28
|
assert.strictEqual(disc.needsRecall("", 0), false, "empty → no recall");
|
|
@@ -82,5 +93,22 @@ const helpers = { packsLib, entitiesLib, mergeMatches, buildPackBlock, buildEnti
|
|
|
82
93
|
});
|
|
83
94
|
assert.strictEqual(typeof out2.packBlock, "string");
|
|
84
95
|
|
|
96
|
+
// tool seeding: a tool matching the user text surfaces as a kept tool match and
|
|
97
|
+
// renders into toolBlock; a turn that matches no tool yields an empty toolBlock
|
|
98
|
+
const outTool = await disc.run({
|
|
99
|
+
userText: "audit the mikrotik router fleet", contextText: "",
|
|
100
|
+
fullContext: "audit the mikrotik router fleet", packLimit: 6, budget: {}, helpers,
|
|
101
|
+
});
|
|
102
|
+
assert.ok(outTool.toolMatches.some((m) => m.name === "mikrotik-audit"), "matching tool is kept (fail-open user seed)");
|
|
103
|
+
assert.ok(/mikrotik-audit/.test(outTool.toolBlock), "kept tool renders into the toolBlock");
|
|
104
|
+
assert.ok(/Tools that may help here/.test(outTool.toolBlock), "toolBlock carries its heading");
|
|
105
|
+
|
|
106
|
+
const outNoTool = await disc.run({
|
|
107
|
+
userText: "help with the mobile app", contextText: "", fullContext: "help with the mobile app",
|
|
108
|
+
packLimit: 6, budget: {}, helpers,
|
|
109
|
+
});
|
|
110
|
+
assert.strictEqual(outNoTool.toolBlock, "", "no tool match → empty toolBlock");
|
|
111
|
+
assert.strictEqual(outNoTool.toolMatches.length, 0, "no tool match → no toolMatches");
|
|
112
|
+
|
|
85
113
|
console.log("recall discoverer OK");
|
|
86
114
|
})().catch((e) => { console.error(e); process.exit(1); });
|
package/test-tool-graph.js
CHANGED
|
@@ -93,5 +93,64 @@ assert.ok(typeof t.pruned === "number" && typeof t.decayed === "number", "tend r
|
|
|
93
93
|
const s = graph.stats();
|
|
94
94
|
assert.strictEqual(s.edges, graph.allEdges().length, "stats edge count matches allEdges");
|
|
95
95
|
|
|
96
|
+
// ══ Pack↔tool contextual edges ══════════════════════════════════════════════
|
|
97
|
+
|
|
98
|
+
// ── shapeCommand: strips literals to placeholders, keeps subcommands + flags ──
|
|
99
|
+
assert.strictEqual(
|
|
100
|
+
graph.shapeCommand("open-claudia tool run spaces docs --task-id 69abf1e25f50dd1fb617d6d0 /tmp/out"),
|
|
101
|
+
"spaces docs --task-id <id> <path>", "id + path reduced to shapes, wrapper stripped");
|
|
102
|
+
assert.strictEqual(
|
|
103
|
+
graph.shapeCommand('spaces comment --task-id 69abf1e25f50dd1fb617d6d0 "draft 1"'),
|
|
104
|
+
"spaces comment --task-id <id> <text>", "quoted text reduced, flag kept");
|
|
105
|
+
assert.ok(graph.shapeCommand("foo 12345").includes("<n>"), "bare number reduced to <n>");
|
|
106
|
+
|
|
107
|
+
// ── parseToolRuns: extracts {name, shape}, splits chained invocations ──
|
|
108
|
+
const runs = graph.parseToolRuns("open-claudia tool run spaces docs --task-id abcdef123456 /tmp/x && open-claudia tool run other list");
|
|
109
|
+
assert.strictEqual(runs.length, 2, "two chained tool runs parsed");
|
|
110
|
+
assert.strictEqual(runs[0].name, "spaces", "first run is spaces");
|
|
111
|
+
assert.strictEqual(runs[0].shape, "spaces docs --task-id <id> <path>", "first run shaped");
|
|
112
|
+
assert.strictEqual(runs[1].name, "other", "second run is other (args did not bleed across &&)");
|
|
113
|
+
assert.strictEqual(graph.parseToolRuns("ls -la").length, 0, "non-tool command yields no runs");
|
|
114
|
+
|
|
115
|
+
// ── recordPackTool: insert, then bump accumulates; packTools surfaces strongest ──
|
|
116
|
+
graph.recordPackTool("san-marco-agreements", "spaces", { command: "spaces docs --task-id <id> <path>", bump: 1 });
|
|
117
|
+
graph.recordPackTool("san-marco-agreements", "spaces", { command: "spaces docs --task-id <id> <path>", bump: 1 });
|
|
118
|
+
graph.recordPackTool("san-marco-agreements", "spaces", { command: "spaces upload --task-id <id> --comment <text> <path>", bump: 1 });
|
|
119
|
+
const pt = graph.packTools("san-marco-agreements", 2);
|
|
120
|
+
assert.strictEqual(pt.length, 2, "two distinct command shapes recorded");
|
|
121
|
+
assert.strictEqual(pt[0].command, "spaces docs --task-id <id> <path>", "most-reinforced shape surfaces first");
|
|
122
|
+
assert.strictEqual(pt[0].weight, 2, "two bumps accumulate to weight 2");
|
|
123
|
+
|
|
124
|
+
// ── intent enrichment: a later non-empty intent fills a blank one ──
|
|
125
|
+
graph.recordPackTool("san-marco-agreements", "spaces", { command: "spaces docs --task-id <id> <path>", intent: "pull contract drafts" });
|
|
126
|
+
assert.strictEqual(graph.packTools("san-marco-agreements", 1)[0].intent, "pull contract drafts", "blank intent enriched later");
|
|
127
|
+
|
|
128
|
+
// ── toolPacks: inverse view (which packs use this tool) ──
|
|
129
|
+
assert.ok(graph.toolPacks("spaces").some((u) => u.pack === "san-marco-agreements"), "toolPacks lists the pack");
|
|
130
|
+
|
|
131
|
+
// ── stats counts pack↔tool edges ──
|
|
132
|
+
assert.strictEqual(graph.stats().packToolEdges, graph.allPackToolEdges().length, "stats packToolEdges matches allPackToolEdges");
|
|
133
|
+
assert.ok(graph.stats().packToolEdges >= 2, "at least two pack↔tool edges present");
|
|
134
|
+
|
|
135
|
+
// ── decay: a year-old pack↔tool edge under the floor is pruned ──
|
|
136
|
+
const ptDb = graph.openDb();
|
|
137
|
+
ptDb.prepare("UPDATE pack_tool_edges SET last_reinforced=? WHERE pack=? AND tool=?")
|
|
138
|
+
.run(new Date(Date.now() - 365 * 86400000).toISOString(), "san-marco-agreements", "spaces");
|
|
139
|
+
const decayRes = graph.decay({ halfLifeDays: 1, pruneBelow: 0.15 });
|
|
140
|
+
assert.ok(decayRes.pruned >= 1, "stale pack↔tool edge pruned by decay");
|
|
141
|
+
assert.strictEqual(graph.packTools("san-marco-agreements").length, 0, "pruned pack↔tool edges gone");
|
|
142
|
+
|
|
143
|
+
// ── pruneOrphans: drops edges for a removed tool and (with packsLib) a removed pack ──
|
|
144
|
+
graph.recordPackTool("live-pack", "live-tool", { command: "live-tool go", bump: 1 });
|
|
145
|
+
graph.recordPackTool("live-pack", "ghost-tool", { command: "ghost-tool go", bump: 1 });
|
|
146
|
+
graph.recordPackTool("ghost-pack", "live-tool", { command: "live-tool go", bump: 1 });
|
|
147
|
+
const toolsLib = { listTools: () => [{ name: "live-tool" }] };
|
|
148
|
+
const packsLib = { listPacks: () => [{ dir: "live-pack" }] };
|
|
149
|
+
const orphans = graph.pruneOrphans(toolsLib, packsLib);
|
|
150
|
+
assert.ok(orphans >= 2, "edges for a missing tool and a missing pack are pruned");
|
|
151
|
+
assert.ok(graph.packTools("live-pack").some((u) => u.tool === "live-tool"), "live pack + live tool edge survives");
|
|
152
|
+
assert.ok(!graph.packTools("live-pack").some((u) => u.tool === "ghost-tool"), "missing-tool edge gone");
|
|
153
|
+
assert.strictEqual(graph.packTools("ghost-pack").length, 0, "missing-pack edge gone");
|
|
154
|
+
|
|
96
155
|
graph._resetForTest();
|
|
97
156
|
console.log("tool-graph OK");
|
package/test-tools.js
CHANGED
|
@@ -85,6 +85,36 @@ assert.strictEqual(
|
|
|
85
85
|
// ── parseHeader returns null for a script with no marker line ──
|
|
86
86
|
assert.strictEqual(tools.parseHeader("#!/bin/sh\necho hi\n"), null, "no marker → not a tool");
|
|
87
87
|
|
|
88
|
+
// ── matchTools: lexical scoring over name/description (strong) vs requires/pack
|
|
89
|
+
// /source (weak); a non-matching query returns nothing ──
|
|
90
|
+
const mHello = tools.matchTools("say hi");
|
|
91
|
+
assert.ok(mHello.some((m) => m.name === "hello"), "matchTools finds 'hello' by its description terms");
|
|
92
|
+
const mFetch = tools.matchTools("fetch a thing");
|
|
93
|
+
assert.ok(mFetch.some((m) => m.name === "fetch"), "matchTools finds 'fetch'");
|
|
94
|
+
assert.ok(!mFetch.some((m) => m.name === "hello"), "an unrelated tool does not match");
|
|
95
|
+
assert.deepStrictEqual(tools.matchTools("zzzznomatchatall"), [], "no terms hit → empty match");
|
|
96
|
+
assert.deepStrictEqual(tools.matchTools(""), [], "empty query → empty match");
|
|
97
|
+
|
|
98
|
+
// ── recordRun telemetry: bumps runCount + stamps lastUsed, visible via readUsage
|
|
99
|
+
// and surfaced on the tool record itself ──
|
|
100
|
+
assert.strictEqual(tools.findTool("hello").runCount, 0, "fresh tool has runCount 0");
|
|
101
|
+
tools.recordRun("hello");
|
|
102
|
+
assert.strictEqual(tools.readUsage().hello.runCount, 1, "recordRun bumps the usage sidecar");
|
|
103
|
+
assert.ok(tools.readUsage().hello.lastUsed, "recordRun stamps lastUsed");
|
|
104
|
+
assert.strictEqual(tools.findTool("hello").runCount, 1, "runCount surfaces on the tool record");
|
|
105
|
+
tools.recordRun("hello");
|
|
106
|
+
assert.strictEqual(tools.findTool("hello").runCount, 2, "a second run increments again");
|
|
107
|
+
|
|
108
|
+
// ── matchTools tie-break: equal score → more-run tool ranks first ──
|
|
109
|
+
for (const n of ["alpha", "beta"]) {
|
|
110
|
+
const s = path.join(tmp, `${n}.sh`);
|
|
111
|
+
fs.writeFileSync(s, "#!/usr/bin/env bash\necho x\n");
|
|
112
|
+
tools.addTool(s, { name: n, description: "common widget gadget" });
|
|
113
|
+
}
|
|
114
|
+
tools.recordRun("beta");
|
|
115
|
+
const ranked = tools.matchTools("common widget gadget").map((m) => m.name);
|
|
116
|
+
assert.ok(ranked.indexOf("beta") < ranked.indexOf("alpha"), "on equal score, the more-run tool ranks first");
|
|
117
|
+
|
|
88
118
|
// ── remove ──
|
|
89
119
|
assert.ok(tools.removeTool("hello"), "removeTool returns the removed tool");
|
|
90
120
|
assert.strictEqual(tools.findTool("hello"), null, "removed tool is gone");
|