@inetafrica/open-claudia 3.1.23 → 3.1.24
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +9 -0
- package/bin/recall.js +9 -1
- package/core/recall/discoverer.js +125 -15
- package/core/recall/query.js +27 -2
- package/core/recall/tuning.js +5 -0
- package/package.json +1 -1
- package/test-provider-language.js +15 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,14 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## v3.1.24 — Catalog-first recall: the walker sees the whole index, nothing is cut before a model judges it
|
|
4
|
+
|
|
5
|
+
- **Root cause (proven, `diag-recall-ranking.js`).** Recall misses were 4-layer candidate starvation, not bad judging: (1) lexical pack scores count distinct matched terms, so vocab-dense packs outbid the true pack; (2) the graph fire-threshold scales off the MAX seed score, so junk seeds suppress true edges; (3) per-node top-K fan-out on hub nodes cuts true edges before they transmit; (4) killer — the dream tuned `walkerMaxCandidates` down to 10 on 2026-07-09 while seeds are 6 pack + 4 entity = exactly 10 and assembly orders all seeds before all graph-activated nodes, so **graph-activated packs/entities got zero walker slots on every full-seeded turn since**. The walker judged well; it just never saw the right nodes.
|
|
6
|
+
- **Catalog-first routing.** The walker's SYSTEM prompt now carries the ENTIRE memory index — every pack, entity, and tool as a one-liner (~50KB for 276 nodes, `buildCatalog()`, 60s TTL cache). Set once per warm-walker session (exempt from the warm per-message char budget) and identical on the cold path. Lexical seeds + graph activation are demoted to per-turn RANKING HINTS: the same top-K candidate assembly, but now it only decides which few nodes carry live excerpts — never membership. The walker may keep ANY catalog id, hinted or not; zero-hint turns still walk. Kept catalog picks inject exactly like hinted ones (packs render extractive lines, un-hinted tools resolve through the registry).
|
|
7
|
+
- **Simpler under catalog mode.** The history cascade is bypassed (an auto-drop would cut on history alone — never-cut principle) and tool pack-riding is removed (every tool is already in view). Both return when the `catalog` knob is off — new dream-tunable enum (`RECALL_CATALOG` env pin) that restores the legacy capped-menu contract as a one-flip rollback.
|
|
8
|
+
- **Honest observability.** `recall query` now reports `catalog` counts + `hintedActivated`, and the CLI banner says "N kept of catalog of 276 nodes (K hinted)" with "graph-activated in hints: X of Y" — the old "+N graph-activated" counted raw expansion and overstated what actually survived into the walker's view (measured: it said +9 while 0 survived). Unjudged fail-open output no longer claims catalog scope. Per-turn metrics log catalog counts, never the full id list.
|
|
9
|
+
- **Validated on the two proven misses.** Sim A (update-app button) now keeps the in-app-updater pack at 85 `[via catalog]` — a node the old pipeline structurally could not surface; sim B (group permission system) keeps the governing pack at 88 as the sole result. Corollary: catalog one-liners make pack descriptions load-bearing for recall — the nightly dream already tightens them.
|
|
10
|
+
- New release-chain test `test-recall-catalog.js` pins the contract: full-index build + TTL cache + stub-safe null, catalog in system prompt (never the user prompt), un-hinted pack/entity/tool keeps injecting end-to-end, score-gate on catalog picks, knob-off legacy filtering, and query provenance (`via: "catalog"`).
|
|
11
|
+
|
|
3
12
|
## v3.1.23 — Scoped external rules (person / channel / everyone), auto-contacts, meet-greet lane
|
|
4
13
|
|
|
5
14
|
- **Make-a-rule now offers three scopes.** The owner's guardrail card gained a middle option: alongside "just this person" and "everyone", rules can now bind to **this channel** — stored in `channel-rules.md` keyed generically on `adapterType:channelId`, so every surface (kazee groups, Spaces tasks, telegram chats, future public domains) participates simply by existing. No adapter-specific policy logic anywhere; caps at 20 rules × 300 chars per channel.
|
package/bin/recall.js
CHANGED
|
@@ -104,7 +104,15 @@ async function queryCmd(args) {
|
|
|
104
104
|
}
|
|
105
105
|
if (flags.json) { console.log(JSON.stringify(res, null, 2)); return; }
|
|
106
106
|
const w = res.walker || {};
|
|
107
|
-
|
|
107
|
+
const cat = res.catalog;
|
|
108
|
+
// Catalog scope only when a judge actually saw the catalog — an unjudged
|
|
109
|
+
// fail-open list is hints only and must not claim full-index coverage.
|
|
110
|
+
const scope = cat && res.judged
|
|
111
|
+
? `catalog of ${cat.packs + cat.entities + cat.tools} nodes (${res.candidates} hinted)`
|
|
112
|
+
: `${res.candidates} candidate(s)`;
|
|
113
|
+
// hintedActivated counts graph pulls that actually reached the walker —
|
|
114
|
+
// res.activated is raw expansion size and used to overstate what survived.
|
|
115
|
+
console.log(`Recall query — ${res.results.length} kept of ${scope} (seeds: ${res.seeds.packs} pack, ${res.seeds.entities} entity, ${res.seeds.tools} tool; graph-activated in hints: ${res.hintedActivated ?? 0} of ${res.activated})`);
|
|
108
116
|
console.log(res.judged
|
|
109
117
|
? `Walker: ${w.provider || "?"}${w.model ? `/${w.model}` : ""} (${w.source}, ${w.ms}ms)`
|
|
110
118
|
: `Walker: ${w.source === "off" ? "off" : "failed"} — UNJUDGED candidate list (fail-open)`);
|
|
@@ -232,6 +232,81 @@ function excerptFor(id, packsLib, entitiesLib) {
|
|
|
232
232
|
} catch (e) { return null; }
|
|
233
233
|
}
|
|
234
234
|
|
|
235
|
+
// ── Catalog-first routing ────────────────────────────────────────────────────
|
|
236
|
+
// The 2026-07-24 root-cause proof (diag-recall-ranking.js): lexical seeding +
|
|
237
|
+
// the walker candidate cap silently CUT the known-true pack before the model
|
|
238
|
+
// ever saw it — graph-activated nodes got zero candidate slots on full-seeded
|
|
239
|
+
// turns. Catalog mode inverts the contract: the walker's SYSTEM prompt carries
|
|
240
|
+
// the ENTIRE index (every pack/entity/tool as a one-liner, ~50KB for 276
|
|
241
|
+
// nodes), so nothing is ever cut before a model sees it. Lexical seeds + graph
|
|
242
|
+
// activation are demoted to per-turn RANKING HINTS (top-K with live excerpts).
|
|
243
|
+
// The system prompt is set once per warm-walker session and is exempt from the
|
|
244
|
+
// warm MAX_CHARS budget (which counts user-prompt chars only); per-turn hints
|
|
245
|
+
// bridge catalog staleness within a session.
|
|
246
|
+
const CATALOG_DESC_CHARS = 160;
|
|
247
|
+
const CATALOG_TTL_MS = Number(process.env.RECALL_CATALOG_TTL_MS || 60000);
|
|
248
|
+
let _catalog = null; // { at, key, text, ids, counts }
|
|
249
|
+
|
|
250
|
+
function catalogLine(id, name, description) {
|
|
251
|
+
const d = String(description || "").replace(/\s+/g, " ").trim();
|
|
252
|
+
return `- ${id}: ${name}${d ? ` — ${clip(d, CATALOG_DESC_CHARS)}` : ""}`;
|
|
253
|
+
}
|
|
254
|
+
|
|
255
|
+
// Full-index catalog for the walker system prompt. Returns { text, ids, counts }
|
|
256
|
+
// or null when the corpus libs can't enumerate (stub libs in old tests).
|
|
257
|
+
// Cached ~60s keyed on the packs lib instance; staleness within the TTL (or a
|
|
258
|
+
// warm walker session) is bridged by hints, which always reflect the live corpus.
|
|
259
|
+
function buildCatalog(packsLib, entitiesLib) {
|
|
260
|
+
if (typeof packsLib.listPacks !== "function" || typeof entitiesLib.listEntities !== "function") return null;
|
|
261
|
+
const now = Date.now();
|
|
262
|
+
if (_catalog && _catalog.key === packsLib && now - _catalog.at < CATALOG_TTL_MS) return _catalog;
|
|
263
|
+
const ids = new Set();
|
|
264
|
+
const packLines = [];
|
|
265
|
+
const entLines = [];
|
|
266
|
+
const toolLines = [];
|
|
267
|
+
try {
|
|
268
|
+
for (const p of packsLib.listPacks()) {
|
|
269
|
+
if (!p || !p.dir || p.archived) continue;
|
|
270
|
+
ids.add(`pack:${p.dir}`);
|
|
271
|
+
packLines.push(catalogLine(`pack:${p.dir}`, p.name || p.dir, p.description));
|
|
272
|
+
}
|
|
273
|
+
for (const e of entitiesLib.listEntities()) {
|
|
274
|
+
if (!e || !e.slug) continue;
|
|
275
|
+
ids.add(`entity:${e.slug}`);
|
|
276
|
+
entLines.push(catalogLine(`entity:${e.slug}`, e.name || e.slug, e.description));
|
|
277
|
+
}
|
|
278
|
+
} catch (e) { return null; }
|
|
279
|
+
try {
|
|
280
|
+
for (const t of toolsLib.listTools()) {
|
|
281
|
+
if (!t || !t.name) continue;
|
|
282
|
+
ids.add(`tool:${t.name}`);
|
|
283
|
+
toolLines.push(catalogLine(`tool:${t.name}`, t.name, t.description));
|
|
284
|
+
}
|
|
285
|
+
} catch (e) { /* tools are optional in the catalog */ }
|
|
286
|
+
if (!ids.size) return null;
|
|
287
|
+
const text = [
|
|
288
|
+
`Packs (${packLines.length}) — topic knowledge; a kept pack injects its live Stance/State plus the lines matching the message:`,
|
|
289
|
+
...packLines,
|
|
290
|
+
"",
|
|
291
|
+
`Entities (${entLines.length}) — people, systems, places:`,
|
|
292
|
+
...entLines,
|
|
293
|
+
"",
|
|
294
|
+
`Tools (${toolLines.length}) — reusable CLIs the assistant can run:`,
|
|
295
|
+
...toolLines,
|
|
296
|
+
].join("\n");
|
|
297
|
+
_catalog = { at: now, key: packsLib, text, ids, counts: { packs: packLines.length, entities: entLines.length, tools: toolLines.length } };
|
|
298
|
+
return _catalog;
|
|
299
|
+
}
|
|
300
|
+
|
|
301
|
+
const WALKER_CATALOG_ADDENDUM = [
|
|
302
|
+
"Below is the COMPLETE catalog of the assistant's memory nodes.",
|
|
303
|
+
"Each message also brings RANKING HINTS: a few catalog nodes pre-ranked by keyword/graph overlap, shown with live excerpts, plus any matching past-conversation episodes.",
|
|
304
|
+
"Hints are hints, NOT the menu: you may keep ANY catalog node by its exact id, hinted or not. Prefer a directly-governing un-hinted node over a weakly-related hinted one.",
|
|
305
|
+
"Judge un-hinted nodes by their catalog line; hinted nodes by their excerpt.",
|
|
306
|
+
"",
|
|
307
|
+
"## Memory catalog",
|
|
308
|
+
].join("\n");
|
|
309
|
+
|
|
235
310
|
const WALKER_SYSTEM = [
|
|
236
311
|
"You are the recall ROUTER for an assistant's long-term memory.",
|
|
237
312
|
"You are given the user's message, recent conversation, and candidate memory nodes (some matched by keyword, some pulled in by graph links).",
|
|
@@ -263,11 +338,12 @@ async function runWalker(prompt, options = {}) {
|
|
|
263
338
|
modelEnvPrefix: "RECALL_DISCOVERER_MODEL",
|
|
264
339
|
legacyClaudeModelEnv: "RECALL_DISCOVERER_MODEL",
|
|
265
340
|
});
|
|
341
|
+
const systemPrompt = options.systemPrompt || WALKER_SYSTEM;
|
|
266
342
|
const opts = {
|
|
267
343
|
provider: selection.provider,
|
|
268
344
|
model: selection.model,
|
|
269
345
|
tier: "low",
|
|
270
|
-
systemPrompt
|
|
346
|
+
systemPrompt,
|
|
271
347
|
timeoutMs: WALKER_TIMEOUT_MS,
|
|
272
348
|
};
|
|
273
349
|
if (warm.isEnabled(selection.provider)) {
|
|
@@ -296,7 +372,7 @@ async function runWalker(prompt, options = {}) {
|
|
|
296
372
|
model: selection.model,
|
|
297
373
|
modelEnvPrefix: "RECALL_DISCOVERER_MODEL",
|
|
298
374
|
legacyClaudeModelEnv: "RECALL_DISCOVERER_MODEL",
|
|
299
|
-
systemPrompt
|
|
375
|
+
systemPrompt,
|
|
300
376
|
readOnly: true,
|
|
301
377
|
timeoutMs: WALKER_TIMEOUT_MS,
|
|
302
378
|
cwd: options.cwd || process.cwd(),
|
|
@@ -349,32 +425,45 @@ function cokeptPairs(keptIds, scoreById, groundedIds, { minScore = 60, maxPairs
|
|
|
349
425
|
// kept is id→why(string); scores is the parallel id→0-100 relevance from the
|
|
350
426
|
// router, consumed by the injection score-gate and asymmetric warming.
|
|
351
427
|
async function walk(userText, contextText, candidates, options = {}) {
|
|
352
|
-
|
|
428
|
+
const catalog = options.catalog || null;
|
|
429
|
+
if (!WALKER_ENABLED || (candidates.length === 0 && !catalog)) {
|
|
353
430
|
return { kept: null, scores: null, costUsd: null, source: "off", walkerMs: 0, provider: options.provider || null, model: null };
|
|
354
431
|
}
|
|
355
432
|
const lines = candidates.map((c) => {
|
|
356
433
|
const tag = c.activated ? ` [linked via ${c.via || "graph"}]` : "";
|
|
357
434
|
return `- ${c.id}: ${c.name}${tag}${c.description ? `\n ${c.description}` : ""}${c.excerpt ? `\n excerpt: ${c.excerpt.replace(/\n/g, " ")}` : ""}`;
|
|
358
435
|
});
|
|
436
|
+
const header = catalog
|
|
437
|
+
? (lines.length
|
|
438
|
+
? "Ranking hints — pre-ranked candidates with live excerpts (the full catalog in your instructions stays fully eligible):"
|
|
439
|
+
: "No pre-ranked hints this turn — route from the catalog in your instructions.")
|
|
440
|
+
: "Candidate memory nodes:";
|
|
441
|
+
const exampleId = candidates.length ? candidates[0].id : "pack:example";
|
|
359
442
|
const prompt = [
|
|
360
443
|
"User message:",
|
|
361
444
|
"<<<", clip(String(userText || ""), 1500), ">>>",
|
|
362
445
|
...(contextText ? ["", "Recent conversation:", "<<<", clip(String(contextText), CONTEXT_CLIP), ">>>"] : []),
|
|
363
446
|
"",
|
|
364
|
-
|
|
447
|
+
header,
|
|
365
448
|
...lines,
|
|
366
449
|
"",
|
|
367
|
-
'Reply ONLY with the JSON array of kept nodes, each with id, why, and score (0–100), e.g. [{"id":"' +
|
|
450
|
+
'Reply ONLY with the JSON array of kept nodes, each with id, why, and score (0–100), e.g. [{"id":"' + exampleId + '","why":"...","score":80}].',
|
|
368
451
|
].join("\n");
|
|
452
|
+
// Catalog rides in the SYSTEM prompt: set once per warm session (exempt from
|
|
453
|
+
// the warm walker's per-message char budget) and identical on the cold path.
|
|
454
|
+
const walkOptions = catalog
|
|
455
|
+
? { ...options, systemPrompt: `${WALKER_SYSTEM}\n${WALKER_CATALOG_ADDENDUM}\n${catalog.text}` }
|
|
456
|
+
: options;
|
|
369
457
|
const t0 = Date.now();
|
|
370
458
|
try {
|
|
371
|
-
const { text, costUsd, source, provider, model } = await runWalker(prompt,
|
|
459
|
+
const { text, costUsd, source, provider, model } = await runWalker(prompt, walkOptions);
|
|
372
460
|
const walkerMs = Date.now() - t0;
|
|
373
461
|
const match = String(text || "").match(/\[[\s\S]*\]/);
|
|
374
462
|
if (!match) return { kept: null, scores: null, costUsd, source, walkerMs, provider, model };
|
|
375
463
|
const arr = JSON.parse(match[0]);
|
|
376
464
|
if (!Array.isArray(arr)) return { kept: null, scores: null, costUsd, source, walkerMs, provider, model };
|
|
377
465
|
const known = new Set(candidates.map((c) => c.id));
|
|
466
|
+
if (catalog && catalog.ids) for (const id of catalog.ids) known.add(id);
|
|
378
467
|
const out = new Map();
|
|
379
468
|
const scores = new Map();
|
|
380
469
|
for (const item of arr) {
|
|
@@ -447,10 +536,19 @@ async function run(ctx) {
|
|
|
447
536
|
}
|
|
448
537
|
const expandMs = Date.now() - expandStarted;
|
|
449
538
|
|
|
539
|
+
// Catalog-first (full tier): the walker's system prompt carries the ENTIRE
|
|
540
|
+
// index, so membership is never decided by lexical scoring or the candidate
|
|
541
|
+
// cap — those now only rank which few nodes get live excerpts as hints.
|
|
542
|
+
let catalog = null;
|
|
543
|
+
if (tier === "full" && tuning.get("catalog") !== "off") {
|
|
544
|
+
try { catalog = buildCatalog(packsLib, entitiesLib); } catch (e) { catalog = null; }
|
|
545
|
+
}
|
|
546
|
+
|
|
450
547
|
// 4: assemble candidates (seeds + activated) in priority order — user seeds
|
|
451
548
|
// by score, then context seeds, then graph-activated by activation — capped
|
|
452
549
|
// at MAX_WALKER_CANDIDATES so the walker prompt stays small and fast. Only
|
|
453
550
|
// the full tier pays for excerpts; the seeds tier never reads them.
|
|
551
|
+
// Under catalog mode these are ranking hints; without it they are the menu.
|
|
454
552
|
const seedRank = new Map();
|
|
455
553
|
for (const m of packSeeds) seedRank.set(`pack:${m.dir}`, { score: m.score || 2, origin: m.origin });
|
|
456
554
|
for (const m of entSeeds) seedRank.set(`entity:${m.slug}`, { score: m.score || 2, origin: m.origin });
|
|
@@ -473,16 +571,16 @@ async function run(ctx) {
|
|
|
473
571
|
}
|
|
474
572
|
const candIds = new Set(tier === "full" ? candidates.map((c) => c.id) : seedIds);
|
|
475
573
|
|
|
476
|
-
// Tool candidates = direct tool seeds + tools that belong
|
|
477
|
-
// pack (declared via the tool's `pack:` header, or used-in-
|
|
478
|
-
// tool-graph).
|
|
479
|
-
//
|
|
480
|
-
//
|
|
574
|
+
// Tool candidates = direct tool seeds + (catalog off only) tools that belong
|
|
575
|
+
// to any candidate pack (declared via the tool's `pack:` header, or used-in-
|
|
576
|
+
// topic per the tool-graph). Under catalog mode every tool is already in the
|
|
577
|
+
// walker's view as a catalog line, so pack-riding is redundant — direct seeds
|
|
578
|
+
// still join as hints because they carry usage/shape excerpts.
|
|
481
579
|
const candPackDirs = new Set();
|
|
482
580
|
for (const id of candIds) if (id.startsWith("pack:")) candPackDirs.add(id.slice(5));
|
|
483
581
|
const toolCand = new Map();
|
|
484
582
|
for (const [name, t] of toolSeedByName) toolCand.set(name, { name, description: t.description, via: null });
|
|
485
|
-
if (tier === "full" && candPackDirs.size) {
|
|
583
|
+
if (tier === "full" && !catalog && candPackDirs.size) {
|
|
486
584
|
try {
|
|
487
585
|
const byPack = new Map();
|
|
488
586
|
for (const t of toolsLib.listTools()) {
|
|
@@ -547,7 +645,10 @@ async function run(ctx) {
|
|
|
547
645
|
// continuously audited against the model it replaces; on shadowed candidates
|
|
548
646
|
// the judge's verdict wins. Top-3 ranked candidates and episodes (no stable
|
|
549
647
|
// history) always go to the judge.
|
|
550
|
-
|
|
648
|
+
// Catalog mode bypasses the cascade: an auto-drop would strip a hint's
|
|
649
|
+
// excerpt on history alone, and history auto-keeps are the sticky-pack leak
|
|
650
|
+
// the score gate exists to stop. Never-cut principle: the judge sees it all.
|
|
651
|
+
const cascadeOn = tier === "full" && !catalog && tuning.get("cascade") !== "off";
|
|
551
652
|
// Score-gate config (WS4). When a relevance floor is set, the walker's real
|
|
552
653
|
// per-turn score decides injection — so the cascade must NOT auto-KEEP from
|
|
553
654
|
// history: a sticky pack's history score is always high and would re-leak the
|
|
@@ -577,7 +678,7 @@ async function run(ctx) {
|
|
|
577
678
|
}
|
|
578
679
|
}
|
|
579
680
|
|
|
580
|
-
const walkRes = await walk(userText, contextText || fullContext, judgeCands, { provider: provider || null, ...(ctx.walkerOptions || {}) });
|
|
681
|
+
const walkRes = await walk(userText, contextText || fullContext, judgeCands, { provider: provider || null, catalog, ...(ctx.walkerOptions || {}) });
|
|
581
682
|
const whyById = walkRes.kept;
|
|
582
683
|
const scoreById = walkRes.scores; // parallel id→0-100 router relevance (null on fail-open)
|
|
583
684
|
|
|
@@ -686,6 +787,14 @@ async function run(ctx) {
|
|
|
686
787
|
toolMatches.push({ name: c.name, description: c.description, why: whyMap ? whyMap.get(`tool:${c.name}`) || "" : "" });
|
|
687
788
|
}
|
|
688
789
|
}
|
|
790
|
+
// Catalog picks: a kept tool the hint set never carried still renders.
|
|
791
|
+
for (const id of keptIds) {
|
|
792
|
+
if (!id.startsWith("tool:") || toolCand.has(id.slice(5))) continue;
|
|
793
|
+
try {
|
|
794
|
+
const t = toolsLib.findTool(id.slice(5));
|
|
795
|
+
if (t) toolMatches.push({ name: t.name, description: t.description, why: whyMap ? whyMap.get(id) || "" : "" });
|
|
796
|
+
} catch (e) {}
|
|
797
|
+
}
|
|
689
798
|
let toolBlock = "";
|
|
690
799
|
if (toolMatches.length) {
|
|
691
800
|
// Tiered injection (Phase 2): a 2-line headline per tool — identity line
|
|
@@ -755,6 +864,7 @@ async function run(ctx) {
|
|
|
755
864
|
seeds: seedNodes,
|
|
756
865
|
activated: [...activated.entries()].map(([id, a]) => ({ id, activation: a.activation, hop: a.hop })),
|
|
757
866
|
cand: candidates.map((c) => ({ id: c.id, act: !!c.activated })),
|
|
867
|
+
catalog: catalog ? catalog.counts : undefined,
|
|
758
868
|
kept: [...keptIds].map((id) => ({ id, why: whyMap ? whyMap.get(id) : "", score: scoreById && scoreById.has(id) ? scoreById.get(id) : null })),
|
|
759
869
|
auto: (autoKept.size || autoDropped.length || shadowChecks)
|
|
760
870
|
? { kept: [...autoKept.keys()], dropped: autoDropped, shadow: { checks: shadowChecks, agree: shadowAgree } }
|
|
@@ -778,4 +888,4 @@ async function run(ctx) {
|
|
|
778
888
|
};
|
|
779
889
|
}
|
|
780
890
|
|
|
781
|
-
module.exports = { name: "discoverer", run, runWalker, needsRecall, recallTier, walk, excerptFor, cascadeVerdict, cokeptPairs, scoutToolGaps, renderToolGaps, episodesAllowedForSpeaker };
|
|
891
|
+
module.exports = { name: "discoverer", run, runWalker, needsRecall, recallTier, walk, excerptFor, buildCatalog, cascadeVerdict, cokeptPairs, scoutToolGaps, renderToolGaps, episodesAllowedForSpeaker };
|
package/core/recall/query.js
CHANGED
|
@@ -90,6 +90,14 @@ async function query(question, opts = {}) {
|
|
|
90
90
|
try { activated = graph.expand(seedNodes, {}); } catch (e) {}
|
|
91
91
|
}
|
|
92
92
|
|
|
93
|
+
// Catalog-first: same contract as the per-turn engine — the walker sees the
|
|
94
|
+
// whole index in its system prompt; seeds/activation below only rank which
|
|
95
|
+
// hints carry excerpts. Null when the corpus libs can't enumerate (stubs).
|
|
96
|
+
let catalog = null;
|
|
97
|
+
if (tuning.get("catalog") !== "off") {
|
|
98
|
+
try { catalog = discoverer.buildCatalog(packsLib, entitiesLib); } catch (e) { catalog = null; }
|
|
99
|
+
}
|
|
100
|
+
|
|
93
101
|
// Candidates: question seeds by score, context seeds, then graph-activated.
|
|
94
102
|
const seedRank = new Map();
|
|
95
103
|
for (const m of packSeeds) seedRank.set(`pack:${m.dir}`, { score: m.score || 2, origin: m.origin });
|
|
@@ -151,18 +159,32 @@ async function query(question, opts = {}) {
|
|
|
151
159
|
} catch (e) { /* episodic recall is best-effort */ }
|
|
152
160
|
}
|
|
153
161
|
|
|
162
|
+
const hintedCount = candidates.length;
|
|
163
|
+
const hintedActivated = candidates.filter((c) => c.activated && (c.id.startsWith("pack:") || c.id.startsWith("entity:"))).length;
|
|
164
|
+
|
|
154
165
|
// Walker judgement (cold spawn only). opts.walker === false skips the spawn.
|
|
155
166
|
let walkRes = { kept: null, scores: null, source: "off", walkerMs: 0, provider: null, model: null, costUsd: null };
|
|
156
|
-
if (opts.walker !== false && candidates.length) {
|
|
167
|
+
if (opts.walker !== false && (candidates.length || catalog)) {
|
|
157
168
|
walkRes = await discoverer.walk(q, context, candidates, {
|
|
158
169
|
warmWalker: COLD_ONLY_WARM,
|
|
159
170
|
cwd: opts.cwd,
|
|
171
|
+
catalog,
|
|
160
172
|
...(opts.walkerOptions || {}),
|
|
161
173
|
});
|
|
162
174
|
}
|
|
163
175
|
const whyMap = walkRes.kept;
|
|
164
176
|
const scores = walkRes.scores;
|
|
165
177
|
|
|
178
|
+
// Catalog picks outside the hint set still return — excerpt them on demand.
|
|
179
|
+
if (whyMap) {
|
|
180
|
+
const candIds = new Set(candidates.map((c) => c.id));
|
|
181
|
+
for (const id of whyMap.keys()) {
|
|
182
|
+
if (candIds.has(id)) continue;
|
|
183
|
+
const base = discoverer.excerptFor(id, packsLib, entitiesLib);
|
|
184
|
+
if (base) candidates.push({ ...base, activated: true, via: "catalog" });
|
|
185
|
+
}
|
|
186
|
+
}
|
|
187
|
+
|
|
166
188
|
const results = [];
|
|
167
189
|
for (const c of candidates) {
|
|
168
190
|
if (whyMap && !whyMap.has(c.id)) continue;
|
|
@@ -205,7 +227,9 @@ async function query(question, opts = {}) {
|
|
|
205
227
|
hasContext: !!context,
|
|
206
228
|
seeds: { packs: packSeeds.length, entities: entSeeds.length, tools: toolSeeds.size },
|
|
207
229
|
activated: activated.size,
|
|
208
|
-
|
|
230
|
+
hintedActivated,
|
|
231
|
+
candidates: hintedCount,
|
|
232
|
+
catalog: catalog ? catalog.counts : null,
|
|
209
233
|
judged: !!whyMap,
|
|
210
234
|
walker: {
|
|
211
235
|
source: walkRes.source,
|
|
@@ -227,6 +251,7 @@ async function query(question, opts = {}) {
|
|
|
227
251
|
seeds: seedNodes,
|
|
228
252
|
activated: [...activated.entries()].map(([id, a]) => ({ id, activation: a.activation, hop: a.hop })),
|
|
229
253
|
cand: candidates.map((c) => ({ id: c.id, act: !!c.activated })),
|
|
254
|
+
catalog: catalog ? catalog.counts : undefined,
|
|
230
255
|
kept: results.map((r) => ({ id: r.id, why: r.why, score: r.score })),
|
|
231
256
|
gated: false,
|
|
232
257
|
latencyMs: out.latencyMs,
|
package/core/recall/tuning.js
CHANGED
|
@@ -18,6 +18,11 @@ const KNOBS = {
|
|
|
18
18
|
seedsTierMaxWords: { def: 4, min: 2, max: 6, env: "RECALL_SEEDS_TIER_MAX_WORDS" },
|
|
19
19
|
decayHalfLifeDays: { def: 60, min: 10, max: 90, env: "RECALL_DECAY_HALF_LIFE_DAYS" },
|
|
20
20
|
cascade: { def: "on", values: ["on", "off"], env: "RECALL_CASCADE" },
|
|
21
|
+
// Catalog-first routing: the walker's system prompt carries the ENTIRE
|
|
22
|
+
// pack/entity/tool index; lexical seeds + graph activation become ranking
|
|
23
|
+
// hints (top-K excerpts), never a membership cut. "off" restores the legacy
|
|
24
|
+
// capped-candidate menu (and re-enables cascade + tool pack-riding).
|
|
25
|
+
catalog: { def: "on", values: ["on", "off"], env: "RECALL_CATALOG" },
|
|
21
26
|
// Extractive on-open injection: surface Stance verbatim + the actual lines
|
|
22
27
|
// that matched this message (from anywhere in the pack), not just a headline.
|
|
23
28
|
extractive: { def: "on", values: ["on", "off"], env: "RECALL_EXTRACTIVE" },
|
package/package.json
CHANGED
|
@@ -288,7 +288,21 @@ assert.match(pkg.description, /provider-agnostic coding-agent harness/i);
|
|
|
288
288
|
// owner) without explicit coverage while disclosure, commitments, and actions
|
|
289
289
|
// stay held for escalation; approval cards say where the message came from
|
|
290
290
|
// via the optional adapter channelLabel (kazee chat name, spaces task title).
|
|
291
|
-
|
|
291
|
+
// 3.1.24 approved by Sumeet 2026-07-24 ("Okay, I think that makes sense, so go
|
|
292
|
+
// ahead and implement that and then push it out"): catalog-first recall — the
|
|
293
|
+
// proven 4-layer candidate-starvation (diag-recall-ranking.js: lexical seed
|
|
294
|
+
// scoring, junk-seed fire threshold, hub fan-out topK, and a dream-tuned
|
|
295
|
+
// walker cap of 10 vs exactly 10 seeds leaving graph-activated nodes ZERO
|
|
296
|
+
// slots since 2026-07-09) is fixed by inverting the contract: the walker's
|
|
297
|
+
// SYSTEM prompt now carries the ENTIRE pack/entity/tool index (one-liners,
|
|
298
|
+
// warm-session cached, exempt from the warm per-message char budget) so
|
|
299
|
+
// nothing is ever cut before a model sees it; lexical seeds + graph
|
|
300
|
+
// activation demote to per-turn ranking HINTS (top-K excerpts); the walker
|
|
301
|
+
// may keep ANY catalog id; cascade + tool pack-riding bypass under catalog
|
|
302
|
+
// mode; recall query mirrors it and the CLI banner reports honest
|
|
303
|
+
// hint-survival instead of raw expansion counts; catalog knob (dream-tunable,
|
|
304
|
+
// RECALL_CATALOG pin) restores the legacy menu.
|
|
305
|
+
assert.strictEqual(pkg.version, "3.1.24", "release version must remain unchanged without explicit approval");
|
|
292
306
|
for (const keyword of ["claude", "codex", "coding-agent", "provider-agnostic"]) {
|
|
293
307
|
assert.ok(pkg.keywords.includes(keyword), `package keyword missing: ${keyword}`);
|
|
294
308
|
}
|