@polycode-projects/the-mechanical-code-talker 0.9.6 → 0.9.8

package/ROADMAP.md

@@ -12,11 +12,28 @@ mapped into them (item numbers retained for traceability); the seven sketches
 formerly held in `code-talker-ideas.txt` are folded into items 8–11 below and
 the file has been deleted.
 
-## Where we are now (2026-07-07)
+## Where we are now (2026-07-08)
 
-`npm test` green (**1042**). **v0.9.5, pushed** (0.8.2 → 0.9.0 for this session's main wave, then
-one patch per shipped playtest-sprint fix — see `HANDOVER.md` for the exact release chain and the
-"Playtest sprint" section there for what each patch fixed).
+`npm test` green (**1055**). **v0.9.6, pushed** (0.8.2 → 0.9.6 across this session — see
+`HANDOVER.md` for the exact release chain).
+
+### Doing right now (5 background clusters, all worktree-isolated, merged sequentially as each lands)
+
+- **Cluster A** — Bug 6 (scoped-listing false-empty) + Bug 7 (modal-in-fuzzy-cascade) + the
+  chat-feel residual pair (items 1-2 of the trio's siblings) + `edgesOfKind` memoization +
+  ontology numeric-vocabulary declaration.
+- **Cluster B** — Bug 8 (goal-reasoner confident-wrong on off-domain requests) + refreshing
+  `PLAN_CAPABILITY_ROUTER.md`'s stale stage-5 status text.
+- **Cluster C** — `PLAN_CHAT_FEEL.md` item 6, the pronoun/temporal/discourse-count trio
+  (measured red sets from an earlier advisor tick; re-measured against the current graded pool
+  before any fix).
+- **Cluster D** — `PLAN_CODE.md` Track 1 (GOAL_RULE/PHRASING_FRAMES synthesis) — operator
+  sign-off given this session; Tracks 2-4 remain unsigned-off and untouched.
+- **Cluster E** — `PLAN_INFERENCE_TESTING.md` Stage 1 (`cax-sco` rule) + Stage 2 (proof-chain
+  receipts) in `src/syllogise.mjs`, targeting the measured chat-A2 50% ceiling.
+
+Also in flight: playtest sprint rounds 4-8 (3/3 rounds so far shipped a real fix), the strategy
+advisor (restarted, 5-min re-arm).
 
 ### Now: shipped this session
 
@@ -44,16 +61,23 @@ one patch per shipped playtest-sprint fix — see `HANDOVER.md` for the exact re
 - **4 new plan docs.** `PLAN_ontology-hierarchies.md`, `PLAN_INFERENCE_TESTING.md` (revised so
   infbench generation is mechanical, not hand-authored), `archive/PLAN_PREDICATE_QUERIES.md`, and
   `PLAN_CODE.md` (new, program synthesis over tmct's closed DSLs, gated on explicit operator
-  sign-off per track, not built yet).
+  sign-off per track — Track 1 signed off, see "Doing" above).
+- **`PLAN_TMCT_ECOSYSTEM_INTEGRATION.md`**, landed — a code-grounded tmct/bedrock-meter/marginalia
+  integration plan. Corrects a standing wrong assumption (the `/v1/messages` shim was believed
+  unmerged; it's been on `main` since 0.8.0) and confirms seonix has already fully migrated onto
+  tmct as a real dependency.
+- **A `/narrate` debug mode.** Opt-in (`/narrate on`, `--narrate`, `TMCT_NARRATE=1`) per-turn
+  decision narrative — which lane/pattern matched, results + sources, a deterministic goal
+  deduction — built for playtest-sprint debugging; zero cost when off (byte-identical regression
+  guard).
 
 ### Next: the open follow-ups
 
 In priority order (full detail and measured targets in `HANDOVER.md`):
 
-1. **Bug 8.** The goal-reasoner's global-mode deduction answers requests unrelated to any deduced
-   goal with false confidence, once no focus entity binds (`src/router/goal-reasoner.mjs:226`).
-   The most serious open issue: a genuine confident-wrong failure in the flagship
-   zero-hallucination capability. Needs a real semantic gate.
+1. **Bug 8 — FIXED.** A global-mode domain gate now requires the request itself (not just the
+   caller's declared toolset) to parse as a query about the deduced goal's domain, reusing
+   `ask.mjs`'s own grammar. Zero-hallucination baseline unchanged; independently re-verified.
 2. **Bug 6.** Scoped listing false-empty: "list modules in `<pkg>`" returns empty though the
    unscoped lister shows matches. Found dogfooding a 191k-entity monorepo graph.
 3. **Bug 7.** A modal auxiliary ("should") survives the fuzzy-correction cascade and misreads as

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@polycode-projects/the-mechanical-code-talker",
-  "version": "0.9.6",
+  "version": "0.9.8",
   "private": false,
   "type": "module",
   "description": "The Mechanical Code Talker (tmct) — a tolerant, offline, $0 chat surface that guides you toward precision queries about a software repository. ELIZA/PARRY-style but domain-obsessed with code. No model calls; no codebase index of its own.",

package/src/ask.mjs

@@ -38,7 +38,7 @@
 // edges (mgx:touchedByCommit / mgx:changeCoupledWith), which is a different (and
 // simpler) question than the browser's time-scrubbing view.
 
-import { relationKind, impactClosure } from "./codegraph.mjs";
+import { relationKind, impactClosure, normPath } from "./codegraph.mjs";
 import {
   VERB_TO_KIND, ENTITY_TO_TYPE, MODIFIER_TO_KIND,
   CONTEXT_PRONOUNS, META_MEANING_VERBS,
@@ -70,12 +70,36 @@ export { normalizeQuery, applyNegationFrames };
 // shipping a ~1MB language model inside the page.
 import { nlpAdapter } from "./ask-nlp.mjs";
 
+/** Per-graph, per-kind memo for THIS file's own edgesOfKind copy — same WeakMap<graph,
+ *  Map<kind, edge[]>> shape as codegraph.mjs's twin (and this file's own qualCache,
+ *  below), kept as an independent cache rather than sharing codegraph.mjs's (same
+ *  commit-boundary reasoning as the function copy itself: both derive the identical
+ *  result from the identical relationKind classification, so two caches can never
+ *  disagree, only duplicate a little memory). Correctness rests on the same
+ *  invariant qualCache already relies on: a loaded graph's `relations` are never
+ *  mutated in place (a refresh always builds a NEW graph object via parseEntities).
+ *  Deliberately NAMED DIFFERENTLY from codegraph.mjs's `edgesOfKindCache` — the
+ *  inlined viewer bundle (viz.mjs's askSource) literally CONCATENATES a stripped
+ *  codegraph.mjs + this file into one classic script (test/ask-nlp.test.mjs pins
+ *  this), so two `const`s with the same name would be a real SyntaxError there. */
+const askEdgesOfKindCache = new WeakMap();
+
 /** All edges of a classified relation kind, flattened across relation groups —
  *  a local copy of codegraph.mjs's private edgesOfKind (kept local rather than
  *  exported+imported to avoid coupling this file's commit boundary to concurrent
  *  in-flight edits elsewhere in codegraph.mjs; both read the same relationKind
- *  classification, so they cannot drift in meaning). */
+ *  classification, so they cannot drift in meaning). Memoized per (graph, kind) —
+ *  perf lever, HANDOVER follow-up #8: this is the query engine's hottest path,
+ *  called repeatedly on the same (graph, kind) pair across a single query's
+ *  compositional evaluation, and at monorepo scale (tens of thousands of modules)
+ *  the repeated O(relations) scan is a real latency/GC cost — not a correctness
+ *  fix (the stack-overflow bug this file's twin comment references is already
+ *  fixed and unrelated). */
 function edgesOfKind(graph, kind) {
+  let byKind = askEdgesOfKindCache.get(graph);
+  if (!byKind) { byKind = new Map(); askEdgesOfKindCache.set(graph, byKind); }
+  const cached = byKind.get(kind);
+  if (cached) return cached;
   const out = [];
   // Plain-loop append, NOT out.push(...g.edges): argument spread overflows the call
   // stack past ~100k edges on graph-scale relation groups (see codegraph.mjs twin).
@@ -83,6 +107,7 @@ function edgesOfKind(graph, kind) {
     if (relationKind(g) !== kind) continue;
     for (const e of g.edges) out.push(e);
   }
+  byKind.set(kind, out);
   return out;
 }
 
@@ -980,6 +1005,19 @@ function uniqueById(inds) {
   return out;
 }
 
+/** The Module individuals whose path lives strictly UNDER the directory named by
+ *  `term` — a proper path-segment prefix match (normPath(label).startsWith(dir +
+ *  "/")), never a bare substring, so "src/lib" cannot spuriously catch
+ *  "src/libfoo/x.mjs". Mirrors renderArchitecture's own pkg-prefix scoping
+ *  (codegraph.mjs) but returns individuals rather than a summary string — this is
+ *  the "membership" AST node's directory-scope branch (see its call site above). */
+function directoryScopeModules(graph, term) {
+  const norm = normPath(term);
+  if (!norm) return [];
+  const prefix = `${norm}/`;
+  return graph.individuals.filter((i) => i.class === "Module" && normPath(i.label).startsWith(prefix));
+}
+
 // Per-graph memo for the qualifier attribute/edge sets (exported symbols, tested
 // modules, symbol→module map) — computed once, so a qualifier filter over a large
 // result set stays cheap and deterministic.
@@ -1242,7 +1280,27 @@ function evalSet(graph, ast, opts) {
       return forwardOverSet(graph, ast.kind, ids);
     }
     case "membership": {
+      // DIRECTORY SCOPE ("modules in src/lib", "files in src/handlers"): a bare
+      // path term with no exact node of its own is a DIRECTORY, not a single
+      // container individual — resolveObject's fuzzy tiers used to land it on ONE
+      // arbitrarily-chosen module whose label merely CONTAINS the path substring
+      // (e.g. "src/lib" fuzzy-matching "src/lib/logger.mjs"), then traversed that
+      // one module's own membership edges for entityType "Module" — which a module
+      // never has, so the answer was a false-empty ("no modules in this index.")
+      // even though several modules genuinely live under the directory. An EXACT
+      // node match (tier 1 — a real file/symbol named that) still wins outright
+      // (unchanged single-container-node behavior, e.g. "methods in widget.mjs");
+      // only when there is no exact match do we try directory-prefix scope first.
       const r = resolveObject(graph, ast.term);
+      if (!(r.match && r.tier === 1)) {
+        const dirMods = directoryScopeModules(graph, ast.term);
+        if (dirMods.length) {
+          if (!ast.entityType || ast.entityType === "Module") return dirMods;
+          const ids = new Set(dirMods.map((m) => m.id));
+          const objs = uniqueById(MEMBERSHIP_KINDS.flatMap((k) => forwardOverSet(graph, k, ids)));
+          return objs.filter((o) => o.class === ast.entityType);
+        }
+      }
       if (!r.match) return [];
       const ids = new Set([r.match.id]);
       const objs = uniqueById(MEMBERSHIP_KINDS.flatMap((k) => forwardOverSet(graph, k, ids)));

package/src/chat.mjs

@@ -1052,12 +1052,22 @@ async function moduleOrientLane(query, { graph }) {
   return { text: moduleOverviewText(graph, ind), via: "meta" };
 }
 
-async function metaLane(query, { graph, memoryDir }) {
+async function metaLane(query, { graph, memoryDir, last = null }) {
   const q = String(query).trim().toLowerCase().replace(/[?.!]+$/, "").replace(/\s+/g, " ");
   if (WHAT_KNOW_RE.test(q) || q === "what have you learned" || q === "what have you learnt") {
     return { text: await memorySummary(memoryDir, graph), via: "meta" };
   }
-  if (META_ORIENT_RE.test(q)) return { text: orientationText(graph), via: "meta" };
+  if (META_ORIENT_RE.test(q)) {
+    // Chat-feel residual (0.8.2 confirmation playtest, follow-up #3): Bug B1 only
+    // taught the isConversational-triggered orientation branch (below, via:"template")
+    // to shorten on an identical repeat — this META_ORIENT_RE branch is a SEPARATE
+    // route to the same class of full-blurb text ("what does this app do" reprinted
+    // orientationText(graph) verbatim on every repeat, never collapsing). Mirrors
+    // ORIENTATION_REPEAT_ONELINER's identity-check pattern exactly, with its own
+    // distinct oneliner text (self-limiting for the same reason).
+    const text = orientationText(graph);
+    return { text: last?.answer === text ? META_ORIENT_REPEAT_ONELINER : text, via: "meta" };
+  }
   // Bug E: an arbitrary "what does <term> do" that META_ORIENT_RE's closed noun
   // list didn't claim — try the module-grain overview before falling through to
   // the author-sha check below (disjoint triggers; order doesn't matter, but
@@ -1272,6 +1282,13 @@ const WALL_REPEAT_ONELINER = "still couldn't parse that — /help lists every qu
  *  re-offers the full orientation instead of droning the one-liner forever. */
 const ORIENTATION_REPEAT_ONELINER = "still the same overview — /help lists every command and query shape.";
 
+/** metaLane's own repeat-suppression twin for META_ORIENT_RE ("what does this app
+ *  do", etc. — see metaLane's doc above) — a genuinely separate route to the same
+ *  orientation-class text that Bug B1 didn't cover. MUST differ from
+ *  ORIENTATION_REPEAT_ONELINER (a distinct string, checked by identity) so the two
+ *  independent repeat-suppression sites can never be confused with one another. */
+const META_ORIENT_REPEAT_ONELINER = "still the same overview — /stats for the full one, /help for commands.";
+
 // ---- repo-root resolution: default the target to the GIT ROOT, not raw cwd ----
 
 /** The git top-level for `cwd`, or null if not in a repo (or git is unavailable).
@@ -2455,7 +2472,7 @@ async function runAsk(query, { config, source, graph, focus, last, templates, me
   // codebase", "how do i start") → a summary / orientation, answered before the
   // fact-dump readers so "what do you know" gets a summary, not raw facts.
   if (miss) {
-    const meta = await metaLane(query, { graph, memoryDir });
+    const meta = await metaLane(query, { graph, memoryDir, last });
     if (meta) {
       answer = meta.text; via = meta.via; recordMiss = false; handled = true;
       note(trace, `lane: (1) META/SELF — bare self/session question recognized, answered via="${meta.via}"`);

package/src/codegraph.mjs

@@ -115,7 +115,7 @@ export function relationKind(group) {
 
 // ---- symbol resolution (exact → normalised path → substring) ------------------
 
-function normPath(s) {
+export function normPath(s) {
   return String(s || "")
     .trim()
     .toLowerCase()
@@ -1166,12 +1166,31 @@ export function renderSearch(graph, query, { limit = SEARCH_LIMIT, kind = "", de
 //      history / call neighbours). Each answers ONE question in one compact call so
 //      the agent need not Read/Grep. All keep the bounded-output discipline. -------
 
+/** Per-graph, per-kind memo for edgesOfKind's own flattened scan — WeakMap<graph,
+ *  Map<kind, edge[]>>, mirroring qualCache's (ask.mjs) established per-graph-object
+ *  caching convention: a loaded graph's `relations` are never mutated in place after
+ *  parseEntities builds it (every refresh constructs a NEW graph object), so caching
+ *  keyed on graph object identity is correctness-safe for a graph's whole lifetime —
+ *  same invariant qualCache already relies on in production. edgesOfKind is called
+ *  repeatedly on the SAME (graph, kind) pair across a single query's traversal
+ *  (evalSet/traverse/adjacencyForKinds/renderArchitecture/… all re-derive it), and at
+ *  monorepo scale (tens of thousands of modules) that repeated O(relations) scan is a
+ *  real latency/GC cost — this collapses every call after the first to an O(1) lookup. */
+const edgesOfKindCache = new WeakMap();
+
 /** All edges whose relation classifies to `kind`, flattened across relation groups. */
 /** All edges of a classified relation kind (imports/calls/defines/tests/touches/inherits/
  *  cochange/reexports/callsSymbol/touchesSymbol/contains — see relationKind/PROP_KIND above),
  *  flattened across every raw relation group that classifies to it. Exported for ask.mjs's
- *  mechanical NL-query engine (PLAN_MECHANICAL_CHAT.md) to orchestrate rather than duplicate. */
+ *  mechanical NL-query engine (PLAN_MECHANICAL_CHAT.md) to orchestrate rather than duplicate.
+ *  Memoized per (graph, kind) — see edgesOfKindCache's own doc above (perf lever, HANDOVER
+ *  follow-up #8: latency/GC on monorepo-scale graphs, not a correctness fix — the earlier
+ *  stack-overflow bug below is already fixed and unrelated). */
 export function edgesOfKind(graph, kind) {
+  let byKind = edgesOfKindCache.get(graph);
+  if (!byKind) { byKind = new Map(); edgesOfKindCache.set(graph, byKind); }
+  const cached = byKind.get(kind);
+  if (cached) return cached;
   const out = [];
   // Plain-loop append, NOT out.push(...g.edges): argument spread materialises every
   // element as a call argument and overflows the stack past ~100k edges (live report:
@@ -1180,6 +1199,7 @@ export function edgesOfKind(graph, kind) {
     if (relationKind(g) !== kind) continue;
     for (const e of g.edges) out.push(e);
   }
+  byKind.set(kind, out);
   return out;
 }
 

package/src/interpret/normalize.mjs

@@ -491,6 +491,13 @@ export const STOPWORDS = new Set([
   // the module alone). Same trade as every other stopword: a symbol literally named
   // "usually" would be the accepted residual cost.
   "usually", "typically", "generally", "normally", "often", "commonly", "mostly",
+  // modal auxiliaries ("what SHOULD i look at first") — found live: with no modal in
+  // this set, "should" reached the cascade's bounded fuzzy-correction step and landed
+  // within edit distance of the unrelated closed-vocab word "hold" ("defines" synonym,
+  // ask-vocab.mjs), corrupting the whole query into "what hold i at". Same trade as
+  // every other stopword: a symbol literally named "should" would be the accepted
+  // residual cost.
+  "should", "would", "could", "can", "will", "shall", "might", "must",
 ]);
 
 /** Split free text into words: trailing "?" run stripped, commas treated as

package/src/router/goal-reasoner.mjs

@@ -40,11 +40,32 @@
 // (queries mutate nothing — the STRIPS closed world), so no step can delete a
 // condition another goal depends on. We compute this from the registry rather
 // than assume it (threatsAmong), so the guarantee is grounded, not asserted.
+//
+// THE GLOBAL-MODE DOMAIN GATE (Bug 8 fix). In SCOPED mode, relevance is already
+// proven structurally: the focus is a REAL bound graph entity (resolveObject
+// found it), so the request is provably about something in the graph. In GLOBAL
+// mode there is no focus to bind, and `applicableRules` alone only screens the
+// CALLER'S DECLARED TOOLSET — a caller-constant fact that says nothing about
+// whether THIS request has any connection to the deduced goal (a caller who
+// declares tmct_untested/tmct_impact once per session would ground
+// coverage-invariant for every off-topic turn). The fix reuses ask.mjs's OWN
+// compositional NL grammar (parseQuery — the SAME primitive the C1 resolver
+// already parses every request with, see resolver.mjs mapParse) as a structural
+// relevance check, never a new keyword table: does the request even COMPILE to a
+// recognized graph-query shape naming a known entity kind, and does that kind
+// match the rule's declared focusClass? A request parseQuery cannot place at all
+// (null — "write a haiku about pizza") or places without landing on any
+// recognized entity kind (a miss with no entity kind — "how many pizzas are
+// there") is an honest "not about this graph" signal; a request parseQuery
+// resolves to a real AST naming the rule's focus class ("which module is the
+// biggest testing risk" -> {node:"superlative", entityType:"Module", ...})
+// stays exactly as reachable as before. Zero request keywords added.
 
 import { backwardChain, extractEntity } from "./resolver.mjs";
 import { capabilityByName, effectsOf } from "./registry.mjs";
 import { hallucinationsIn } from "./call-validator.mjs";
 import { intersect } from "./set-algebra.mjs";
+import { parseQuery } from "../ask.mjs";
 
 // Hard OUTER-tick budget — the meta-loop runs at most this many ticks, then
 // REFUSES (escalate). Independent of BDI convergence and of the monotone
@@ -178,6 +199,25 @@ function focusOf(request, ctx) {
   return r && r.match && !r.ambiguous ? r.match : null;
 }
 
+/** The GLOBAL-MODE DOMAIN GATE's primitive: what entity CLASS (if any) did
+ *  ask.mjs's own compositional NL grammar recognize in the request? Walks
+ *  parseQuery's AST (the same shapes resolver.mjs's mapParse/mapFrame already
+ *  consume) for its declared `entityType` field, unwrapping the wrapper nodes
+ *  (`clause`, `inner`, `base`) that carry no entityType of their own. Returns the
+ *  class name, or null when the grammar placed nothing (an outright non-parse) or
+ *  placed a MISS with no recognized entity kind at all ("how many pizzas are
+ *  there" -> {node:"miss", reason:"count needs a known entity kind..."} carries no
+ *  entityType, same as a flat null). Pure; no request-string keyword table — it
+ *  reads a field ask.mjs's grammar already computes for every request. */
+function parsedEntityType(node) {
+  if (!node || typeof node !== "object") return null;
+  if (typeof node.entityType === "string") return node.entityType;
+  if (node.clause) return parsedEntityType(node.clause);
+  if (node.inner) return parsedEntityType(node.inner);
+  if (node.base) return parsedEntityType(node.base);
+  return null;
+}
+
 /** Ground ONE epistemic sub-goal (a topic + optional bound entity) into a
  *  grounded, EXECUTED call, or null when it is not groundable in the declared
  *  toolset (=> the meta-loop escalates). Backward-chains topic->capability, binds
@@ -237,13 +277,31 @@ export async function goalReason(request, tools, ctx, { driver = "goal-0.8.1" }
   // request's scope in this toolset); >1 => an AMBIGUOUS meta-goal (arbitration
   // between meta-goals is undeclared) — both are honest refusals, never a guess.
   const applicable = applicableRules(declared, focus, mode);
-  if (!applicable.length) {
+
+  // THE GLOBAL-MODE DOMAIN GATE (Bug 8 fix, see the module header). SCOPED mode
+  // already proved relevance via a bound graph entity; GLOBAL mode has not, so
+  // `applicable` alone (a pure function of the caller's DECLARED TOOLSET) is not
+  // enough — it says nothing about whether THIS request is even about the graph.
+  // Screen it against ask.mjs's own NL grammar: the request must parse to a shape
+  // naming the candidate rule's declared focusClass, or it is refused as honestly
+  // off-domain rather than answered with someone else's goal.
+  let domainRelevant = applicable;
+  if (mode === "global" && applicable.length) {
+    const requestClass = parsedEntityType(parseQuery(request));
+    domainRelevant = applicable.filter((r) => requestClass === r.focusClass);
+    if (!domainRelevant.length) {
+      const classes = [...new Set(applicable.map((r) => r.focusClass))].join("/");
+      return refuse(`open-world: the request does not parse as a query about ${classes} (ask.mjs's own NL grammar names ${requestClass || "no recognized entity kind"} in it) — global goal deduction needs the REQUEST ITSELF to be about the deduced goal's domain, not just a declared toolset that happens to ground it — escalate`, driver);
+    }
+  }
+
+  if (!domainRelevant.length) {
     return refuse(`open-world: no declared goal-rule is applicable in ${mode} mode (each needs a sub-goal capability outside the declared toolset, or a scope it does not declare) — escalate`, driver);
   }
-  if (applicable.length > 1) {
-    return refuse(`ambiguous meta-goal: ${applicable.length} declared goal-rules apply (${applicable.map((r) => r.id).join(", ")}) — meta-goal arbitration is undeclared, refuse rather than guess — escalate`, driver);
+  if (domainRelevant.length > 1) {
+    return refuse(`ambiguous meta-goal: ${domainRelevant.length} declared goal-rules apply (${domainRelevant.map((r) => r.id).join(", ")}) — meta-goal arbitration is undeclared, refuse rather than guess — escalate`, driver);
   }
-  const rule = applicable[0];
+  const rule = domainRelevant[0];
 
   // the glass-box WHY, citing the declared goal-rule by backward-chain (the C2
   // twin of resolver.mjs's "backward-chain => <capability>" provenance).