@polycode-projects/the-mechanical-code-talker 0.9.11 → 1.0.0

package/ROADMAP.md

@@ -890,6 +890,21 @@ stop signs (full detail + full citation lists in each owning doc):
   nobody has published the specific combination with tmct's multi-trust-tier, hard-budget
   requirement. Speculative angle: an ATMS-lite extension to `syllogise.mjs`'s currently-flat
   provenance tag, sketched but unbuilt.
+- **A shared ~2M-word cross-domain ontology (1M general-English base + 1M
+  technical/scientific/engineering/programming-language/slang)** — `PLAN_ontology-hierarchies.md`
+  §7, additive to (not a revision of) that doc's existing track (e), which stays about importing
+  raw WordNet into tmct's own small tier-1 corpus specifically. Walked into, not avoided: merging
+  two 1M-word vocabularies collides senses of lexically-shared words (`class`, `cache`, `thread`,
+  `wave`, `cell`, `field`, `state`, …) across general/CS/physics/biology/slang registers —
+  knowledge-based (non-neural) WSD is real but measurably weaker than supervised/neural WSD (Lesk
+  1986; Raganato, Camacho-Collados & Navigli, EACL 2017), and BabelNet (Navigli & Ponzetto,
+  *Artificial Intelligence* 193, 2012) proves automatic cross-resource sense merging at this scale
+  is achievable — but its own pipeline moved toward statistical/graph-ML methods as it scaled,
+  solves the cross-*lingual* not cross-*domain* axis, and carries a non-commercial licence, so it
+  is a precedent, not a usable vehicle. Speculative angle recorded: mutual disambiguation from
+  already-resolved neighbouring terms in tmct's own closed graph (a structurally-bounded,
+  deterministic reading of Gale/Church/Yarowsky's "one sense per discourse/collocation"
+  regularities) — not published anywhere found for this application.
 
 Every item above is honestly labeled speculative — a direction recorded so it isn't
 re-discovered from scratch, not a committed build plan. None of it is scheduled; the phases above

package/corpus/seon/relations.jsonl

@@ -6,3 +6,4 @@
 {"relation":"defines","definition":"A definition is where a name (a class, function, or variable) is introduced.","sense":"software"}
 {"relation":"touches","definition":"A touch is a commit changing a file or a symbol in the codebase.","sense":"software"}
 {"relation":"cochange","definition":"Change-coupling is two files that tend to be changed together in the same commits.","sense":"software"}
+{"relation":"reexports","definition":"A re-export is a module passing another module's definition through as part of its own public API.","sense":"software"}

package/data/templates/responses.jsonl

@@ -60,10 +60,12 @@
 {"id":"conversational-greeting-good-evening","class":"conversational","register":"friendly","template":"Good evening. Ask me about this codebase, or /help."}
 {"id":"conversational-thanks","class":"conversational","register":"friendly","template":"Any time. Ask another, or /help for what I can do."}
 {"id":"conversational-farewell","class":"conversational","register":"friendly","template":"Bye — flushing the session log. Come back with a question any time."}
-{"id":"orientation-friendly","class":"orientation","register":"friendly","template":"I answer questions about THIS codebase's structure — imports, calls, definitions,\nhistory and counts. For example:\n  which modules import {example1}\n  what calls {example2}\n  how many classes are there\n/help for commands, /stats for an overview of the graph."}
+{"id":"orientation-friendly","class":"orientation","register":"friendly","template":"I'm tmct — a deterministic, offline code-graph assistant (no LLM). I answer questions about THIS codebase's structure — imports, calls, definitions,\nhistory and counts. For example:\n  which modules import {example1}\n  what calls {example2}\n  how many classes are there\n/help for commands, /stats for an overview of the graph."}
 {"id":"miss-no-previous-answer","class":"miss","register":"friendly","template":"No previous answer to expand yet — ask me a question first, then say \"why\" or \"say more\"."}
-{"id":"conversational-greeting-empty","class":"conversational","register":"friendly","template":"Hi. There's no code graph loaded here — for code structure (imports, calls, definitions) I need a `.tmct/graph.json`: point me at one with `--repo <path>`, or try the shipped example `npm run example:mini`. (tmct reads graphs; it doesn't index code itself.) For general vocabulary, `tmct init` seeds concepts — try \"what is a cache\". /help for commands."}
-{"id":"orientation-empty","class":"orientation","register":"friendly","template":"There's no code graph loaded here, so I can't answer structure questions (imports, calls, definitions) yet.\nFor those I need a `.tmct/graph.json` produced by a graph producer — point me at one with `--repo <path>`, or try the shipped example `npm run example:mini`. tmct reads graphs; it doesn't index code itself.\nFor general vocabulary, `tmct init` seeds concepts — try \"what is a cache\". /help for commands, /memory for what I remember."}
+{"id":"conversational-greeting-empty","class":"conversational","register":"friendly","template":"Hi. I'm tmct. {vocabHint} Point me at a repo with `--repo <path>` for code-structure questions too (imports, calls, definitions). /help for commands."}
+{"id":"orientation-empty","class":"orientation","register":"friendly","template":"I'm tmct — a deterministic, offline chat assistant (no LLM). {vocabHint} /memory for what I remember.\nFor code structure (imports, calls, definitions) point me at a repo: `--repo <path>`, or try the shipped example `npm run example:mini`. tmct reads graphs; it doesn't index code itself.\n/help for commands."}
+{"id":"identity-self","class":"conversational","register":"friendly","template":"I'm tmct — a deterministic, offline chat assistant. No LLM: wink-nlp parsing over a seeded ontology/lexicon, plus a code graph when you point me at a repo with `--repo <path>`. /help for commands, /stats for an overview."}
+{"id":"identity-not-an-llm","class":"conversational","register":"friendly","template":"No — no LLM involved. tmct is deterministic: wink-nlp parsing over a graph/ontology, not a language model. /help for commands."}
 {"id":"technical-density","class":"count","register":"technical","template":"{subject} carries {count} {noun} across {scope} — a concentration well above what a codebase of this size typically sustains ({provenance})."}
 {"id":"technical-comparison","class":"count","register":"technical","template":"At {count} {noun}, {subject} sits {comparison} the comparable-project baseline, a divergence that reflects deliberate structure rather than measurement noise ({provenance})."}
 {"id":"technical-superlative","class":"count","register":"technical","template":"No {noun} in {scope} is more {metric} than {subject}; it leads the next candidate by a clear margin of {count} ({provenance})."}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@polycode-projects/the-mechanical-code-talker",
-  "version": "0.9.11",
+  "version": "1.0.0",
   "private": false,
   "type": "module",
   "description": "The Mechanical Code Talker (tmct) — a tolerant, offline, $0 chat surface that guides you toward precision queries about a software repository. ELIZA/PARRY-style but domain-obsessed with code. No model calls; no codebase index of its own.",

package/src/ask-vocab.mjs

@@ -316,6 +316,10 @@ export const MISSPELLINGS = Object.freeze({
   "extands": "extends", "extneds": "extends",
   "depnds": "depends",
   "touchs": "touches", "tuoches": "touches", "touhced": "touched",
+  // WHERE_MARKERS typo (0.9.13 Tier-1 playtest): "defined" itself had no typo
+  // entry, so "where is it defned" fell through to the bare-object search path
+  // instead of the where-shape ("no module matching 'it defned' found").
+  "defned": "defined",
   "chagned": "changed", "chnaged": "changed",
   "chagnes": "changes", "chnages": "changes",
   "calles": "calls",
@@ -323,6 +327,16 @@ export const MISSPELLINGS = Object.freeze({
   "tets": "tests",
   // grammar anchor words
   "whcih": "which", "wich": "which", "whihc": "which",
+  // "wehre"/"whre" (0.9.13 Tier-1 playtest, "where is it defined" drill-down):
+  // the WHERE-DEFINED shape's own anchor word had NO typo tolerance at all
+  // (unlike which/what/does/the above), so a plain dropped/transposed letter
+  // fell straight through resolveObject and hit either the grammar wall or a
+  // bogus "no module matching 'it defined'" search. "were" (the missing-h
+  // homophone slip) is NOT curated here — it's a real word already load-bearing
+  // as the TEMPORAL_AUX auxiliary ("when were the modules last touched"), so
+  // that one typo is handled by its own anchored phrasing frame instead
+  // (normalize.mjs PHRASING_FRAMES) to avoid clobbering the legitimate reading.
+  "wehre": "where", "whre": "where",
   // "wat" (chatbench cycle 2, tf-wat-calls): the internet-casual spelling of
   // "what" — neither curated noise nor a restorable trigger typo, so "wat calls
   // fnAlpha" used to die as "couldn't resolve one of the terms". Restored here
@@ -381,12 +395,21 @@ export const G_DROP = /\b([a-z]{3,})in'/gi;
 /** Words stripped during normalization/keyword-spotting once they carry no
  *  grammatical weight for this grammar — greetings, politeness, hedges, and
  *  the discourse fillers a spoken-style question picks up. Never strips a
- *  relation verb, entity noun, or modifier (those are checked first). */
+ *  relation verb, entity noun, or modifier (those are checked first).
+ *  Bare "you" (0.9.14 Tier-2 playtest, §3b ESL angle): a non-native word-order
+ *  slip ("please you tell me what is Class", "you tell me what is Class") left
+ *  a leading "you" that "could you"/"can you"/"would you" don't cover (those
+ *  are anchored WRAPPERS requiring the verb-first order) — the leftover
+ *  pronoun broke the bare "what is Class" no-article count reading, which
+ *  (unlike the "what is a Class" meta form) requires the WHOLE normalized
+ *  string to match, not just a substring. "you" carries no grammatical weight
+ *  in this code-graph grammar (never a real entity/relation term), so it is
+ *  safe to strip anywhere, same trade as every other word in this list. */
 export const FILLER_WORDS = Object.freeze([
   "um", "uh", "erm", "so", "like", "yo", "hey", "bru", "bro", "fam", "mate",
   "please", "could you", "can you", "would you", "tell me", "i wonder",
   "just wondering", "quickly", "real quick", "kinda", "sorta",
-  "btw", "by the way",
+  "btw", "by the way", "you",
 ]);
 
 /** Deictic/pronoun terms that refer to a context entity rather than naming one

package/src/ask.mjs

@@ -280,7 +280,16 @@ const MAX_COMPOSE_DEPTH = 4;
 // treats it as an ordinary object term rather than dropping it.
 const NEST_SENTINEL = "zzinnerset";
 // Filler words dropped at the front of a relative predicate / anaphora filter.
-const PRED_LEAD_SKIP = new Set(["that", "which", "who", "are", "is", "was", "were", "do", "does", "also", "still", "both", "and"]);
+// "then"/"though" (Tier-2 playtest, 5th pass): a trailing discourse tag on an
+// otherwise-bare anaphora follow-up — "how many of those THEN", "which of
+// them THOUGH" — used to be read as an (uncompilable) filter clause instead
+// of being dropped as filler, so the follow-up MISSED at PARSE time with a
+// generic "the follow-up filter didn't parse" instead of reaching the
+// friendly eval-time "needs a previous answer" nudge when there was truly no
+// prior set (or the correct count/list when there was one) — the exact same
+// discourse-tag tolerance WHAT_ABOUT_RE already carries for "what about X
+// then"/"what about X though".
+const PRED_LEAD_SKIP = new Set(["that", "which", "who", "are", "is", "was", "were", "do", "does", "also", "still", "both", "and", "then", "though"]);
 const FRAME_WORDS = new Set(["which", "what", "who", "list", "show", "find", "give", "me", "us", "all"]);
 
 const entityNoun = (w) => (ENTITY_TO_TYPE[w] ? { entityType: ENTITY_TO_TYPE[w], placeholder: false }
@@ -519,6 +528,19 @@ function parseTemporal(w, lc, nlp, depth = 0) {
  *  fires). Returns a {node:"anaphora"} (mode count|list), a miss (filter present but
  *  uncompilable), or null. */
 function parseAnaphora(w, lc, nlp) {
+  // "which ones"/"which one" — a bare anaphoric re-LIST of the previous result set,
+  // phrased as a QUESTION rather than the imperative ("list them") or pronoun-tail
+  // ("count them") shapes the loop below already covers. Found live (0.9.14 Tier-2
+  // playtest, third pass, numeric/quantifier relation touches): after "how many
+  // modules import app/lib/a.mjs" / "which modules import app/lib/a.mjs", the
+  // completely natural follow-up "which ones" fell straight through to the generic
+  // orientation card — "ones" isn't an ANAPHORA_TRIGGERS pronoun and bare "which"
+  // isn't a LIST_TRIGGERS head, so neither existing branch below ever fires for it.
+  // Pinned to the WHOLE query (exactly two words) so it never shadows an ordinary
+  // "which one of these two functions …" clause, which has more words after "one".
+  if (lc.length === 2 && lc[0] === "which" && (lc[1] === "ones" || lc[1] === "one")) {
+    return { node: "anaphora", mode: "list", filter: { type: "all" } };
+  }
   let p = -1;
   let viaOf = false;
   for (let i = 1; i < lc.length; i += 1) {
@@ -564,10 +586,16 @@ function parsePredicateFilter(words, nlp) {
  *  and it must NOT be mistaken for a restrictor (that's the exact bug behind "how many
  *  classes are there" → the count-restrictor miss). Combined with STOPWORDS (which
  *  already carries are/there/is/in/the/…) at the call site, so only the non-stopword
- *  extras live here. A tail with ANY word outside this ∪ STOPWORDS is a real restrictor. */
+ *  extras live here. A tail with ANY word outside this ∪ STOPWORDS is a real restrictor.
+ *  "this"/"that" (0.9.14 Tier-2 playtest): "which classes exist IN THIS codebase"/
+ *  "which methods exist in this codebase" used to miss — "codebase" alone was already
+ *  filler, but the demonstrative right before it ("this"/"that") is not a STOPWORDS
+ *  entry, so the tail read as non-filler and the whole thing fell through to the
+ *  grammar wall instead of the bare list. */
 const AGG_TAIL_FILLER = new Set([
   "total", "altogether", "overall", "exist", "exists", "existing", "present",
   "here", "now", "currently", "graph", "index", "codebase", "repo", "repository",
+  "this", "that",
 ]);
 
 /** AGGREGATE / COUNT: "how many <entity> [<restrictor>]", "count <entity>",
@@ -605,6 +633,12 @@ const LIST_SKIP = new Set(["the", "a", "an", "all", "me", "us"]);
 const LIST_TRIGGERS_SORTED = [...LIST_TRIGGERS].sort((a, b) => b.split(" ").length - a.split(" ").length);
 // The listable node classes, named in the honest miss and the empty-index message.
 const LISTABLE_KINDS = "functions, classes, methods, modules, attributes, variables, or commits";
+// SCOPE PREPOSITIONS (HANDOVER item 12.2/11.1): a leading "in"/"inside"/"under" right
+// after the entity noun (past an optional copula) is an unambiguous LOCATION-SCOPE
+// tail, never a reverse-clause predicate object — "which modules import X" has no
+// preposition there at all. Narrowly scoped to just these three words so the
+// interrogative exception below can't be mistaken for a general tail-acceptance.
+const SCOPE_PREPOSITIONS = new Set(["in", "inside", "under"]);
 
 /** LIST: "list <kind>", "show me the <kind>s", "what are the <kind>", "list <kind> in
  *  <module>". A sibling of the count node — it enumerates the individuals of a class
@@ -616,7 +650,14 @@ const LISTABLE_KINDS = "functions, classes, methods, modules, attributes, variab
  *  (reusing membership/relational/boolean), and its `scoped` flag suppresses the
  *  "narrow with …" hint. An unknown kind after a clear imperative trigger ("list
  *  bananas") is an honest miss naming the listable kinds; anything less certain falls
- *  through (null) to the existing parser/cascade rather than guessing. */
+ *  through (null) to the existing parser/cascade rather than guessing.
+ *
+ *  A single narrowly-scoped EXCEPTION to the interrogative gate (HANDOVER item
+ *  12.2/11.1): "what/which <kind> [is|are] in|inside|under <scope>" — a leading scope
+ *  preposition, past an optional copula, straight after the entity noun — is routed the
+ *  SAME way the imperative "list <kind> in <scope>" already is. This does NOT widen the
+ *  general decline: "which modules import X"/"which functions call X" have a VERB there,
+ *  not a scope preposition, so they still fall through untouched. */
 function parseList(w, lc, nlp, depth) {
   let i = 0;
   let interrogative = false;
@@ -650,18 +691,33 @@ function parseList(w, lc, nlp, depth) {
   i += 1;
   const tail = w.slice(i);
   const tailMeaningful = lc.slice(i).some((t) => !STOPWORDS.has(t) && !AGG_TAIL_FILLER.has(t));
+  // SCOPE-PREPOSITION exception (HANDOVER 12.2/11.1): past an optional copula
+  // ("is"/"are"), does the tail open with "in"/"inside"/"under"? If so this is a
+  // location-scope tail ("what modules ARE IN app/lib"), not a reverse-clause
+  // predicate — strip only the copula (the imperative form never has one: "list
+  // modules in app/lib") so the membership-tail text this builds below is byte-
+  // identical in shape to what the imperative path already builds.
+  let scopeTailLc = lc.slice(i);
+  let scopeTailWords = tail;
+  if (scopeTailLc[0] === "is" || scopeTailLc[0] === "are") {
+    scopeTailLc = scopeTailLc.slice(1);
+    scopeTailWords = scopeTailWords.slice(1);
+  }
+  const scopedException = interrogative && SCOPE_PREPOSITIONS.has(scopeTailLc[0]);
   // The bare interrogative "what/which <kind>" is a list ONLY with an explicit
-  // list-confirming filler tail ("… are there", "… that exist"): a real predicate
-  // ("which functions call X") is a reverse query, and a *bare* "which methods" is left
-  // alone deliberately — otherwise the relaxation cascade could drop an unknown
-  // qualifier ("which shiny methods" → "which methods") and silently list everything,
-  // erasing the honest "unknown qualifier" miss. Imperative triggers ("list methods")
-  // carry their own list intent, so they need no such tail.
-  if (interrogative && (tailMeaningful || tail.length === 0)) return null;
+  // list-confirming filler tail ("… are there", "… that exist") or the scoped
+  // exception just above: a real predicate ("which functions call X") is a reverse
+  // query, and a *bare* "which methods" is left alone deliberately — otherwise the
+  // relaxation cascade could drop an unknown qualifier ("which shiny methods" →
+  // "which methods") and silently list everything, erasing the honest "unknown
+  // qualifier" miss. Imperative triggers ("list methods") carry their own list
+  // intent, so they need no such tail.
+  if (interrogative && ((tailMeaningful && !scopedException) || tail.length === 0)) return null;
   let base;
   let scoped = false;
   if (tailMeaningful) {
-    const setAst = parseSetPhrase(`which ${[...quals, entWord, ...tail].join(" ")}`, nlp, (depth || 0) + 1);
+    const useTail = scopedException ? scopeTailWords : tail;
+    const setAst = parseSetPhrase(`which ${[...quals, entWord, ...useTail].join(" ")}`, nlp, (depth || 0) + 1);
     if (!setAst || setAst.node === "miss") return { node: "miss", reason: (setAst && setAst.reason) || "the list filter didn't parse" };
     base = setAst;
     scoped = true;