@polycode-projects/the-mechanical-code-talker 0.2.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (53) hide show
  1. package/README.md +77 -3
  2. package/ROADMAP.md +416 -3
  3. package/bin/tmct.mjs +308 -12
  4. package/corpus/README.md +52 -0
  5. package/corpus/conceptnet/LICENSE-NOTICE +37 -0
  6. package/corpus/conceptnet/README.md +103 -0
  7. package/corpus/conceptnet/fetch-slice.mjs +136 -0
  8. package/corpus/conceptnet/filter-dump.mjs +89 -0
  9. package/corpus/conceptnet/slice.jsonl +14258 -0
  10. package/data/phrasebook/software-phrases.txt +231 -0
  11. package/data/templates/grammar-rules.toml +89 -0
  12. package/data/templates/responses.jsonl +68 -0
  13. package/package.json +40 -3
  14. package/src/ask-nlp.mjs +22 -10
  15. package/src/ask-vocab.mjs +35 -1
  16. package/src/ask.mjs +171 -494
  17. package/src/chat.mjs +709 -81
  18. package/src/corpus/conceptnet-map.toml +251 -0
  19. package/src/corpus/conceptnet.mjs +167 -0
  20. package/src/corpus/templates.mjs +188 -0
  21. package/src/finish.mjs +443 -0
  22. package/src/grammar/ace.mjs +341 -0
  23. package/src/grammar/assert.mjs +40 -0
  24. package/src/grammar/lexicon-core.json +287 -0
  25. package/src/grammar/lexicon.mjs +202 -0
  26. package/src/hash.mjs +32 -0
  27. package/src/index.mjs +21 -5
  28. package/src/init.mjs +264 -0
  29. package/src/interpret/fuzzy.mjs +89 -0
  30. package/src/interpret/merge.mjs +148 -0
  31. package/src/interpret/normalize.mjs +151 -0
  32. package/src/interpret/pipeline.mjs +112 -0
  33. package/src/interpret/strategies/grammar.mjs +137 -0
  34. package/src/interpret/strategies/keywords.mjs +241 -0
  35. package/src/interpret/strategies/noise-strip.mjs +114 -0
  36. package/src/memory/blocks.mjs +221 -0
  37. package/src/memory/core.mjs +533 -0
  38. package/src/memory/fold.mjs +0 -0
  39. package/src/memory/inspect.mjs +141 -0
  40. package/src/memory/trust.mjs +113 -0
  41. package/src/prose-nlp.mjs +14 -16
  42. package/src/providers/bootstrap.mjs +24 -0
  43. package/src/providers/fixture.mjs +118 -0
  44. package/src/providers/graph-service.mjs +312 -0
  45. package/src/repository-interface.mjs +318 -0
  46. package/src/server.mjs +44 -28
  47. package/src/sessions.mjs +137 -4
  48. package/src/source.mjs +44 -5
  49. package/src/syllogise.mjs +0 -0
  50. package/src/toml-config.mjs +14 -0
  51. package/src/tui/app.mjs +173 -0
  52. package/src/wink-model.mjs +74 -0
  53. package/bin/cli.mjs +0 -226
package/README.md CHANGED
@@ -36,6 +36,20 @@ statements it can store, retrieve, and answer from later. Text that doesn't
36
36
  fit the grammar still gets the tolerant strategies; nothing is rejected for
37
37
  being loose, fuzzy, or misspelled.
38
38
 
39
+ **Negation and passive.** "Which modules do *not* import X?" computes a bounded
40
+ **set complement** over the graph — and an honestly empty result stays a miss
41
+ rather than a fabricated list (a non-enumerable type, like arbitrary *changes*,
42
+ is refused outright). Reversible-passive questions traverse the right direction:
43
+ "what is imported by Y" and "what does Y import" are understood as opposite
44
+ edges, not the same one.
45
+
46
+ **Response finishing.** Before an answer is printed it is segmented into typed
47
+ spans — prose versus *protected* entities, paths, numbers, code, provenance, and
48
+ receipts — and a small data-driven grammar pass runs on the prose spans only,
49
+ under a guard that proves the protected spans came through byte-for-byte. Today
50
+ that pass fixes the a/an article defect; broader voice and agreement rules are
51
+ implemented but parked until they earn their place on the benchmark.
52
+
39
53
  ## How it remembers
40
54
 
41
55
  tmct's memory has two layers, both fed by every parsed request and response and
@@ -51,6 +65,29 @@ With no graph at all, tmct starts empty and remembers what you tell it — the
51
65
  vocabulary; a filtered **ConceptNet slice** (CC-BY-SA 4.0) is planned — see
52
66
  `ROADMAP.md` Phase 2.
53
67
 
68
+ ### Provenance and trust
69
+
70
+ Every fact and text block records **where it came from and when**. Sources are
71
+ first-class individuals — operator chat, a curated corpus, a provider graph, a
72
+ web scrape, a rule-derived entailment — and a fact links back to *all* of them
73
+ (`mgx:derivedFrom` / `mgx:statedBy` / `mgx:canonicalisedFrom`), timestamped with
74
+ `mgx:createdAt`. From those links tmct computes a **deterministic, explainable
75
+ trust score** — a source-type prior combined with corroboration (how many
76
+ independent sources agree) and recency — never hand-set, always traceable to its
77
+ inputs. Retrieval then ranks by **relevance × trust**, so a corroborated,
78
+ operator-stated fact outranks a lone web scrape on the same question. When two
79
+ trusted sources *disagree*, the `/memory` inspector shows **both sides with their
80
+ provenance** rather than silently picking a winner.
81
+
82
+ ### Speculative inference (a maintenance job, not a chat cost)
83
+
84
+ `tmct syllogise [--depth n] [--budget n]` is an offline, bounded, deterministic
85
+ batch that forward-chains the memory's `rdfs:subClassOf` closure into new
86
+ **entailed** facts — pre-deriving what the trusted sources already imply. It runs
87
+ once automatically after seeding and on demand; the entailed facts are
88
+ **low-trust and retractable** (never outranking a stated fact) and this never runs
89
+ on the chat's hot path.
90
+
54
91
  ## What tmct deliberately is NOT
55
92
 
56
93
  - **It is not an indexer.** tmct keeps no codebase index of its own. It
@@ -58,7 +95,9 @@ vocabulary; a filtered **ConceptNet slice** (CC-BY-SA 4.0) is planned — see
58
95
  a code graph is out of scope. tmct's job is the *conversation*.
59
96
  - **It is not a reasoning model.** Where it "reasons", it does so by
60
97
  *calculation* surfaced as prose ("there are a lot of tests for a codebase of
61
- that size") — deterministic, explainable, cheap. There is **no LLM anywhere
98
+ that size") — deterministic, explainable, cheap. Even its forward-chaining
99
+ entailment (`tmct syllogise`) is mechanical OWL rule materialization applied
100
+ offline, rule-by-rule and retractable — not an LLM. There is **no LLM anywhere
62
101
  in the product**. (An LLM-as-judge exists only in the offline eval harness
63
102
  that tunes tmct — see `SKILL_TUNING_CYCLE.md` — never in the product path.)
64
103
  - **It never guesses silently.** When it cannot resolve your question it says
@@ -70,10 +109,18 @@ vocabulary; a filtered **ConceptNet slice** (CC-BY-SA 4.0) is planned — see
70
109
  npm install -g @polycode-projects/the-mechanical-code-talker
71
110
  tmct # bare = chat (the headline)
72
111
  tmct chat --repo /abs/path/to/repo # chat over a specific repo's graph
112
+ tmct init # scaffold .tmct/, tmct.toml, seed + provenance
113
+ tmct syllogise # offline: pre-derive entailed facts (maintenance)
73
114
  ```
74
115
 
75
- Inside the chat: `/help` lists commands, `/exit` leaves. `TMCT_GRAPH_FILE`
76
- overrides the graph location.
116
+ Inside the chat: `/help` lists commands, `/memory` inspects what tmct remembers
117
+ (grouped by OWL class, with provenance and any contradictions), `/exit` leaves.
118
+ `TMCT_GRAPH_FILE` overrides the graph location.
119
+
120
+ `tmct init` is the onboarding surface for the repository interface below: it
121
+ creates the `.tmct/` directory, writes the externalized `tmct.toml`
122
+ configuration, seeds the tier-1 corpus, and records provenance — a host package
123
+ or a bare user gets a working install in one command.
77
124
 
78
125
  > Install-size note: tmct depends on wink-nlp's deterministic English language
79
126
  > model (~3.8 MB installed). That model is a lookup table, not an LLM.
@@ -88,6 +135,33 @@ The `exports` map and the chat primitives (`ask`, `resolveObject`,
88
135
  `relationKind`, `impactClosure`, `dispatchTool`, `fetchEntities`) are the
89
136
  extension surface.
90
137
 
138
+ ## The repository interface
139
+
140
+ tmct is not an indexer, so it consumes a graph through a typed contract any
141
+ producer can implement. That contract is now first-class: a **versioned
142
+ (1.0.0), OWL-grounded, machine-readable service definition**
143
+ (`docs/repository-interface.md` + a JSON schema) of every service, its
144
+ arguments, result types, and error contract. A **miss is a value, not a throw** —
145
+ the interface models "no answer" explicitly. tmct ships **reference providers**
146
+ (a fixture graph and the empty/bootstrap graph) that implement every service,
147
+ and a **runnable conformance suite**: tmct's own providers pass it in `npm test`,
148
+ and any external graph producer (seonix first) runs the *same* suite against its
149
+ native implementation to claim conformance. Conformance is the suite, not prose.
150
+ The design deliberately inverts the original relationship — tmct was lifted out
151
+ of seonix; now seonix reorients as a *user* that imports the tmct library and
152
+ exposes its graph to tmct as a service, with the LLM agent staying outside tmct
153
+ exactly as the no-LLM ethos requires.
154
+
155
+ ## Security and supply chain
156
+
157
+ tmct is $0 to run and meant to be trusted offline, so the supply chain is
158
+ hardened accordingly: CI runs **SAST and secret detection**, a **nightly
159
+ `npm audit` + OSV-Scanner** job watches dependencies, releases are published with
160
+ **npm provenance** (`--provenance`), and there is a coordinated-disclosure
161
+ `SECURITY.md` policy. The content-address hash is single-sourced in
162
+ `src/hash.mjs` so the cross-version-stable fact-id contract has exactly one
163
+ definition.
164
+
91
165
  ## Provenance
92
166
 
93
167
  tmct began as a whole-package lift of the seonix chat surface (v0.1.0, then
package/ROADMAP.md CHANGED
@@ -115,8 +115,11 @@ Keep the `exports` map and the primitives stable and documented as the
115
115
  internals are refactored.
116
116
 
117
117
  ### Shell work
118
- - **OpenTUI console shell** around `runTurn`; readline `runChat` stays as
119
- `--plain` and as the test surface.
118
+ - **Ink console shell** (`src/tui/app.mjs`, ink + react, no build step) around
119
+ the shared session sink; readline `runChat` stays as `--plain` and as the
120
+ test surface. *(Decision: OpenTUI ruled out — `@opentui/core` depends on Bun
121
+ FFI (`bun-ffi-structs`, native Zig renderer), not Node-clean; revisit when it
122
+ runs under plain Node.)*
120
123
  - Fold the surviving `bin/cli.mjs` arms into `bin/tmct.mjs`; delete `cli.mjs`.
121
124
 
122
125
  ---
@@ -170,6 +173,30 @@ in `SKILL_TUNING_CYCLE.md`:
170
173
  harness only** — the product stays no-LLM;
171
174
  - `CHATBENCH_0NN.md` artifacts and an autonomous cycle loop (no hard pause;
172
175
  each cycle logs its ranked decision menu and continues).
176
+ - **The graded benchmark** (case-set v2, operator-specified 2026-07-04): a
177
+ scaled ladder fitted to HUMAN LANGUAGE STANDARDS, not AI-benchmark mechanics
178
+ (bAbI explicitly rejected — it tests expected-AI mechanics and overfits the
179
+ same way our own cases would). Every case carries a **CEFR band (A1–C2)** +
180
+ a **construction specialization** tag (TROG-2/CELF-5 style blocks, adapted:
181
+ naming/vocabulary, SVO queries, pronoun binding, reversible/passive,
182
+ relative/embedded clauses, quantifiers+counting, negation, temporal,
183
+ coordination/compositional, multi-turn discourse reference, declarative
184
+ assert+recall). Multiple questions per grade × specialization with a little
185
+ overlap — authored as a POOL ~10× the per-run need (deterministic generator;
186
+ each run samples ~10% stratified, ≥5 items per populated cell, recorded
187
+ seed), with COMBINATION cells alongside single-area cells so weakness is
188
+ attributable to an area alone or to a specific pairing.
189
+ **Ladder gating:** advanced grades are SKIPPED until every grade
190
+ below passes reliably; when a grade reaches unit-test-level reliability its
191
+ cases become ALWAYS-RUN deterministic tests (judge-free, promoted alongside
192
+ test/showcase.test.mjs). Winograd/WinoGrande items stay as permanent
193
+ ceiling markers; CHILDES as naturalistic easy-band input source. Licence
194
+ rule: TROG/CELF are commercial — borrow the construction taxonomy and
195
+ grading structure, author ORIGINAL items, never copy instrument content.
196
+ A case at 0% is a ceiling marker, not a failure.
197
+ - **Retained showcase** (landed): the five most complex achieved sequences are
198
+ frozen as unit-timescale regression tests in `test/showcase.test.mjs`; a
199
+ showcase regression voids a cycle PASS regardless of the mean.
173
200
 
174
201
  Inside this loop, two earlier ambitions become **tuning levers** rather than
175
202
  standalone features:
@@ -183,7 +210,7 @@ calculations, not inference: deterministic, explainable, cheap.
183
210
  Let tmct run linters or tests to observe whether something actually worked,
184
211
  reporting the observation — measurement, not reasoning.
185
212
 
186
- ### Item 11 — Formal logical reasoning over the ontology (Prolog / Progol) — exploratory, gated
213
+ ### Item 11 — Formal logical reasoning over the ontology (Prolog / Progol) — exploratory, gated; matured into Phase LATER tier-5 "entailment-on-miss"
187
214
  Apply real rules of inference (modus tollens, etc.) to formulas extracted from
188
215
  parsed prose, checked against the axioms in the ontology (item 3) — a step
189
216
  beyond item 5's arithmetic. The sketch: map OWL constructs into templates over
@@ -199,6 +226,392 @@ then use Progol to theorem-prove against parsed prose".)*
199
226
 
200
227
  ---
201
228
 
229
+ ## Phase 4 — The wiring wave (operator-directed 2026-07-04)
230
+
231
+ Five subsystems are built, tested, and consumed by NOTHING in the answer path — they measured
232
+ zero on case-set v1 because no case could see them. This phase wires each into answering, with
233
+ **unit tests at the seam AND graded-benchmark cells that measure it** (the graded pool creates
234
+ the cases that make each lever visible). Wired as one operator-directed wave; cycle-level
235
+ attribution resumes per-lever afterwards.
236
+
237
+ | # | Wiring | Seam | Unit test | Bench coverage |
238
+ |---|---|---|---|---|
239
+ | W1 | **Templates → render path** | answer rendering consumes `data/templates/responses.jsonl` via `src/corpus/templates.mjs` instead of hardcoded strings (same output first — byte-stable swap — then variation) | render parity + slot lint | every existing case re-measures the swap; `via:"template"` provenance |
240
+ | W2 | **retrieveBlocks → miss path** | a bare-question miss consults the memory block index before the honest miss; a hit answers with the recalled block + provenance ("you asked this on …") | recall hit/miss seam | memory-recall cells (mr-asked-before flips) |
241
+ | W3 | **seedMemory → bootstrap** | first run in a graph-less repo seeds a capped corpus slice (limit ~500) into `.tmct/memory/`; banner says so honestly | seeded-bootstrap test | bootstrap-empty + vocabulary cells ("what is a cache?") |
242
+ | W4 | **Asserted Facts → answers** | "what is a module?" / "is a module a component?" consults remembered `rdfs:subClassOf`/`rdf:type` facts alongside the code graph, cited with provenance | fact-lookup seam | assert-recall cells |
243
+ | W5 | **Corpus on-demand** | unknown-term misses may consult the corpus slice (local first; network tier only behind an explicit flag) | on-demand seam, offline-degrades test | naming-vocabulary cells at higher grades |
244
+
245
+ Answer-path **provenance** lands with W1: every turn record carries `via`
246
+ (composed | template | count | recall | fact | corpus) — the field the dual-banding
247
+ benchmark (Phase 5) and the memory inspector read.
248
+
249
+ ### Corpus tiering policy (the committed/seeded/on-demand cutoff)
250
+
251
+ - **Tier 1 — committed & shipped in the npm tarball**: small, load-bearing, licence-clean,
252
+ diffable — the lexicon, response templates, phrasebook, the relation→OWL map, and the CORE
253
+ ConceptNet slice. Budget: **~2 MB total tarball**; rule: what the product needs to be useful
254
+ offline out of the box.
255
+ - **Tier 2 — fetched at seed time into install-local folders** (`.tmct/corpus/` per repo, or a
256
+ user-level cache): growable corpora — extended ConceptNet neighbourhoods, acquired template
257
+ libraries (Phase 5), any corpus > ~2 MB. Fetched once by `tmct seed` (or first bootstrap with
258
+ consent), checksummed, provenance-recorded, never committed.
259
+ - **Tier 3 — on-demand at question time**: unbounded/live sources (ConceptNet API for unknown
260
+ terms, paper phrase-mining), consulted ONLY behind an explicit opt-in flag, cached down into
261
+ tier 2 after use. **Network failure degrades to the honest miss** — the $0-offline default is
262
+ inviolable; tiers 2-3 are additive, never required.
263
+ - **Tier 4 — learn-on-miss (Phase LATER, not now)**: the term IS in the lexicon and the query
264
+ built cleanly, but the graph+memory+corpus all return nothing → web search, clean the fetched
265
+ text into tmct's own dialect (the ACE-ish controlled grammar), store on disk (tier 2), ingest,
266
+ THEN answer — the full acquisition loop at question time. See Phase LATER.
267
+
268
+ ### Memory inspection (seeing into the memory)
269
+
270
+ Graph-vis exploration hasn't earned its keep; the in-ethos answer is TEXT. A `/memory` chat
271
+ command + `tmct memory` CLI: the memory graph grouped by **OWL superclass** (Utterance, Fact,
272
+ Session; code classes when present), counts per class with **balanced samples scaled to class
273
+ size** (log-scaled so a 10,000-fact class shows ~8 exemplars and a 3-session class shows all 3),
274
+ top facts ranked by provenance breadth (corpus+chat-agreed facts first), recent utterance pairs,
275
+ and the block-index summary (blocks, tokens, top PageRank blocks). Same renderer serves
276
+ `/stats`-style terse and `why`-style verbose.
277
+
278
+ ## Phase 5 — The cycle-4+ tuning arc (near-term: make the floor reliable)
279
+
280
+ > **STATUS: in progress — the two headline B1 levers shipped.** Negation as a bounded SET
281
+ > COMPLEMENT ("which X do not <verb> Y") and reversible-passive traversal ("X is imported by Y")
282
+ > are live; the harness meta-fixes and the rest of the ranked lever board continue.
283
+
284
+ > Detailed plans: **PLAN_CYCLE_4.md** (the tuning arc) and **PLAN_DEPENDENCY_STRATEGY.md**
285
+ > (the pre-build dependency audit — verdict: no dep changes now; a standing adoption
286
+ > register + avoid-list for phases 6-9; two near-term actions).
287
+
288
+ The immediate work: drive the graded benchmark up the CEFR ladder, one lever per cycle, per
289
+ `SKILL_TUNING_CYCLE.md`. Cycle 3 (post-wiring-wave, CHATBENCH_003) gave the first full-spectrum
290
+ reading — A1 1.72 / A2 1.70 / **B1 0.77 (the cliff)** / B2 0.97 / C1 1.07 / C2 0.69 — and two
291
+ META-fixes gate everything:
292
+
293
+ - **Meta-1 — fix the harness artifact BEFORE trusting groundedness.** The judge systematically
294
+ scores TRUTHFUL product output (/describe attributes, recall frames, session ids) as
295
+ fabrication because `FIXTURE_CONTEXT` omits the detail the product legitimately emits.
296
+ Measurement integrity: lands in its own cycle, re-measured, before any product lever — else
297
+ every groundedness delta is confounded. A harness correction, logged as such (like cycle-2's
298
+ H1a/H1b), not a product change.
299
+ - **Meta-2 — the ladder rule: get B1 reliable before judging C-grades.** Don't pay to judge a
300
+ ceiling while the floor leaks. A/B grades carry the judged spend; C1/C2 stay tier-1-only
301
+ ceiling markers, judged only occasionally to confirm they're still ceilings (the existing
302
+ `--ladder`/`--grade` flags).
303
+
304
+ Then the product levers, one per cycle, in ranked order: **(1) B1 negation operator** — the
305
+ deepest gap, the engine tokenizes "not"/"don't" as an entity ("no module matching 'not' found");
306
+ add set-complement to traversal; predicted B1 0.77→~1.05, ~10 hard fails cleared. **(2)
307
+ reversible-passive** ("is imported by" reverses edge direction). **(3) under-covered pool
308
+ growth** (B1 pronoun/temporal, C1 temporal — instrument fix, parallelizable). **(4)
309
+ assert-recall read-back**, **(5) quantifier+temporal composition**, **(6) the help-text honesty
310
+ leak** (hardcoded examples naming non-fixture entities — a real product fabrication, distinct
311
+ from meta-1's harness artifact), **C2 ceiling LAST**. Operator decision 2026-07-05: **do all of
312
+ them**. Exit criterion in the plan (roughly: B1 grade mean ≥ ~1.5 with all cells dual-draw
313
+ agreeing unlocks C-grade judging).
314
+
315
+ ## Near-term actions (from the dependency audit, PLAN_DEPENDENCY_STRATEGY.md)
316
+
317
+ > **STATUS: both shipped.** The wink browser-loader seam is added (shared model loader with a
318
+ > browser registration path) and `fnv1a` is single-sourced into `src/hash.mjs`.
319
+
320
+ Two concrete, low-risk actions the audit surfaced — not features, not dep changes:
321
+
322
+ 1. **Fix wink's `createRequire` browser-loader gap** — the wink model IS the browser build, but
323
+ our adapters load it via `createRequire(import.meta.url)` (`ask-nlp.mjs:29`, `prose-nlp.mjs:31`),
324
+ which is Node-only. Browser mode needs a bundler `import` path. **A Phase 8 (browser-mode)
325
+ blocker** — budget it into `PLAN_REPOSITORY_INTERFACE.md`; it is a wiring fix, not a dependency
326
+ change (the model is already browser-capable).
327
+ 2. **Single-source `fnv1a`** — extract the content-address hash to one `src/hash.mjs` so the
328
+ cross-version-stable fact-id contract has a single definition. Trivial refactor, do any time;
329
+ no dependency (the audit confirmed home-grown FNV-1a is the correct choice — sync + browser +
330
+ version-stable, which every library candidate fails).
331
+
332
+ ## Provenance & trust — the unified source-link primitive (cross-cutting)
333
+
334
+ > **STATUS: shipped.** `mgx:createdAt` universal; `Source` first-class individuals linked by
335
+ > `mgx:derivedFrom` / `mgx:statedBy` / `mgx:canonicalisedFrom`; a deterministic `computeTrust`
336
+ > (source-type prior × corroboration × recency); retrieval weighted by relevance × trust; the
337
+ > `/memory` inspector surfaces contradictions with provenance. Legacy `mgx:factProvenance` kept
338
+ > as a compat shim.
339
+
340
+ > Detailed plan: **PLAN_PROVENANCE_TRUST.md**.
341
+
342
+ *(Operator-specified 2026-07-05, from the observation that Phase-6 canonicalise-and-link,
343
+ tier-4 learn-on-miss, and the ConceptNet slice all share one shape: raw source preserved,
344
+ derived form linked back.)* Promote that shape to a FIRST-CLASS primitive used everywhere a
345
+ fact enters memory:
346
+
347
+ - **Every fact/block carries a `Source` and a link to it** — one predicate family
348
+ (`mgx:derivedFrom` / `mgx:canonicalisedFrom` / `mgx:statedBy`) instead of the current
349
+ per-writer `mgx:factProvenance` string. Sources are first-class individuals (class `Source`:
350
+ operator-chat, corpus:conceptnet, learned:web:<url>, entailed:<rule>, provider:seonix), so a
351
+ fact can cite MANY sources (the existing "|"-union becomes real edges).
352
+ - **Everything created is TIMESTAMPED** (`mgx:createdAt`), universally — Facts don't carry one
353
+ today (only Utterances do), a Phase-6-trust gap to close: recency is a trust input and the
354
+ novelty signal (below) needs it. Backfill on write; the timestamp is itself provenance.
355
+ - **Calculable trust scores per source**, deterministic and explainable: a source-type prior
356
+ (operator > provider graph > curated corpus > web > unverified entailment) combined with
357
+ corroboration (how many independent sources assert the same fact — the union already tells us)
358
+ and recency/agreement signals. Trust is a computed attribute, never hand-set, always
359
+ traceable to its inputs.
360
+ - **Trust as RETRIEVAL WEIGHTING**: `retrieveBlocks` / fact lookup / the memory inspector rank
361
+ by relevance × trust, not relevance alone — a corroborated operator-stated fact outranks a
362
+ lone web scrape on the same query. Contradiction becomes visible (two high-trust sources
363
+ disagree → surface both with their provenance, never silently pick).
364
+ - **Feeds tier-5**: the Syllogist's entailed facts get a derived trust (min/product of premise
365
+ trusts × rule confidence) — a conclusion is only as trustworthy as its weakest premise, and
366
+ that number is computed, not asserted.
367
+
368
+ ## Phase 6 — Formulaic competence: the template-acquisition learning loop
369
+
370
+ > **STATUS: shipped.** A technical (C1) register of templates and productive/performance
371
+ > dual-banding (computed from the `via` provenance) are live in the benchmark.
372
+
373
+ > Detailed plan: **`PLAN_FORMULAIC_COMPETENCE.md`**.
374
+
375
+ The operator's insight upgraded to the strategy: a consistently-failed C1/C2 graded cell whose
376
+ answer EXISTS as a stable phrasing in technical prose is not a ceiling — it is a
377
+ **template-acquisition lever**. tmct learns the way human learners do: formulaic chunks first
378
+ (Wray's formulaic sequences), productive competence later.
379
+
380
+ - **Dual banding**: every graded score splits into a **productive band** (composed answers only)
381
+ and a **performance band** (templates allowed), computed from the `via` provenance (W1). The
382
+ band GAP is a first-class metric: how much fluency is memorized vs generated.
383
+ - **Template-lane benchmarking**: cases that target templated capability are TAGGED as such —
384
+ a template-carried C1 pass counts in the performance band and never inflates the productive
385
+ band; template-lane cells get their own agreement/reliability treatment (they are additional
386
+ benchmarking, not replacements — a level we would otherwise expect to fail at is being
387
+ deliberately faked, and the bench must say so).
388
+ - **The shopping list**: each cycle, the write-up extracts consistently-failed C1/C2 cells and
389
+ ranks them by template-acquirability (does a stable technical-prose phrasing exist? is the
390
+ slot structure mechanical — counts, comparisons, provenance we already compute?). Acquiring
391
+ the template IS the lever; the graded bench measures the flip in the performance band.
392
+ - **Mechanical conclusions at paragraph grade**: counting + comparison + superlatives (item 5)
393
+ composed through acquired C1-register templates — "X has 340 tests across 12 suites, unusually
394
+ dense for a codebase this size" — tech-domain answers can be genuinely advanced while the CEFR
395
+ banding tells us honestly how good the conversation AROUND them is.
396
+ - **Generalization path**: fixed tech domain first (templates hand-picked from technical-paper
397
+ register); then template acquisition generalizes — mining candidate templates from corpus
398
+ blocks (tier-2), scored by slot-fillability, promoted into `data/templates/` with provenance.
399
+
400
+ ## Phase 7 — Response finishing: the grammar pass (tone of voice dropped for now)
401
+
402
+ > **STATUS: shipped.** Answers segment into typed spans (prose vs protected); the grammar-rule
403
+ > pass runs on prose spans only under a protected-span invariance guard. The a/an article fix is
404
+ > active; broader voice/agreement rules are implemented-but-parked.
405
+
406
+ > Detailed plan: **`PLAN_RESPONSE_FINISHING.md`**.
407
+
408
+ *(Refined 2026-07-05; decisions settled with the operator. Fact invariance is achieved by
409
+ CONSTRUCTION, not by hope — finishing operates over a SEGMENTED answer, never a raw string.
410
+ Tone-of-voice synonym substitution is DROPPED: once every term with technical significance is
411
+ protected — entities, paths, vocabulary, receipts, provenance — the substitutable surface is
412
+ mostly connectives: high accuracy risk, thin reward. "Keen on the trickery to make a helpful
413
+ product, but not at the cost of accuracy." Moved to Phase LATER should a provably-safe subset
414
+ ever emerge.)*
415
+
416
+ - **The segmentation IR (the foundation, lever 1)**: every answer becomes a list of typed spans
417
+ before it becomes text — `prose` vs PROTECTED (`entity`, `path`, `number`, `code`,
418
+ `provenance`, `receipt`). Protected spans are byte-copied through finishing; only prose spans
419
+ are ever touched. The W1 template renderer is already slot-aware (slots ARE the protected
420
+ spans); composed renders adopt segmentation progressively via a conservative masker. Phase 5's
421
+ dual banding reads the same spans.
422
+ - **Grammar pass (lever 2)**: a data-driven rule table (TOML, item-7 formats) over prose spans —
423
+ article selection ("a artifact" → "an artifact", an observed defect class), subject–verb
424
+ agreement against slot plurality, capitalization, list/terminal punctuation. Grammar
425
+ corrections IMPROVE accuracy (they fix our own generated defects); that is why they survive
426
+ the tone cut. Neutral behavior is byte-stable except where a rule fixes a genuine defect —
427
+ each rule lands as a bench-measured lever.
428
+ - **Memory decision (settled)**: memory stores BOTH — the **as-spoken** turns live as larger
429
+ prose blocks on the graph (the honest record), and the **canonical** form is derived and
430
+ LINKED to its source prose blocks (canonise + link, never replace). Recall and folding read
431
+ canonical; provenance walks back to as-spoken.
432
+ - **Verification**: unit invariance checker (protected-span multiset identical pre/post) +
433
+ golden files per rule + the graded bench measuring each grammar rule as a lever.
434
+
435
+ ## Phase 8 — The Repository Interface (seonix inverts to a tmct user)
436
+
437
+ > **STATUS: shipped.** A versioned (1.0.0), OWL-grounded service contract
438
+ > (`docs/repository-interface.md` + `.schema.json`); a typed graph-service with a first-class
439
+ > miss contract (a miss is a value, not a throw); fixture + bootstrap reference providers; a
440
+ > runnable conformance/compatibility suite; and `tmct init` (scaffold `.tmct/`, `tmct.toml`,
441
+ > tier-1 seed, provenance).
442
+
443
+ > Detailed plan: **`PLAN_REPOSITORY_INTERFACE.md`**.
444
+
445
+ *(Operator-specified 2026-07-05; upgraded from research item to a build phase. tmct was spun OUT
446
+ of seonix; this inverts the relationship: seonix reorients as a USER that imports the tmct
447
+ library and exposes its graph to tmct as a typed service. Grows item 14's provider adapter from
448
+ a passive payload loader into the product's primary integration surface.)*
449
+
450
+ **Phase deliverables — define, reference-implement, and test the interface:**
451
+ 1. **The interface DEFINITION**: the typed, OWL-grounded service contract as a versioned
452
+ document + machine-readable shape (docs/repository-interface.md + a JSON-schema/typedef of
453
+ every service, its arguments, result types, and error contract) — tmct owns and versions it.
454
+ 2. **A REFERENCE IMPLEMENTATION tmct ships itself**: the in-repo provider (fixture graph +
455
+ bootstrap/empty graph) implementing EVERY service of the interface — the executable
456
+ specification any external producer reads first.
457
+ 3. **The contract test suite (the compatibility kit)**: a runnable suite any implementation is
458
+ tested against — tmct's reference implementation passes it in `npm test`; seonix runs the
459
+ SAME suite against its native implementation to claim conformance. Conformance = the suite,
460
+ not prose.
461
+ 4. **The session-handle lifecycle**, implemented: create/dispose context handles (focus, last,
462
+ memory dir, lexicon), provider-owned caching, documented re-entrancy — proven by the
463
+ contract suite's concurrent-session cases.
464
+ 5. **`tmct init`** shipped as part of this phase (it is the interface's onboarding surface).
465
+
466
+ - **tmct defines the adapter shape** — not the producer. Rationale: tmct is the brittle side
467
+ (query interpretation), so it must own and optimize around a STABLE interface; because the
468
+ vocabulary is OWL-grounded, the human/code world is already quantized into types both sides
469
+ understand — the interface is built from those shared types, not ad-hoc JSON.
470
+ - **A rich instruction set, translated from what seonix already exposes**: survey seonix's
471
+ native tool surface (describe / members / subclasses / impact / callers / callees / tests-for /
472
+ untested / history / exports / architecture / search / context / snippet / locate / digest —
473
+ the dispatchTool catalog tmct carried at the lift) and translate it into tmct's language as
474
+ the REPOSITORY INTERFACE: a consistent set of typed services any graph producer implements
475
+ natively (seonix first; the empty/bootstrap and fixture providers are degenerate
476
+ implementations tmct ships itself).
477
+ - **The flow** (LLM-agent front door): Claude Code et al. is briefed to use seonix → when the
478
+ agent judges it useful, seonix's "ask" tools pass NATURAL LANGUAGE to tmct → seonix calls the
479
+ tmct library in-process with the query PLUS a callbacks object (functions implementing the
480
+ repository interface over its native graph) → tmct resolves the query mechanically, calling
481
+ back into seonix's services for graph truth → results return through seonix to the LLM agent.
482
+ The mechanical interpreter becomes the NL front-end for any agent-facing graph tool; the LLM
483
+ stays outside tmct, exactly as the no-LLM ethos requires.
484
+ - **In-process lifecycle research (the hard part)**: seonix calls tmct directly, and the
485
+ interface is wider than the in-house chat — so define explicitly what is HELD IN MEMORY
486
+ between function calls: an explicit session/context handle (focus, last-answer, memory dir,
487
+ loaded lexicon) created and disposed by the caller instead of process-global state; graph
488
+ caching delegated to the provider (tmct never caches provider truth — the known source.mjs
489
+ process-cache staleness in long-lived servers becomes the provider's concern, by contract);
490
+ re-entrancy and concurrent-session guarantees documented per service.
491
+ - **seonix chat becomes tmct chat + a pointer**: seonix's chat surface loads tmct's chat with
492
+ the repository-interface handle — one chat implementation, N graph backends.
493
+ - **Browser mode**: the same inversion works in seonix's browser/code-browser surface — seonix
494
+ finds its own graph (it already ships one to the page) and embeds an OFF-THE-SHELF tmct: the
495
+ engine core (interpret / ask / render, lexicon, templates) is already pure JS with no
496
+ node-only dependency — wink's eng-lite-web-model is literally the browser build — so the
497
+ repository interface + a browser storage seam for memory (or provider-supplied persistence)
498
+ is all that separates the npm package from running in the page. The fs/readline/child_process
499
+ seams stay node-side; the browser gets the library surface, not the shell.
500
+ - **Distribution: `tmct init`** — a CLI command that initializes a local directory for tmct:
501
+ seeds/links the text corpuses (tier-1/2 policy applies), writes the externalized configuration
502
+ (tmct.toml — the seonix.toml pattern), creates `.tmct/`, and records provenance — so a host
503
+ package (seonix) or a bare user gets a working install with one command.
504
+
505
+ ## Phase 9 — Speculative inference: a step toward the Syllogist
506
+
507
+ > **STATUS: shipped.** `tmct syllogise [--depth n] [--budget n]` — an offline, bounded,
508
+ > deterministic maintenance job that forward-chains the `rdfs:subClassOf` closure into low-trust,
509
+ > retractable entailed facts; runs once after seeding, never on the chat hot path.
510
+
511
+ *(Operator-specified 2026-07-05. Tier-5 entailment answers a MISS on demand; this is the step
512
+ before it — PROACTIVELY extending memory with inferences that will be useful later, forward and
513
+ backward chaining over the OWL base during idle/fold time rather than at query time.)*
514
+
515
+ > Detailed plan: **`PLAN_SPECULATIVE_INFERENCE.md`**.
516
+
517
+ **A maintenance job, not a query-time cost.** Speculative inference runs as an explicit
518
+ batch — `npx tmct syllogise --depth <N>` (default depth bounded, e.g. 32) — and **once
519
+ automatically after seeding** (the W3 bootstrap seed is the natural trigger: a fresh corpus is
520
+ exactly when pre-deriving the useful closure pays off). Never on the chat's hot path.
521
+
522
+ **The selection criterion, sharpened by the operator (2026-07-05):** the guiding question is
523
+ *"what do the assertions of the sources I TRUST allow me to infer about this topic that is of
524
+ RELEVANCE"* — so **novelty × trust is the primary driver**, not an afterthought: the pass walks
525
+ outward from high-trust premises (the provenance primitive) toward novel, relevant conclusions,
526
+ timestamping each so recency and novelty stay computable. The mechanics are the easy half
527
+ (bounded forward chaining materializes entailments; backward chaining from frequent query shapes
528
+ pre-derives likely answers). **The residual hard half is still the FRAME PROBLEM / relevance
529
+ realization — unsolved in the general case and not pretended otherwise;** trust+novelty+relevance
530
+ are the tractable approximation, not a solution. The plan's job is
531
+ to make it TRACTABLE in tmct's narrow, closed world, not to solve it: usefulness is approximated
532
+ from what the system actually gets asked (query-shape frequency), what connects to recent focus,
533
+ what a cheap forward step yields that isn't already stored, and — the honest guardrail — a hard
534
+ budget (inference is bounded, its output trust-scored via the provenance primitive, and anything
535
+ speculative is retractable and never outranks a stated fact). Everything else is deferred to the
536
+ plan's open questions, where the relevance problem is named as the open research risk it is.
537
+
538
+ ### Open-source the ACE-OWL parser as a standalone library
539
+ > **STATUS: deferred follow-up** — not yet started; still gated on the Phase 8 library-surface
540
+ > work settling the extraction boundary. See `PLAN_OSS_ACE_PARSER.md`.
541
+
542
+ *(Operator-specified 2026-07-05, from the dependency audit's publish-not-replace finding.)* The
543
+ pure-JS, ESM, dependency-free ACE-OWL controlled-grammar parser (`src/grammar/ace.mjs` +
544
+ `lexicon.mjs`) that turns controlled-English sentences into OWL-labelled triples is a RARE thing:
545
+ the reference implementation (APE) is GPL + SWI-Prolog (native), so there is no permissive,
546
+ browser-capable, npm-installable ACE→OWL parser in the JS ecosystem. tmct's is exactly that.
547
+ Extract it to its own MPL-2.0 package (tmct depends on it back), so the wider RDF/OWL/semantic-web
548
+ JS community gains a controlled-natural-language front-end that runs in the browser. Gated on the
549
+ Repository Interface library-surface work (Phase 8) settling the extraction boundary; see
550
+ `PLAN_OSS_ACE_PARSER.md`. Sibling publish-candidates (the bounded-Damerau fuzzy matcher, the
551
+ PageRank+IDF block ranker) may follow the same path if there is demand.
552
+
553
+ ## Phase LATER — recognized, deferred, not now
554
+
555
+ Features we have deliberately shaped seams for but will not build until the phases above have
556
+ earned them:
557
+
558
+ ### Tone-of-voice adaptation (dropped from Phase 6, 2026-07-05)
559
+ Per-voice synonym/phrase substitution over prose spans. Dropped because tmct's protected-span
560
+ analysis leaves too little safely-substitutable text: any term with technical significance is
561
+ untouchable, and accuracy outranks helpfulness trickery. Revisit only if a provably-safe
562
+ substitutable subset emerges (e.g. connective-only voice profiles, or per-voice template
563
+ overrides authored as whole alternatives rather than substitutions). The grammar-preference
564
+ half of the idea survives inside Phase 6's rule table.
565
+
566
+ ### Tier-4 corpus: learn-on-miss acquisition
567
+ The strongest miss signal tmct can emit is: *lexicon term recognized, query built cleanly,
568
+ zero matches anywhere* — the question was well-formed and the knowledge is simply absent. The
569
+ tier-4 loop answers it by learning: web search on the resolved term → clean the fetched text
570
+ into tmct's own dialect (normalize into the ACE-OWL controlled grammar; whatever survives the
571
+ grammar becomes Facts, whatever doesn't becomes tier-2 text blocks under the PageRank index) →
572
+ store on disk with source provenance → ingest → answer the original question from the newly
573
+ learned material, citing what was just learned and from where. Strictly opt-in, network tier
574
+ rules apply (offline default inviolable; failure degrades to the honest miss). Prerequisites:
575
+ W1-W5 wired and measured, the Phase-5 template/dialect cleaning machinery (the "clean dialect"
576
+ IS the acquisition format), and a provenance-trust policy for web-sourced facts (never blended
577
+ silently with graph/operator facts — the `via`/provenance discipline extends to "learned:web").
578
+
579
+ ### Tier-5: entailment-on-miss — "the Syllogist" (deductive inference over the OWL base)
580
+ *(Item 11 matured from exploratory sketch to a designed tier; the "theorem-prove against
581
+ parsed prose" thread of the original code-talker ideas.)*
582
+
583
+ **The concept, classically:** answering from the **deductive closure** of a knowledge base —
584
+ KB ⊨ φ ("the knowledge base *entails* φ") — content that is nowhere ASSERTED in the graph,
585
+ memory, or corpus, but is a logical CONSEQUENCE of what is. Deductive inference (modus ponens,
586
+ modus tollens, syllogistic chains) predates ELIZA by ~2,300 years (Aristotle's syllogisms →
587
+ Frege's predicate logic → Robinson's resolution principle 1965 → Kowalski's "logic as a
588
+ programming language" → Prolog's SLD resolution; on the rules side, forward-chaining production
589
+ systems and the Rete algorithm; on the OWL side, description-logic reasoners and the RDFS/OWL
590
+ entailment regimes). tmct's version: a well-formed query misses everywhere → run the inference
591
+ layer over the OWL-encoded facts + axioms → if the answer is ENTAILED, materialize it as a Fact
592
+ with `via:"entailed"` and a **proof-chain provenance** (the applied rules + premise facts,
593
+ renderable as a chain of thought in words: "every cache is a store; every store is a component;
594
+ so a cache is a component") → the same query now yields an answer that shows its derivation.
595
+
596
+ **Worked shape (modus tollens over the code graph):** axiom "every tested module is covered by
597
+ a suite"; fact "m.mjs is covered by no suite" ⊨ "m.mjs is not tested" — never asserted,
598
+ honestly derived, provenance = the two premises + the rule name.
599
+
600
+ **Engine choice (the Prolog / graph-query question):** the classical candidates are embedded
601
+ Prolog (SLD, backward-chaining, item 11's original sketch), a graph query syntax (SPARQL under
602
+ entailment regimes / datalog / openCypher), or a description-logic tableau reasoner. The
603
+ recommended target is **OWL 2 RL** — the profile DESIGNED to be implemented as forward-chaining
604
+ rules (datalog-style semi-naive materialization, polynomial, decidable): pure-JS implementable,
605
+ mechanical, explainable rule-by-rule — exactly in ethos. Prolog-style backward chaining stays
606
+ the fallback for query-time-only derivation if materialization proves too eager. Progol/ILP
607
+ (learning NEW rules from examples) remains a separate, further-out spike.
608
+
609
+ **Gates:** the full-domain lexicon + OWL encoding in a queriable structure (Phases 2+4+5 and
610
+ tier-4's acquisition feed it), the provenance-trust policy (entailed facts must never silently
611
+ mix with asserted ones — a wrong axiom poisons the closure, so entailments are retractable by
612
+ provenance), and bench cells that measure inference specifically (premises in, conclusion
613
+ asked, derivation shown).
614
+
202
615
  ## Explicitly out of scope (for now)
203
616
 
204
617
  - No AWS, no benchmark rig — tmct is a published npm library + CLI with a