@polycode-projects/the-mechanical-code-talker 6.0.20 → 6.0.22

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (38) hide show
  1. package/README.md +5 -15
  2. package/bin/tmct.mjs +8 -3
  3. package/package.json +1 -5
  4. package/src/adapters/memory/core.mjs +108 -96
  5. package/src/adapters/memory/corpus-bands.mjs +6 -0
  6. package/src/domain/agent-traits.mjs +1 -1
  7. package/src/domain/answer-variants.json +1 -1
  8. package/src/domain/cli-verbs.mjs +1 -0
  9. package/src/domain/memory/causal-stability.mjs +22 -5
  10. package/src/domain/memory/fact-order.mjs +3 -3
  11. package/src/domain/memory/provenance-time.mjs +35 -0
  12. package/src/domain/memory/retraction.mjs +3 -5
  13. package/src/domain/memory/trust.mjs +11 -11
  14. package/src/domain/news-feed.mjs +1 -1
  15. package/src/domain/seeded-random.mjs +6 -6
  16. package/src/services/adventure.mjs +10 -41
  17. package/src/services/chat-page-viz.mjs +12 -1111
  18. package/src/services/chat-session.mjs +20 -6
  19. package/src/services/chat.mjs +269 -197
  20. package/src/services/extract-facts.mjs +1 -1
  21. package/src/services/import-file.mjs +1 -1
  22. package/src/services/mud-viz.mjs +21 -1082
  23. package/src/services/mudiii-viz.mjs +0 -4
  24. package/src/services/news.mjs +44 -24
  25. package/src/services/pill-complete.mjs +5 -8
  26. package/src/services/predator-prey.mjs +3 -5
  27. package/src/surfaces/web/memory-ask-browser.bundle.js +107 -107
  28. package/src/surfaces/web/mud-browser-entry.mjs +8 -48
  29. package/test-benchmarks/agentbench/README.md +7 -10
  30. package/src/adapters/p2p/webrtc-transport.mjs +0 -169
  31. package/src/domain/p2p/facts.mjs +0 -102
  32. package/src/domain/p2p/peer-id.mjs +0 -47
  33. package/src/domain/p2p/provenance-relabel.mjs +0 -37
  34. package/src/domain/p2p/sync-filter.mjs +0 -43
  35. package/src/domain/p2p/wire.mjs +0 -126
  36. package/src/services/p2p-room.mjs +0 -848
  37. package/src/services/share-overlay-viz.mjs +0 -623
  38. package/src/surfaces/web/p2p-browser-entry.mjs +0 -39
package/README.md CHANGED
@@ -131,8 +131,8 @@ table. Choose the architecture that fits your use case.
131
131
  | `docs/` | reference docs: the adapter/repository-interface contracts, bibliography |
132
132
  | `test/` | the unit, corpus, and estate-guard test suite (`npm test`) |
133
133
  | `test-e2e/` | the end-to-end suite: real CLI/TUI spawns and Playwright browser journeys (`npm run test:e2e`) |
134
- | `test-benchmarks/` | the benchmark harnesses (agentbench, chatbench, idxbench, infbench, ingestbench, researchbench, synthbench) and their shared `benchlib/` |
135
- | `reports/` | benchmark write-ups (`BENCHMARK_*.md`) and `PAGE_WEIGHTS.md` — see the root `STATUS.md` for the one-page summary these feed |
134
+ | `test-benchmarks/` | the benchmark harnesses (agentbench, idxbench, infbench, ingestbench, researchbench, synthbench) and their shared `benchlib/` |
135
+ | `reports/` | benchmark write-ups (`BENCHMARK_*.md`) — see the root `STATUS.md` for the one-page summary these feed |
136
136
  | `playtests/` | numbered playtest session logs, one edge found and fixed per entry |
137
137
  | `archive/` | delivered `PLAN_*.md`/`BENCHMARK_*.md` docs, kept for history |
138
138
  | `public/` | the demo site: the hand-written home page, `help.html`, the six about pages, `receipts.html`, `claims.html`, the shared stylesheet and the model/screenshot assets. The demo pages and browser bundles beside them are gitignored build outputs of `npm run demo:build` |
@@ -443,11 +443,7 @@ every class's `rdfs:subClassOf` parent drawn as a connector line rather than
443
443
  listed. The chat page and the ledger take the same paste-or-drop text in
444
444
  place; every page that holds a fact store exports it as JSONL.
445
445
 
446
- Two browsers can also link directly, no server in between: chat's share
447
- overlay walks a five-step WebRTC handshake (mint an invite, send it, create
448
- the reply, send that back, paste it to connect), and the mesh grows past a pair — three peers converge
449
- on the same fact store through a deterministic merge, real browsers driving
450
- real WebRTC in CI, not a mock. `help.html` walks asking, teaching, sharing,
446
+ `help.html` walks asking, teaching,
451
447
  and what to do when something goes wrong. `receipts.html` and `claims.html`
452
448
  carry the site's own numbers and claims, each figure naming the committed
453
449
  file it was measured or rendered from, including the ones that don't flatter
@@ -1619,19 +1615,15 @@ and transcripts are in the linked write-ups.
1619
1615
  |---|---|---|---|
1620
1616
  | Multi-hop entailment | 379/379 chat cases and 100/100 kernel cases, 0% fabrication, all bands pass | The case set is unchanged from 2.6.0 (same templates, same counts). The one real move this cycle is INF-4's ceiling-graded count dropping 35→30: five cases that now pass as genuine capability instead of against the declared honest-miss floor. | `archive/BENCHMARK_INFERENCE_2.7.12.md` |
1621
1617
  | Tool-call planning | 68/68 cases, 100% plan-completion, 100% result-completion, 0% hallucination, every rung A0→C2 | Goal driver. 2.6.0 gated at TOOL-7 (62/66, 94%). This cycle's router uplift (a guarded RECOVER step, a tied-candidate composer) cleared it: a real capability move, not a ruler change. | `archive/BENCHMARK_AGENT_2.7.12.md` |
1622
- | Groundedness | Every answer carries a source, and an empty graph reports itself empty. Judge-scored mean 1.809/2 over 138 cases, 5 hard fails, 136/138 tier-1. | Judged (`claude-haiku-4-5-20251001`, `judge-prompt-v2`) at N=1. The judge prompt moved v1→v2 since 2.6.0, so this is a measurement, not a clean lever comparison against the prior cycle. The judge runs in the offline eval harness, never in the product. | `archive/BENCHMARK_CEFR_ENGLISH_2.7.12.md` |
1623
1618
  | Abstention (the honest miss) | 0% fabrication across 479 inference rows (379 chat + 100 kernel) and 0% hallucination across 272 agent rows | Structural, not a tuned threshold. tmct abstains because nothing matched, so the rows test a property of a no-model design rather than a score. | `archive/BENCHMARK_INFERENCE_2.7.12.md`, `archive/BENCHMARK_AGENT_2.7.12.md` |
1624
1619
  | Determinism | Byte-identical on rerun: a 379-case `--replay` clean across 2 runs, no LLM, no network, $0 per turn | A property of the no-model pipeline. | `archive/BENCHMARK_INFERENCE_2.7.12.md` |
1625
1620
  | Dialogue robustness (persona sweep) | A 6-persona sweep (textbook logician, casual newcomer, new developer, adversarial sceptic, returning user, planning user) fixed 25 of the prior cycle's 29 routed findings (21 clean, 4 with a residual noted); 4 remain broken, 2 in a shape distinct from the original complaint | Free exploration across all six personas surfaced roughly 60 fresh findings beyond the ratchet check. The single highest-signal pattern: tmct's own suggested repair text was itself frequently broken when followed verbatim (since fixed, see `NEXT.md`). | `archive/BENCHMARK_CONVERSATION_2.7.11.md` |
1626
1621
 
1627
- Seven offline benchmark rigs live in a clone (the harnesses themselves are
1622
+ Six offline benchmark rigs live in a clone (the harnesses themselves are
1628
1623
  not in the npm package; AGENTBENCH's capability envelope is, see "As a
1629
1624
  library" above). Each replays a committed case set through the real product
1630
1625
  and writes graded rows you can diff between runs:
1631
1626
 
1632
- - `npm run chatbench:run` measures chat quality against CEFR-graded English
1633
- cases with deterministic tier-1 checks (the full tuning loop is in
1634
- `.claude/skills/benchmark-cefr-english/SKILL.md`);
1635
1627
  - `npm run infbench` generates inference cases, then runs each through both
1636
1628
  drive points, the reasoning kernel and the chat surface;
1637
1629
  - `npm run agentbench:run` measures the tool-loop behaviour, and every
@@ -1648,7 +1640,6 @@ The smallest real slice of each, the same invocations the test suite's
1648
1640
  bench-smoke lane replays:
1649
1641
 
1650
1642
  ```bash cwd=repo
1651
- node test-benchmarks/chatbench/run.mjs --stamp smoke --only g-a1-naming-1 --out /tmp/chatbench-smoke
1652
1643
  node test-benchmarks/infbench/generate-cases.mjs --out /tmp/infbench-cases.jsonl
1653
1644
  node test-benchmarks/infbench/run.mjs --cases /tmp/infbench-cases.jsonl --only inf-1-lookup-subClassOf-001 --stamp smoke --out /tmp/infbench-smoke
1654
1645
  node test-benchmarks/agentbench/run.mjs --stamp smoke --driver stub --only ab-a0-describe-widget --out /tmp/agentbench-smoke
@@ -1658,7 +1649,7 @@ Grading beyond tier 1 uses an LLM as judge. The offline eval harness is the
1658
1649
  one place an LLM is allowed, never the product:
1659
1650
 
1660
1651
  ```bash skip=offline-eval-only
1661
- npm run chatbench:judge -- --product /tmp/chatbench-smoke/product.jsonl
1652
+ npm run ingestbench:judge -- --product /tmp/ingestbench-smoke/product.jsonl
1662
1653
  ```
1663
1654
 
1664
1655
  ## Security and supply chain
@@ -1735,7 +1726,6 @@ edition, the retrieval date, the terms tmct uses, and what could not be verified
1735
1726
 
1736
1727
  | source | edition | what tmct uses it for |
1737
1728
  |---|---|---|
1738
- | Council of Europe, CEFR — Companion volume | 2020, ISBN 978-92-871-8621-8 | The band labels A1–C2 the chat benchmark grades against. CEFR measures what a *person* can do communicatively; grading the difficulty of *prompts* by band is tmct's adaptation, not a CEFR-validated use. The band descriptions in `test-benchmarks/chatbench/GRADED.md` are tmct's own prose. |
1739
1729
  | Reiter, "On Closed World Data Bases" | *Logic and Data Bases*, Plenum, 1978, pp. 55–76 | Both halves of the honest miss. The planner's operator model is **closed-world**, which is what makes a plan checkable. The chat layer is **open-world**: it will not read "no matching rule" as "the answer is no". |
1740
1730
  | Chow, "On optimum recognition error and reject tradeoff" | *IEEE Trans. Information Theory* 16(1), 1970 | Prior art for the goal. The literature calls a refusal **abstention**, or selective prediction, and Chow's reject option is its root. Those methods threshold a confidence score; tmct has none, and abstains because nothing matched — which is why the row above names the mechanism. |
1741
1731
  | Ji et al., "Survey of Hallucination in Natural Language Generation" | *ACM Computing Surveys* 55(12), 2023 | Groundedness, and what tmct is avoiding by having no model to hallucinate with. |
package/bin/tmct.mjs CHANGED
@@ -710,6 +710,11 @@ async function main() {
710
710
  // supplement on. Default OFF; `/wiki on`/`/wiki off` toggles it
711
711
  // mid-session.
712
712
  const liveReference = rest.includes("--live-wikipedia");
713
+ // `--code-graph` (or TMCT_CODE_GRAPH=1, read inside createSession): start
714
+ // the session reading every line as a question about the code graph.
715
+ // Default OFF; `/code-graph on`/`/code-graph off` toggles it mid-session.
716
+ // Nothing infers this from the graph or from what you type.
717
+ const codeGraphMode = rest.includes("--code-graph");
713
718
  // `--graph <path>` (repeatable) / `--config <path>` (src/services/cli-args.mjs): threaded
714
719
  // through as the new top tier of createSession's graph-resolution order (above
715
720
  // TMCT_GRAPH_FILE — see chat.mjs's own docblock at createSession). Omitted from
@@ -770,7 +775,7 @@ async function main() {
770
775
  if (prompt) {
771
776
  const { createSession } = await import("../src/services/chat.mjs");
772
777
  const { splitSentences } = await import("../src/services/sentences.mjs");
773
- const session = await createSession({ repoPath, ephemeral, narrate, liveReference, ...extra });
778
+ const session = await createSession({ repoPath, ephemeral, narrate, liveReference, codeGraphMode, ...extra });
774
779
  let finalAnswer = "";
775
780
  let finalPlan = null;
776
781
  let parts;
@@ -812,10 +817,10 @@ async function main() {
812
817
  const plain = rest.includes("--plain") || !process.stdin.isTTY || !process.stdout.isTTY;
813
818
  if (plain) {
814
819
  const { runChat } = await import("../src/services/chat.mjs");
815
- await runChat({ repoPath, ephemeral, narrate, liveReference, ...extra });
820
+ await runChat({ repoPath, ephemeral, narrate, liveReference, codeGraphMode, ...extra });
816
821
  } else {
817
822
  const { runTui } = await import("../src/surfaces/tui/app.mjs");
818
- await runTui({ repoPath, ephemeral, narrate, liveReference, ...extra });
823
+ await runTui({ repoPath, ephemeral, narrate, liveReference, codeGraphMode, ...extra });
819
824
  }
820
825
  return;
821
826
  }
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@polycode-projects/the-mechanical-code-talker",
3
- "version": "6.0.20",
3
+ "version": "6.0.22",
4
4
  "private": false,
5
5
  "type": "module",
6
6
  "description": "The Mechanical Code Talker (tmct) — a tolerant, offline, $0 chat surface that guides you toward precision queries about a software repository. ELIZA/PARRY-style but domain-obsessed with code. No model calls; indexes a repo on request (tmct index) or reads any producer's graph.",
@@ -165,10 +165,6 @@
165
165
  "serve:public": "node --disable-warning=ExperimentalWarning bin/tmct.mjs serve --host 0.0.0.0 --port 8787",
166
166
  "example:mini": "node --disable-warning=ExperimentalWarning bin/tmct.mjs chat --repo examples/mini-webapp --ephemeral",
167
167
  "example:polyglot": "node --disable-warning=ExperimentalWarning bin/tmct.mjs chat --repo examples/polyglot --ephemeral",
168
- "chatbench:run": "node test-benchmarks/chatbench/run.mjs",
169
- "chatbench:judge": "node test-benchmarks/chatbench/judge.mjs",
170
- "chatbench:judge:cached": "node test-benchmarks/chatbench/judge.mjs --cache test-benchmarks/chatbench/verdict-cache.json --promoted test-benchmarks/chatbench/promoted.json",
171
- "chatbench:distill": "node test-benchmarks/chatbench/distill.mjs",
172
168
  "agentbench:run": "node test-benchmarks/agentbench/run.mjs",
173
169
  "infbench": "node test-benchmarks/infbench/generate-cases.mjs && node test-benchmarks/infbench/run.mjs",
174
170
  "idxbench:run": "node test-benchmarks/idxbench/run.mjs",
@@ -122,7 +122,7 @@ export const SOURCE_RELIABILITY_PROP = "mgx:sourceReliability"; // actor-level (
122
122
  // per-session Source instead (`${ID}:<sessionId>`, sourceIdFor below).
123
123
  export const OPERATOR_SOURCE_ID = "src:operator-chat";
124
124
  const TEACH_SOURCE_ID = "src:teach-chat";
125
- // One Source per peer NODE, keyed by the stable id its relabeled tag carries.
125
+ // One Source per authoring NODE, keyed by the stable id its tag carries.
126
126
  const TEACH_NODE_SOURCE_ID = "src:teach-node";
127
127
 
128
128
  const ROLES = new Set(["visitor", "tmct"]);
@@ -384,11 +384,11 @@ export function factGroupId(recordId) {
384
384
  }
385
385
 
386
386
  /** The record id one provenance tag keys for a triple. A store files an
387
- * assertion under the Source its OWN tag derives, and a broadcast relabel
388
- * changes that Source — locally a chat session, at the peer the node that
389
- * sent it. So two stores hold one assertion under two ids, and anything that
390
- * has to name a record ACROSS the wire (a retraction does) resolves the id
391
- * through the tag rather than assuming both ends agree. */
387
+ * assertion under the Source its OWN tag derives, and a relabelled tag names a
388
+ * different Source — locally a chat session, elsewhere the node that authored
389
+ * it. So two stores hold one assertion under two ids, and anything that has to
390
+ * name a record ACROSS stores (a retraction does) resolves the id through the
391
+ * tag rather than assuming both ends agree. */
392
392
  export function factRecordIdForTag(groupId, tag) {
393
393
  return `${groupId}@${assertionSourceFor(tag).id}`;
394
394
  }
@@ -449,7 +449,7 @@ function assertionTimestampFor(tags, fallback = "", pick = Math.max) {
449
449
 
450
450
  /** Does an incoming assertion replace this source's own current record?
451
451
  * A genuinely newer embedded timestamp does; an exact re-delivery never does,
452
- * which is what keeps a re-seed and a duplicate mesh delivery no-ops. Both
452
+ * which is what keeps a re-seed and a duplicate delivery no-ops. Both
453
453
  * sides must carry a real embedded timestamp — an unstamped corpus row
454
454
  * asserted a second time is the same hop saying the same thing, not a new
455
455
  * version. The one tie-break: at equal instants a record carrying an
@@ -784,7 +784,6 @@ export async function openSqliteSeedStore(dbPath) {
784
784
  const ROW_META_MEMORY_KEY = "memory";
785
785
  const ROW_META_PREFIXES_KEY = "prefixes";
786
786
  const ROW_SYLLOGISE_STATE_KEY = "syllogiseState";
787
- const ROW_NODE_ID_KEY = "nodeId";
788
787
 
789
788
  /** Bind a row backend as a `memoryDir` token: `{ backend: "row", impl,
790
789
  * cachedPayload, basePayload }`. `basePayload` is the read-only seed overlay;
@@ -1398,15 +1397,41 @@ function buildSqlitePayloadFromRows(handle) {
1398
1397
  // ---- handle.cachedPayload mirrors: applied in lockstep with each SQL write
1399
1398
  // so the cache always matches a fresh SQL reconstruction. ----------------
1400
1399
 
1400
+ const cacheArrayPositions = new WeakMap();
1401
+
1402
+ /** Key -> first position in one of the cache's own arrays, so a mirror finds
1403
+ * the entry it is about to replace without scanning for it. Every mirror that
1404
+ * REMOVES entries replaces the array rather than editing it, and the identity
1405
+ * check here is what stops an index outliving the rows it named. */
1406
+ function cachePositionsOf(owner, array, keyOf) {
1407
+ const held = cacheArrayPositions.get(owner);
1408
+ if (held && held.array === array) return held.byKey;
1409
+ const byKey = new Map();
1410
+ for (let i = 0; i < array.length; i += 1) {
1411
+ const key = keyOf(array[i]);
1412
+ if (!byKey.has(key)) byKey.set(key, i);
1413
+ }
1414
+ cacheArrayPositions.set(owner, { array, byKey });
1415
+ return byKey;
1416
+ }
1417
+
1418
+ const individualIdOf = (ind) => ind?.id;
1419
+
1420
+ /** An edge's (subject,object) identity in the cache. NUL-delimited, matching
1421
+ * the SQL diff beside it — collision-proof, unlike a space. */
1422
+ const edgeCacheKey = (e) => `${e?.subject}\u0000${e?.object}`;
1423
+
1401
1424
  /** Mirrors `INSERT OR REPLACE INTO individuals(...)`: an existing id is
1402
1425
  * replaced IN PLACE (same array position, matching how SQL keeps that row's
1403
1426
  * `ord` — and so its sort position — unchanged on an update); a new id is
1404
1427
  * appended (matching a fresh row getting the next `ord`). */
1405
1428
  function cacheUpsertIndividual(cache, ind) {
1406
1429
  const clone = cloneJson(ind);
1407
- const i = cache.individuals.findIndex((x) => x?.id === ind.id);
1408
- if (i >= 0) cache.individuals[i] = clone;
1409
- else cache.individuals.push(clone);
1430
+ const positions = cachePositionsOf(cache, cache.individuals, individualIdOf);
1431
+ const at = positions.get(ind.id);
1432
+ if (at !== undefined) { cache.individuals[at] = clone; return; }
1433
+ positions.set(ind.id, cache.individuals.length);
1434
+ cache.individuals.push(clone);
1410
1435
  }
1411
1436
 
1412
1437
  /** Mirrors `DELETE FROM individuals`: drop every cached individual the write
@@ -1428,23 +1453,31 @@ function cacheGroupFor(cache, prop) {
1428
1453
 
1429
1454
  /** Mirrors `INSERT OR REPLACE INTO edges(...)`: a changed/new row sorts LAST
1430
1455
  * under `ORDER BY rowid`, so this moves the entry to the end of `examples`
1431
- * rather than replacing it in place. NUL-delimited (subject,object) key,
1432
- * matching the SQL diff beside it — collision-proof, unlike a space. */
1456
+ * rather than replacing it in place. */
1433
1457
  function cacheUpsertEdge(group, edge, extraKeys) {
1434
- const key = `${edge.subject}\u0000${edge.object}`;
1435
- group.examples = group.examples.filter((e) => `${e.subject}\u0000${e.object}` !== key);
1458
+ const key = edgeCacheKey(edge);
1459
+ const positions = cachePositionsOf(group, group.examples, edgeCacheKey);
1436
1460
  const cached = {
1437
1461
  subject: edge.subject, object: edge.object,
1438
1462
  subjectLabel: edge.subjectLabel ?? null, objectLabel: edge.objectLabel ?? null,
1439
1463
  };
1440
1464
  if (extraKeys.length) Object.assign(cached, cloneJson(Object.fromEntries(extraKeys.map((k) => [k, edge[k]]))));
1465
+ if (positions.has(key)) {
1466
+ // Rewriting an edge the group already holds moves every position after the
1467
+ // one it vacates, so the array is replaced and the index built again on the
1468
+ // next call rather than patched here.
1469
+ group.examples = group.examples.filter((e) => edgeCacheKey(e) !== key);
1470
+ group.examples.push(cached);
1471
+ return;
1472
+ }
1473
+ positions.set(key, group.examples.length);
1441
1474
  group.examples.push(cached);
1442
1475
  }
1443
1476
 
1444
1477
  /** Mirrors the per-group edge delete loop: drop any cached edge in this group
1445
1478
  * whose (subject,object) key isn't in the just-persisted group's key set. */
1446
1479
  function cacheDropEdgesExcept(group, newKeys) {
1447
- group.examples = group.examples.filter((e) => newKeys.has(`${e.subject}\u0000${e.object}`));
1480
+ group.examples = group.examples.filter((e) => newKeys.has(edgeCacheKey(e)));
1448
1481
  }
1449
1482
 
1450
1483
  /** Mirrors `DELETE FROM relations`: drop every cached edge group the write just
@@ -1805,10 +1838,23 @@ function recomputeDerivedTables(ctx, foldSource, headIndex) {
1805
1838
  recordObjectSupersessions(ctx.db, foldCtx, ctx.touchedPairs);
1806
1839
  }
1807
1840
 
1841
+ /** How many rows one write may change and still be mirrored into
1842
+ * `handle.cachedPayload` row by row. Above this the write drops the cache and
1843
+ * lets the next read rebuild from the tables instead: mirroring clones every
1844
+ * row it writes into a payload that already holds a copy of the whole store,
1845
+ * and a corpus import writes tens of thousands of rows in a single call, so
1846
+ * the mirror costs more than the one rebuild it saves — and holds two copies
1847
+ * of the seed in memory while it does it. The bar sits well above any
1848
+ * interactive write (a chat turn, a teach, a world load move rows in the tens
1849
+ * to low hundreds) so an ordinary session keeps its warm cache. */
1850
+ const CACHE_MIRROR_ROW_LIMIT = 2000;
1851
+
1808
1852
  /** Persist a mutated payload into a Backend C handle: the payload projected to
1809
1853
  * rows, diffed against the rows already stored, and only the difference
1810
- * written, in one transaction. Patches `handle.cachedPayload` in lockstep; a
1811
- * rolled-back write invalidates the cache instead of leaving a partial patch.
1854
+ * written, in one transaction. Patches `handle.cachedPayload` in lockstep,
1855
+ * unless the write moves more rows than `CACHE_MIRROR_ROW_LIMIT`, where it
1856
+ * drops the cache instead; a rolled-back write invalidates the cache rather
1857
+ * than leaving a partial patch.
1812
1858
  *
1813
1859
  * The 4 KB wire cap the row contract enforces is not applied here: it protects
1814
1860
  * a network hop, and a local file store holds a seed's whole statedBy group in
@@ -1816,9 +1862,14 @@ function recomputeDerivedTables(ctx, foldSource, headIndex) {
1816
1862
  function persistSqlitePayload(handle, payload) {
1817
1863
  const db = handle.db;
1818
1864
  const empty = emptyMemory();
1819
- const cache = handle.cachedPayload || null;
1820
1865
  db.exec("BEGIN IMMEDIATE");
1821
1866
  try {
1867
+ const before = sqlitePayloadStoreRows(handle);
1868
+ const { rows: after, partsByKey } = sqlitePayloadRows(payload, before);
1869
+ const { puts, deletes } = diffRows(before, after);
1870
+ const mirrorsIntoCache = puts.length + deletes.length <= CACHE_MIRROR_ROW_LIMIT;
1871
+ const cache = mirrorsIntoCache ? handle.cachedPayload || null : null;
1872
+
1822
1873
  const setMeta = db.prepare("INSERT OR REPLACE INTO meta(k, v) VALUES (?, ?)");
1823
1874
  setMeta.run("generated_at", JSON.stringify(payload.generated_at ?? empty.generated_at));
1824
1875
  setMeta.run("memory", JSON.stringify(payload.memory ?? empty.memory));
@@ -1836,14 +1887,15 @@ function persistSqlitePayload(handle, payload) {
1836
1887
  }
1837
1888
 
1838
1889
  const ctx = sqliteWriteContext(handle, { cache });
1839
- const before = sqlitePayloadStoreRows(handle);
1840
- const { rows: after, partsByKey } = sqlitePayloadRows(payload, before);
1841
- const { puts, deletes } = diffRows(before, after);
1842
1890
  writeSqliteRows(ctx, puts, partsByKey);
1843
1891
  deleteSqliteRows(ctx, deletes);
1844
1892
  recomputeDerivedTables(ctx, payload, handle.cachedFactHeads);
1845
1893
 
1846
1894
  db.exec("COMMIT");
1895
+ if (!mirrorsIntoCache) {
1896
+ handle.cachedPayload = null;
1897
+ handle.cachedFactHeads = null;
1898
+ }
1847
1899
  } catch (e) {
1848
1900
  db.exec("ROLLBACK");
1849
1901
  // The cache may hold a partially-applied patch at this point (some of the
@@ -2361,47 +2413,6 @@ export async function saveSyllogiseState(dir, state) {
2361
2413
  await atomicWriteJson(file, state);
2362
2414
  }
2363
2415
 
2364
- // ---- Node id: the stable per-store P2P identity, a second sidecar ----------
2365
- // 16 hex, minted the first time a store joins a room and never regenerated.
2366
- // Persisted beside the store rather than inside the graph so it survives a
2367
- // store that gets re-seeded, and so nothing about it ever replicates: a node
2368
- // id is this store's own name for itself, not a fact about the world.
2369
-
2370
- export const NODE_ID_REL = join(MEMORY_DIR_REL, "node-id.json");
2371
- const SQLITE_NODE_ID_KEY = "nodeId";
2372
-
2373
- /** This store's node id, or null when it has never joined a room. */
2374
- export async function loadNodeId(dir) {
2375
- if (isMemoryHandle(dir)) return dir.nodeId || null;
2376
- if (isSqliteHandle(dir)) {
2377
- const row = dir.db.prepare("SELECT v FROM meta WHERE k = ?").get(SQLITE_NODE_ID_KEY);
2378
- return row?.v ? JSON.parse(row.v).nodeId || null : null;
2379
- }
2380
- if (isRowHandle(dir)) return (await readRowMeta(dir, ROW_NODE_ID_KEY, null))?.nodeId || null;
2381
- try {
2382
- return JSON.parse(await readFile(join(dir, NODE_ID_REL), "utf8")).nodeId || null;
2383
- } catch (e) {
2384
- if (e?.code === "ENOENT") return null;
2385
- throw e;
2386
- }
2387
- }
2388
-
2389
- /** Record this store's node id — atomic file write (Backend A), a handle field
2390
- * (Backend B), a meta-table row (Backend C), or a meta value in the injected
2391
- * store (Backend D). Callers mint through resolveStoreNodeId, which never
2392
- * overwrites an id a store already holds, so this value is written once. */
2393
- export async function saveNodeId(dir, nodeId) {
2394
- if (isMemoryHandle(dir)) { dir.nodeId = nodeId; return; }
2395
- if (isSqliteHandle(dir)) {
2396
- dir.db.prepare("INSERT OR REPLACE INTO meta(k, v) VALUES (?, ?)").run(SQLITE_NODE_ID_KEY, JSON.stringify({ nodeId }));
2397
- return;
2398
- }
2399
- if (isRowHandle(dir)) { await dir.impl.putMeta(ROW_NODE_ID_KEY, JSON.stringify({ nodeId })); return; }
2400
- const file = join(dir, NODE_ID_REL);
2401
- await mkdir(dirname(file), { recursive: true });
2402
- await atomicWriteJson(file, { nodeId });
2403
- }
2404
-
2405
2416
  /** Fresh read -> mutate -> atomic write. Serialized per call; every public
2406
2417
  * append goes through here, including the lazy legacy-provenance migration
2407
2418
  * and actor-level Source reliability recompute. `fn` may be async (the
@@ -2685,11 +2696,11 @@ function sourceIdFor(desc) {
2685
2696
  switch (desc?.kind) {
2686
2697
  case "operator": return { id: desc.sessionId ? `${OPERATOR_SOURCE_ID}:${desc.sessionId}` : OPERATOR_SOURCE_ID, type: "operator" };
2687
2698
  case "teach": return { id: desc.sessionId ? `${TEACH_SOURCE_ID}:${desc.sessionId}` : TEACH_SOURCE_ID, type: "teach" };
2688
- // One Source per peer NODE, keyed on the stable id the tag carries rather
2689
- // than the display name beside it: names are user-chosen and collidable, so
2690
- // two peers who picked the same one would otherwise collapse into a single
2691
- // Source and corroborate each other for free. Scores at the teach tier —
2692
- // a peer teaching is still a person telling us something.
2699
+ // One Source per authoring NODE, keyed on the stable id the tag carries
2700
+ // rather than the display name beside it: names are user-chosen and
2701
+ // collidable, so two nodes that picked the same one would otherwise collapse
2702
+ // into a single Source and corroborate each other for free. Scores at the
2703
+ // teach tier — a node teaching is still a person telling us something.
2693
2704
  case "teachNode": return { id: `${TEACH_NODE_SOURCE_ID}:${desc.nodeId}`, type: "teach" };
2694
2705
  case "provider": return { id: `src:provider:${desc.name}`, type: "provider" };
2695
2706
  case "corpus": return { id: `src:corpus:${desc.name}`, type: "corpus" };
@@ -2861,11 +2872,11 @@ function migrateLegacyProvenance(payload) {
2861
2872
  }
2862
2873
 
2863
2874
  /** A Source id that names one actor and can therefore hold a track record —
2864
- * the `${SINGLETON}:<id>` shape, for a local operator/teach session or for a
2865
- * peer NODE across the mesh. A corpus/web/provider/entailed Source names a
2866
- * document or a derivation, so there is no actor to score.
2875
+ * the `${SINGLETON}:<id>` shape, for a local operator/teach session or for
2876
+ * another NODE that authored the tag. A corpus/web/provider/entailed Source
2877
+ * names a document or a derivation, so there is no actor to score.
2867
2878
  *
2868
- * A peer node counts for the same reason a local session does, and it is the
2879
+ * Another node counts for the same reason a local session does, and it is the
2869
2880
  * reason this matters most: a node that asserts junk drags its own every-fact
2870
2881
  * prior toward half, so minting fresh identities to corroborate yourself stops
2871
2882
  * being free the moment any of those claims is contradicted. */
@@ -2978,7 +2989,7 @@ function recomputeSourceReliability(payload) {
2978
2989
  /**
2979
2990
  * Two rollup summaries at one id JOIN instead of overwriting: union the ids
2980
2991
  * they absorbed, then re-derive count, bounds and prior from that union. This
2981
- * is what lets two peers that compacted the same group at different moments
2992
+ * is what lets two stores that compacted the same group at different moments
2982
2993
  * converge — union, min and max are all joins, so the result is the same in
2983
2994
  * either order and applying it twice changes nothing. Re-writing a summary that
2984
2995
  * already holds everything the incoming one does is therefore a no-op, which is
@@ -3210,12 +3221,12 @@ function assertionGroupsFor(payload, groupId, provenance, createdAt = "") {
3210
3221
  if (groups.length === 1 && groups[0].sourceId === NO_SOURCE_ID) {
3211
3222
  if (factRecordIdsFor(payload, groupId).length) return [];
3212
3223
  }
3213
- // A source whose assertion was retracted here does not come back on the next
3214
- // sync. This is the ingest half of the enforcement: every delivery path for a
3215
- // fact lands in this function, so a re-sent copy of a retracted assertion is
3216
- // recognized and dropped rather than re-materialized. The comparison is
3217
- // against the assertion's OWN embedded instant, so the same source saying the
3218
- // thing again — a fresh tag, a later moment — still lands.
3224
+ // A source whose assertion was retracted here does not come back the next
3225
+ // time those rows land. This is the ingest half of the enforcement: every
3226
+ // delivery path for a fact lands here, so a re-sent copy of a retracted
3227
+ // assertion is recognized and dropped rather than re-materialized. The
3228
+ // comparison is against the assertion's OWN embedded instant, so the same
3229
+ // source saying the thing again — a fresh tag, a later moment — still lands.
3219
3230
  const retractions = retractionsFor(payload, groupId);
3220
3231
  if (retractions.length) {
3221
3232
  groups = groups.filter((group) => !isRetractedRecord(
@@ -3327,7 +3338,7 @@ function normalizeExtractionFindings(extraction) {
3327
3338
  * record it replaces kept whole under `#v<n>` and linked both ways;
3328
3339
  * - the same source saying the same thing again: its tags union onto the head
3329
3340
  * and its first write's stamps stand. An exact re-delivery changes nothing,
3330
- * which is what keeps a re-seed and a duplicate mesh path idempotent.
3341
+ * which is what keeps a re-seed and a duplicate delivery path idempotent.
3331
3342
  */
3332
3343
  function planFactAssertion(payload, spec) {
3333
3344
  const { groupId, s, p, o, label, tokens, group, createdAt, observedAt, quantifier, environments, extraction = [] } = spec;
@@ -4163,10 +4174,10 @@ export async function foldedFactRows(dir) {
4163
4174
  *
4164
4175
  * The rows come out in content order, not in the order the payload happened to
4165
4176
  * hold them. A reader that takes the first of several equally-ranked rows, or
4166
- * sorts by a key that ties, otherwise answers by arrival order — and two peers
4177
+ * sorts by a key that ties, otherwise answers by arrival order — and two stores
4167
4178
  * holding one fact set arrive at it differently. Sorting here is what lets a
4168
4179
  * reader inherit the guarantee instead of re-earning it: the fold is the one
4169
- * place every fact read passes through. p2p-room.mjs's sortFactIndividualsById
4180
+ * place every fact read passes through. rows.mjs's sortFactIndividualsById
4170
4181
  * does the same job one level down, over the stored records. */
4171
4182
  function foldFactRows(memory, ctx, opts = {}) {
4172
4183
  // A materialised head, when the backend keeps one, replaces the group's own
@@ -4218,7 +4229,7 @@ const FACT_ID_PREFIX = "fact:";
4218
4229
  * Pure: the speakers come back sorted, so a claim two outlets attributed reads
4219
4230
  * the same whichever order the attributions arrived in, and an attribution
4220
4231
  * whose claim the fold never saw simply hangs on nothing — which is the case
4221
- * every time rows arrive over p2p out of order. */
4232
+ * every time rows arrive out of order. */
4222
4233
  function attachSpeakers(rows) {
4223
4234
  // An attribution names its claim as its SUBJECT, and a claim's own subject is
4224
4235
  // a term, so the prefix rules nearly every row out on one comparison. The
@@ -4304,11 +4315,11 @@ function factFoldContext(memory, { pairs = null, scopedGroups = new Set() } = {}
4304
4315
  else groups.set(groupId, [ind]);
4305
4316
  }
4306
4317
 
4307
- // The read half of retraction enforcement. A record a peer re-delivered before
4308
- // its retraction arrived is still sitting in the payload, and this is what
4309
- // keeps it out of the answer: the fold is a pure function of the fact set, so
4310
- // both peers read the same row whichever order the two arrived in. Only a
4311
- // group that actually carries a retraction pays anything for the check.
4318
+ // The read half of retraction enforcement. A record re-delivered before its
4319
+ // retraction arrived is still sitting in the payload, and this is what keeps
4320
+ // it out of the answer: the fold is a pure function of the fact set, so both
4321
+ // stores read the same row whichever order the two arrived in. Only a group
4322
+ // that actually carries a retraction pays anything for the check.
4312
4323
  for (const [groupId, retractions] of retractionsByGroup) {
4313
4324
  const members = groups.get(groupId);
4314
4325
  if (!members) continue;
@@ -4323,7 +4334,7 @@ function factFoldContext(memory, { pairs = null, scopedGroups = new Set() } = {}
4323
4334
  }
4324
4335
 
4325
4336
  // Codepoint order on the record id, which sorts by source key — the same
4326
- // locale-free determinism the P2P layer's own sort insists on, so two peers
4337
+ // locale-free determinism rows.mjs's own sort insists on, so two stores
4327
4338
  // holding the same records read the same row.
4328
4339
  for (const members of groups.values()) members.sort((a, b) => (a.id < b.id ? -1 : a.id > b.id ? 1 : 0));
4329
4340
 
@@ -4566,8 +4577,8 @@ function factGroupsReferencing(payload, goneGroupIds) {
4566
4577
  *
4567
4578
  * The delete leaves a RETRACTION RECORD behind, one per (triple, source),
4568
4579
  * carrying the record ids it suppressed and the moment it did. That record is
4569
- * what makes the retraction survive a sync: a plain delete against a grow-only
4570
- * set comes straight back from any peer that still holds the fact. It also
4580
+ * what makes the retraction survive a merge: a plain delete against a grow-only
4581
+ * set comes straight back from any store that still holds the fact. It also
4571
4582
  * keeps the retraction on record rather than erasing the fact that something
4572
4583
  * was asserted at all. A retraction record is never itself removed here.
4573
4584
  *
@@ -4673,8 +4684,9 @@ export async function removeFacts(dir, ids, { provenance = "", retractedAt = ""
4673
4684
  }
4674
4685
 
4675
4686
  /** Every retraction record the store holds, as the wire facts that carry them.
4676
- * The P2P layer's own diff and sync response read this: a retraction is not a
4677
- * fact row, so nothing that walks readFactRows would ever find one. */
4687
+ * Anything that ships one store's retractions to another reads this: a
4688
+ * retraction is not a fact row, so nothing that walks readFactRows would ever
4689
+ * find one. */
4678
4690
  export function readRetractions(memory) {
4679
4691
  const out = [];
4680
4692
  for (const ind of memory?.individuals || []) {
@@ -4734,12 +4746,12 @@ export async function appendRetractions(dir, wireFacts) {
4734
4746
  /** What this store could retire, and the roster it has to convince first.
4735
4747
  *
4736
4748
  * `roster` is the world's admission graph, folded to a set of node ids —
4737
- * replicated, grow-only, and the same on every peer holding the same facts.
4738
- * `retirable` is the tombstones every peer on that roster is known to hold.
4749
+ * replicated, grow-only, and the same on every replica holding the same facts.
4750
+ * `retirable` is the tombstones every node on that roster is known to hold.
4739
4751
  * `acknowledgedBy(nodeId)` is what supplies that evidence; nothing produces it
4740
4752
  * yet, so `retirable` reads empty and this is a report rather than a sweep.
4741
4753
  * Retiring nothing is the current behaviour, and it is the safe one: a
4742
- * tombstone dropped one peer early lets that peer's copy resurrect a retracted
4754
+ * tombstone dropped one node early lets that node's copy resurrect a retracted
4743
4755
  * fact. See docs/references/papers/crdt.md. */
4744
4756
  export function retirableRetractions(memory, { self = "", acknowledgedBy = null } = {}) {
4745
4757
  const roster = admittedNodes(readFactRows(memory));
@@ -29,6 +29,8 @@ export const MANIFEST_SORT_KEY = "manifest";
29
29
 
30
30
  /** The bands this repo ships a build pipeline and a loader for. */
31
31
  export const FIRST_CLASS_BANDS = Object.freeze([
32
+ "child",
33
+ "conceptnet",
32
34
  "wordnet-complete",
33
35
  ]);
34
36
 
@@ -38,6 +40,10 @@ export const FIRST_CLASS_BANDS = Object.freeze([
38
40
  * `notice` is a repo-relative path to the human-readable attribution file;
39
41
  * null when the licence carries no attribution burden. */
40
42
  export const BAND_LICENSES = Object.freeze({
43
+ // The child pack is ConceptNet-derived, so it carries ConceptNet's own
44
+ // share-alike terms rather than the maintainer-owned seed script's.
45
+ child: Object.freeze({ license: "CC-BY-SA-4.0", notice: "corpus/child/LICENSE-NOTICE" }),
46
+ conceptnet: Object.freeze({ license: "CC-BY-SA-4.0", notice: "corpus/conceptnet/LICENSE-NOTICE" }),
41
47
  "wordnet-complete": Object.freeze({ license: "CC-BY-4.0", notice: "corpus/wordnet/LICENSE-NOTICE" }),
42
48
  });
43
49
 
@@ -15,7 +15,7 @@
15
15
  //
16
16
  // Every list this module returns is sorted in codepoint order before it is
17
17
  // returned, and every resolver here is a pure function of the row SET, never
18
- // of the order the rows arrived in — the same rule p2p-room.mjs's
18
+ // of the order the rows arrived in — the same rule rows.mjs's
19
19
  // sortFactIndividualsById holds for a merged fact store, so two callers who
20
20
  // built the same rows in different orders never read a different answer.
21
21
 
@@ -10,7 +10,7 @@
10
10
  "ask.mjs metaFallbackEntityAnswer (\"X is a Y in this codebase, defined in Z ...\")"
11
11
  ],
12
12
  "deliberatelyNotUsedBy": [
13
- "ask.mjs renderCore whereShape (\"X is defined in Y at line(s) Z\" / \"X is defined in Y (no line span...)\") -- test-benchmarks/chatbench/graded-pool-max.jsonl pins this exact substring as ground truth for 11 'where is X defined' cases (2 in the always-run promoted subset), and that pool is append-only/never-edited-mid-arc per .claude/skills/benchmark-cefr-english/SKILL.md."
13
+ "ask.mjs renderCore whereShape (\"X is defined in Y at line(s) Z\" / \"X is defined in Y (no line span...)\") -- the grammar, messy-user, compositional and drilldowns corpus lanes pin this exact substring as ground truth for 'where is X defined', so varying it there breaks those rows."
14
14
  ]
15
15
  },
16
16
  "is-dated": {
@@ -38,6 +38,7 @@ export const CLI_VERBS = [
38
38
  { flag: "[--plain]", prose: ["force the plain readline shell (the default when", "stdin/stdout is not a terminal)"] },
39
39
  { flag: "[--memory-backend <default|memory|sqlite>]", prose: ["storage backend for taught facts this", "session (CLI flag > TMCT_MEMORY_BACKEND env > tmct.toml's", "[memory] backend > sqlite, .tmct/memory/graph.sqlite)"] },
40
40
  { flag: "[--research-source <wikipedia|wikidata|simple-wikipedia-pack>]", prose: ["where \"research <topic>\" fetches from this session", "(/wikipedia|/wikidata in chat > this flag > tmct.toml's", "[research] source > wikipedia)"] },
41
+ { flag: "[--code-graph]", prose: ["read every line as a question about the code graph", "(default off, and never inferred; also TMCT_CODE_GRAPH=1;", "toggle mid-session with /code-graph on|off)"] },
41
42
  ],
42
43
  },
43
44
  {
@@ -9,13 +9,13 @@
9
9
  //
10
10
  // The literature calls the missing rule CAUSAL STABILITY: a record is safe to
11
11
  // drop once every replica that could still send a conflicting copy has it. Two
12
- // inputs, and the mesh has one of them.
12
+ // inputs.
13
13
  //
14
14
  // - The ROSTER. `node:<joiner> mgx:invitedBy node:<inviter>` is an ordinary
15
15
  // replicated fact, so the set of node ids ever admitted to a world is a
16
- // grow-only union every peer computes the same way. `admittedNodes` reads
17
- // it. Grow-only is exactly right here: a roster that could shrink would let
18
- // a forgotten node's stale copy back in.
16
+ // grow-only union every replica computes the same way. `admittedNodes`
17
+ // reads it. Grow-only is exactly right here: a roster that could shrink
18
+ // would let a forgotten node's stale copy back in.
19
19
  // - The ACKNOWLEDGEMENT. Nothing yet records that a named node holds a named
20
20
  // record. `stableRecordIds` takes it as an argument rather than inventing
21
21
  // it, and answers "nothing is stable" when it is absent — which is the
@@ -32,7 +32,24 @@
32
32
  // invariant every read-time resolver over the store has to meet.
33
33
  // docs/references/papers/crdt.md carries the full design and the options it
34
34
  // rejected.
35
- import { INVITED_BY_PREDICATE } from "../p2p/facts.mjs";
35
+
36
+ export const INVITED_BY_PREDICATE = "mgx:invitedBy";
37
+
38
+ /** The term a node id takes as a fact subject or object. */
39
+ export const nodeTerm = (nodeId) => `node:${nodeId}`;
40
+
41
+ /** One admission edge: who let a node in. There is no open discovery here, so
42
+ * this records a social admission graph that an identity cannot mint for
43
+ * itself. The joiner writes it, because the joiner is the only side that
44
+ * knows both node ids at the moment it decides to join. */
45
+ export function invitedByFact(joinerNodeId, inviterNodeId, timestamp) {
46
+ return {
47
+ subject: nodeTerm(joinerNodeId),
48
+ predicate: INVITED_BY_PREDICATE,
49
+ object: nodeTerm(inviterNodeId),
50
+ provenance: `ace:p2p:${joinerNodeId}@${timestamp}`,
51
+ };
52
+ }
36
53
 
37
54
  /** The node ids a world has ever admitted, from its admission edges. Both ends
38
55
  * of each edge count: the joiner wrote the edge about itself, and it names the
@@ -12,9 +12,9 @@
12
12
  // key. Codepoint order throughout, never localeCompare — two locales have to
13
13
  // land on the same order.
14
14
  //
15
- // The store's own precedent is p2p-room.mjs's sortFactIndividualsById, which
16
- // sorts Fact individuals by content-addressed id after every merge for exactly
17
- // this reason. This is that discipline carried through to the read side.
15
+ // The store's own precedent is rows.mjs's sortFactIndividualsById, which sorts
16
+ // Fact individuals by content-addressed id after every merge for exactly this
17
+ // reason. This is that discipline carried through to the read side.
18
18
 
19
19
  /** A fact row's content-derived sort key. */
20
20
  export const factOrderKey = (f) => [