skynet-graph 1.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (213) hide show
  1. package/CONTRIBUTING.md +22 -0
  2. package/LICENSE +661 -0
  3. package/README.md +245 -0
  4. package/bin/sg +20 -0
  5. package/concepts/README.md +23 -0
  6. package/concepts/_substrate/Claim/Confidence/Trusted.json +6 -0
  7. package/concepts/_substrate/Claim/Confidence.json +10 -0
  8. package/concepts/_substrate/Claim/Freshness/Stale.json +7 -0
  9. package/concepts/_substrate/Claim/Freshness.json +5 -0
  10. package/concepts/_substrate/Claim/Verification/Refuted.json +6 -0
  11. package/concepts/_substrate/Claim/Verification.json +6 -0
  12. package/concepts/_substrate/Claim.json +4 -0
  13. package/concepts/_substrate/Frontier/InBeam.json +6 -0
  14. package/concepts/_substrate/Frontier/Stuck.json +6 -0
  15. package/concepts/_substrate/Frontier.json +5 -0
  16. package/concepts/_substrate/Intake/ToTask.json +22 -0
  17. package/concepts/_substrate/Intake/Typed.json +10 -0
  18. package/concepts/_substrate/Intake.json +27 -0
  19. package/concepts/_substrate/Task/Answer.json +12 -0
  20. package/concepts/_substrate/Task/Complexity/Atomic.json +5 -0
  21. package/concepts/_substrate/Task/Complexity/Compound/Expansion.json +7 -0
  22. package/concepts/_substrate/Task/Complexity/Compound.json +6 -0
  23. package/concepts/_substrate/Task/Complexity.json +6 -0
  24. package/concepts/_substrate/Task/EvalComplexity.json +13 -0
  25. package/concepts/_substrate/Task/ReportUp.json +6 -0
  26. package/concepts/_substrate/Task/Rollup.json +10 -0
  27. package/concepts/_substrate/Task.json +5 -0
  28. package/concepts/clinical/Diagnosis/Medication.json +6 -0
  29. package/concepts/clinical/Diagnosis.json +15 -0
  30. package/concepts/clinical/Observation/LabValue/OutOfRange.json +6 -0
  31. package/concepts/clinical/Observation/LabValue.json +5 -0
  32. package/concepts/clinical/Observation.json +6 -0
  33. package/concepts/common/Document/pathBasket.json +6 -0
  34. package/concepts/common/Document.json +3 -0
  35. package/concepts/common/Edge/Distance.json +13 -0
  36. package/concepts/common/Edge/Stay/LongStay.json +8 -0
  37. package/concepts/common/Edge/Stay.json +8 -0
  38. package/concepts/common/Edge/Travel/LongTravel.json +16 -0
  39. package/concepts/common/Edge/Travel/ShortTravel.json +11 -0
  40. package/concepts/common/Edge/Travel/targetNode.json +7 -0
  41. package/concepts/common/Edge/Travel.json +7 -0
  42. package/concepts/common/Edge.json +3 -0
  43. package/concepts/common/Vertice.json +3 -0
  44. package/concepts/supply/Fulfillment.json +10 -0
  45. package/concepts/supply/Inventory/Reorder.json +7 -0
  46. package/concepts/supply/Inventory.json +5 -0
  47. package/concepts/supply/Procurement/SupplierConfirm.json +6 -0
  48. package/concepts/supply/Procurement.json +8 -0
  49. package/concepts/supply/Transport.json +5 -0
  50. package/doc/API.md +556 -0
  51. package/doc/CAPABILITIES.md +373 -0
  52. package/doc/MODELISATION.md +511 -0
  53. package/doc/architecture.md +267 -0
  54. package/doc/concept-as-graph.md +379 -0
  55. package/doc/concept-learning.md +168 -0
  56. package/doc/creative-loop-map.md +202 -0
  57. package/doc/original-2016-doc.md +148 -0
  58. package/doc/usage.md +501 -0
  59. package/lib/authoring/abstract.js +401 -0
  60. package/lib/authoring/abstraction.js +190 -0
  61. package/lib/authoring/adapt.js +393 -0
  62. package/lib/authoring/ancestry.js +216 -0
  63. package/lib/authoring/author.js +147 -0
  64. package/lib/authoring/bounded-merge.js +57 -0
  65. package/lib/authoring/canon.js +158 -0
  66. package/lib/authoring/clock.js +123 -0
  67. package/lib/authoring/combinator.js +92 -0
  68. package/lib/authoring/compete.js +76 -0
  69. package/lib/authoring/compose-hotspot.js +314 -0
  70. package/lib/authoring/compress.js +279 -0
  71. package/lib/authoring/concept-net.js +254 -0
  72. package/lib/authoring/concepts.js +66 -0
  73. package/lib/authoring/context-project.js +277 -0
  74. package/lib/authoring/contract.js +476 -0
  75. package/lib/authoring/corpus-pack.js +125 -0
  76. package/lib/authoring/cost-probe.js +86 -0
  77. package/lib/authoring/crystallize.js +555 -0
  78. package/lib/authoring/dag-decompose.js +256 -0
  79. package/lib/authoring/dataset-adapter.js +253 -0
  80. package/lib/authoring/debug-provider.js +92 -0
  81. package/lib/authoring/decompose.js +343 -0
  82. package/lib/authoring/emittability.js +236 -0
  83. package/lib/authoring/equilibrium.js +160 -0
  84. package/lib/authoring/extract.js +153 -0
  85. package/lib/authoring/forest.js +55 -0
  86. package/lib/authoring/forge-fallback.js +116 -0
  87. package/lib/authoring/givens.js +121 -0
  88. package/lib/authoring/glossary.js +177 -0
  89. package/lib/authoring/grammar-graph.js +191 -0
  90. package/lib/authoring/granularity.js +80 -0
  91. package/lib/authoring/graph-net.js +108 -0
  92. package/lib/authoring/ground.js +148 -0
  93. package/lib/authoring/higher-order.js +85 -0
  94. package/lib/authoring/hotspot.js +132 -0
  95. package/lib/authoring/hysteresis.js +85 -0
  96. package/lib/authoring/index.js +106 -0
  97. package/lib/authoring/lattice-morphism.js +61 -0
  98. package/lib/authoring/lattice-pack.js +148 -0
  99. package/lib/authoring/leaf-io.js +77 -0
  100. package/lib/authoring/library.js +156 -0
  101. package/lib/authoring/lifecycle.js +92 -0
  102. package/lib/authoring/loop.js +193 -0
  103. package/lib/authoring/master-loop.js +132 -0
  104. package/lib/authoring/mdl.js +209 -0
  105. package/lib/authoring/memo-stability.js +122 -0
  106. package/lib/authoring/method-explorer.js +138 -0
  107. package/lib/authoring/method-pack.js +181 -0
  108. package/lib/authoring/method.js +202 -0
  109. package/lib/authoring/mine.js +510 -0
  110. package/lib/authoring/mount.js +119 -0
  111. package/lib/authoring/negotiate.js +100 -0
  112. package/lib/authoring/parametric.js +139 -0
  113. package/lib/authoring/reaggregate.js +75 -0
  114. package/lib/authoring/rebalance.js +157 -0
  115. package/lib/authoring/recall.js +135 -0
  116. package/lib/authoring/registry.js +350 -0
  117. package/lib/authoring/relearn.js +147 -0
  118. package/lib/authoring/retention.js +89 -0
  119. package/lib/authoring/segment-proxy.js +114 -0
  120. package/lib/authoring/serve-leaf.js +64 -0
  121. package/lib/authoring/slot-aware-serve.js +77 -0
  122. package/lib/authoring/sound-invoke.js +59 -0
  123. package/lib/authoring/split-serve.js +74 -0
  124. package/lib/authoring/ste.js +239 -0
  125. package/lib/authoring/stock.js +101 -0
  126. package/lib/authoring/store.js +94 -0
  127. package/lib/authoring/supervise.js +138 -0
  128. package/lib/authoring/support.js +131 -0
  129. package/lib/authoring/task-mirror.js +51 -0
  130. package/lib/authoring/typed-loop.js +160 -0
  131. package/lib/authoring/validate.js +569 -0
  132. package/lib/authoring/widen.js +88 -0
  133. package/lib/combos/appliance.js +189 -0
  134. package/lib/combos/critique.js +377 -0
  135. package/lib/combos/defaults.js +86 -0
  136. package/lib/combos/durable-runner.js +115 -0
  137. package/lib/combos/forge.js +209 -0
  138. package/lib/combos/index.js +86 -0
  139. package/lib/combos/learning-library.js +253 -0
  140. package/lib/combos/mixture-serve.js +193 -0
  141. package/lib/combos/plan-loop.js +192 -0
  142. package/lib/combos/proxy-cache.js +272 -0
  143. package/lib/combos/self-mod.js +94 -0
  144. package/lib/durable/audit.js +102 -0
  145. package/lib/durable/checkpoint-store.js +550 -0
  146. package/lib/durable/fold.js +62 -0
  147. package/lib/durable/interpreter.js +242 -0
  148. package/lib/durable/xlate.js +185 -0
  149. package/lib/graph/Graph.js +2486 -0
  150. package/lib/graph/expr.js +307 -0
  151. package/lib/graph/index.js +26 -0
  152. package/lib/graph/log.js +134 -0
  153. package/lib/graph/objects/Concept.js +382 -0
  154. package/lib/graph/objects/Entity.js +503 -0
  155. package/lib/graph/objects/Node.js +57 -0
  156. package/lib/graph/objects/PathMap.js +557 -0
  157. package/lib/graph/objects/Segment.js +100 -0
  158. package/lib/graph/tasks/stabilize.js +59 -0
  159. package/lib/graph/tasks/taskflow.js +247 -0
  160. package/lib/index.js +122 -0
  161. package/lib/load.js +132 -0
  162. package/lib/providers/backends.js +117 -0
  163. package/lib/providers/borderline.js +129 -0
  164. package/lib/providers/cache.js +226 -0
  165. package/lib/providers/canonicalize.js +198 -0
  166. package/lib/providers/constat.js +107 -0
  167. package/lib/providers/geo.js +53 -0
  168. package/lib/providers/index.js +116 -0
  169. package/lib/providers/intake.js +263 -0
  170. package/lib/providers/llm-local.js +88 -0
  171. package/lib/providers/llm.js +236 -0
  172. package/lib/providers/local-host.js +197 -0
  173. package/lib/providers/merge-consistency.js +162 -0
  174. package/lib/providers/nogood.js +149 -0
  175. package/lib/providers/reason-loop.js +223 -0
  176. package/lib/providers/semiring.js +285 -0
  177. package/lib/providers/solver-fork.js +163 -0
  178. package/lib/providers/stats.js +228 -0
  179. package/lib/providers/verify.js +120 -0
  180. package/lib/runtime/index.js +135 -0
  181. package/lib/runtime/invoke-pool.js +92 -0
  182. package/lib/runtime/protocol.js +210 -0
  183. package/lib/runtime/transport-socket.js +77 -0
  184. package/lib/runtime/worker-entry.js +31 -0
  185. package/lib/sg/cli.js +601 -0
  186. package/lib/sg/log-sinks.js +214 -0
  187. package/lib/sg/mcp.js +395 -0
  188. package/lib/sg/proxy-run.js +66 -0
  189. package/lib/sg/serve-demo.js +185 -0
  190. package/lib/sg/serve.js +205 -0
  191. package/lib/sg/trace.js +118 -0
  192. package/lib/studio/protocol.js +45 -0
  193. package/lib/studio/public/app.js +310 -0
  194. package/lib/studio/public/components/ConceptEditor.js +43 -0
  195. package/lib/studio/public/components/ConceptTree.js +43 -0
  196. package/lib/studio/public/components/CorpusPanel.js +68 -0
  197. package/lib/studio/public/components/ForkTree.js +36 -0
  198. package/lib/studio/public/components/GrammarGraph.js +99 -0
  199. package/lib/studio/public/components/GraphCanvas.js +156 -0
  200. package/lib/studio/public/components/Inspector.js +48 -0
  201. package/lib/studio/public/components/LearningPanel.js +85 -0
  202. package/lib/studio/public/components/PromptConsole.js +46 -0
  203. package/lib/studio/public/components/ProviderTrace.js +39 -0
  204. package/lib/studio/public/components/SessionSplit.js +47 -0
  205. package/lib/studio/public/components/TilingOverlay.js +40 -0
  206. package/lib/studio/public/components/Timeline.js +57 -0
  207. package/lib/studio/public/index.html +26 -0
  208. package/lib/studio/public/styles.css +185 -0
  209. package/lib/studio/public/ws.js +58 -0
  210. package/lib/studio/server.js +130 -0
  211. package/lib/studio/session.js +397 -0
  212. package/lib/studio/studio.js +149 -0
  213. package/package.json +42 -0
@@ -0,0 +1,202 @@
1
+ # The Creative Loop — a map of the distillation & dispatch bricks
2
+
3
+ > **What this is.** A practical, code-grounded map of the *creative / distillation loop* — the host-side layer that turns
4
+ > LLM/engine execution **traces** into a library of typed, defeasible, **composable** concept-Methods, and dispatches
5
+ > them deterministically. It answers the three questions the theory-heavy docs (`concept-as-graph.md`, `MODELISATION.md`)
6
+ > don't: **which bricks exist, which are core vs experimental, and how they chain end-to-end.** Every brick lives in
7
+ > `lib/authoring/` or `lib/providers/` and is **ZERO-CORE** (the `lib/graph/` engine is untouched). Companion to
8
+ > `architecture.md` (the engine) — this is the authoring/learning layer above it.
9
+ >
10
+ > Grounded by a per-module read of the real code (verdicts below cite who imports each brick + which tests cover it).
11
+ > Read alongside the barrel `lib/authoring/index.js` (curated one-line roles + the declared shelved tier) and the
12
+ > standing design direction `concept-as-graph.md` §9.
13
+
14
+ ## The mental model (one paragraph)
15
+
16
+ The engine is a rule-driven knowledge graph: *concepts* cast/uncast transformations onto objects until a fixpoint
17
+ (*stabilization*). The **creative loop sits on top**: when an LLM-driven run produces a recurring reasoning step, the loop
18
+ **forges** it into a *Method* — a typed, contracted, library-indexed sub-graph — so the *next* time that step is needed
19
+ it is **retrieved and mounted at zero model calls** instead of re-asked. Methods **compose** (a method's body slot is
20
+ filled by another dispatched method), are **verified** by a born-with-it defeasible contract at runtime, and are
21
+ **un-learned** when they drift. The whole thing is bounded by the **K1 barrier**: only the *recurrent + typed +
22
+ canonicalizable* fraction of behavior amortizes into the library; everything else stays in the model (the honest floor).
23
+ Two sub-systems meet: a **front door** (prose → a typed dispatch key, deterministically, LLM last-resort) and a
24
+ **distillation loop** (trace → mined structure → crystallized Method → composed → indexed).
25
+
26
+ ## The end-to-end spine
27
+
28
+ ```
29
+ ┌──────────────────────── FRONT DOOR — library-science dispatch (prose → typed key) ────────────────────────┐
30
+ │ prose/task ─▶ intake ────▶ canonicalize ───▶ (miss?) borderline ───▶ registry (Σ_sep) ──┐ │
31
+ │ (NLU: 27B) exact-match + LLM last-resort, curated + versioned │ │
32
+ │ → natural confluent synonym re-canonicalized, vocab CATALOG where │ │
33
+ │ words RING lookup PROVISIONAL/propose the rings LIVE │ │
34
+ │ │ hard miss │ propose-only │ │
35
+ │ ▼ CanonMiss ▼ (autonomous RegistryMerge loop) │ │
36
+ │ (fail-closed → ring grows ⇄ retractRingAlias (un-learn) │ │
37
+ │ host escalate) ▼ │
38
+ └────────────────────────────────────────────────────────────────────────▶ dispatch (library.js byKey) │
39
+ O(1) libraryKey bucket, no search│
40
+ ┌──────────────────── DISTILLATION LOOP — trace → Method ────────────────────┐ │ retrieve-or- │
41
+ │ traces ─▶ mine / antiUnify ─▶ gate ─▶ crystallize ─▶ compose ─────────────┼─▶ indexMethod─┤ forge │
42
+ │ (firing recurrent (2 paths) typed method blendMethods / │ (amortize: │ adaptOrForge │
43
+ │ records) STRUCTURE (LGG) +contract? synthesizeByBlend │ next hit=0) │ (adapt.js) │
44
+ │ depth-3, 0 forge │ ▼ │
45
+ │ │ runtime assertPost │
46
+ │ │ (contract.js = the MOAT) │
47
+ │ │ │ drift → blame │
48
+ │ └──▶ relearn (un-learn) + revise │
49
+ └────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
50
+ ```
51
+
52
+ The two halves share one object: the **typed dispatch key**. The front door's job is to make prose land on the *same*
53
+ key across paraphrases (so the memo hits); the distillation loop's job is to make a key *worth* having (a reusable Method
54
+ behind it). Break the typedness and the index degrades to linear matching or NP composition — the design law of
55
+ `concept-as-graph.md` §9.
56
+
57
+ ### ⚠ The one structural fact to internalize: there are TWO retrieve-or-forge stacks, over TWO libraries
58
+
59
+ This is the non-obvious thing a reader (and the cont.⁶ session) misses, so it is stated up front:
60
+
61
+ - **The dispatch / creative-loop stack** — `crystallize.js` → `library.js` (`byKey` index) → `adapt.js` (`adaptOrForge`
62
+ + `synthesizeByBlend`) → `combinator.js`. This is where **composition actually lives** (`synthesizeByBlend` grafts a
63
+ depth-3 method from existing parts at **0 forge calls**). It is proven by **real-engine integration tests** and reached
64
+ through `Graph.authoring`, but it is **NOT wired into the always-on loop** — nothing under `lib/` calls `adaptOrForge`
65
+ except the barrel.
66
+ - **The always-on master loop** — `master-loop.js` → `recall.js` (fuzzy-recall → exact-verify) + a cache + `mount.js`.
67
+ A cost-ordered arm ladder (escalate / match / recall-full / recall-partial / forge). It runs on the **recall index +
68
+ cache**, a *different* library structure from `library.js#byKey` — and it is what `method-pack.js` packages.
69
+
70
+ So "the library" is two things. **Unifying these two stacks on the system's own trace is precisely NEXT #1** (the
71
+ end-to-end distillation loop, below) — the bricks all exist and are individually tested; the *whole chain* has never
72
+ been run and measured as one live loop.
73
+
74
+ ## The bricks, by stage
75
+
76
+ > Each table: **module** · the verbs it exports · **feeds →** · **tier** (defined below). `file:line` anchors are in the
77
+ > notes, not the tables, to keep them scannable. **bold module** = the stage's linchpin.
78
+
79
+ ### 1 · DETECT — trace → scored generalization candidate
80
+
81
+ | module | exports (verbs) | feeds → | tier |
82
+ |---|---|---|---|
83
+ | **`mine.js`** | `traceMiner`/`methodTrace` (collect firings) · `mineChains` (producer→consumer edges) · `mineMethods` (recurrent structural candidates) | crystallize / mdl+abstraction | CORE |
84
+ | `abstract.js` | `antiUnify` (Plotkin-LGG stability) · `relativize`/`instantiate` (id-hole rewrite) · `generalizeContent`/`fillContentHoles`/`blendAtSegment` | mine, adapt, crystallize, combinator | SHARED-INSTRUMENT |
85
+ | `mdl.js` | `mdlGain`/`rankCandidates` (bits ΔL) · `makeMdlGate` | crystallize (edge path only) | CORE (narrow) |
86
+ | `abstraction.js` | `evaluate` (double engine-boot MDL/utility admit) · `interfaceRegression` | crystallize (the admit authority) | CORE |
87
+ | `decompose.js` | `treeDecomposition`/`forkPlan` · `bagInterface`/`separatorGate` (Σ_sep) | abstraction, extract, registry | SHARED-INSTRUMENT |
88
+ | `ancestry.js` | `decideLeaf`/`promoteContentVars`/`enhanceCandidateWithAncestry` (bake/promote/forge) | *(test-only; enhances a candidate)* | EXPERIMENTAL |
89
+
90
+ **Notes.** `antiUnify` is the actual scorer and lives in **`abstract.js:144`**, not `mine.js` (which delegates to it).
91
+ There are **two mining paths**: `mineChains` (data-flow *edges*, MDL-ranked by `mdl.js` then admitted by
92
+ `abstraction.evaluate`'s double engine boot) and `mineMethods` (recurrent *structure*, gated by antiUnify-stability + the
93
+ K1 `signatureDetermined` check, **no MDL**). `ancestry.js` is a research post-processor: **zero `lib/` importers, absent
94
+ from the barrel** — reached only by its own integration tests. Dead/superseded: `emitMethodAsSubgraph` (kept as
95
+ `emitEquivalence` instrumentation), `inferCtx` (superseded by `declaredCtx`).
96
+
97
+ ### 2 · COMPOSE & CRYSTALLIZE — candidate → indexed, dispatchable, contracted Method
98
+
99
+ | module | exports (verbs) | feeds → | tier |
100
+ |---|---|---|---|
101
+ | **`crystallize.js`** | `crystallizeStructural` (→ candidate: frontier + contract? + `libraryKey` + `templatesBySig`) · `synthesizeContract` (defeasible post, or null) · `reifyFrontier`/`libraryKey` | library.indexMethod | CORE |
102
+ | `library.js` | `indexMethod` (file under `byKey`) · `dispatch` (O(1) bucket + app-cond refine) · `dispatchInterface` (loosened recall) · `appConditionsHold` | adapt, combinator | CORE |
103
+ | `adapt.js` | `adaptOrForge` (retrieve/adapt/forge → verifier-gate → index-back) · `synthesizeByBlend` (depth-3 compose @0 calls) · `blendMethods`/`composeContract` · `antiUnifyAdapt` | library.indexMethod (amortize) | CORE-of-loop |
104
+ | `combinator.js` | `dispatchConcept`/`buildDispatchProvider` (0-call cross-concept mount; durable `Done` guard) | engine mount | CORE |
105
+ | `method.js` | `lintMethod` (decidability/frame lint) · `applySubgraphArg`/`mapSubgraph` · `selectCluster` | crystallize.lintFrontier | SHARED-INSTRUMENT |
106
+ | `method-pack.js` | `packMethods`/`unpackMethods`/`loadMethods` (`.sgc` portable, B8 version gate) | the master-loop recall library | SHARED-INSTRUMENT |
107
+
108
+ **Notes.** **Composition already exists** and lives here: `synthesizeByBlend` (`adapt.js:200`) + `blendMethods` +
109
+ `abstract.blendAtSegment` graft a depth-3 method from existing `templatesBySig` parts at `calls:0`, under a terminating
110
+ μ-descent on `methodDepth` — "a cache cannot (it replays); crystallize cannot without ≥2 traces." `adaptOrForge` indexes
111
+ the forged/adapted method back (`adapt.js:168`) so the *next* encounter hits at 0 calls (the amortization).
112
+ **Correction to the folklore:** the "refuse a method with no sound contract" gate is **here** (`adapt.js#hasSoundContract`,
113
+ reject at `:159`), **not** in `crystallizeStructural` — that admits a contractless method (`synthesizeContract` may return
114
+ null at `:263`, yet `:502`/`:506` still return `admitted:true`); it refuses on *other* soundness grounds
115
+ (untyped-behavioral-param, echoed-require, ambiguity, leak). The `adapt.js:16` comment claiming it mirrors
116
+ crystallize's refusal is inaccurate.
117
+
118
+ ### 3 · FRONT DOOR — prose → deterministic typed dispatch key (LLM last-resort)
119
+
120
+ | module | exports (verbs) | feeds → | tier |
121
+ |---|---|---|---|
122
+ | `intake.js` (C0) | `createIntake` → `Intake::type` (prose → typed facts + digest; own miss-loop; `resolveFacts` seam) | canonicalize; borderline (on miss); registry | CORE |
123
+ | **`canonicalize.js`** | `canonValue`/`canonFacts` (exact + confluent ring snap; **no embedding**) · `digest` (memo key) · `compileEnumMap` · `normToken` | intake, borderline, registry, +8 authoring | CORE |
124
+ | `borderline.js` | `makeBorderlineSnap` (LLM last-resort on a miss; re-canonicalized; provisional + propose-only) · `enumGbnf` · `pickMember` | intake (opt-in seam); registry (proposals) | CORE |
125
+ | `registry.js` (Σ_sep) | `deriveRegistry`/`freezeRegistry` · `specForKey`/`resolveFactsSchema` · `mergeRingProposals` (admit + confluence re-check) · `retractRingAlias` (un-learn) · `registryLoopTree`/`Reg::merge` (autonomous loop) | intake (resolveFacts); canonicalize (specForKey) | CORE |
126
+ | `emittability.js` | `profile`/`perTaskStats`/`fleissKappa`/`poolAgreement` (paraphrase-stability profiler) | *(tests only)* | SHARED-INSTRUMENT |
127
+
128
+ **Notes.** Prose crosses into the typed world **exactly once**, at `Intake::type`. `canonicalize` is the single
129
+ soundness barrier (the most-imported module in `lib/`) and does a **one-probe** exact + case/ws-normalized + curated-ring
130
+ lookup — never an embedding or edit-distance (the HARD RULE, `canonicalize.js:32`). A reusable `FactsDigest` is minted
131
+ **only** when `IntakeStatus==='typed'` (`intake.js:191`), so a miss can never false-hit the memo; a borderline guess rides
132
+ an untracked `<name>Borderline` fact and stays un-cacheable. The vocabulary **self-grows** — a propose-only ring proposal
133
+ is deposited on a proxy node, and the autonomous `RegistryMerge` loop admits it (member ∈ enum ∧ confluence re-checked) —
134
+ and stays **reversible** via `retractRingAlias`, which de-locks first-writer-wins confluence and version-bumps to
135
+ invalidate anything typed through the retracted alias.
136
+
137
+ ### 4 · SOUNDNESS SPINE, GATES & CONTROL
138
+
139
+ | module | exports (verbs) | role | tier |
140
+ |---|---|---|---|
141
+ | **`contract.js`** | `assertPost` (runtime monitor) · `checkCompose`/`entailsKey`/`footprintCycles` (static ⊨) · `reviseOnBlame` (CEGIS) · `widenOnVerified` · `holdsAtoms` (live admission) | defeasible soundness | CORE (runtime) |
142
+ | `relearn.js` | `makeRelearnProviders` (`Lib::blame`/`Lib::revise`) · `relearnTree` | the reference **un-learn** loop | CORE (opt-in) |
143
+ | `recall.js` | `createRecallIndex` (fuzzy cosine) · `verify` (structure-exact → full/partial/reject) · `recallAndVerify` | the master-loop RETRIEVE arm | CORE |
144
+ | `master-loop.js` | `createMasterLoop` → `{solve, drift, stats}` (cost-ordered arm ladder) | the **always-on driver** | CORE |
145
+ | `loop.js` | `loopConceptTree`/`reactiveLoopConceptTree` · `makeDecomposeProviders` · `synthesize` | decompose→answer DAG ("answer a huge prompt") | CORE |
146
+ | `extract.js` | `extractSubgraph` (k-hop slice + frozen frontier) · `mergeSlice` (single-writer + assumption-recheck) | the fork / multi-process ship lever | SHARED-INSTRUMENT |
147
+ | `widen.js` | `widenTree`/`makeWidenProviders` | test-under-drift **widen** (pairs with relearn) | SHARED-INSTRUMENT |
148
+ | `compose-hotspot.js` | `composeHotspots` (4-way verdict) · `provenanceChains` · `anyComposeCandidate` | `compress.js` gate — STRUCTURAL half | INSTRUMENT (not wired) |
149
+ | `cost-probe.js` | `costProbe`/`paysToCompress` | `compress.js` gate — COST half | INSTRUMENT (not wired) |
150
+
151
+ **Notes.** The contract's soundness is **asymmetric**: the **runtime** half is the load-bearing moat — `assertPost` is
152
+ wired into the durable interpreter's assert-before-commit (`interpreter.js:86`) and `holdsAtoms` gates live admission
153
+ (`library.js`) — while the **static** compose-time half (`checkCompose`/`footprintCycles`) has **no live caller**
154
+ (instrument + tests only; its own docstring says the runtime assert is the load-bearing one). `relearn.js` is the
155
+ autonomous un-learn loop that **`registry.js`'s `RegistryMerge` and `widen.js` both copy** (blame → revise →
156
+ `patchConcept`, as reactive concepts). `compress.js` **does not exist on disk**; its two gates are decided *instruments*,
157
+ never wired into dispatch — and per the cont.⁶ correction their standing "NO-GO" is **reopened** (see below): the
158
+ `cost-probe` framing measured elided *forge* calls, but `synthesizeByBlend` already composes at 0 forge, so the baseline
159
+ was wrong. The instruments are sound; the decision is open.
160
+
161
+ ## The three tiers (core vs experimental) — the honest split
162
+
163
+ - **CORE — load-bearing and exercised by real-engine integration tests.** Front door: `intake`, `canonicalize`,
164
+ `borderline`, `registry`. Detect: `mine`, `mdl`, `abstraction`. Compose: `crystallize`, `library`, `combinator`, and
165
+ `adapt` (**core *of the creative loop*, but reached via `Graph.authoring` + tests, not the always-on master loop**).
166
+ Spine/control: `contract` (runtime half), `relearn`, `recall`, `master-loop`, `loop`.
167
+ - **SHARED-INSTRUMENT — measurement, gates, and primitives, not on a live runtime path.** `abstract` (the LGG primitive
168
+ everyone builds on), `decompose` (Σ_sep), `method`, `method-pack`, `emittability`, `extract`, `widen`, and the two
169
+ `compress.js` gates `compose-hotspot` + `cost-probe`. Sound *as instruments*; a green instrument is not a shipped
170
+ feature.
171
+ - **SHELVED / EXPERIMENTAL.** Marked `(shelved)` in the barrel: `conceptNet`, `graphNet`, `equilibrium`, `ste` (the
172
+ probabilistic concept-net line — additive research, studies filed, off the critical path). Plus `ancestry.js`
173
+ (unwired research post-processor).
174
+
175
+ ## Where the open questions sit (mapping the roadmap onto the bricks)
176
+
177
+ The owner-agreed NEXT is **integration + measurement, not new operators** — the bricks exist; two are un-chained/un-decided:
178
+
179
+ 1. **The end-to-end distillation loop, MEASURED on a narrow real domain** *(needs owner go — touches the program core).*
180
+ Every brick in stages 1→2→4 exists and is tested *in isolation*; what has never been done is running the *whole chain*
181
+ on the system's **own emergent trace** and measuring reuse (elided calls, composability, before/after). Concretely
182
+ this is also **wiring the dispatch stack into a live loop** — closing the "two stacks, two libraries" gap above.
183
+ 2. **Re-open `compress.js` EMPIRICALLY, against `synthesizeByBlend`** *(needs owner go).* The cont.⁶ verdict
184
+ ("compress.js filed on the forge-cost axis") is **void**: `adapt.js#synthesizeByBlend` already composes at depth-3 for
185
+ 0 forge calls and `indexMethod` amortizes, so `cost-probe` (which counts elided *forge* calls) measured the wrong
186
+ baseline. The honest question is whether *offline proactive mining* beats *on-demand blend + index-on-first-encounter*
187
+ on **search cost / latency / coverage** — measured on the trace question 1 produces. `composeHotspot` + `costProbe`
188
+ remain the right *instruments*; only the framing was wrong.
189
+ 3. **Debt (no go needed).** A **scalability bench** N=1000+ (never measured; `extract.js` is the fork lever to test), and
190
+ the **C0 back-check** depth (a CEGIS re-check of the intake result vs the prose). The **NUL/CI** gap is closed
191
+ (`tests/unit/source-hygiene.test.js`).
192
+
193
+ **The standing guardrail:** any "new idea" from an external LLM about this repo must first be grepped in `lib/authoring/`
194
+ before treating it as work — the already-built rate is very high (both external reviews re-derived the *existing* archi).
195
+ Assemble + measure; don't re-code.
196
+
197
+ ## Reading order for the next agent
198
+
199
+ 1. This map (the substrate in one view) → 2. `concept-as-graph.md` §9 (the formal frame / standing direction) →
200
+ 3. `adapt.js` (`adaptOrForge` / `synthesizeByBlend`) + `library.js` (dispatch) → 4. `contract.js` (the moat) +
201
+ `relearn.js` (un-learn) → 5. the front door (`intake.js` → `canonicalize.js` → `borderline.js` → `registry.js`) →
202
+ 6. `doc/WIP/HANDOFF.md` (the live ledger) for the current fork.
@@ -0,0 +1,148 @@
1
+ > **Note (EN).** This is the concept-schema reference (the rule language), kept in French. For an English overview of the same schema, see [architecture.md](architecture.md) §3 and [API.md](API.md).
2
+
3
+ ## Objectifs
4
+
5
+ Les principaux objectifs du graphe sont :
6
+ - Permettre un enrichissement générique des chemins via l'exploration de sources de données multiples.
7
+ - Permettre une géneration de nouveaux chemins générique en se basant sur les données obtenue dynamiquement
8
+ - Permettre une synchronisation constante entre le serveur et le client afin, entre autre, de permettre l'edition collaborative de graphe. Mais aussi de répartir les requêtes générées.
9
+
10
+ ## Principes, architecture, Concepts & entities
11
+
12
+
13
+ Afin de permettre un enrichissement dynamique et générique des données, le graphe utilise un système théoriquement proche des grammaires en théorie des langages.
14
+
15
+ En "théorie" chaque régle décrit une serie de symboles qui, si ils sont présent et vérifie certaines contraintes, généreront d'autre symboles.
16
+
17
+ Dans le graphe, des entités appelé 'concepts' joue le role de régle, et les objets du graphe (noeuds, segments, records), nommé "entity" sont les contextes sur lesquels s'appliquent ces concepts.
18
+ Les symboles étant quant à eux, assimilé aux clefs ou valeurs accessible dans le graphe.
19
+
20
+ Le graphe applique donc, tant que possible, pour chaque objet les concepts applicable en fonction des concepts & données déjà présents.
21
+
22
+ Quant plus aucun concept n'est applicable, le graphe est considéré comme 'stable'.
23
+
24
+ Lors de l'application d'un concepts, une série de mutations est appliquée. Celles ci peuvent soit être implicitement définie dans le concept, soit être généré par une fonction provider synchrone ou asynchrone.
25
+
26
+ Les concepts peuvent aussi être dynamique, si les conditions de leur application ne sont plus remplie, ils peuvent être automatiquement supprimé / nettoyée.
27
+
28
+ Le graphe est synchronisé "en temps réél" / atomiquement entre le serveur et le client, différents sets de concepts sont disponible et s'appliquent en fonction du contexte (client, serveur, graphe d'exploration)
29
+
30
+ A noter que toute mutation appliqué au graphe serveur est automatiquement diffusé à tout les clients utilisant le graphe courant. A l'inverse, le graphe client, ne push ses mutations qu'à la demande.
31
+
32
+ 3 types d'objets source sont consideré par le graphe et les jeux de concepts :
33
+
34
+ ### 1 - Noeuds
35
+
36
+ {
37
+ Node : true // entry point
38
+ }
39
+
40
+ ### 2 - Segments
41
+
42
+
43
+ {
44
+ Segment : true// concept's EP
45
+ originNode : {objId} // auto référencé
46
+ targetNode : {objId}
47
+ }
48
+
49
+ ### 3 - Documents
50
+
51
+
52
+ {
53
+ Record : true
54
+ }
55
+
56
+
57
+ ## Références
58
+
59
+ Afin de cibler & référencer simplement et relativement les valeurs dans le graphe on utilise une syntaxe dédié:
60
+
61
+ (\$someId)?((hashMapKey.)|(referenceKey:))*(anyTargetKey)
62
+
63
+ Dans les assert, ensure & follow, on rajoute un $ pour différencier les références des valeurs numériques
64
+
65
+ \$((hashMapKey.)|(referenceKey:))*(anyTargetKey)
66
+
67
+
68
+ ## Template de mutations
69
+
70
+ Afin de permettre de muter le graphe de façon "atomique" et scalable, un langage de templates y est dédié.
71
+
72
+ ### Exemple/schéma — [Graph.js](../lib/graph/Graph.js)
73
+
74
+ [
75
+ {
76
+ "$_id": "_parent", // ajoute les clefs de cet objet sur l'objet ou s'applique le concept d'origine de cette mutation
77
+ "SomeConcept": true,// applique "de force" un concept (ses providers & tpl seront ignorés)
78
+ "Distance":null,// null will uncast the concept
79
+ "$originNode": "localReference",
80
+ "$targetNode": "$someGraphObjId" // value of key starting with '$' will be evaluated from the _parent scope
81
+ },
82
+ {
83
+ "_id": "localReference", // new node
84
+ "Node":true,
85
+ "$pathDescriptor": "someDescriptor"
86
+ },
87
+ {
88
+ "_id": "someDescriptor", // path descriptor are included when parsing paths
89
+ "Record":true,
90
+ "$parentPathDescriptor": "someParentDescriptor"// parent path descriptors are recursivly included too
91
+ },
92
+ {
93
+ "_id": "someParentDescriptor",
94
+ "Record":true
95
+ }
96
+ ]
97
+
98
+ ## Concepts
99
+ ### Définition
100
+
101
+ Les concepts sont groupé par "concept set", chacun s'appliquant dans des contextes différents.
102
+
103
+ Dans ces sets de concepts, les concepts sont organisée hiérarchiquement; si un répertoire porte le nom d'un concept et que ce concept existe (cad une clefs du nom du concept existe sur le contexte d'application), les concepts présent dans ce répertoire deviennent potentiellement applicable.
104
+
105
+ Enfin si un concept est supprimé, ses concepts enfant sont préalablement supprimés.
106
+
107
+ ### Schéma
108
+
109
+ Un concept est considéré comme appliqué si une clefs (du nom du concept) existe sur l'objet avec une valeur != de undefined
110
+
111
+ ```
112
+ {
113
+ "autoCast":false,// si false le concepts n'est pas appliqué automatiquement
114
+ //par cette définition/contexte
115
+
116
+ "syncAfter":true,// provoque une synchronisation du client vers le serveur
117
+ // dés que le graphe est stable (des concepts serveurs pourront alors s'appliquer)
118
+
119
+ "require": [ // N'applique ce concept que si les valeurs suivante existe :
120
+ "originNode:Position",// Position sur l'objet référencé dans originNode
121
+ "targetNode:Position"// same
122
+ ],
123
+ "assert" : [// N'appliquer ce concept que si les condition suivantes sont remplie
124
+ "$value == 8"
125
+ ],
126
+ "ensure" : [// Pareil qu'assert mais supprime le concept si la condition n'est plus remplie
127
+ "$value == 8"
128
+ ],
129
+ "follow" : [// Réapplique le concept à chaque update de stuffSomewhere
130
+ "$someRef:stuffSomewhere"
131
+ ],
132
+ "provider": [// fonction à appeler lors de l'application du concept
133
+ "Common::Distance",
134
+ {},"arg"// arguments...
135
+ ],
136
+ "cleaner": [ // fonction à appeller lors de la suppression d'un concept
137
+ "User::UnCastWidget"
138
+ ],
139
+ "defaultValue": ["some", "values"], // valeur du concept par default si appliqué sans tpl/provider
140
+
141
+ "applyMutations": [// template de mutation appliqué par default lors du cast du concept
142
+ {
143
+ "$_id":"_parent",
144
+ "MyConcept" : true // l'application d'un concept doit créer une clefs à son nom
145
+ }
146
+ ]
147
+ }
148
+ ```