@hviana/sema 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (110) hide show
  1. package/AGENTS.md +470 -0
  2. package/AUTHORS.md +12 -0
  3. package/COMMERCIAL-LICENSE.md +19 -0
  4. package/CONTRIBUTING.md +15 -0
  5. package/HOW_IT_WORKS.md +4210 -0
  6. package/LICENSE.md +117 -0
  7. package/README.md +290 -0
  8. package/TRADEMARKS.md +19 -0
  9. package/example/demo.ts +46 -0
  10. package/example/train_base.ts +2675 -0
  11. package/index.html +893 -0
  12. package/package.json +25 -0
  13. package/src/alphabet.ts +34 -0
  14. package/src/alu/README.md +332 -0
  15. package/src/alu/src/alu.ts +541 -0
  16. package/src/alu/src/expr.ts +339 -0
  17. package/src/alu/src/index.ts +115 -0
  18. package/src/alu/src/kernel-arith.ts +377 -0
  19. package/src/alu/src/kernel-bits.ts +199 -0
  20. package/src/alu/src/kernel-logic.ts +102 -0
  21. package/src/alu/src/kernel-nd.ts +235 -0
  22. package/src/alu/src/kernel-numeric.ts +466 -0
  23. package/src/alu/src/operation.ts +344 -0
  24. package/src/alu/src/parser.ts +574 -0
  25. package/src/alu/src/resonance.ts +161 -0
  26. package/src/alu/src/text.ts +83 -0
  27. package/src/alu/src/value.ts +327 -0
  28. package/src/alu/test/alu.test.ts +1004 -0
  29. package/src/bytes.ts +62 -0
  30. package/src/config.ts +227 -0
  31. package/src/derive/README.md +295 -0
  32. package/src/derive/src/deduction.ts +263 -0
  33. package/src/derive/src/index.ts +24 -0
  34. package/src/derive/src/priority-queue.ts +70 -0
  35. package/src/derive/src/rewrite.ts +127 -0
  36. package/src/derive/src/trie.ts +242 -0
  37. package/src/derive/test/derive.test.ts +137 -0
  38. package/src/extension.ts +42 -0
  39. package/src/geometry.ts +511 -0
  40. package/src/index.ts +38 -0
  41. package/src/ingest-cache.ts +224 -0
  42. package/src/mind/articulation.ts +137 -0
  43. package/src/mind/attention.ts +1061 -0
  44. package/src/mind/canonical.ts +107 -0
  45. package/src/mind/graph-search.ts +1100 -0
  46. package/src/mind/index.ts +18 -0
  47. package/src/mind/junction.ts +347 -0
  48. package/src/mind/learning.ts +246 -0
  49. package/src/mind/match.ts +507 -0
  50. package/src/mind/mechanisms/alu.ts +33 -0
  51. package/src/mind/mechanisms/cast.ts +568 -0
  52. package/src/mind/mechanisms/confluence.ts +268 -0
  53. package/src/mind/mechanisms/cover.ts +248 -0
  54. package/src/mind/mechanisms/extraction.ts +422 -0
  55. package/src/mind/mechanisms/recall.ts +241 -0
  56. package/src/mind/mind.ts +483 -0
  57. package/src/mind/pipeline-mechanism.ts +326 -0
  58. package/src/mind/pipeline.ts +300 -0
  59. package/src/mind/primitives.ts +201 -0
  60. package/src/mind/rationale.ts +275 -0
  61. package/src/mind/reasoning.ts +198 -0
  62. package/src/mind/recognition.ts +234 -0
  63. package/src/mind/resonance.ts +0 -0
  64. package/src/mind/trace.ts +108 -0
  65. package/src/mind/traverse.ts +556 -0
  66. package/src/mind/types.ts +281 -0
  67. package/src/rabitq-hnsw/README.md +303 -0
  68. package/src/rabitq-hnsw/src/database.ts +492 -0
  69. package/src/rabitq-hnsw/src/heap.ts +90 -0
  70. package/src/rabitq-hnsw/src/hnsw.ts +514 -0
  71. package/src/rabitq-hnsw/src/index.ts +15 -0
  72. package/src/rabitq-hnsw/src/prng.ts +39 -0
  73. package/src/rabitq-hnsw/src/rabitq.ts +308 -0
  74. package/src/rabitq-hnsw/src/store.ts +994 -0
  75. package/src/rabitq-hnsw/test/hnsw.test.ts +1213 -0
  76. package/src/store-sqlite.ts +928 -0
  77. package/src/store.ts +2148 -0
  78. package/src/vec.ts +124 -0
  79. package/test/00-extract.test.mjs +151 -0
  80. package/test/01-floor.test.mjs +20 -0
  81. package/test/02-roundtrip.test.mjs +83 -0
  82. package/test/03-recall.test.mjs +98 -0
  83. package/test/04-think.test.mjs +627 -0
  84. package/test/05-concepts.test.mjs +84 -0
  85. package/test/08-storage.test.mjs +948 -0
  86. package/test/09-edges.test.mjs +65 -0
  87. package/test/11-universality.test.mjs +180 -0
  88. package/test/13-conversation.test.mjs +228 -0
  89. package/test/14-scaling.test.mjs +503 -0
  90. package/test/15-decomposition-gap.test.mjs +0 -0
  91. package/test/16-bridge.test.mjs +250 -0
  92. package/test/17-intelligence.test.mjs +240 -0
  93. package/test/18-alu.test.mjs +209 -0
  94. package/test/19-nd.test.mjs +254 -0
  95. package/test/20-stability.test.mjs +489 -0
  96. package/test/21-partial.test.mjs +158 -0
  97. package/test/22-multihop.test.mjs +130 -0
  98. package/test/23-rationale.test.mjs +316 -0
  99. package/test/24-generalization.test.mjs +416 -0
  100. package/test/25-embedding.test.mjs +75 -0
  101. package/test/26-cooperative.test.mjs +89 -0
  102. package/test/27-saturation-drop.test.mjs +637 -0
  103. package/test/28-unknowable.test.mjs +88 -0
  104. package/test/29-counterfactual.test.mjs +476 -0
  105. package/test/30-conflict-resolution.test.mjs +166 -0
  106. package/test/31-audit.test.mjs +288 -0
  107. package/test/32-confluence.test.mjs +173 -0
  108. package/test/33-multi-candidate.test.mjs +222 -0
  109. package/test/34-cross-region.test.mjs +252 -0
  110. package/tsconfig.json +16 -0
@@ -0,0 +1,4210 @@
1
+ # How Sema Works
2
+
3
+ **A complete account of the theory and the algorithm — from the mathematical
4
+ foundations to the full inference pipeline, in plain language.**
5
+
6
+ This document explains _concepts_ and _algorithms_. It deliberately contains no
7
+ repository-level detail (file layout, module names, build and test concerns):
8
+ for that, see [AGENTS.md](AGENTS.md), the development manual. Everything here is
9
+ stated so that a careful reader with no background in machine learning — human
10
+ or machine — can follow it from first principles.
11
+
12
+ ---
13
+
14
+ ## Table of contents
15
+
16
+ - **Part I — Foundations**
17
+ - [1. What kind of AI is this?](#1-what-kind-of-ai-is-this)
18
+ - [2. Vector Symbolic Architectures](#2-vector-symbolic-architectures)
19
+ - [3. Content-addressable memory and the Merkle DAG](#3-content-addressable-memory-and-the-merkle-dag)
20
+ - [4. Distributional structure](#4-distributional-structure)
21
+ - [5. Automated deduction: the lightest derivation](#5-automated-deduction-the-lightest-derivation)
22
+ - [6. Approximate nearest-neighbour search](#6-approximate-nearest-neighbour-search)
23
+ - **Part II — The big picture**
24
+ - [7. How the five foundations connect](#7-how-the-five-foundations-connect)
25
+ - [8. Derived thresholds: the geometry of every decision](#8-derived-thresholds-the-geometry-of-every-decision)
26
+ - [9. The concept inventory](#9-the-concept-inventory)
27
+ - **Part III — The ingestion pipeline**
28
+ - [10. Perception: from bytes to a tree](#10-perception-from-bytes-to-a-tree)
29
+ - [11. Deposition: interning the tree into the graph](#11-deposition-interning-the-tree-into-the-graph)
30
+ - [12. Learning relations: edges and halos](#12-learning-relations-edges-and-halos)
31
+ - [13. Ingestion, end to end](#13-ingestion-end-to-end)
32
+ - **Part IV — The inference pipeline**
33
+ - [14. The shape of an answer](#14-the-shape-of-an-answer)
34
+ - [15. Recognition: decomposing the query](#15-recognition-decomposing-the-query)
35
+ - [16. Computation: extensions and the ALU](#16-computation-extensions-and-the-alu)
36
+ - [17. The consensus climb: points of attention](#17-the-consensus-climb-points-of-attention)
37
+ - [18. Grounding I — counterfactual transfer (CAST)](#18-grounding-i--counterfactual-transfer-cast)
38
+ - [19. Grounding II — cover: the graph search](#19-grounding-ii--cover-the-graph-search)
39
+ - [20. Grounding III — extraction by skill](#20-grounding-iii--extraction-by-skill)
40
+ - [21. Grounding IV — recall by resonance](#21-grounding-iv--recall-by-resonance)
41
+ - [22. Reasoning: the multi-hop chain](#22-reasoning-the-multi-hop-chain)
42
+ - [23. Fusion: multi-topic answers](#23-fusion-multi-topic-answers)
43
+ - [24. Articulation: answering in the asker's words](#24-articulation-answering-in-the-askers-words)
44
+ - [25. Disambiguation: choosing among alternatives](#25-disambiguation-choosing-among-alternatives)
45
+ - [26. Auditability: provenance and the rationale](#26-auditability-provenance-and-the-rationale)
46
+ - **Part V — The whole algorithm in pseudocode**
47
+ - [27. End-to-end pseudocode](#27-end-to-end-pseudocode)
48
+ - **Part VI — Reference**
49
+ - [28. Glossary](#28-glossary)
50
+ - [29. Complexity summary](#29-complexity-summary)
51
+ - [30. Bibliography](#30-bibliography)
52
+
53
+ ---
54
+
55
+ ---
56
+
57
+ # Part I — Foundations
58
+
59
+ Sema rests on five independent bodies of theory, each decades old and each well
60
+ established in the academic literature. None of them is a neural network, and
61
+ none of them requires training in the gradient-descent sense. Part I presents
62
+ each one on its own terms; Part II shows how they interlock.
63
+
64
+ ---
65
+
66
+ ## 1. What kind of AI is this?
67
+
68
+ ### 1.1 The classification problem
69
+
70
+ Sema is not a large language model: it has no learned weight matrices, no
71
+ gradient descent, no probabilistic sampling. But calling it simply "symbolic AI"
72
+ is also inaccurate — classical symbolic AI (logic programming, production
73
+ systems, description logics) operates on discrete tokens with exact matching and
74
+ has no notion of _similarity_, _generalization by proximity_, or _graceful
75
+ degradation_, all of which Sema exhibits. And industry labels such as "RAG +
76
+ Reasoner" describe _pipelines of separate components_ (a retriever feeding a
77
+ generator), which misrepresents Sema's single-mechanism design and has no
78
+ standing in the academic literature as a category of system.
79
+
80
+ The academically precise classification is:
81
+
82
+ > **Sema is a non-parametric, instance-based reasoning system: a Vector Symbolic
83
+ > Architecture (VSA) coupled to a content-addressable memory, with inference
84
+ > performed by weighted automated deduction.**
85
+
86
+ Each term in that sentence is a recognised concept with its own literature:
87
+
88
+ | Term | Meaning | Field |
89
+ | :-------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------- |
90
+ | **Non-parametric** | The system has no fixed-size parameter vector that training adjusts. Capacity grows with the data itself; the stored instances _are_ the model. | Statistics; machine learning |
91
+ | **Instance-based** (memory-based) | Learning is storing experiences; generalization happens at _query time_ by comparing the query to stored instances, not at training time by fitting a function. | Machine learning; cognitive science exemplar theory |
92
+ | **Vector Symbolic Architecture** | Structured knowledge (sequences, trees, role–filler bindings) is encoded into fixed-width high-dimensional vectors using algebraic operations (binding, superposition), so that _similarity of structure_ becomes _proximity of vectors_. | Connectionist/cognitive modelling (Plate 1995; Kanerva 2009; Gayler 2003) |
93
+ | **Content-addressable memory** | Items are retrieved by _what they are_ (their content or something similar to it), not by _where they are_ (an address or key assigned externally). | Computer architecture and associative-memory theory |
94
+ | **Weighted automated deduction** | Answers are _derived_ by applying inference rules with costs, and the system returns the derivation of minimal total cost — a strict generalization of shortest-path search. | Automated reasoning; parsing theory (Knuth 1977; Goodman 1999; Felzenszwalb & McAllester 2007) |
95
+
96
+ ### 1.2 What this system is _not_
97
+
98
+ To sharpen the category, the contrasts matter as much as the definition:
99
+
100
+ - **Not a parametric statistical model.** An LLM compresses its corpus into a
101
+ fixed number of floating-point weights and answers by sampling from a learned
102
+ conditional distribution. Sema stores its corpus as an explicit, losslessly
103
+ reconstructible graph and answers by _deduction over that graph_. There is no
104
+ distribution to sample; identical inputs always produce identical outputs.
105
+
106
+ - **Not classical (crisp) symbolic AI.** In a Prolog-style system, `mona_lisa`
107
+ and `monalisa` are unrelated atoms; nothing matches unless it unifies exactly.
108
+ In Sema, every stored structure also carries a high-dimensional vector (its
109
+ _gist_), and geometric proximity of gists gives the system the soft matching,
110
+ analogy, and noise tolerance that crisp symbols lack — while the underlying
111
+ identities remain exact and content-addressed.
112
+
113
+ - **Not a retrieval pipeline with a bolted-on reasoner.** In a
114
+ retrieval-augmented generation system, retrieval and generation are two
115
+ different mechanisms with an interface between them. In Sema there is one
116
+ mechanism: a single lightest-derivation search whose _axioms_ come from
117
+ recognition and resonance (the "retrieval") and whose _rules_ are the learned
118
+ edges and compositions (the "reasoning"). Retrieval and inference are two
119
+ descriptions of one search.
120
+
121
+ - **Not neuro-symbolic in the usual sense.** The term "neuro-symbolic" usually
122
+ denotes hybrids of neural networks with symbolic components. Sema contains no
123
+ neural network. Its vectors are constructed by deterministic algebra (random
124
+ projections, permutations, sums), not learned by backpropagation. The correct
125
+ lineage for its vector side is _hyperdimensional computing / VSA_, which grew
126
+ out of connectionism but does not require learning.
127
+
128
+ ### 1.3 Where it sits in the literature
129
+
130
+ Systems close in spirit, each sharing one facet:
131
+
132
+ - **Holographic Reduced Representations** (Plate 1995) and **hyperdimensional
133
+ computing** (Kanerva 2009): the representational substrate. Sema shares the
134
+ algebra (superposition + binding); it differs by anchoring every vector to an
135
+ exact, reconstructible symbolic structure in a Merkle DAG.
136
+ - **Semiring-weighted deduction** (Goodman 1999) and the **Generalized A\***
137
+ architecture (Felzenszwalb & McAllester 2007): the inference engine. Sema
138
+ shares the formalism exactly; its contribution is _what_ the items and rules
139
+ are — spans of a query, nodes of a learned graph, learned continuations.
140
+ - **Case-based reasoning** (Kolodner 1992): solve new problems by retrieving and
141
+ adapting stored cases. Sema's counterfactual-transfer mechanism (Section 18)
142
+ is a formalized, byte-level version of case adaptation.
143
+
144
+ The short label used in the rest of this document: **a vector-symbolic,
145
+ memory-based reasoning system**.
146
+
147
+ ---
148
+
149
+ ## 2. Vector Symbolic Architectures
150
+
151
+ ### 2.1 The problem VSAs solve
152
+
153
+ Fixed-width vectors are attractive as a representation: comparing two of them
154
+ (one dot product) is trivially fast, and "similar" has a natural meaning (small
155
+ angle between them). But a bag of numbers has no obvious way to represent
156
+ _structure_ — the difference between "the dog bit the man" and "the man bit the
157
+ dog" is not a difference in _which_ words occur but in _how they are arranged_.
158
+
159
+ A **Vector Symbolic Architecture** (VSA; also _hyperdimensional computing_) is
160
+ an algebra over high-dimensional vectors that solves exactly this problem. It
161
+ provides:
162
+
163
+ 1. **Atoms.** Elementary symbols are assigned (usually random) high-dimensional
164
+ vectors.
165
+ 2. **Superposition** (bundling): a way to combine several vectors into one
166
+ vector that is _similar to each of its inputs_ — typically element-wise
167
+ addition followed by normalization. Superposition represents _sets_: the
168
+ result "contains" its inputs in the sense that each input can be detected in
169
+ it by a similarity test.
170
+ 3. **Binding**: a way to combine vectors into one that is _dissimilar to its
171
+ inputs_ — it represents an _association_ (a role filled by a filler, a
172
+ position occupied by an item) rather than a collection. Crucially, binding is
173
+ invertible (or at least, structurally distinguishable), so bound structure
174
+ can be probed.
175
+ 4. **A similarity measure**, usually cosine similarity, under which all of the
176
+ above has meaning.
177
+
178
+ Different VSA families use different binding operators: circular convolution in
179
+ Plate's Holographic Reduced Representations (1995), element-wise XOR in
180
+ Kanerva's binary spatter codes, element-wise multiplication in Gayler's
181
+ Multiply–Add–Permute (2003). Sema uses **permutation binding**: applying a fixed
182
+ random permutation of the vector's coordinates.
183
+
184
+ ### 2.2 Why high dimensions work: quasi-orthogonality
185
+
186
+ The entire edifice rests on a fact of high-dimensional geometry sometimes called
187
+ _concentration of measure_: **two independently chosen random unit vectors in D
188
+ dimensions are almost exactly orthogonal**. Their expected cosine similarity is
189
+ 0, with standard deviation approximately 1/√D. At D = 1024, that standard
190
+ deviation is about 0.031 — so unrelated random vectors reliably score within a
191
+ few hundredths of zero, while a vector scores 1.0 against itself. There is an
192
+ enormous, dependable gap between "same" and "unrelated", and an entire usable
193
+ band in between for "partially similar".
194
+
195
+ This has three consequences that Sema uses constantly:
196
+
197
+ - **Capacity.** A superposition of a handful of random vectors is still clearly
198
+ similar to each of them and clearly dissimilar to everything else, because the
199
+ cross-talk between the components is O(1/√D) noise.
200
+ - **Statistical decision thresholds.** "Is this similarity real or chance?" has
201
+ a principled answer: a cosine of k/√D is k standard deviations above what
202
+ chance produces. Sema's significance bar is exactly 3/√D — three sigma
203
+ (Section 8).
204
+ - **Robustness.** Corrupting a few coordinates of a high-dimensional vector
205
+ barely moves it; every comparison degrades gracefully rather than breaking.
206
+
207
+ ### 2.3 Permutation binding and why order becomes visible
208
+
209
+ A permutation π rearranges a vector's coordinates: the value at position π(i)
210
+ moves to position i. Two properties make permutations excellent binding
211
+ operators:
212
+
213
+ - **They preserve lengths and angles** (they are orthogonal linear maps), so
214
+ permuting a vector produces an equally well-behaved vector.
215
+ - **A random permutation decorrelates.** For a random vector v, the permuted
216
+ vector πv is (with overwhelming probability) nearly orthogonal to v — the
217
+ permutation "hides" the vector's identity behind the role.
218
+
219
+ Sema keeps a fixed **keyring** of independent random permutations π₀, π₁, π₂, …
220
+ — one per _seat_ (ordinal position in a group). To encode an ordered group of
221
+ children (c₀, c₁, …, cₖ), each child's vector is bound to its seat and the
222
+ results are superposed:
223
+
224
+ ```
225
+ encode(c₀, c₁, …, cₖ) = π₀·v(c₀) + π₁·v(c₁) + … + πₖ·v(cₖ)
226
+ ```
227
+
228
+ Because the seats are _different_ permutations, "A in seat 0, B in seat 1" and
229
+ "B in seat 0, A in seat 1" produce nearly orthogonal encodings — **order is part
230
+ of the representation**. And because independent permutations do not commute,
231
+ nesting the operation encodes _paths_: "the x that sits in seat 2 of the thing
232
+ in seat 1" has a distinct signature from "the x in seat 1 of the thing in seat
233
+ 2". A whole tree can thus be folded, level by level, into one fixed-width vector
234
+ whose geometry reflects the tree's entire shape and content. Sema calls the
235
+ result of this fold the tree's **gist**.
236
+
237
+ Note that this encoding step is **not** followed by a normalize: unlike the
238
+ classical VSA recipe (which renormalizes after every superposition), Sema's fold
239
+ leaves every interior sum at its natural length and only ever normalizes the
240
+ _root_ of a fold — see §2.6, where this is the basis of the system's
241
+ angle-and-magnitude semantics.
242
+
243
+ The trade this makes is deliberate and important to understand: the fold is
244
+ **lossy** as a vector (a 1024-dimensional gist cannot losslessly hold a kilobyte
245
+ of text), but Sema never needs to _decode_ a gist — the exact content is always
246
+ recoverable from the symbolic side (the DAG, Section 3). Gists exist purely to
247
+ make _similarity of structured content computable in one dot product_. This
248
+ division of labour — vectors for geometry, the DAG for identity — is the single
249
+ most important design fact in Sema.
250
+
251
+ ### 2.4 The alphabet: atoms with graded similarity
252
+
253
+ The atoms of Sema's VSA are the 256 possible byte values. Each byte value is
254
+ assigned a fixed unit vector at construction time, deterministically from a
255
+ seed. But the assignment is not uniformly random: the 256 vectors are built by
256
+ **recursive refinement** — 16 coarse random directions are each refined into 4
257
+ intermediate directions, each of which is refined into 4 final directions (16 →
258
+ 64 → 256). Refinement means: keep a weighted portion of the parent direction and
259
+ mix in fresh randomness.
260
+
261
+ The result is an alphabet with _graded similarity_: byte values that share a
262
+ refinement ancestor have moderately similar vectors, while distant byte values
263
+ are quasi-orthogonal. This gives perception a mild, structured tolerance at the
264
+ very lowest level, while preserving the global quasi-orthogonality the algebra
265
+ needs. (The mixing ratio is the alphabet's _roughness_; the construction is
266
+ deterministic given the seed, which is what makes all of Sema reproducible.)
267
+
268
+ ### 2.5 What the VSA contributes to Sema, in one sentence
269
+
270
+ > The VSA turns "how similar are these two _structures_?" into "what is the
271
+ > cosine of these two vectors?" — a question answerable in microseconds and
272
+ > indexable at scale — without ever being trusted to _store_ the structures
273
+ > themselves.
274
+
275
+ Sema's word for cosine similarity between gists is **resonance**, and this
276
+ document uses both terms interchangeably.
277
+
278
+ ### 2.6 Magnitude: the second axis of similarity
279
+
280
+ A fold whose interior sums are never renormalized (§2.3) carries more
281
+ information than its angle alone. Because the leaf vectors it superposes are
282
+ close to orthogonal, the length of an unnormalized interior gist grows with the
283
+ _amount_ of content folded into it: a span of `len` bytes has a gist whose norm
284
+ is approximately √len. Only the root of a completed fold is normalized to unit
285
+ length (so that content and halo indexes, which compare roots, still compare
286
+ directions on the unit sphere); every gist below the root retains this natural,
287
+ byte-proportional magnitude.
288
+
289
+ This turns the fold into what the codebase calls a **linear** fold: it is a
290
+ genuine linear operator (superposition of seat-bound leaf vectors, nothing
291
+ else), and a resonance score between two such unnormalized quantities reads as
292
+ **byte-proportional overlap**, not scale-free cosine. Concretely: the cosine
293
+ between two spans' gists is (shared content) / √(len₁ · len₂) — the geometric
294
+ mean of their lengths in the denominator, exactly as an inner product between
295
+ two sums of near-orthogonal unit vectors predicts. Reading a magnitude back out
296
+ of a stored gist is therefore reading a byte count: a node's gist norm (or,
297
+ equivalently, its stored content length) IS its size in the fold's own units.
298
+
299
+ This is Sema's **angle-and-magnitude** semantics: the ANGLE between two gists
300
+ carries the _fraction_ of content they share; the MAGNITUDE (recovered from the
301
+ stored span length) converts that fraction into an absolute count of shared
302
+ bytes. Every mechanism that must ask a scale-sensitive question — "does this
303
+ score mean the query is entirely explained, or merely brushed by a much larger
304
+ stored form?" — reads both axes explicitly rather than trusting the cosine
305
+ alone. Two examples used constantly in Part IV:
306
+
307
+ - The **identity bar** for a whole-span claim (§8.1) tightens as the span grows,
308
+ because a fixed cosine tolerates a growing number of foreign bytes once the
309
+ span is long — the magnitude correction keeps "near-identical" meaning the
310
+ same absolute thing at every scale.
311
+ - **Recall's last-resort tier** (§21) and the **consensus climb's** per-region
312
+ vote (§17.3) both convert a raw resonance score into a _query-relative
313
+ fraction_ — how much of the smaller side the larger side's content actually
314
+ accounts for — using exactly this norm-as-byte-count reading, rather than
315
+ trusting the raw cosine, which conflates "small thing fully inside a big
316
+ thing" with "big thing loosely touching a small one".
317
+
318
+ Nothing about content addressing or the DAG changes: identity is still decided
319
+ by exact, byte-verified lookup (§3.1), never by a magnitude reading. Magnitude
320
+ is strictly a refinement of the _geometric_ half of the system — it makes
321
+ resonance scores honest about scale, it does not replace the exact half.
322
+
323
+ ---
324
+
325
+ ## 3. Content-addressable memory and the Merkle DAG
326
+
327
+ ### 3.1 Content addressing
328
+
329
+ In a conventional memory, an item lives at an _address_, and you must know the
330
+ address to retrieve the item. In a **content-addressable memory** (CAM), the
331
+ item's own content determines where it is: to ask "do I know this?" you present
332
+ the content itself, and the memory answers with the stored item (or its
333
+ identity) directly. Associative memories of this kind have been studied since
334
+ the earliest days of computer architecture.
335
+
336
+ Sema's long-term store is content-addressed in the strict sense: **a node's
337
+ identity is a pure function of its content**. Storing the same content twice
338
+ yields the same node, always. This gives three properties for free:
339
+
340
+ - **Idempotent learning.** Re-ingesting a document changes nothing; there is no
341
+ duplicate to create.
342
+ - **Intrinsic deduplication.** Storage grows with _distinct_ content, not with
343
+ volume. A phrase seen a million times occupies one node.
344
+ - **Exact identity tests.** "Is this span something I have seen?" is a lookup,
345
+ not a similarity estimate. Sema leans on this constantly: soft (vector)
346
+ evidence _suggests_, but identity decisions are always made by
347
+ content-addressed lookup, never by a similarity score crossing 1.0.
348
+
349
+ ### 3.2 The Merkle DAG
350
+
351
+ A **Merkle structure** (Merkle 1987) is one in which a composite object's
352
+ identifier is derived from the identifiers of its parts. If two composites have
353
+ the same parts in the same arrangement, they _are_ the same object. Applied to
354
+ trees this is also known in programming-language circles as **hash-consing**:
355
+ construct each node "modulo equality", so structurally equal subtrees are
356
+ physically shared.
357
+
358
+ Sema's memory is a Merkle **DAG** (directed acyclic graph) of nodes:
359
+
360
+ - A **leaf** node is a span of raw bytes.
361
+ - A **branch** node is an ordered list of child node identities.
362
+ - A node's identity is determined by exactly that content (bytes, or the ordered
363
+ child list). Equal content ⇒ same node.
364
+
365
+ Because identical subtrees collapse into one shared node, the "tree" of any
366
+ single perceived input becomes, in storage, a subgraph woven into every other
367
+ input that shares material with it. A sentence deposited yesterday and a
368
+ paragraph deposited today that contains that sentence _share the sentence's
369
+ node_; the paragraph's branch simply points at it. Three consequences:
370
+
371
+ 1. **Every span ever perceived is individually addressable.** Not just whole
372
+ documents: every intermediate grouping the perception process produced is a
373
+ node with an identity, and can be the target of an association or a
374
+ similarity probe.
375
+ 2. **The graph can be climbed.** From any node one can ask "which larger
376
+ structures contain me?" (its _parents_) — the reverse of the child lists.
377
+ Recognising a fragment of an experience thus gives a path _upward_ to the
378
+ whole experiences that contain it. This upward climb is the backbone of
379
+ Sema's attention mechanism (Section 17).
380
+ 3. **Reconstruction is exact.** Concatenating a node's leaves, left to right,
381
+ reproduces the original bytes losslessly. The memory is not an approximation
382
+ of the corpus; it _is_ the corpus, shared and structured.
383
+
384
+ ### 3.3 The marriage of CAM and VSA
385
+
386
+ Each node carries both halves of Sema's dual representation:
387
+
388
+ | Facet | Representation | Answers |
389
+ | :----------- | :------------------------------------------------- | :------------------------------------------------------------------------ |
390
+ | **Identity** | content-addressed node in the Merkle DAG | "Is this exactly something I know? What contains it? What are its parts?" |
391
+ | **Geometry** | the gist vector (the VSA fold of the same content) | "What does this _resemble_? How strongly?" |
392
+
393
+ The store maintains a vector index over gists (Section 6) so that "find the
394
+ stored nodes most similar to this vector" is fast. The two facets discipline
395
+ each other: geometric search proposes candidates cheaply; content-addressed
396
+ structure verifies and grounds them exactly. Nothing in Sema ever acts on a
397
+ similarity score alone when an exact structural check is available — a rule
398
+ stated once here and honoured throughout the pipeline.
399
+
400
+ One refinement deserves mention because it is conceptually load-bearing:
401
+ **near-deduplication is byte-verified**. When a freshly perceived experience is
402
+ geometrically almost identical to one just stored (cosine above the _merge
403
+ threshold_, Section 8), Sema considers treating them as the same node — but
404
+ geometric closeness alone is scale-blind, so the decision is made by the bytes:
405
+ the two contents must be identical except for **one local span no wider than the
406
+ perception window**. Geometric evidence proposes; bytes dispose.
407
+
408
+ ---
409
+
410
+ ## 4. Distributional structure
411
+
412
+ ### 4.1 The distributional hypothesis
413
+
414
+ The **distributional hypothesis** (Harris 1954) — often summarized as "you shall
415
+ know a word by the company it keeps" — holds that linguistic items with similar
416
+ meanings occur in similar contexts — that _meaning_, to a useful approximation,
417
+ is _distribution of use_. It is the theoretical foundation of every modern
418
+ word-embedding method, but the hypothesis itself is prior to and independent of
419
+ neural networks: it is a claim about language, testable by counting.
420
+
421
+ Sema implements the distributional hypothesis directly and transparently.
422
+ Alongside its gist (which encodes _what a node is made of_), a node that takes
423
+ part in learned associations accumulates a second vector: its **halo** — a
424
+ superposition of **company signatures** of the _partners it appeared with_ (what
425
+ preceded it, what followed it, bound to a role seat so that "appears as context"
426
+ and "appears as answer" are distinguishable). The halo encodes _the company the
427
+ node keeps_.
428
+
429
+ A company signature is a deterministic unit vector derived from the partner's
430
+ **node identity** (a seeded function of the node id), not from the partner's
431
+ gist. This decouples content similarity from company similarity: two halos
432
+ correlate exactly as much as their episode-participation histories overlap,
433
+ never because their partners merely contain similar bytes. Pouring raw partner
434
+ gists instead would let any byte-overlap between partners leak _content_
435
+ similarity into _distributional_ similarity, silently shifting the halo null
436
+ model that the concept threshold's derivation (unrelated halos ⇒ cosine 0 ±
437
+ 1/√D) depends on.
438
+
439
+ Two nodes whose halos are similar have occurred in similar circumstances — they
440
+ are **distributional siblings**: synonyms, paraphrases, items of the same
441
+ category, two names for one thing. Note the complementarity:
442
+
443
+ | Vector | Encodes | Two nodes are close when… |
444
+ | :------- | :----------------------------------- | :--------------------------------------------------------------------------- |
445
+ | **Gist** | the node's own content and structure | they are _made of_ similar material ("colour" ≈ "colours") |
446
+ | **Halo** | the node's contexts of use | they are _used_ the same way ("colour" ≈ "hue"), even with zero shared bytes |
447
+
448
+ ### 4.2 What halos do in the pipeline
449
+
450
+ Halos give Sema its capacity for synonymy and analogy without any trained
451
+ embedding model:
452
+
453
+ - **Concept hops.** A recognised form that has no learned continuation of its
454
+ own can borrow the continuation of a distributional sibling — the system
455
+ answers about "hue" using what it learned about "colour" (Section 19's concept
456
+ rule).
457
+ - **Articulation.** An answer is re-voiced in the asker's own vocabulary by
458
+ substituting answer forms with query forms that share a halo (Section 24).
459
+ - **Analogy strength.** Whether two entities are genuinely analogous — the gate
460
+ on counterfactual comparison (Section 18) — is measured by halo similarity,
461
+ directly or through shared siblings (a second-order distributional test).
462
+ - **Evidence weight.** How many episodes poured into a node's halo (its _mass_)
463
+ is a direct count of distributional corroboration, consulted when choosing
464
+ among competing continuations (Section 25).
465
+
466
+ Like gists, halos live in a vector index of their own so that "which nodes keep
467
+ this kind of company?" is a fast query.
468
+
469
+ ### 4.3 A note on scientific hygiene
470
+
471
+ Because the halo is an explicit superposition of explicit episode signatures,
472
+ distributional claims in Sema are _auditable_: one can enumerate exactly which
473
+ learning events contributed to a halo and with what role. This distinguishes
474
+ Sema's distributional layer from learned embeddings, whose geometry is real but
475
+ whose provenance is diffused across an entire training run.
476
+
477
+ ---
478
+
479
+ ## 5. Automated deduction: the lightest derivation
480
+
481
+ ### 5.1 Weighted deduction systems
482
+
483
+ **Automated deduction** is the field concerned with deriving conclusions from
484
+ premises by mechanical application of inference rules. A **weighted deduction
485
+ system** attaches a non-negative cost to each rule application:
486
+
487
+ ```
488
+ premise₁ ∧ premise₂ ∧ … ∧ premiseₖ --(cost c)--> conclusion
489
+ ```
490
+
491
+ A **derivation** of an item is a proof tree: leaves are axioms, and each
492
+ internal node is a rule application whose children derive its premises. The
493
+ derivation's cost is the sum of the costs of the rules it uses. The **lightest
494
+ derivation** of a goal item is the derivation of minimal total cost. This
495
+ formalism, developed principally in parsing theory (Goodman 1999 gave the
496
+ general semiring formulation), strictly generalizes shortest-path search: a
497
+ graph is the special case in which every rule has exactly one premise.
498
+
499
+ Rules with _multiple_ premises are what make the formalism powerful. A
500
+ two-premise rule is a **conjunction**: it composes two independently derived
501
+ results into one, paying a join cost. The search space is therefore an AND/OR
502
+ **hypergraph**, not a graph — and finding the lightest derivation is the
503
+ hypergraph analogue of finding a shortest path.
504
+
505
+ ### 5.2 Knuth's algorithm and the A\* generalization
506
+
507
+ Knuth (1977) showed that Dijkstra's algorithm generalizes from graphs to
508
+ weighted deduction: process items in order of cost; when an item is removed from
509
+ the priority queue, its cost is final (given non-negative, monotone rules).
510
+ Felzenszwalb & McAllester (2007) then generalized A\* the same way — **A\*
511
+ Lightest Derivation (A\*LD)**: if an admissible heuristic (a lower bound on the
512
+ cost remaining from an item to the goal) is available, the queue is ordered by
513
+ _cost so far + lower bound_, and provably no item is expanded whose lightest
514
+ derivation costs more than the goal's. The search is **output-sensitive**: its
515
+ work is proportional to the answer, not to the size of the (implicit,
516
+ potentially enormous) space of derivations.
517
+
518
+ Sema's inference engine is an implementation of A\*LD, with one deliberate
519
+ extension described next. Four standard disciplines keep it tractable:
520
+
521
+ 1. **Chart memoization** — equivalent partial derivations collapse to one
522
+ canonical entry (the cheapest).
523
+ 2. **Lazy rule generation** — rules are enumerated only when one of their
524
+ premises has been finalised, never up front.
525
+ 3. **Demand filtering** — rules whose conclusions cannot reach the goal are
526
+ never emitted.
527
+ 4. **Admissible heuristic pruning** — the A\* bound keeps the frontier focused
528
+ on the goal.
529
+
530
+ ### 5.3 The semiring extension: evidence pooling
531
+
532
+ Classic lightest-derivation search operates in the **tropical semiring** (min,
533
+ +): among competing derivations of the same conclusion, only the cheapest
534
+ survives. That is the right regime for _choosing_ — one best answer, one best
535
+ parse.
536
+
537
+ But some of Sema's decisions are not choices; they are _accumulations of
538
+ evidence_. When several independent regions of a query each independently point
539
+ at the same stored fact, the fact should be credited with the _sum_ of their
540
+ support, not merely the strongest single vote. For those decisions the engine
541
+ supports a second combining mode operating in the **arithmetic semiring** (+,
542
+ +): every derivation of a marked conclusion _adds_ its cost into a pooled total,
543
+ and every contribution is recorded rather than discarded. Semiring-general
544
+ deduction is standard theory (Goodman 1999); running both regimes in one search
545
+ — minimum-cost for structure, sum for evidence — is how Sema keeps consensus
546
+ formation (Section 17) inside the same formal system as everything else, rather
547
+ than as an ad-hoc tally alongside it.
548
+
549
+ ### 5.4 Why deduction, and not generation
550
+
551
+ The choice of weighted deduction as the inference engine is what makes Sema's
552
+ central claims true rather than aspirational:
553
+
554
+ - **Auditability.** The answer _is_ a proof tree. Every byte of output is the
555
+ conclusion of an explicit chain of rule applications over explicit stored
556
+ facts, and that chain can be read back (Section 26).
557
+ - **Determinism.** Lightest derivation is an optimization with a well-defined
558
+ optimum (ties broken by fixed conventions), not a sample from a distribution.
559
+ - **Honest silence.** If no derivation of the goal exists, the search returns
560
+ nothing. The system cannot "make something up": fabrication is not expressible
561
+ in the formalism.
562
+
563
+ ---
564
+
565
+ ## 6. Approximate nearest-neighbour search
566
+
567
+ ### 6.1 The role of ANN search
568
+
569
+ Both of Sema's vector relations — gists and halos — need the same primitive:
570
+ _given a query vector, find the k stored vectors with highest cosine
571
+ similarity_, over a store that may hold millions of vectors, in milliseconds, on
572
+ a CPU, without holding everything in RAM. Exact search is linear in the
573
+ collection size; **approximate nearest-neighbour (ANN)** search trades a small,
574
+ controlled amount of recall for sub-linear query time.
575
+
576
+ Sema uses two established techniques in combination:
577
+
578
+ - **HNSW** — Hierarchical Navigable Small World graphs (Malkov & Yashunin 2018).
579
+ Vectors are connected into a multi-layer proximity graph; a query descends
580
+ greedily from a sparse top layer to a dense bottom layer, examining only a
581
+ logarithmic-ish neighbourhood of the collection. Empirically the work grows
582
+ around N^0.3 in Sema's configuration — decisively sub-linear.
583
+ - **RaBitQ 1-bit quantization** (Gao & Long 2024). Each stored vector is
584
+ randomly rotated and reduced to one _sign bit_ per dimension — a 32×
585
+ compression — with an unbiased, theoretically-grounded estimator of the
586
+ original cosine computable from the code alone. Sema stores _only_ the codes;
587
+ the original float vectors are never kept by the index.
588
+
589
+ ### 6.2 The epistemological consequence
590
+
591
+ This layer is the one place where Sema's answers to "what is similar?" are
592
+ _estimates_: the scores returned by the index are RaBitQ estimates over 1-bit
593
+ codes, not exact cosines, and the ranking is approximate. Sema's discipline
594
+ about this is strict and worth stating as a principle, because it shapes several
595
+ pipeline decisions:
596
+
597
+ > **Approximate scores may rank and propose; they may never decide identity or
598
+ > be compared against exactness.** Any decision of the form "this _is_ that" is
599
+ > made by content-addressed resolution in the DAG. Thresholds compared against
600
+ > estimated scores gate broad regions (three-sigma bands, half-window bars),
601
+ > never knife-edge equalities.
602
+
603
+ ### 6.3 Space-filling curves: geometry as reading order
604
+
605
+ One more piece of classical machinery belongs to this layer of fundamentals.
606
+ Sema's perception consumes _streams of bytes_; images, video, and other
607
+ grid-shaped data must first become a stream. Sema linearizes n-dimensional grids
608
+ along a **Hilbert curve** — the space-filling curve with the strongest locality
609
+ guarantees (points close on the curve are close in the grid, and vice versa to
610
+ the extent topology allows). This means spatial neighbourhoods in an image
611
+ become contiguous runs in the stream, so the same stream-folding perception that
612
+ reads text reads pixels — _geometry is only a reading order_, and every modality
613
+ meets the same memory.
614
+
615
+ ---
616
+
617
+ ---
618
+
619
+ # Part II — The big picture
620
+
621
+ ## 7. How the five foundations connect
622
+
623
+ ### 7.1 One structure, two verbs, one memory
624
+
625
+ Everything Sema does reduces to two operations over one store:
626
+
627
+ - **Deposit** (learn): perceive an input into a tree, intern the tree into the
628
+ Merkle DAG, and record its relations (continuation edges, halo pours).
629
+ - **Ask** (think): perceive the query the same way, and run one
630
+ lightest-derivation search whose axioms come from recognising the query
631
+ against the store and whose rules are the store's learned relations.
632
+
633
+ There is no third operation. There is no training phase distinct from
634
+ depositing, no fine-tuning, no consolidation pass required for correctness. The
635
+ store _is_ the model; a deposit is immediately available to every subsequent
636
+ ask.
637
+
638
+ ### 7.2 The division of labour
639
+
640
+ Each foundation from Part I owns one aspect of the system, and the seams between
641
+ them are explicit:
642
+
643
+ ```
644
+ ┌─────────────────────────────────────────┐
645
+ │ INPUT (any modality) │
646
+ │ text · bytes · images · video │
647
+ └──────────────────┬──────────────────────┘
648
+ │ Hilbert linearization (§6.3)
649
+
650
+ PERCEPTION ┌─────────────────────────────────────────┐
651
+ (VSA, §2) │ the river fold: bytes → leaves → tree │
652
+ │ every node gets a GIST (permutation- │
653
+ │ bind + superpose + normalize) │
654
+ └──────────────────┬──────────────────────┘
655
+ │ identical bytes ⇒ identical tree
656
+
657
+ MEMORY ┌─────────────────────────────────────────┐
658
+ (CAM/Merkle, §3) │ the DAG: hash-consed nodes │
659
+ │ identity = content; │
660
+ │ parents ↑ / kids ↓ climbable │
661
+ ├─────────────────────────────────────────┤
662
+ RELATIONS │ continuation edges (what follows what) │
663
+ (distributional, §4) │ halos (what company each node keeps) │
664
+ ├─────────────────────────────────────────┤
665
+ INDEXES │ gist index + halo index │
666
+ (ANN, §6) │ (HNSW over 1-bit RaBitQ codes) │
667
+ └──────────────────┬──────────────────────┘
668
+ │ axioms & rule candidates
669
+
670
+ INFERENCE ┌─────────────────────────────────────────┐
671
+ (deduction, §5) │ ONE lightest-derivation search: │
672
+ │ cover the query · follow edges · │
673
+ │ hop concepts · fuse & recompose · │
674
+ │ splice connectors · pool evidence │
675
+ └──────────────────┬──────────────────────┘
676
+
677
+ ┌─────────────────────────────────────────┐
678
+ │ answer bytes + provenance + rationale │
679
+ │ (a readable proof tree) │
680
+ └─────────────────────────────────────────┘
681
+ ```
682
+
683
+ Read the seams carefully, because they are where the design earns its
684
+ properties:
685
+
686
+ - **Perception → Memory.** Perception is a _pure, deterministic function of
687
+ bytes_. The same bytes always fold into the same tree with the same gists.
688
+ This is what makes content addressing possible at all: if perception were
689
+ stochastic or context-dependent, the same content would not reproduce the same
690
+ nodes.
691
+ - **Memory → Indexes.** The vector indexes are _derived_ data — pure
692
+ accelerators. Every fact they suggest is verified against the DAG before it is
693
+ acted on. Deleting the indexes loses speed, never knowledge.
694
+ - **Memory → Inference.** The deduction system's rules are read off the store: a
695
+ continuation edge is a one-premise rule; a learned composite is a two-premise
696
+ fusion rule; a distributional sibling licenses a (more expensive) concept-hop
697
+ rule. Inference has no rules of its own beyond the cost algebra — _everything
698
+ it can do, it can do only because something was learned_ (plus the manual
699
+ computation rules of Section 16).
700
+
701
+ ### 7.3 The two vector relations, side by side
702
+
703
+ It is worth fixing firmly, once, the two distinct vector spaces in play —
704
+ confusing them is the commonest way to misunderstand the system:
705
+
706
+ | | **Gist (content) space** | **Halo (concept) space** |
707
+ | :---------------------- | :-------------------------------------------- | :----------------------------------------------------------------- |
708
+ | A node's vector encodes | its own bytes and structure | the episodes it took part in |
709
+ | Built by | the perception fold (deterministic) | superposing seat-bound partner company signatures at learning time |
710
+ | Two nodes close means | similar content | similar usage (synonymy, categoryhood) |
711
+ | Typical query | "what stored form resembles this query span?" | "which nodes are used like this one?" |
712
+ | Indexed in | the content index | the halo index |
713
+
714
+ ### 7.4 What "learning" and "generalizing" mean here
715
+
716
+ - **Learning a fact** = depositing a (context, continuation) pair: both sides
717
+ are interned, one continuation edge is recorded from the context's root to the
718
+ continuation's root, and each side's signature is poured into the other's
719
+ halo.
720
+ - **Generalization** happens at query time, by three distinct, inspectable
721
+ mechanisms rather than one opaque one: _geometric proximity_ (a query near a
722
+ learned form resonates with it), _distributional substitution_ (a form can
723
+ stand in for its halo siblings), and _structural analogy_ (a query that
724
+ aligns, byte-wise, with the shapes of several learned experiences can have
725
+ structure transferred between them — Section 18).
726
+ - **Forgetting** does not happen implicitly. Nodes are never silently discarded;
727
+ the store only ever grows more connected. (Index maintenance can prune
728
+ _acceleration_ entries, but that affects speed, not knowledge.)
729
+
730
+ ---
731
+
732
+ ## 8. Derived thresholds: the geometry of every decision
733
+
734
+ A system that makes soft (geometric) decisions needs thresholds, and thresholds
735
+ are where hidden empiricism usually creeps in ("0.7 worked on the dev set").
736
+ Sema's design rule is strict: **every threshold is derived from the geometry of
737
+ the representation itself — from the dimension D, the perception window W, or
738
+ the corpus size N — never tuned.** Any constant that cannot be derived is not
739
+ used. The five bars below govern the entire pipeline; each is stated with its
740
+ derivation.
741
+
742
+ Throughout: D is the vector dimension, W ("maxGroup") is the maximum number of
743
+ children a perception fold groups at once, N is the number of learned contexts
744
+ (nodes bearing at least one outgoing continuation edge).
745
+
746
+ ### 8.1 The merge threshold: 1 − 1/√D — "geometrically the same"
747
+
748
+ The standard deviation of chance resonance between unrelated vectors is 1/√D
749
+ (§2.2). A cosine within one such unit _of 1.0_ is closer to identity than chance
750
+ can measure apart: the store treats two gists this close as candidates for being
751
+ the _same_ node (subject to the byte verification of §3.3). Recall reuses the
752
+ same bar to accept "the query is essentially a stored form".
753
+
754
+ This fixed bar is the identity threshold for two gists of comparable size (in
755
+ particular, two roots, which are always unit vectors — §2.6). A claim of the
756
+ form "this whole SPAN of `len` bytes is essentially identical to that stored
757
+ form" needs a **scale-aware** version of the same idea, because under the linear
758
+ fold a cosine reads as a byte-proportional overlap fraction (§2.6): a fixed
759
+ cosine bar admits a foreign-byte budget that _grows_ with the span, so naively
760
+ reusing 1 − 1/√D on a long span would tolerate far more corruption than
761
+ "essentially the same" should mean. The scale-aware bar instead fixes the
762
+ tolerated foreign-byte budget at one perception window W — the same
763
+ single-window budget near-dedup's byte check grants (§3.3, §11.1) — and converts
764
+ it into a cosine floor for the span's own length: **1 − W/len**, floored at the
765
+ fixed bar 1 − 1/√D (below which the RaBitQ estimator cannot certify identity
766
+ regardless of scale). The scale-aware bar `1 − W/len` tolerates exactly one
767
+ perception window W of foreign content — the same single-window budget the byte
768
+ check grants, now expressed in cosine space; a span barely longer than W
769
+ tolerates almost none. Derived from W, D, and the span length; never tuned.
770
+
771
+ ### 8.2 The reach threshold: 1 − 1/(2W) — "related at all"
772
+
773
+ Perception folds children in groups of at most W. Two structures that differ in
774
+ _one whole child_ — the smallest difference perception can express — sit at
775
+ cosine ≈ 1 − 1/W (one of W superposed, permuted components differs). Half that
776
+ quantum, 1 − 1/(2W), is therefore _closer than any real single-child difference
777
+ can be_: anything scoring above it is a positional echo of the same content, and
778
+ anything whose _best_ match in the store falls below it is structurally
779
+ unrelated to everything stored. The reach threshold is Sema's confidence floor:
780
+ rather than answer from an unrelated neighbour, the system returns nothing.
781
+ **Silence is a first-class output.**
782
+
783
+ ### 8.3 The significance bar: 3/√D — "not chance"
784
+
785
+ Chance resonance has mean 0 and standard deviation 1/√D, so a cosine of 3/√D is
786
+ three standard deviations above chance — the conventional statistical bar for
787
+ "this relationship is real". Whole-query evidence below this bar is not followed
788
+ into the more trusting inference tiers.
789
+
790
+ ### 8.4 The estimator noise floor: 1/√D — "above quantisation noise"
791
+
792
+ One standard deviation of the cosine between two independent random vectors in D
793
+ dimensions (§2.2). It is the smallest difference in cosine that is
794
+ distinguishable from the rotation-uniformised RaBitQ estimation error: a
795
+ contrastive margin below it is quantisation noise, not evidence. The consensus
796
+ climb gates a region's vote on its _discriminative margin_ — the score gap
797
+ between the best and second-best anchor — clearing this floor. One σ, not the
798
+ stricter 3σ relatedness bar: the minimal "above noise" threshold. Derived from
799
+ D, never tuned.
800
+
801
+ ### 8.5 The concept threshold: ½ + 1/(2√D) — "same concept"
802
+
803
+ Halos are superpositions of episode signatures. The structural midpoint 0.5
804
+ separates "more similar than dissimilar"; the added half-sigma 1/(2√D) widens
805
+ the bar slightly at low dimension (where chance noise is broader) and vanishes
806
+ as D grows. Two halos above this bar mark their nodes as distributional siblings
807
+ — eligible for concept hops, articulation substitutions, and analogy.
808
+
809
+ ### 8.6 The consensus floor: ln N + ½ — "corroborated, not echoed"
810
+
811
+ In the consensus climb (Section 17), a query region's vote for an anchor is
812
+ weighted by _inverse document frequency_: reaching an anchor through c of the N
813
+ learned contexts is worth ln(N/c), so the maximum any single region can
814
+ contribute is ln N (a maximally specific region, c = 1). Requiring a pooled vote
815
+ to exceed ln N + ½ therefore demands _strictly more than any one region could
816
+ say alone_ — genuine multi-region corroboration at the current corpus scale —
817
+ before an anchor is trusted as an independent point of attention. The floor
818
+ grows with the corpus exactly as the maximum single-region vote does.
819
+
820
+ ### 8.7 The half-dominance test: ½ — "a part that swallows its whole"
821
+
822
+ A span covering strictly more than half of its whole can no longer discriminate
823
+ the whole's own content — the test behind three pipeline decisions: liftAnswer
824
+ keeps the framing when a single recognised span dominates the query (the rest of
825
+ the cover is scaffolding), collectRegions excludes a wrapper region that would
826
+ drown multi-topic queries, and CAST's frame-depth majority classifies shared
827
+ material as non-discriminative structure. Derived from the structural midpoint:
828
+ half is the threshold at which the part outweighs what remains. Never tuned.
829
+
830
+ ### 8.8 The hub bound: √N — "stop at non-discriminative fan-out"
831
+
832
+ Not a similarity bar but the same spirit: any walk over the graph's fan-out (a
833
+ node's parents, a context's continuations, an answer's reverse fan-in) is capped
834
+ at √N candidates. A node connected to more than √N others is a _hub_ — its
835
+ connections are so numerous that each individual one carries almost no
836
+ discriminative information. The cap is applied at the _store level_: the store
837
+ provides LIMITed read operations so that no per-query read ever materialises a
838
+ corpus-sized fan-out list. Consumers of partial fan-outs use the LIMITed reads
839
+ to decide their question exactly — "hub or not?", "saturated or voted?" —
840
+ without ever expanding past √N distinct contexts. The full materialising reads
841
+ remain available for maintenance and inspection paths, but every hot-path
842
+ decision consults only the LIMITed prefix or an indexed existence probe. Every
843
+ fan-out-limited decision in the pipeline uses this one bound, so the trade is
844
+ made once, consistently, and the cost of inference stays bounded by √N rather
845
+ than growing with the corpus.
846
+
847
+ ### 8.9 The cost ladder: the one currency of every decision
848
+
849
+ The deduction system's rule costs form the **single cost currency of the whole
850
+ mind**: every grounding mechanism's candidate answer is weighed in these same
851
+ units, so a mechanism-level choice (should the answer come from CAST, the cover
852
+ search, or recall?) and a byte-level choice (should this span be a recognised
853
+ completion or a carried literal?) are the _same kind of decision_ — a lightest
854
+ derivation. The ladder is:
855
+
856
+ ```
857
+ ε (MICRO: bridge a recognised span into the — essentially free; the
858
+ cover) per-byte unit of the
859
+ admissible A* heuristic
860
+ 1 (STEP: follow one learned edge; one computed — the unit of inference
861
+ result; one projection; one frame location)
862
+ 10 (CONCEPT: borrow a sibling's edge; one halo- — one order dearer than a
863
+ mediated act; one consensus climb) literal continuation
864
+ 1000·bytes (PASS: carry an unrecognised literal) — coverage dominates
865
+ everything: the search
866
+ prefers to recognise
867
+ ```
868
+
869
+ The constants are chosen as a strict _ordering_ — any set preserving the order
870
+ yields identical lightest derivations. The one quantitative role of ε: it is the
871
+ cheapest per-position cost, so "ε × (bytes remaining)" is an admissible A\*
872
+ lower bound on the cost to finish covering the query — the heuristic that keeps
873
+ the search output-sensitive (§5.2).
874
+
875
+ The grounding decider (§14.1) uses the same ladder: a candidate answer's weight
876
+ is its mechanism's moves (STEP per projection, CONCEPT per halo-mediated act)
877
+ plus PASS for every query byte the mechanism did _not_ account for (did not
878
+ match against learnt structure). The lightest grounding candidate wins — the
879
+ same elementary decision, lifted to the mechanism level.
880
+
881
+ ### 8.10 Two measures of commonality
882
+
883
+ Every mechanism that asks "is this content discriminative?" must choose a
884
+ **reference set** — the population over which _commonality_ is measured. The
885
+ system provides exactly two, and they are formally independent: neither quantity
886
+ bounds the other, and no derived threshold can convert one into the other. The
887
+ choice between them is the single most consequential design decision a mechanism
888
+ makes, because it determines what counts as _scaffolding_ and what counts as
889
+ _evidence_.
890
+
891
+ #### Corpus-global commonality
892
+
893
+ The reference set is **every learned context in the store** — the durable,
894
+ corpus-wide population of edge-bearing nodes (counted by `corpusN`). A node's
895
+ corpus-global commonality is the number of distinct contexts whose
896
+ containment/edge climb reaches it — `reachOf(id, N)`, read through
897
+ `edgeAncestors`, capped at √N.
898
+
899
+ Corpus-global commonality is a **property of a node**, stable across queries.
900
+ The same node always reaches the same number of contexts (modulo new deposits).
901
+ It answers: _does this content discriminate anything in what the system has
902
+ learned?_
903
+
904
+ Content reaching a corpus **minority** of contexts (¬dominates(reach, N))
905
+ discriminates — it is an entity, a filler, a name. Content reaching a corpus
906
+ **majority** (dominates(reach, N)) is frame scaffolding — it discriminates
907
+ nothing anywhere. This is the half-dominance convention of §8.7, applied to the
908
+ entire store.
909
+
910
+ The climb's IDF weighting (§17.3), confluence's filler/scaffolding gate (§18.5),
911
+ and every decision of the form "is this node a hub?" use corpus-global
912
+ commonality. The halo index (§4, §12.2) is also corpus-global: a node's
913
+ distributional signature is the superposition of ALL episodes it took part in,
914
+ not just those relevant to the current query.
915
+
916
+ #### Weave-local commonality
917
+
918
+ The reference set is **the structures aligned with this query** — the transient,
919
+ query-specific population of anchors the consensus climb ranked and whose
920
+ contexts produced literal or distributional runs against the query bytes. The
921
+ population size is `aligned`, counted fresh per query; commonality at byte
922
+ position `i` is `depth[i]`, the sum of alignment weights covering that byte.
923
+
924
+ Weave-local commonality is a **property of a query-byte position**, not of a
925
+ node. The same byte can be frame for one query and content for another, because
926
+ the aligned population changes. It answers: _does this content discriminate
927
+ among the structures THIS query activates?_
928
+
929
+ A byte covered by a weave-local **minority** of aligned structures
930
+ (¬dominates(depth[i], aligned)) discriminates among them — it differentiates one
931
+ aligned context from another. A byte covered by a weave-local **majority**
932
+ (dominates(depth[i], aligned)) is shared scaffolding of the weave — it carries
933
+ no information about which aligned structure is which, regardless of how rare or
934
+ common it is in the corpus.
935
+
936
+ CAST's frame gate (§18.3) uses weave-local commonality. The grounding decider's
937
+ `unaccounted` bytes (§14.1) are also weave-local in spirit: they measure what
938
+ THIS query's mechanisms did not explain, priced against the query's own length.
939
+
940
+ #### Independence
941
+
942
+ The two measures are computed over **different data structures** with
943
+ **different stopping criteria**:
944
+
945
+ - Corpus-global: `edgeAncestors` walks the DAG's parent edges (`parentsFirst`,
946
+ `prevFirst`), counting distinct edge-bearing contexts, capped at √N. It
947
+ answers a question about a node's position in the permanent store.
948
+
949
+ - Weave-local: `alignGraded` aligns raw bytes (literal W-gram seed-and-extend,
950
+ then halo-matched recognised sites), incrementing a per-byte `depth` array. It
951
+ answers a question about this query's transient alignment.
952
+
953
+ Neither computation is a special case of the other. A phrase common to 2 of 3
954
+ aligned exemplars but rare in the corpus (low reach, high weave-local share)
955
+ **is** frame for the weave — it is shared scaffolding of this particular
956
+ analogy, not differentiating content. A phrase with high corpus reach (common
957
+ everywhere) that happens to appear in only 1 of 3 aligned exemplars **is**
958
+ content for the weave — it differentiates that exemplar from the others. The two
959
+ coincide often (semantically rich exemplars tend to share corpus-wide
960
+ scaffolding), but neither derives the other. They cannot be treated as
961
+ interchangeable: replacing CAST's weave-local gate with the structural IDF lets
962
+ the substitution branch fire on reordered single-fact queries.
963
+
964
+ #### Which measure for which question
965
+
966
+ The system provides both measures. Each mechanism picks the one that answers its
967
+ question:
968
+
969
+ | Mechanism | Question | Measure |
970
+ | ------------------------- | ------------------------------------------------------------------------- | ---------------------------------------------- |
971
+ | Consensus climb (§17) | Does this region's anchor discriminate among learned contexts? | Corpus-global (IDF weight) |
972
+ | Confluence (§18.5) | Does this shared content name an entity or is it scaffolding? | Corpus-global (dominates(reach, N)) |
973
+ | CAST frame gate (§18.3) | Do the aligned structures share this byte, or does it differentiate them? | Weave-local (dominates(depth[i], aligned)) |
974
+ | Grounding decider (§14.1) | Which mechanism explains more of THIS query? | Weave-local (unaccounted bytes / query length) |
975
+ | Recognition (§15) | Is this span a stored form? | Neither — exact, content-addressed |
976
+ | Cover search (§19) | Can the query be covered by recognised completions? | Neither — cost-ladder, output-sensitive |
977
+
978
+ A mechanism that uses the wrong measure answers the wrong question. The system
979
+ cannot prevent this — both measures are available, and the architecture does not
980
+ enforce which one a mechanism consults. The distinction is a design discipline,
981
+ not a type-level guard.
982
+
983
+ Concretely: in a query that weaves 3 painting exemplars, the phrase " describe
984
+ it" has high weave-local depth (all 3 aligned contexts contain it) but low
985
+ corpus-global reach (only those 3 of 20 learned contexts do).
986
+ `dominates(depth, 3)` says frame — shared scaffolding of this particular
987
+ analogy. `dominates(reach, 20)` says content — a minority of the corpus, hence
988
+ discriminative. The frame gate correctly ignores the corpus and asks the
989
+ weave-local question. A mechanism that asked the corpus-global question here
990
+ would classify " describe it" as content and let substitution fire on a
991
+ reordered single-fact query. The same node, the same bytes, two different
992
+ answers — because two different questions were asked over two different
993
+ populations.
994
+
995
+ ---
996
+
997
+ ## 9. The concept inventory
998
+
999
+ Every named concept in the system, one line each, with its home section. This is
1000
+ the vocabulary the rest of the document (and the codebase) speaks.
1001
+
1002
+ **Representation**
1003
+
1004
+ - **Gist** — the VSA fold of a span's content; content similarity in one dot
1005
+ product (§2.3).
1006
+ - **Seat / keyring** — the fixed random permutations that bind ordinal position
1007
+ into a fold (§2.3).
1008
+ - **Alphabet** — the 256 deterministic byte vectors with graded similarity
1009
+ (§2.4).
1010
+ - **River fold** — the level-by-level grouping of leaves into a tree, W at a
1011
+ time (§10). **Linear**: only the fold's root is normalized, so every interior
1012
+ gist keeps a byte-proportional magnitude — the basis of angle-and-magnitude
1013
+ semantics (§2.6).
1014
+ - **Magnitude / contentLen** — the byte-proportional length an unnormalized
1015
+ interior gist carries (norm ≈ √len); read back from the store as a span's
1016
+ content length and used to convert a raw cosine into a query-relative or
1017
+ scale-aware fraction (§2.6, §8.1, §17.3, §21).
1018
+ - **Stable prefix** — the already-known head of a stream, folded independently
1019
+ so its structure is reproducible (§10.3).
1020
+
1021
+ **Memory**
1022
+
1023
+ - **Node** — a leaf (bytes) or branch (ordered children); identity = content
1024
+ (§3.2).
1025
+ - **isChunk** — the predicate for "children are all leaves" — the perception
1026
+ tree's smallest grouped unit, behind region collection, canonical seams, and
1027
+ sub-span indexing (§10, §11.3, §17.2).
1028
+ - **Interning / hash-consing** — storing a tree bottom-up so equal subtrees
1029
+ share one node (§11).
1030
+ - **Near-dedup** — merging a fresh root onto a geometrically identical,
1031
+ byte-verified stored root (§11.2).
1032
+ - **Containment edge** — a durable "this window of bytes occurs inside that
1033
+ chunk" record for sub-spans that are not structural children (§11.3).
1034
+ - **Transparent chain (chainRun)** — a run of nodes each with exactly one
1035
+ structural parent and no continuation edges in or out; climbed in a single
1036
+ bounded read instead of one probe per node. Used by the structural DAG climb
1037
+ to skip scaffolding when ascending to edge-bearing contexts.
1038
+ - **Continuation edge** — the learned relation "this followed that"; the atom of
1039
+ factual knowledge (§12.1).
1040
+ - **Company signature** — a deterministic unit vector derived from a node's
1041
+ identity (seeded by its id), used as the halo-pour unit instead of the node's
1042
+ gist (§4, §12.2). Decouples content similarity from distributional similarity.
1043
+ - **Halo / halo mass** — a node's distributional signature (superposition of
1044
+ partner company signatures) and the count of episodes poured into it (§4,
1045
+ §12.2).
1046
+ - **Resonance target** — a node whose gist is admitted to the content index
1047
+ (lazily: roots, edge/halo bearers, and interior forms of experiences) (§12.3).
1048
+ - **Junction** — a learnt whole that literally contains two forms, found by
1049
+ content-addressed DAG ascent (parents + containment links) from the two sides'
1050
+ canonical identities — not by a resonance guess. The walk is **order-free** (a
1051
+ junction evidences that two forms were learnt together; which one the query
1052
+ mentions first is a fact about the query, not the learnt whole — the
1053
+ byte-containment test probes both orders, costing two indexOf calls per
1054
+ visited node, never a second walk). Overlapping or abutting occurrences are
1055
+ accepted (grid fragments of one whole legitimately overlap inside it), with a
1056
+ strict-super-form requirement (holding both must be more than restating either
1057
+ side). The bridge's Tier 1 connector search (§19.5) and cross-region
1058
+ attention's joint-context recovery (§17.6) ascend by the same shared, bounded,
1059
+ cached walk. A per-response walk cache memoises every identity read across all
1060
+ walks of one response, and junction seeds are computed once per candidate and
1061
+ reused across all its pairs. Synonym junctions extend the ascent to halo
1062
+ siblings (Tier 2.5), sharing one expansion budget across all sibling walks.
1063
+
1064
+ **Inference**
1065
+
1066
+ - **Match-and-project** — the ONE elementary operation every generalising
1067
+ mechanism configures: match a learned structure under a matcher, project along
1068
+ a learned relation in a direction, accept past a derived gate (§14.4).
1069
+ - **Matcher** — the matching relation of a match-and-project: exact
1070
+ (content-addressed), locate (the graded exact→halo→gist ladder), aligned
1071
+ (literal W-gram runs), or distributional (analogy strength) (§14.4).
1072
+ - **Projection / direction** — the learned relation a matched node is projected
1073
+ along: forward (`follow`, to the continuation fixpoint), reverse
1074
+ (`reverseContext`, to the establishing context), both (`project`), read-out,
1075
+ insert, or substitute (§14.4).
1076
+ - **Hub cap** — the one √N fan-out convention (§8.8), applied in two forms:
1077
+ `hubBound` (≥ 2, the numerical cap passed to the store's LIMITed reads) and
1078
+ `hubCap` (the list-side reading). Every fan-out walk and disambiguation uses
1079
+ one of them; the store enforces the cap at read time so no per-query cost
1080
+ grows with the corpus.
1081
+ - **Estimator noise floor** — 1/√D, one standard deviation of chance cosine
1082
+ between random vectors. The smallest difference distinguishable from RaBitQ
1083
+ quantisation error (§8.4). The consensus climb gates a region's vote on its
1084
+ discriminative margin clearing this floor.
1085
+ - **Half-dominance** — a part covering strictly more than half of its whole can
1086
+ no longer discriminate it (§8.7). The structural midpoint, derived, never
1087
+ tuned. Used by liftAnswer, region collection, and CAST frame classification.
1088
+ - **Commonality (corpus-global vs. weave-local)** — the TWO reference sets over
1089
+ which "is this content shared or discriminative?" is measured (§8.10).
1090
+ Corpus-global: the durable population of all learned contexts (N); answers
1091
+ "does this discriminate anything in the store?" Weave-local: the transient
1092
+ population of structures aligned with this query (aligned); answers "does this
1093
+ discriminate among the structures this query activates?" The two are formally
1094
+ independent — a phrase rare in the corpus can be scaffolding of the weave, and
1095
+ a phrase common everywhere can differentiate two aligned exemplars. The system
1096
+ provides both; each mechanism chooses the one that answers its question.
1097
+ - **Corpus N** (`corpusN`) — the count of distinct learned contexts floored at
1098
+ 2, so its derived readings (ln N, √N) stay meaningful on a near-empty store.
1099
+ Defined once; every consumer of the corpus scale reads it. (§8.8, §17)
1100
+ - **Expand-until-decided** — the climb's work is bounded by stopping the moment
1101
+ the answer (saturated vs. voted) is known, through LIMITed store reads only
1102
+ (§17.4). The walk is exact below √N distinct contexts and stops at the first
1103
+ proof of saturation past it.
1104
+ - **Canonical contract** — the write/read convention for the store's
1105
+ segmentation: the write side interns W−1 and W sliding windows and a
1106
+ whole-stream flat branch; the read side chains leaf ids up to W² positions and
1107
+ probes every prefix as a flat branch. Defined in one module; a drift between
1108
+ the sides silently breaks recognition. (§10.3, §11.3, §15.2)
1109
+ - **Window IDs** — the canonical content-addressed identity of every W-sized
1110
+ slice of a byte stream, offset → node id. Under this mapping, any content two
1111
+ deposits share IS the same node (hash-consing paid the comparison at write
1112
+ time). Confluence's meet and CAST's frame detection read shared content
1113
+ through this — never through a byte scan. (§18.1, §18.5)
1114
+ - **Reach (structural IDF)** — the number of distinct learnt contexts a single
1115
+ node's containment/edge climb reaches, or Infinity when it reaches none or
1116
+ saturates. Paired with the half-dominance convention: content reaching a
1117
+ corpus minority of contexts discriminates (an entity, a filler); content
1118
+ reaching a majority is frame scaffolding. (§18.3, §18.5)
1119
+ - **Recognition** — decomposing a query into every stored form it contains, by
1120
+ structural and canonical readings (§15).
1121
+ - **Site** — one recognised form: a query span plus the node it names (§15).
1122
+ - **Cover** — the lightest-derivation goal: the query covered left to right by
1123
+ recognised completions and carried literals (§19).
1124
+ - **Fuse / recompose** — the search's discovery that adjacent fragments spell a
1125
+ deeper learned form (§19.4).
1126
+ - **Connector (bridge)** — learned material that belongs _between_ two spans,
1127
+ found by a graded junction ladder: Tier 1 containment ascent by
1128
+ content-addressed identity, then Tier 2 edge junctions (a continuation/context
1129
+ carrying the glue), then Tier 2.5 synonym junctions (the same ascent over halo
1130
+ siblings), then resonance as last resort; disambiguated by the response guide,
1131
+ with the shortest interior preferred. The junction ascent is shared with
1132
+ cross-region attention (§19.5, §17.6).
1133
+ - **Concept hop** — borrowing a distributional sibling's continuation via
1134
+ `haloSiblings` (the unified halo-sibling enumeration) and `guidedFirst` (the
1135
+ guided-or-first convention for edge picks) (§19.3).
1136
+ - **Join with bridge** — the shared composition step for out-of-search assembly:
1137
+ a learned connector between two spans, or a bare join with a visible
1138
+ `bridgeMiss` trace step. Used by multi-topic fusion and CAST. (§19.5, §23)
1139
+ - **Recompletion** — covering a produced answer's own bytes with the same
1140
+ machinery, recursively, to let composites resolve deeper (§19.6).
1141
+ - **Consensus climb / point of attention** — regions of the query vote, through
1142
+ the DAG's parents, for the learned contexts they belong to. Regions come from
1143
+ TWO sources: perceived subtrees (the river fold's positional chunks) and
1144
+ recognised sites (content-addressed nodes — exact anchors that skip the ANN
1145
+ resonance step). Sites capture whole words that cross W-boundaries, which
1146
+ perceived chunks alone miss. Pooled votes select the query's independent
1147
+ topics (§17).
1148
+ - **Saturation** — a region whose upward climb hits hub fan-out abstains rather
1149
+ than voting noise (§17.4).
1150
+ - **Cross-region attention** — direct region-to-region interaction: two regions
1151
+ that independently voted (at least one strongly) pair to recover their joint
1152
+ context — the learnt whole containing both — by the same order-free junction
1153
+ ascent the bridge uses. Corpus-independent: any voted region composes, and a
1154
+ known but non-voting region may serve as the weak side of a pair whose other
1155
+ side voted (a word never trained standalone still binds through its stored
1156
+ byte fragments); two non-voting regions never pair (the shared-prefix trap).
1157
+ N-ary: pair containers are filtered by the remaining candidate forms — the
1158
+ container covering the most of the query's composable forms wins, so three
1159
+ cross-cutting attributes resolve to their unique triple at the cost of one
1160
+ cached read + indexOf per (container, extra), never an extra walk. Explaining
1161
+ away: when a junction binds, any individual vote whose bytes the joint
1162
+ container literally contains and whose roots are fully disjoint from the
1163
+ junction's is superseded — the exact joint evidence explains those bytes away;
1164
+ partial agreement corroborates and is kept. Self-evidence guard: a container
1165
+ whose joined occurrence is itself a substring of the query is rejected —
1166
+ binding is only evidence when the query mentions the forms apart. Consumed
1167
+ candidates never re-pair. A joint container is exact evidence, voting at full
1168
+ strength. Additive pooling alone cannot surface a context zero regions
1169
+ individually voted for; cross-region evidence fills that gap (§17.6).
1170
+ - **CAST (counterfactual transfer)** — substitution / redirection / comparison
1171
+ between independently learned structures the query weaves together. Alignment
1172
+ is **graded** (literal W-grams → halo-matched sites); frame gates are
1173
+ **derived** (`MIN_WEAVE` from the weave minimum, `dominates` from
1174
+ half-dominance) and **weave-local** (majority of _aligned_ structures, not
1175
+ corpus-global IDF). (§18)
1176
+ - **Confluence join** — the meet of independent constraint streams by
1177
+ content-addressed identity: window IDs present in both anchors and absent from
1178
+ the query name the entity satisfying all constraints at once. Answers
1179
+ conjunctive queries ("Which X is A and B?") that no single-fact mechanism can
1180
+ resolve. (§18.5)
1181
+ - **Skill / exemplar** — a learned fact shaped "answer-is-a-span-of-context",
1182
+ reusable as an extraction template on unseen text (§20).
1183
+ - **Recall tiers** — the graded fallback for whole-query resonance, from exact
1184
+ self-match to honest echo. Each tier reports _what it matched_ (`accounted`),
1185
+ its _moves_, and `unexplained` — a human-readable label for the query bytes it
1186
+ left on the table — so the grounding decider can compare it against every
1187
+ other mechanism in the same currency with full diagnostic visibility. (§21)
1188
+ - **Accounted spans** — the query byte ranges a mechanism's own structural
1189
+ evidence explains (aligned runs, located frames, voted regions, constraint
1190
+ content). Query bytes outside them are priced at PASS each — the same rate the
1191
+ cover search pays for a literal connective — so "which mechanism explains more
1192
+ of the query with learnt structure" is the primary axis of the grounding
1193
+ decider. (§14.1, §20, §21)
1194
+ - **Open seat (read-out content is not evidence)** — the span extraction reads
1195
+ between located frames is structurally explained (we know _where_ to read it)
1196
+ but content-novel (we do not know _what_ it says). It is the variable being
1197
+ read, not the structure doing the reading — the same role the cover's
1198
+ unrecognised literals play, and priced the same way (PASS each, by exclusion).
1199
+ Counting it as explained would let a mechanism claim credit for bytes it
1200
+ merely copied from the query. (§20)
1201
+ - **Forward asymmetry (reverse is not derivation)** — the deduction system has
1202
+ no backward rule; a reading against the edge direction (`reverseContext`)
1203
+ produces bytes but no forward derivation. The grounding decider expresses this
1204
+ exactly: reverse readings get `accounted = []`, their weight the full
1205
+ PASS·|query|. The decider derives this from the evidence the formalism itself
1206
+ declares. (§21)
1207
+ - **Weave-local vs. corpus-global commonality** — CAST's frame gates are
1208
+ weave-local (majority of _aligned_ points, per query), not corpus-global
1209
+ (majority of contexts, across the store). The two quantities are formally
1210
+ independent — a phrase common to 2 of 3 aligned exemplars but rare in the
1211
+ corpus IS frame for CAST's purposes; substituting global IDF misfires on
1212
+ reordered single-fact queries. (§18.2)
1213
+ - **Free-will architecture** — the grounding decider as a market: mechanisms are
1214
+ decoupled (zero cross-imports), self-gating (binary structural preconditions),
1215
+ budget-capped (√N, k, LIMITed reads), and evidence-carrying (`accounted`,
1216
+ `moves`, `unexplained`). The decider compares weights in one currency; it does
1217
+ not know which mechanism produced which candidate. The same four constraints —
1218
+ decoupling, declared competence, visible budget, traveling evidence — are the
1219
+ structural principle that makes any budget-limited reasoner honest, from
1220
+ Sema's `√N` to a model's `max_tokens`. (§14.5)
1221
+ - **Grounding decider** — the unified choice among grounding mechanisms: every
1222
+ self-gating mechanism yields a candidate answer weighed in the one cost
1223
+ ladder, and the lightest grounding derivation wins. Moves (STEP per
1224
+ projection, CONCEPT per halo-mediated act) discriminate residually; PASS per
1225
+ unexplained byte dominates. Ties keep the mechanism list's order (cover, CAST,
1226
+ confluence, extract, recall). The decider uses admissible-floor pruning (a
1227
+ mechanism whose best-case floor cannot beat the incumbent is never run) — and
1228
+ a mechanism whose floor itself needs expensive precomputation to refine checks
1229
+ the SAME incumbent before paying for it (§14.1, §14.2).
1230
+ - **Pivot** — the longest unconsumed learned context contained in the current
1231
+ answer; the stepping stone of multi-hop reasoning (§22).
1232
+ - **Fusion** — grounding each independent point of attention and joining the
1233
+ results with learned connectors (§23).
1234
+ - **Articulation** — re-voicing the answer in the asker's own words via halo
1235
+ siblings (§24).
1236
+ - **Echo** — the last-resort output that returns a stored form verbatim,
1237
+ explicitly flagged as not grounded (§21.4, §26).
1238
+ - **Provenance** — which grounding mechanism produced the answer; part of every
1239
+ response (§26).
1240
+ - **Rationale** — the replayable trace of every rule application behind an
1241
+ answer (§26).
1242
+
1243
+ **Computation**
1244
+
1245
+ - **PipelineMechanism** — the ONE uniform interface every grounding mechanism
1246
+ (CAST, confluence, cover, extraction, recall, the ALU, any user extension)
1247
+ implements: an optional `parse` (authoritative computed spans, pre-loop), a
1248
+ `floor` (admissible lower bound), and a `run` (candidate answers). The
1249
+ pipeline never special-cases any mechanism by name or kind (§14.1, §16).
1250
+ - **Precomputed** — the shared, response-scoped container every mechanism's
1251
+ `floor`/`run` (and the post-grounding stages) receive: eager fields
1252
+ (recognition, computed spans, guide) plus lazily-cached methods for the
1253
+ expensive structural analyses (the consensus climb `attention()`, the weave,
1254
+ span-shape classification, the identity-window reads), each computed at most
1255
+ once, reused across every consumer, and never computed at all when no
1256
+ mechanism asks (§14.1, §14.2).
1257
+ - **Extension** — a user- or built-in-supplied `PipelineMechanism` whose `parse`
1258
+ recognises computations (arithmetic, logic, …) the mind should not have to
1259
+ learn fact-by-fact; joins via `mechanismFactories` (§16).
1260
+ - **ALU** — the built-in extension: arithmetic, logic, and numerical computation
1261
+ derived from an irreducible kernel (§16.2).
1262
+ - **Masking** — computed spans override colliding learned facts for exactly
1263
+ their bytes ("computation always wins") (§16.3).
1264
+
1265
+ ---
1266
+
1267
+ ---
1268
+
1269
+ # Part III — The ingestion pipeline
1270
+
1271
+ Ingestion is the learning half of the system: input bytes in, a more
1272
+ knowledgeable graph out. It has three stages — perception, deposition, relation
1273
+ learning — and the whole of it is deterministic: the same inputs in the same
1274
+ order always produce a structurally identical store.
1275
+
1276
+ ```
1277
+ THE INGESTION PIPELINE
1278
+
1279
+ input (text / bytes / grid / frames)
1280
+
1281
+ │ 1. modality flattening (grids → Hilbert-ordered byte stream)
1282
+
1283
+ byte stream b₀ b₁ b₂ … bₙ
1284
+
1285
+ │ 2. leaf lift: each byte → a leaf carrying its alphabet vector
1286
+
1287
+ leaves [l₀][l₁][l₂] … [lₙ]
1288
+
1289
+ │ 3. RIVER FOLD: group ≤ W siblings per level, seat-bind + superpose
1290
+ │ (splitting at the stable-prefix boundary when the head of the
1291
+ │ stream is already known)
1292
+
1293
+ perceived tree (every node: bytes-or-kids + gist vector)
1294
+
1295
+ │ 4. INTERN bottom-up into the Merkle DAG
1296
+ │ exact dedup → near-dedup (byte-verified) → mint new node
1297
+
1298
+ root node id + id of every subtree
1299
+
1300
+ ├─ 5a. sub-span windows + containment edges (recognition seams)
1301
+ ├─ 5b. whole-stream flat branch (canonical byte identity)
1302
+
1303
+ │ 6. RELATIONS
1304
+ │ single input: chain part → part continuation edges
1305
+ │ pair (context, continuation):
1306
+ │ edge context-root ──▶ continuation-root
1307
+ │ halos: pour each side's seat-bound company signature into the other
1308
+
1309
+ updated store (DAG + edges + halos + lazily-updated vector indexes)
1310
+ ```
1311
+
1312
+ ---
1313
+
1314
+ ## 10. Perception: from bytes to a tree
1315
+
1316
+ ### 10.1 Every modality is a stream
1317
+
1318
+ Perception's contract is minimal: it accepts a _byte stream_ and returns a tree.
1319
+ Text becomes bytes by UTF-8 encoding; raw binary is already bytes; an image, a
1320
+ volume, or a stack of video frames is linearized along a Hilbert curve (§6.3) so
1321
+ that spatial locality becomes stream locality. Nothing downstream knows or cares
1322
+ which modality produced the stream — _geometry is only a reading order_.
1323
+
1324
+ ### 10.2 The river fold
1325
+
1326
+ The fold builds the tree level by level, like a river merging tributaries:
1327
+
1328
+ ```
1329
+ riverFold(leaves):
1330
+ level ← leaves
1331
+ while |level| > 1:
1332
+ next ← []
1333
+ for each complete group g of W consecutive items in level:
1334
+ next.append( foldGroup(g) ) # one parent node
1335
+ append the trailing incomplete items (fewer than W) unchanged
1336
+ level ← next # recurse upward
1337
+ normalize( level[0].gist ) # ONLY the finished root
1338
+ return level[0]
1339
+
1340
+ foldGroup(children c₀ … cₖ): # k < W
1341
+ gist ← Σᵢ πᵢ · gist(cᵢ) # seat-bind + superpose (§2.3)
1342
+ # — NOT normalized here
1343
+ return branch node with kids (c₀ … cₖ) and that gist
1344
+ ```
1345
+
1346
+ Properties worth noting:
1347
+
1348
+ - **Determinism.** Grouping is purely positional; the same stream always yields
1349
+ the same tree shape and, therefore (given the fixed alphabet and keyring), the
1350
+ same gists everywhere.
1351
+ - **Logarithmic depth.** Each level shrinks by roughly a factor of W, so a
1352
+ stream of n bytes folds in ⌈log_W n⌉ levels.
1353
+ - **Every level is meaningful.** Intermediate nodes are not scaffolding to be
1354
+ discarded; each one is a content-addressable span with a gist — perception
1355
+ manufactures the _addressable sub-structure_ that recognition and attention
1356
+ later depend on.
1357
+ - **Linear, not renormalized per level.** Only the completed root is normalized
1358
+ to unit length; every interior gist is left at its raw superposed length. This
1359
+ is a deliberate choice of similarity semantics, not a shortcut: an interior
1360
+ node's magnitude grows with the amount of content folded into it (§2.6), so
1361
+ the fold gives every span both an angle (what it resembles) and a magnitude
1362
+ (how much of it there is) for free, in the same vector.
1363
+ - **W is the resolution quantum.** W bounds how much material one fold step
1364
+ mixes; it reappears throughout the system as the "one perceptual step" unit
1365
+ (the reach threshold, the near-dedup window, the fusible span ceiling,
1366
+ alignment seed size).
1367
+
1368
+ ### 10.3 The stable prefix
1369
+
1370
+ One refinement protects structure across growing inputs. Consider training on a
1371
+ dialogue where each turn's context is the previous context plus one more
1372
+ exchange. Folded naively, adding bytes at the end can shift every group
1373
+ boundary, so the shared prefix folds _differently_ in each deposit — the store
1374
+ would never notice that the prefix is the same knowledge.
1375
+
1376
+ Perception therefore checks, before folding, whether some head of the stream is
1377
+ _already a known form_ (a store-recognised sequence of leaves — an exact,
1378
+ content-addressed check, not a similarity guess). If a known **proper** prefix
1379
+ of length p exists, the fold is split at p: the prefix folds exactly as it did
1380
+ when it was learned, the suffix folds independently, and the two join only at
1381
+ the top. Identical prefixes thus produce identical subtrees — and hash-consing
1382
+ then collapses them to the very same nodes — regardless of what follows them.
1383
+
1384
+ (The prefix must be _proper_ — shorter than the whole input — because a
1385
+ full-length match would mean the entire input is already stored, and splitting
1386
+ there would hide the true internal structure.)
1387
+
1388
+ ### 10.4 Perception pseudocode, complete
1389
+
1390
+ ```
1391
+ perceive(input):
1392
+ bytes ← flatten(input) # UTF-8 / identity / Hilbert
1393
+ if bytes is empty: return the empty tree
1394
+ leaves ← [ leaf(bᵢ, alphabet[bᵢ]) for each byte bᵢ ]
1395
+ p ← longest proper prefix of `bytes` whose leaf-sequence is a known form
1396
+ return riverFold(leaves, splitAt = p if p > 0 else none)
1397
+
1398
+ # riverFold with a split: at every level, items are partitioned at the
1399
+ # boundary containing byte-offset p; each side folds as if standalone.
1400
+ ```
1401
+
1402
+ ---
1403
+
1404
+ ## 11. Deposition: interning the tree into the graph
1405
+
1406
+ ### 11.1 Bottom-up interning
1407
+
1408
+ The perceived tree is interned into the DAG bottom-up:
1409
+
1410
+ ```
1411
+ intern(tree node n) → node id:
1412
+ if n is a leaf:
1413
+ return internLeaf(n.bytes, n.gist)
1414
+ kidIds ← [ intern(k) for k in n.kids ] # children first
1415
+ return internBranch(kidIds, n.gist)
1416
+ ```
1417
+
1418
+ Both interning operations follow the same ladder:
1419
+
1420
+ ```
1421
+ internLeaf / internBranch(content, gist):
1422
+ 1. EXACT DEDUP if a node with this exact content exists → return it
1423
+ 1b. CROSS-REP if this is a branch whose kid ids flatten to a known
1424
+ flat branch's bytes, reuse that flat branch's id
1425
+ (§3.2 — content addressing across representations)
1426
+ 2. NEAR DEDUP (branches only, against whole-experience roots only)
1427
+ if some fresh root's gist is within the merge
1428
+ threshold (§8.1) AND the two byte strings differ by
1429
+ at most ONE local span of ≤ W bytes → return that id
1430
+ 3. MINT otherwise create a new node; record, for each child,
1431
+ the reverse (child → parent) structural edge
1432
+ ```
1433
+
1434
+ Points of principle:
1435
+
1436
+ - **Exact dedup is the primary compression** and is intrinsic: it is what makes
1437
+ identity a function of content (§3.1). It works for leaves and branches alike.
1438
+ - **Near-dedup is deliberately narrow.** It applies only to branches, only
1439
+ against _genuine whole experiences_ (roots that bear edges or halos — never
1440
+ interior scaffolding), and only with byte verification. The geometric bar
1441
+ alone is scale-blind: in a deep fold, a large localized difference dilutes
1442
+ toward cosine 1, so _any_ fixed bar below 1 would eventually merge things that
1443
+ differ in exactly the span that matters. The byte check — identical except one
1444
+ window of at most W bytes — is the perception system's own definition of "the
1445
+ smallest real difference", so the merge can never corrupt reconstruction.
1446
+ - **The parent edges minted in step 3 are the climb.** Every branch records
1447
+ itself as a parent of each distinct child. These reverse edges are what later
1448
+ lets a recognised fragment climb to the experiences containing it (§17).
1449
+ Single-byte leaves are exempt: a byte occurs in nearly everything, so its
1450
+ parent set would be a useless corpus-sized hub.
1451
+
1452
+ ### 11.2 Why interior nodes matter
1453
+
1454
+ After interning, _every_ subtree of the deposit — not just the root — is an
1455
+ addressable node. This is not an implementation convenience; it is the mechanism
1456
+ behind three capabilities:
1457
+
1458
+ - **Partial recall**: a query naming only a slice of an experience can resonate
1459
+ with that slice's node directly.
1460
+ - **Multi-topic attention**: different regions of one query can anchor to
1461
+ interior nodes of _different_ experiences (§17).
1462
+ - **Compositional generalization**: fresh input that shares any sub-span with
1463
+ old input shares nodes with it, so the new is literally built out of the old.
1464
+
1465
+ ### 11.3 Sub-span windows and containment
1466
+
1467
+ Perception's grouping is positional, so a meaningful unit (say, a name) may
1468
+ straddle a group boundary and never be a node of any tree. Deposition therefore
1469
+ additionally interns **sliding windows** of W and W−1 leaves across the stream,
1470
+ as flat branches. A window that does not coincide with a structural child of any
1471
+ chunk is linked to the chunk(s) it overlaps by a durable **containment edge** —
1472
+ a second, weaker parent relation meaning "these bytes occur inside that chunk".
1473
+ When a later climb starts from such a window (which has no structural parents of
1474
+ its own), it climbs through its containment parents instead. This closes the
1475
+ recognition seams that pure positional chunking would leave.
1476
+
1477
+ Deposition also interns the **whole stream as one flat branch** (the sequence of
1478
+ its byte-leaves). This gives every deposit a canonical byte-level identity
1479
+ independent of tree shape — the form the stable-prefix check of §10.3 looks up,
1480
+ and a second content-addressed route to the same experience.
1481
+
1482
+ ---
1483
+
1484
+ ## 12. Learning relations: edges and halos
1485
+
1486
+ ### 12.1 Continuation edges: the atom of factual knowledge
1487
+
1488
+ A **fact**, in Sema, is an ordered association: _this_ was followed by _that_.
1489
+ Depositing a pair (context, continuation) records one continuation edge from the
1490
+ context's root node to the continuation's root node. Edges are:
1491
+
1492
+ - **Idempotent** — the same pair deposited twice is one edge (though its halo
1493
+ evidence accumulates; see below).
1494
+ - **Directional** — "what follows X" (forward) and "what does X follow"
1495
+ (reverse) are both readable, and both are used: forward for answering, reverse
1496
+ for recognising that a query _is_ some context's answer (reverse recall,
1497
+ §21.1) and for counting evidence (§25).
1498
+ - **Plural** — a context may accumulate many continuations (the same question
1499
+ answered differently across a corpus). Choosing among them is a first-class
1500
+ disambiguation problem (§25), not an error.
1501
+
1502
+ A _single_ input (no pair) still learns sequence: the parts of its root are
1503
+ chained by edges at stride W, so a long document is traversable as a sequence of
1504
+ its chunks.
1505
+
1506
+ ### 12.2 Halo pours: distributional bookkeeping
1507
+
1508
+ When a pair (context, continuation) is deposited, each side's **company
1509
+ signature** — a deterministic unit vector derived from the partner's node
1510
+ identity — is superposed ("poured") into the other side's halo, bound to a role
1511
+ seat so that "I appeared as context" and "I appeared as answer" are
1512
+ geometrically distinct:
1513
+
1514
+ ```
1515
+ pour( halo(contextPart) , π₁ · companySignature(continuation) )
1516
+ pour( halo(continuation) , π₀ · companySignature(contextPart) )
1517
+ ```
1518
+
1519
+ The company signature is seeded by the partner's node id. Node ids are
1520
+ content-addressed (mint order), stable for a given corpus (including
1521
+ checkpoint/resume, which re-derives identical ids), so the same partner always
1522
+ contributes the same signature. But two partners with the _same bytes_ and
1523
+ _different ids_ contribute nearly orthogonal signatures — so two halos correlate
1524
+ only through shared episode history, never through accidental byte-level content
1525
+ overlap.
1526
+
1527
+ Over many episodes, a node's halo becomes the superposition of everything it has
1528
+ kept company with — the distributional signature of §4. Each pour also
1529
+ increments the node's **halo mass**, the direct count of corroborating episodes.
1530
+
1531
+ One subtlety guards the signal's quality: in a _tracked_ sequence of deposits
1532
+ (e.g. a growing dialogue), the context's pour targets only the **changed nodes**
1533
+ — the subtree that is new relative to the previous deposit — so a boilerplate
1534
+ prefix repeated in every turn does not soak up halo mass and drown the
1535
+ discriminating content.
1536
+
1537
+ ### 12.3 Lazy indexing: what enters the vector indexes, and when
1538
+
1539
+ The DAG holds every node, but the _content index_ (the ANN structure over gists,
1540
+ §6) holds only nodes worth resonating to — and admission is lazy:
1541
+
1542
+ - A node's gist is _captured_ at intern time but _indexed_ only when the node
1543
+ becomes a **resonance target**: it gains an edge, gains a halo, is a deposit
1544
+ root, or is an interior form of an experience (when either end of an edge is
1545
+ learned, that experience's whole subtree is admitted, because partial queries
1546
+ must be able to resonate with its interior).
1547
+ - A node that structurally **bridges** two experiences (its parent count crosses
1548
+ 1 → 2) is promoted at exactly that moment — the moment it becomes useful for
1549
+ cross-experience recall.
1550
+ - Pure intermediate scaffolding that never becomes any of those is never indexed
1551
+ at all. It still exists in the DAG (identity, reconstruction, climbing all
1552
+ work); it simply is not a resonance destination.
1553
+
1554
+ The halo index is maintained on a geometric schedule (a node's halo is
1555
+ re-indexed when its mass is small or crosses a power of two), since a halo's
1556
+ _direction_ stabilizes as mass grows.
1557
+
1558
+ The principle: **the DAG is the truth; the indexes are lazy, rebuildable views
1559
+ of the parts of the truth that queries actually land on.**
1560
+
1561
+ ---
1562
+
1563
+ ## 13. Ingestion, end to end
1564
+
1565
+ The complete deposit algorithm, in pseudocode:
1566
+
1567
+ ```
1568
+ ingest(input, second = none):
1569
+
1570
+ # ── forms ────────────────────────────────────────────────────────────
1571
+ if input is a list of items / (context, continuation) pairs:
1572
+ for each element: ingest it by the rules below
1573
+ return
1574
+
1575
+ if second is given: ingestPair(input, second)
1576
+ else: ingestOne(input)
1577
+
1578
+
1579
+ deposit(input, tracked):
1580
+ bytes ← flatten(input)
1581
+ tree ← perceive(bytes) # §10
1582
+ ids ← intern every node of tree, bottom-up # §11.1
1583
+ intern sliding W / W−1 windows; record containment edges # §11.3
1584
+ intern the whole stream as a flat branch # §11.3
1585
+ changed ← if tracked and a previous deposit exists:
1586
+ the maximal new subtree vs. the previous deposit # §12.2
1587
+ else: [ tree ]
1588
+ return (tree, rootId, ids, changed)
1589
+
1590
+
1591
+ ingestOne(input): # a bare experience
1592
+ (tree, root, ids, _) ← deposit(input, tracked = true)
1593
+ mark root as a resonance target # §12.3
1594
+ parts ← the root's immediate children
1595
+ if |parts| > W:
1596
+ link parts[i] ──▶ parts[i+W] for each stride-W step # §12.1
1597
+ else:
1598
+ mark each part as a resonance target
1599
+
1600
+
1601
+ ingestPair(context, continuation): # a fact
1602
+ (ctxTree, ctxRoot, ctxIds, changed) ← deposit(context, tracked = true)
1603
+ (conTree, conRoot, _, _ ) ← deposit(continuation, tracked = false)
1604
+
1605
+ link ctxRoot ──▶ conRoot # the fact itself
1606
+ for each part in changed: # distributional evidence
1607
+ pour halo(part) += π₁ · companySignature(conRoot)
1608
+ pour halo(conRoot) += π₀ · companySignature(part)
1609
+ # linking / pouring admits both subtrees' interiors to the content
1610
+ # index (lazily), per §12.3
1611
+ ```
1612
+
1613
+ Costs, in broad strokes: perception is linear in the input length; interning is
1614
+ one content-addressed lookup per tree node (the tree has O(n/W · logᵂ n) nodes
1615
+ but is dominated by its O(n) leaves); relation learning is O(1) edges plus
1616
+ O(changed parts) halo pours. Nothing in the deposit path scans the corpus.
1617
+ **Training a fact takes one pass over the fact.**
1618
+
1619
+ ### 13.1 Why storage stays viable: the economics of the store
1620
+
1621
+ The claims above ("one pass", "nothing scans the corpus", "bounded RAM") are not
1622
+ free consequences of the data model — they are earned by a specific set of
1623
+ cost-control mechanisms in the store. They deserve their own account, because
1624
+ each one is a _deliberate trade_ whose failure mode is well understood, and
1625
+ together they are what makes a corpus-scale store run on ordinary hardware. The
1626
+ organizing principle:
1627
+
1628
+ > **Exactness is mandatory only for identity and reconstruction. Everything else
1629
+ > — caches, indexes, buffers — is a bounded, rebuildable accelerator whose miss
1630
+ > costs work, never correctness.**
1631
+
1632
+ The mechanisms, each with its trade:
1633
+
1634
+ **1. Implicit leaves and flat branches (representation compression).** Single
1635
+ bytes are not stored at all — a byte's node id is derived from its value (the
1636
+ negative range), so the 256 most common nodes in existence cost zero rows. A
1637
+ branch whose children are all single-byte leaves (the vast majority of small
1638
+ spans) stores its _bytes_ (1 byte per child) instead of a packed child-id list
1639
+ (4 bytes per child) — a 4× saving on the store's most numerous row shape,
1640
+ content-addressed through the same lookup path.
1641
+
1642
+ **2. Bounded caches everywhere (RAM viability).** Every in-memory acceleration
1643
+ structure — the exact-dedup key maps, the reconstructed-bytes cache, the
1644
+ node-record cache, the pending-gist capture, the exact halo accumulators — is an
1645
+ LRU map with a _byte budget_, not an entry count. Reconstruction caches evict
1646
+ smallest-first (protecting entries that are expensive to rebuild); all others
1647
+ evict least-recently used. A miss re-derives from durable state. Resident memory
1648
+ is therefore capped by configuration regardless of corpus size.
1649
+
1650
+ **3. Lazy, selective vector indexing (index viability).** The ANN index is the
1651
+ most expensive thing the store maintains — every entry costs an encode, graph
1652
+ edges, and future query work. So admission is lazy and selective (§12.3): a
1653
+ node's gist is _captured_ cheaply at intern time into a bounded buffer, and
1654
+ _indexed_ only at the moment the node demonstrably becomes a resonance
1655
+ destination — when it gains an edge (which also admits its whole subtree's
1656
+ interior forms), gains a halo, is a deposit root, or crosses the 1→2 parent
1657
+ transition that makes it a bridge between experiences. Pure scaffolding is never
1658
+ indexed. The trade: an evicted-before-promotion gist means that node is
1659
+ reachable only by the structural climb until a batch repair pass regenerates it
1660
+ — reduced reach, never wrong answers.
1661
+
1662
+ **4. No per-branch ANN probes on the write path (write viability).**
1663
+ Near-deduplication (§11.1 step 2) consults only the _write buffer's_ few
1664
+ whole-experience roots — an O(buffer) exact scan — never the flushed ANN index.
1665
+ Probing the ANN index for every new branch is both the dominant potential
1666
+ training cost and _unsound_: 1-bit estimates can rank a byte-distinct branch
1667
+ nearest, and merging on that corrupts reconstruction. The principle:
1668
+ **approximate structures are kept off the write path entirely.**
1669
+
1670
+ **5. Write batching with deferred durability (I/O viability).** Node rows,
1671
+ edges, halos, containment sets, and vector-index entries accumulate in buffers
1672
+ and commit in coalesced batches (one transaction, one index upsert) once a size
1673
+ threshold is reached — turning what would be per-node fsync-bounded writes into
1674
+ large sequential ones. Within a batch, repeated halo pours to the same node
1675
+ coalesce to one index write.
1676
+
1677
+ **6. Two quantizations, two purposes (halo viability).** Halo accumulators would
1678
+ otherwise be the largest table in an episodically-trained store (one float
1679
+ vector per fact-bearing node). Gists avoid this problem entirely — a gist is a
1680
+ deterministic function of a node's content (`perceive → fold`), so it can live
1681
+ in a volatile buffer and be regenerated on demand. A halo is a function of
1682
+ _training history_ (the sum of every episode signature poured into it) — it
1683
+ cannot be regenerated from the node's bytes and must be durable. The system uses
1684
+ two different quantizations because durability and search have different
1685
+ requirements:
1686
+
1687
+ - **Durable row: 2-bit Lloyd–Max, reversible.** The halo vector is stored on
1688
+ disk at 16× compression (260 bytes for D=1024 vs. 4096 for float32). It must
1689
+ be decodable back to an approximate float32 vector because it serves as an
1690
+ _accumulator_: a session loads it, adds new pours to it in float32, and writes
1691
+ it back. The round-trip through the quantizer preserves ≥ 0.88 correlation
1692
+ with the exact accumulator — the coarsest grain that survives repeated
1693
+ load→accumulate→flush cycles without the direction drifting. One bit would
1694
+ _not_ suffice for this purpose: decoded back, a sign-only vector is binary
1695
+ (±1), and accumulating on top of it degrades with every cycle.
1696
+
1697
+ - **ANN index: 1-bit RaBitQ, irreversible.** The same halo vector, projected
1698
+ through a random rotation and reduced to one sign bit per dimension (32×
1699
+ compression), serves as a search code in the HNSW index. This code is _never_
1700
+ decoded back — it only answers "which halos are near this query?" The
1701
+ estimator is unbiased (expected cosine is recoverable from the bit count), so
1702
+ ranking quality is preserved despite the loss of reversibility.
1703
+
1704
+ The session's actively poured accumulators are kept exact in a bounded float32
1705
+ cache, so within-session accumulate-then-compare never round-trips through
1706
+ either quantizer. The ANN index re-enters only when a halo's mass is small or
1707
+ crosses a power of two (mechanism 7 below), at which point the durable 2-bit row
1708
+ is decoded to float32, normalized, rotated, and re-encoded to 1-bit RaBitQ.
1709
+
1710
+ **7. Geometric re-index schedule (index-write viability).** A halo's _direction_
1711
+ stabilizes as mass grows, so re-indexing it on every pour is waste. Halos
1712
+ re-enter the index only when their mass is small or crosses a power of two —
1713
+ O(log mass) index writes per node over its lifetime instead of O(mass) — while
1714
+ the durable (quantized) row is always current.
1715
+
1716
+ **8. Counting instead of materialising (read viability).** Evidence questions on
1717
+ the hot path ("how many distinct contexts predict this?", "how many learned
1718
+ contexts exist?", "does this lead somewhere?") are answered by indexed counts
1719
+ (`prevCount`, an incrementally-maintained distinct-source count for `corpusN`,
1720
+ `hasNext`, `hasHalo`) — never by materialising corpus-sized edge lists. Full
1721
+ materialising reads (`prev`, `parents`) exist for maintenance and inspection but
1722
+ are kept off every hot path. The hub bound √N (§8.8) is enforced at the _store
1723
+ level_: every fan-out read on the hot path uses a LIMITed variant
1724
+ (`nextFirst(id, √N)`, `prevFirst(id, √N)`, `parentsFirst(id, √N+1)`,
1725
+ `containersSlice(id, offset, √N)`) whose work is bounded by the cap regardless
1726
+ of the actual fan-out size. A consumer that needs the full fan-out (the
1727
+ consensus climb, §17) instead uses these to decide its question exactly — "does
1728
+ this reach cross √N distinct contexts?" — without ever materialising a
1729
+ corpus-sized list. The principle: **a per-query read must never grow with the
1730
+ corpus.**
1731
+
1732
+ **9. Compaction on write-volume cadence (long-run viability).** Updates and
1733
+ deletions in the vector indexes leave tombstones. After every configured volume
1734
+ of index writes, an index whose physical size exceeds 2× its live size is
1735
+ rebuilt from its surviving codes (lossless — the build is code-based).
1736
+ Post-training, a batch pass can additionally remove index entries for
1737
+ structurally isolated nodes (single-parent, no edges, no halo — they bridge
1738
+ nothing), and a converse repair pass re-indexes bridges that were missed.
1739
+ Failures of these best-effort passes are counted visibly, never silent.
1740
+
1741
+ ### 13.2 The store's cost machinery, in pseudocode
1742
+
1743
+ ```
1744
+ # ── the intern ladder with its caches (§11.1, refined) ────────────────
1745
+ intern(content, gist): # content = bytes | kidIds
1746
+ # 1. exact dedup: bounded LRU first, durable probe second — so dedup
1747
+ # survives a cold cache (a resumed run still recognises old content)
1748
+ id ≔ dedupCache[key(content)] ?? durableFind(content)
1749
+ if id ≠ ∅:
1750
+ dedupCache[key(content)] ≔ id
1751
+ captureIfUnindexed(id, gist) # keep the gist available for
1752
+ return id # lazy indexing (mech. 3)
1753
+ # 1b. content addressing across representations: a branch spanning the
1754
+ # same bytes as a stored flat branch reuses that id
1755
+ if content is kidIds and their leaf-id flattening is a known branch:
1756
+ return that id (capture as above)
1757
+ # 2. near-dedup: BUFFERED whole-experience roots only — never an ANN
1758
+ # probe (mech. 4); geometric proposal + byte verification (§11.1)
1759
+ if content is kidIds:
1760
+ best ≔ argmax over nearDedupBuffer of dot(gist, root.gist)
1761
+ if best ≥ MERGE and differsByOneWindow(content, best, W):
1762
+ return best.id
1763
+ # 3. mint
1764
+ id ≔ nextId++ # dense ids; leaves are implicit
1765
+ write node row (flat-branch encoding when applicable — mech. 1)
1766
+ for each real child c: # (byte leaves get no parent
1767
+ insert kid edge c → id # rows — hub avoidance)
1768
+ if |parents(c)| just became 2: # the 1→2 bridge transition
1769
+ indexGist(c) # promote NOW (mech. 3)
1770
+ pendingGist[id] ≔ gist # captured, not yet indexed
1771
+ maybeFlush()
1772
+ return id
1773
+
1774
+ # ── lazy index admission (mech. 3) ────────────────────────────────────
1775
+ indexGist(id, dedupTarget = false):
1776
+ if id already indexed (session set, else one durable point query):
1777
+ return # a resumed run replays deposits
1778
+ # at read speed — no re-upserts
1779
+ v ≔ pendingGist[id]; if v = ∅: return # evicted ⇒ retry on a future
1780
+ # encounter or repair pass
1781
+ contentBuffer += (id, v); mark indexed
1782
+ if dedupTarget: nearDedupBuffer += (id, v)
1783
+
1784
+ indexSubtree(root): # fired by link() on both ends
1785
+ indexGist(root, dedupTarget = true) # only the ROOT may be merged
1786
+ walk the subtree, pruning at already-classified nodes:
1787
+ indexGist(interior) # reach-only: partial queries
1788
+ # must resonate with interiors
1789
+
1790
+ # ── halo pour (mech. 6–7) ─────────────────────────────────────────────
1791
+ pourHalo(id, addVec):
1792
+ indexGist(id, dedupTarget = true) # a halo-bearer is a target
1793
+ acc ≔ exactCache[id] ?? dequantize(durableRow[id]) ?? 0⃗
1794
+ acc += addVec; exactCache[id] ≔ acc # exact in-session
1795
+ mass += 1
1796
+ durableRow[id] ≔ quantize2bit(acc), mass # always current, 16× small
1797
+ if mass ≤ 4 or mass is a power of two: # geometric schedule
1798
+ haloBuffer[id] ≔ normalize(acc) # O(log mass) index writes
1799
+
1800
+ quantize2bit(v): # Lloyd–Max for unit Gaussian
1801
+ store ‖v‖ exactly; per coordinate: sign bit + |x| ≷ 0.9816·σ bit
1802
+ (σ ≔ ‖v‖/√D; decode to ±0.4528σ / ±1.5104σ, rescaled to ‖v‖)
1803
+
1804
+ # ── batching and compaction (mech. 5, 9) ──────────────────────────────
1805
+ maybeFlush():
1806
+ if |contentBuffer| + |haloBuffer| ≥ batchSize: flush()
1807
+
1808
+ flush():
1809
+ merge containment buffer into packed rows
1810
+ upsert contentBuffer into the content index (one batch)
1811
+ upsert haloBuffer into the halo index (one batch)
1812
+ clear nearDedupBuffer (mirrors contentBuffer)
1813
+ commit the deferred write transaction (one commit per batch)
1814
+ writtenSinceCompact += batch size
1815
+ if writtenSinceCompact ≥ compactEvery:
1816
+ for each vector index with physicalSize > 2 × liveSize:
1817
+ rebuild it from its live codes (lossless; code-based)
1818
+
1819
+ # ── post-hoc maintenance (mech. 9) ────────────────────────────────────
1820
+ compactContentIndex(minParents = 2): # archived-store trade:
1821
+ remove entries with < minParents parents and no edges and no halo
1822
+ # they bridge nothing
1823
+ repairContentIndex(regenerate, minParents = 2):
1824
+ for each branch node not in the index with ≥ minParents parents and
1825
+ (edges or a halo):
1826
+ re-perceive its bytes; add its gist to the index
1827
+ ```
1828
+
1829
+ Two summary facts fall out of this machinery. First, the _asymptotics_: storage
1830
+ is O(distinct subtrees); the ANN index is bounded by the number of distinct byte
1831
+ patterns that ever became resonance destinations (not by deposits, and not by
1832
+ corpus volume — hash-consing sees to that); every per-deposit cost is a bounded
1833
+ number of O(1) probes and amortized-O(1) buffered writes. Second, the
1834
+ _degradation order_: under memory pressure or eviction, what is lost is always
1835
+ acceleration (a duplicate probe, a re-perception, reduced resonance reach until
1836
+ repair) and never identity, reconstruction, or an already-learned relation.
1837
+
1838
+ ---
1839
+
1840
+ ---
1841
+
1842
+ # Part IV — The inference pipeline
1843
+
1844
+ ## 14. The shape of an answer
1845
+
1846
+ ### 14.1 The pipeline at a glance
1847
+
1848
+ Every ask travels one road:
1849
+
1850
+ ```
1851
+ THE INFERENCE PIPELINE
1852
+
1853
+ query bytes
1854
+
1855
+ │ perceive (same fold as ingestion — the query gets a tree & gists)
1856
+
1857
+ ┌───────────────────────────────────────────────────────────────────┐
1858
+ │ RECOGNISE (§15) every stored form the query contains │
1859
+ │ COMPUTE (§16) every mechanism's parse() evaluates spans it │
1860
+ │ is authoritative for (arithmetic, logic, …) │
1861
+ │ PRECOMPUTE build Precomputed: recognition, computed │
1862
+ │ spans, the response's gist — shared, │
1863
+ │ response-scoped data every mechanism (and the │
1864
+ │ post-grounding stages) read; every expensive │
1865
+ │ analysis (consensus climb, weave, span-shape) │
1866
+ │ is a lazily-cached method, computed at most │
1867
+ │ once and only if some consumer asks │
1868
+ └──────────────────────────────┬────────────────────────────────────┘
1869
+
1870
+ ┌───────────────────────────────────────────────────────────────────┐
1871
+ │ GROUND — ONE lightest-derivation choice among UNIFORM mechanisms: │
1872
+ │ │
1873
+ │ The pipeline sees every mechanism through the SAME │
1874
+ │ PipelineMechanism interface (parse?/floor/run) — no branch │
1875
+ │ anywhere names a specific mechanism or asks "is this an │
1876
+ │ extension?". Each mechanism's `floor` yields an admissible │
1877
+ │ lower bound (or null when it structurally cannot fire); each │
1878
+ │ `run` yields CANDIDATEs weighed in the one cost ladder (§8.9), │
1879
+ │ and the lightest grounding derivation wins. A candidate's │
1880
+ │ weight is: │
1881
+ │ │
1882
+ │ moves + PASS · (query bytes the mechanism did not account │
1883
+ │ for — did not match against learnt │
1884
+ │ structure) │
1885
+ │ │
1886
+ │ — PASS per unexplained byte is the cover's own price for a │
1887
+ │ literal connective, so the primary axis is "which mechanism │
1888
+ │ explains more of the query", and move costs (STEP per │
1889
+ │ projection, CONCEPT per halo-mediated act) discriminate │
1890
+ │ residually. Ties keep the mechanism list's own order. │
1891
+ │ │
1892
+ │ Admissible-floor pruning, uniformly: `floor` is called for │
1893
+ │ EVERY mechanism, every time; `run` only for one whose floor │
1894
+ │ can still beat the incumbent. A mechanism whose floor itself │
1895
+ │ needs expensive precomputation to refine (CAST's weave │
1896
+ │ alignment) checks the SAME incumbent, via a `worthRunning` │
1897
+ │ predicate passed into `floor`, before paying for it — the │
1898
+ │ pipeline's own pruning tool, exposed one level earlier, so no │
1899
+ │ mechanism special-cases what beat it. │
1900
+ │ │
1901
+ │ The mechanisms, in list order (cover runs first: a computed │
1902
+ │ span — from the ALU or any user extension — masks in at near- │
1903
+ │ zero cost, so a cheap incumbent is established before CAST/ │
1904
+ │ confluence would otherwise invest in their own precomputation): │
1905
+ │ │
1906
+ │ 1. COVER (§19) the query's own decomposition composes an │
1907
+ │ answer — ONE lightest-derivation search; │
1908
+ │ computed spans (§16) mask colliding sites │
1909
+ │ and enter the search at zero cost │
1910
+ │ 2. CAST (§18) the query weaves ≥2 independent learned │
1911
+ │ structures → transfer structure between them │
1912
+ │ 3. CONFLUENCE (§18.5) the query carries ≥2 independent │
1913
+ │ constraints → intersect their evidence │
1914
+ │ 4. EXTRACT (§20) a learned span-in-context skill reads the │
1915
+ │ analogous span out of the query │
1916
+ │ 5. RECALL (§21) whole-query resonance, four graded tiers │
1917
+ │ (…or NOTHING — silence below the reach bar) │
1918
+ └──────────────────────────────┬────────────────────────────────────┘
1919
+
1920
+ ┌───────────────────────────────────────────────────────────────────┐
1921
+ │ REASON (§22) extend the grounded answer across facts, hop by hop │
1922
+ │ FUSE (§23) ground the query's OTHER points of attention and │
1923
+ │ join them with learned connectors │
1924
+ │ ARTICULATE(§24) re-voice the result in the asker's own vocabulary │
1925
+ └──────────────────────────────┬────────────────────────────────────┘
1926
+
1927
+ answer bytes + provenance + (optionally) the full rationale
1928
+ ```
1929
+
1930
+ The decider also emits three diagnostic signals — purely observational, never
1931
+ affecting the decision itself:
1932
+
1933
+ - **Unexplained label** — every candidate carries `unexplained`, a
1934
+ human-readable label for the query bytes its evidence left on the table.
1935
+ Appears in the rationale trace; does not affect the weight (the PASS-per-byte
1936
+ pricing already accounts for it arithmetically).
1937
+ - **Narrow decision** — when the winner beats the runner-up by ≤ 1 grade unit
1938
+ (⌊w/STEP⌋), the rationale records a `narrowDecision` step. A margin of 0 means
1939
+ the tie-break (the mechanism list's own order: cover, cast, confluence,
1940
+ extract, recall) decided — the answer could change with one more training
1941
+ fact.
1942
+ - **Thin grounding** — when the winning candidate's density (fraction of query
1943
+ bytes actually accounted for by learnt structure) falls below `1/W` (the
1944
+ smallest fraction the store's perceptual window can discriminate), the
1945
+ rationale records a `thinGrounding` step. The answer stands; the label is a
1946
+ signal for downstream consumers that the grounding is sparse.
1947
+
1948
+ ### 14.2 Design invariants of the pipeline
1949
+
1950
+ Four rules hold everywhere and are worth reading the rest of Part IV against:
1951
+
1952
+ 1. **Uniform mechanisms, self-gating, weighed together.** Every grounding
1953
+ mechanism (CAST, confluence, cover, extraction, recall, the ALU, any user
1954
+ extension) implements the SAME `PipelineMechanism` interface — an optional
1955
+ `parse` (pre-loop computed spans), a `floor` (admissible lower bound), and a
1956
+ `run` (candidate answers). The pipeline (`think`) never imports a
1957
+ mechanism-specific type and never branches on which mechanism it is holding;
1958
+ adding or removing a mechanism means adding or removing one object from the
1959
+ list. Each mechanism checks its own _structural preconditions_ (does the
1960
+ query weave two structures? did a cover compose? is any anchor a skill
1961
+ exemplar?) inside `floor`/`run` and abstains when they fail — no mechanism
1962
+ runs on a query it cannot structurally explain. Every mechanism whose gate
1963
+ passes yields a candidate answer **weighed in the one cost ladder** (§8.9):
1964
+ its moves plus PASS per query byte it did not account for. The lightest
1965
+ grounding derivation wins — the same elementary decision the cover search
1966
+ makes among spans, lifted to the mechanism level. Ties keep the mechanism
1967
+ list's own order (cover, cast, confluence, extract, recall).
1968
+
1969
+ `floor` is called for EVERY mechanism, every response, in list order, BEFORE
1970
+ any `run`; `run` is called only for a mechanism whose floor can still beat
1971
+ the incumbent (`worthRunning`). Every expensive analysis a floor might need
1972
+ (the consensus climb `pre.attention()`, the weave `pre.weave()`, span-shape
1973
+ classification) is a lazily-cached method on the response's `Precomputed`:
1974
+ computed at most once, shared by every consumer — mechanisms AND the
1975
+ post-grounding stages — and never computed at all if nobody asks (a query an
1976
+ extension decided outright never pays for a climb). The INVESTMENT DISCIPLINE
1977
+ closes the loop: `worthRunning` only gates `run` on the pipeline's side, so a
1978
+ `floor` that would first-touch an expensive analysis (CAST's climb + weave,
1979
+ which decide only whether its floor EXISTS — the number is always exactly
1980
+ 2·STEP, never below) checks `worthRunning(cheapestBound)` first, and when
1981
+ that already fails it RETURNS THE BOUND uninvested — still admissible, and
1982
+ the pipeline's own check then prunes `run` and records the truthful "cannot
1983
+ beat incumbent" trace note. This is the same admissible-floor pruning the A\*
1984
+ search lives by, applied uniformly — no mechanism asks "did an extension
1985
+ already decide?"; it only ever asks "can I still beat what already won?", and
1986
+ cover running first in the list means a computed span's near-zero cost prunes
1987
+ everything after it the same way any other cheap incumbent would.
1988
+ 2. **Read-only store.** Asking never writes. All the memoization inference uses
1989
+ is per-response and is possible _because_ the store cannot change mid-answer.
1990
+ 3. **One guide.** The whole response shares the query's gist as its
1991
+ disambiguation guide, and shares its per-context choices, so every mechanism
1992
+ of one answer follows the _same_ reading of every ambiguous fact.
1993
+ 4. **Honesty outlets.** At every stage there is a sanctioned way to say less:
1994
+ mechanisms abstain when their structural preconditions fail, recall returns
1995
+ silence below the reach bar, an un-grounded echo is labelled as such, a
1996
+ missing connector joins pieces bare and says so in the trace.
1997
+ 5. **Bounded inference.** No per-query read grows with the corpus. Every fan-out
1998
+ walk is capped at √N, and the cap is enforced at the store level through
1999
+ LIMITed reads and indexed existence probes. The climb uses
2000
+ expand-until-decided: it stops the moment saturation or a concrete vote is
2001
+ determined, with work bounded by √N distinct contexts regardless of corpus
2002
+ size. The cost of answering a query is dominated by the query's own
2003
+ structure, not by how much the system has learned.
2004
+
2005
+ ### 14.3 How the mind's mechanisms integrate
2006
+
2007
+ The pipeline of §14.1 shows the _order of execution_; this diagram shows the
2008
+ _order of dependency_ — which mechanism is built on which. It is a strict
2009
+ layering: every arrow points downward, a mechanism only ever calls mechanisms in
2010
+ layers below it, and there are no cycles. (The layers correspond one-to-one to
2011
+ the modules of the implementation; see AGENTS.md, "Where things live".)
2012
+
2013
+ ```
2014
+ THE MIND'S MECHANISMS — DEPENDENCY LAYERS
2015
+ (arrows = "is built on"; every arrow points to a lower layer)
2016
+
2017
+ L6 ORCHESTRATION ┌───────────────────────────────────────────────┐
2018
+ │ respond ─▶ think (the grounding decider, §14.1)│
2019
+ │ ─▶ articulate │
2020
+ │ rationale/trace: cross-cuts every layer │
2021
+ └──────┬───────┬──────────┬──────────┬──────────┘
2022
+ │ │ │ │
2023
+ L5 GROUNDING & ┌──────▼──┐ ┌──▼────┐ ┌───▼─────┐ ┌──▼──────┐
2024
+ POST-GROUNDING │ cover │ │ CAST │ │ confl. │ │ extract │
2025
+ (§18–§23) │ (§19) │ │ (§18) │ │ (§18.5) │ │ (§20) │
2026
+ └──┬───┬──┘ └──┬───┬─┘ └──┬───┬──┘ └──┬───┬──┘
2027
+ ┌──▼───▼───────▼──────────▼──────────────▼──────┐
2028
+ │ recall (§21) · reason (§22) · fuse (§23) │
2029
+ └──┬────────────────┬───────────────────┬──────┘
2030
+ │ │ │
2031
+ L4 QUERY-LEVEL ┌──▼────────────────▼───┐ ┌───────────▼──────┐
2032
+ EVIDENCE │ consensus climb (§17) │ │ graph search: │
2033
+ │ regions → votes → │ │ the deduction │
2034
+ │ cross-region → pool → │ │ system (§19.1–6) │
2035
+ │ commit │ │ │
2036
+ └──┬──────────┬─────────┘ └──┬────────┬──────┘
2037
+ │ │ │ │
2038
+ L3 MATCH & PROJECT ┌──▼──────────▼────────────────▼───┐ ┌──▼──────┐
2039
+ (the elementary │ match (§14.4): │ │ derive: │
2040
+ operation, §14.4) │ matchers: locate · alignGraded ·│ │ A*LD │
2041
+ │ analogyStrength │ │ engine │
2042
+ │ projections: follow · │ │ (§5) │
2043
+ │ conceptHop · reverseContext ·│ │ │
2044
+ │ project │ │ │
2045
+ ├───────────────────────────────────┤ └─────────┘
2046
+ │ resonance (§19.5, §21, §22): │
2047
+ │ bridge (→ junction) · │
2048
+ │ recallByResonance · │
2049
+ │ pivotInto · meaningOf │
2050
+ └──┬──────────────┬────────────────┘
2051
+ │ │
2052
+ L2 DECOMPOSITION ┌──▼──────────┐ ┌─▼────────────────────────────┐
2053
+ & TRAVERSAL │ recognition │ │ traverse: edgeAncestors · │
2054
+ │ (§15): │ │ nextOf/prevOf · contains · │
2055
+ │ sites/ │ │ chooseNext / chooseAmong │
2056
+ │ leaves/ │ │ (§25) · hubCap (§8.8) │
2057
+ │ splits │ │ │
2058
+ └──┬──────────┘ └─┬────────────────────────────┘
2059
+ │ │
2060
+ L1 PRIMITIVES ┌──▼──────────────▼────────────────────────────┐
2061
+ │ perceive · gistOf · resolve · read (§10, §27)│
2062
+ └──┬───────────────────────────────────────────┘
2063
+
2064
+ L0 SUBSTRATE ┌──▼───────────────────────────────────────────┐
2065
+ │ the store: DAG (nodes, parents, containment) │
2066
+ │ + edges + halos + the two vector indexes │
2067
+ └──────────────────────────────────────────────┘
2068
+
2069
+ Sideways (same-layer) collaborations, all mediated by lower layers:
2070
+ · the ALU and any user extension are ORDINARY L5 mechanisms — the same
2071
+ `PipelineMechanism` shape as CAST, confluence, cover, extraction, and
2072
+ recall (§16). Only `parse()` (pre-loop, collected from every mechanism
2073
+ that implements it) makes them distinctive: its computed spans feed into
2074
+ cover's masking, which hands them INTO the L4 graph search as axioms —
2075
+ an extension never talks to another mechanism directly, and think never
2076
+ branches on "is this an extension?".
2077
+ · CAST, confluence, extract, and recall all consume the SAME memoised
2078
+ consensus climb (via the response's shared `Precomputed`); cover instead
2079
+ consumes recognition directly (its axioms are the query's own
2080
+ decomposition).
2081
+ · reason and fuseAttention run AFTER whichever grounding mechanism
2082
+ fired, and reuse its consumed-node set — the one piece of state that
2083
+ crosses between L5 mechanisms.
2084
+ · articulate re-enters the L4 graph search in substitution mode: the
2085
+ revoiced answer is itself a lightest derivation.
2086
+ ```
2087
+
2088
+ How to read the layers:
2089
+
2090
+ - **L0–L1** are the shared substrate every mechanism stands on: the store's
2091
+ exact relations, and perception as a pure function. Nothing above them touches
2092
+ bytes or vectors except through them.
2093
+ - **L2** produces the two elementary readings of anything: _what stored forms
2094
+ does this byte string contain_ (recognition) and _what does this node connect
2095
+ to_ (traversal — raw edge reads, the two disambiguation regimes of §25, and
2096
+ the one √N fan-out cap of §8.8). The shared **junction** ascent belongs here:
2097
+ it climbs the DAG from content-addressed seeds to find containers holding two
2098
+ forms, serving both the bridge (L3) and cross-region attention (L4) from the
2099
+ same bounded, cached walk.
2100
+ - **L3** is the elementary **match-and-project** operation (§14.4): the matcher
2101
+ family (graded locate, literal alignment, distributional analogy) and the
2102
+ projection family (forward fixpoint, concept hop, reverse context, and their
2103
+ composition), plus the resonance patterns built directly on them.
2104
+ - **L4** holds the two big composite engines: the consensus climb (which turns
2105
+ L2/L3 readings into _pooled evidence_ through the derive engine's arithmetic
2106
+ semiring — voting each region independently, then recovering joint contexts
2107
+ through cross-region junction ascent), and the graph search (which turns them
2108
+ into a _cover_ through the tropical semiring). Both are clients of the same
2109
+ A\*LD engine — that is the sense in which all of Sema's thinking is one kind
2110
+ of computation. The junction ascent sits as a shared utility consumed by both
2111
+ L3 (the bridge's Tier 1 connector search) and L4 (cross-region attention's
2112
+ joint-context recovery).
2113
+ - **L5** are the five grounding strategies plus the two post-grounding
2114
+ extenders. Their candidates are **weighed together** by the grounding decider
2115
+ of §14.1 — no fixed priority ladder; the lightest grounding derivation wins.
2116
+ Their _dependency_ structure is what the diagram shows — e.g. extraction
2117
+ depends on the climb (to find an exemplar) and on resonance (to locate
2118
+ frames), but never on cover or CAST. Confluence (like CAST) depends on the
2119
+ climb and on canonical window identity.
2120
+ - **L6** is orchestration only: `think` sequences L5, `articulate` closes the
2121
+ loop, and the rationale tracer observes every layer without being depended on
2122
+ by any.
2123
+
2124
+ The layering is also the _isolation_ structure: a defect in one grounding
2125
+ strategy cannot corrupt another, because they share nothing above L4 except the
2126
+ memoised climb (read-only) and the consumed set (explicitly handed forward).
2127
+
2128
+ ### 14.4 The elementary operation: match → project, under a gate
2129
+
2130
+ Every generalising mechanism in Part IV is a configuration of **one** elementary
2131
+ operation:
2132
+
2133
+ > **Match** a learned structure against bytes under some _matching relation_,
2134
+ > bind whatever did not match as the variable, then **project** along a learned
2135
+ > relation in some _direction_ — accepting the result only past a derived
2136
+ > **gate**.
2137
+
2138
+ The three parameters, and their complete value sets:
2139
+
2140
+ **The matcher** — how strictly "this query material IS that learned structure"
2141
+ is decided, from strictest to loosest:
2142
+
2143
+ | Matcher | Decides by | Cost | Used as |
2144
+ | :------------------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------- | :--------------------------------------------------------------- |
2145
+ | **exact** | content-addressed identity (the DAG) | O(1) probe | recognition's sites; recall tier 0; the pivot's byte containment |
2146
+ | **locate** | the graded ladder exact → halo → gist: literal bytes first, then distributional role (gate: concept threshold), then perceived gist (gate: merge threshold) | one resonance per relaxation | extraction locating an exemplar's frames in the query |
2147
+ | **aligned** | maximal literal runs by seed-and-extend over W-grams — every span two whole structures share, not one position | O(\|a\|+\|b\|+runs) | CAST detecting the structures a query weaves |
2148
+ | **distributional** (`analogyStrength`) | graded three-tier test: direct halo cosine, then mutual halo sibling, then shared learnt W-window frames (gate: significance bar) | ≤ 2 halo-index queries + O(\|a\|+\|b\|) scan | CAST validating a genuine analog |
2149
+
2150
+ **The direction** — which learned relation the matched node is projected along,
2151
+ and which way:
2152
+
2153
+ | Direction | Operation | Meaning |
2154
+ | :----------------------------- | :---------------------------------------------------------------------------------------------------- | :-------------------------------------------------------- |
2155
+ | **forward** (`follow`) | walk continuation edges to the fixpoint; the first hop may cross a concept (halo) link | "what does this lead to?" |
2156
+ | **reverse** (`reverseContext`) | the context this continuation follows — guide-resonance pick with a guide, halo-mass pick without one | "what establishes this?" |
2157
+ | **both** (`project`) | forward, else reverse | the universal grounding step |
2158
+ | **read-out** | read the query span the matched frame delimits | extraction: the variable comes _out_ of the query |
2159
+ | **insert** | place the matched filler into the learned structure's seat | CAST substitution: the variable goes _into_ the structure |
2160
+ | **substitute** | replace the matched form with its concept sibling's bytes | articulation's revoicing |
2161
+
2162
+ **The gate** — the derived threshold (§8) the match/projection must clear:
2163
+ concept threshold for anything distributional, merge threshold for gist
2164
+ identity, significance bar for analogy, reach threshold for accepting any final
2165
+ answer, the consensus floor for a further point of attention. Never a tuned
2166
+ constant.
2167
+
2168
+ Under this decomposition, the mechanism catalogue of Part IV reads as a
2169
+ configuration table:
2170
+
2171
+ | Mechanism | Matcher | Direction | Gate |
2172
+ | :---------------------- | :-------------------------------- | :----------------------- | :---------------------------- |
2173
+ | cover follow-edge (§19) | exact | forward | — (cost ladder) |
2174
+ | concept hop (§19.3) | halo sibling | forward | concept threshold |
2175
+ | recall tiers 0–1 (§21) | identity / whole-query gist | both | merge threshold |
2176
+ | skill extraction (§20) | locate (on the exemplar's frames) | read-out | per-step ladder gates |
2177
+ | CAST substitution (§18) | graded (literal → halo) | insert | frame (MIN_WEAVE + dominates) |
2178
+ | CAST redirection (§18) | graded (literal → halo) | both (on the substitute) | frame (MIN_WEAVE + dominates) |
2179
+ | CAST comparison (§18) | graded + distributional | reverse, juxtaposed | significance bar |
2180
+ | multi-hop pivot (§22) | exact (byte containment) | forward | — |
2181
+ | articulation (§24) | halo sibling | substitute | concept threshold |
2182
+
2183
+ **Why this is stated here, prominently.** The machinery is factored exactly
2184
+ once: the matcher and projection families are one shared family (with the one √N
2185
+ fan-out convention beside the disambiguators), and each mechanism states _only
2186
+ its configuration_. This is a standing architectural rule:
2187
+
2188
+ > **Before adding a new generalising mechanism, express it as a (matcher,
2189
+ > direction, gate) triple.** If its matcher, projection, and gate already exist,
2190
+ > the mechanism is a configuration — write only the configuration. If it
2191
+ > genuinely needs a new matcher or a new direction, add that _to the shared
2192
+ > family_ (with its gate derived in the geometry, §8), never as a private helper
2193
+ > inside the mechanism. A mechanism file that re-implements locating, aligning,
2194
+ > edge-following to a fixpoint, predecessor-picking, or fan-out capping is
2195
+ > re-introducing the exact duplication this decomposition removed.
2196
+
2197
+ Two boundaries of the unification are deliberate, not omissions:
2198
+
2199
+ - **The cover search keeps its own internal edge-chain.** Inside the deduction
2200
+ system (§19), following an edge is a _rule application_ — one step of the
2201
+ proof tree, individually costed and traced. Collapsing it into the shared
2202
+ forward projection would erase the derivation's granularity and cross the
2203
+ synchronous/asynchronous seam (the search is synchronous; the projections may
2204
+ touch the ANN indexes). The two implementations of "follow an edge" are
2205
+ therefore one _concept_ with two _obligations_: proof-grained inside the
2206
+ search, fixpoint-grained outside it.
2207
+ - **Arbitration between mechanisms is one lightest-derivation decision.** The
2208
+ grounding decider (§14.1) weighs every mechanism's candidate in the one cost
2209
+ ladder, so a mechanism-level choice and a byte-level choice are the same kind
2210
+ of decision. Ties (at STEP resolution — sub-STEP costs like MICRO are
2211
+ non-ordering bookkeeping and must not decide a cross-mechanism choice) keep
2212
+ the mechanism list's own order (cover, cast, confluence, extract, recall) —
2213
+ cover runs first not because it is prioritised over the others by fiat, but
2214
+ because a computed span (§16) masks in at near-zero cost, which then prunes
2215
+ the rest through the SAME admissible-floor mechanism every mechanism is
2216
+ subject to (§14.2), not a special rule.
2217
+
2218
+ ### 14.5 The free-will architecture
2219
+
2220
+ The grounding decider is not a priority ladder with a special-case branch for
2221
+ every mechanism. It is a **market**: mechanisms produce candidates, each priced
2222
+ in one currency, and the lightest wins. This works because every mechanism obeys
2223
+ four constraints that are the same constraints a budgeted reasoner — human,
2224
+ language model, or automated deduction engine — must obey to be composable:
2225
+
2226
+ **1. Decoupling.** A mechanism imports nothing from other mechanisms — and
2227
+ implements exactly the same `PipelineMechanism` shape as every other one, so the
2228
+ pipeline imports nothing mechanism-specific either. CAST does not know
2229
+ extraction exists. Extraction does not know confluence exists (or that an
2230
+ extension might have fired). Each receives `ctx`, `query`, and the response's
2231
+ shared `Precomputed` (recognition, computed spans, the gist — plus lazily-cached
2232
+ methods for every expensive analysis a mechanism opts into: the consensus climb,
2233
+ the weave, span-shape classification, the identity-window reads), and returns a
2234
+ candidate or nothing. Adding a mechanism never touches an existing mechanism
2235
+ file.
2236
+
2237
+ **2. Declared competence.** A mechanism gates itself with a structural
2238
+ precondition — a binary, auditable condition, not a learned score — inside its
2239
+ own `floor`. CAST checks `query.length < 2W`. Extraction checks whether any
2240
+ ranked anchor is span-shaped. When a mechanism does not fire, the rationale says
2241
+ exactly why. A mechanism whose floor needs its own expensive precomputation to
2242
+ refine also receives `worthRunning`, so declaring "structurally impossible" and
2243
+ declaring "cannot beat the incumbent, don't bother computing further" cost the
2244
+ same: nothing.
2245
+
2246
+ **3. Visible budget.** Every loop over corpus-scale data is capped at a named
2247
+ constant: `k = 2·recallQueryK` for the alignment loop, `√N` for every fan-out
2248
+ walk. The cap is enforced at the store level through LIMITed reads and indexed
2249
+ existence probes. No per-query cost grows with the corpus.
2250
+
2251
+ **4. Evidence travels with the answer.** Every candidate carries `accounted`
2252
+ (what it explained), `moves` (what its acts cost), and `unexplained` (what it
2253
+ left on the table). The decider does not know which mechanism produced which
2254
+ candidate — it only sees weights in the one cost ladder.
2255
+
2256
+ These four constraints are the same structural principle that makes a
2257
+ budget-limited reasoner honest. A language model has a context window (visible
2258
+ budget), stops when it converges (declared competence via early-stopping),
2259
+ reports its chain of thought (evidence traveling with the answer), and composes
2260
+ with tools via structured output schemas (decoupling). The architecture is the
2261
+ same. What Sema calls `√N`, a model calls `max_tokens`. What Sema calls
2262
+ `admissible floor`, a model calls `skip-if-unpromising`. What Sema calls
2263
+ `unexplained`, a model calls `I don't know`.
2264
+
2265
+ The difference is that Sema's caps are **derived** from the system's own
2266
+ geometry (D, W, N) rather than chosen by an external budget. A mechanism's
2267
+ admissible floor is computed from the memoised climb before the mechanism runs.
2268
+ A fan-out cap is `Math.ceil(Math.sqrt(corpusN))` — the point at which further
2269
+ evidence no longer discriminates. The principle is the same; the derivation is
2270
+ internal.
2271
+
2272
+ ---
2273
+
2274
+ ## 15. Recognition: decomposing the query
2275
+
2276
+ Recognition answers: _which stored forms does this byte string contain, and
2277
+ where?_ Its output — the **sites** (span → node), the query's perceived
2278
+ **leaves**, and the **split** positions where a form boundary falls inside a
2279
+ leaf — is the raw material of every downstream mechanism.
2280
+
2281
+ Two complementary readings run over the query:
2282
+
2283
+ ### 15.1 The structural reading
2284
+
2285
+ Perceive the query (the same fold as ingestion) and walk its own tree, asking
2286
+ the store, bottom-up, to name each subtree: leaves by their bytes, branches by
2287
+ their children's ids. Because perception is deterministic, any part of the query
2288
+ that was ever deposited _as it appears here_ folds into the identical subtree
2289
+ and is named exactly. Additionally, within each leaf-parent chunk, every
2290
+ contiguous sub-run of leaves is probed, so forms smaller than a chunk are found
2291
+ too.
2292
+
2293
+ ### 15.2 The canonical reading
2294
+
2295
+ The query's own fold may cut the stream differently from how training cut it
2296
+ (different surroundings ⇒ different group boundaries). The canonical reading
2297
+ re-derives the _store's_ segmentation directly on the query's bytes: at each
2298
+ position, chain the known single-byte leaves forward and probe each growing
2299
+ sequence as a flat branch — recovering forms _as training stored them_,
2300
+ regardless of how the query's tree happens to chunk. Where such a form's
2301
+ boundary falls strictly inside one of the query's leaves, that position is
2302
+ recorded as a **split**: the search may later cut the leaf there (§19.4).
2303
+
2304
+ ### 15.3 What counts as a site
2305
+
2306
+ A recognised span is admitted as a site only if its node **leads somewhere** —
2307
+ it bears a continuation edge or a halo. A form that leads nowhere contributes
2308
+ nothing to any derivation, so recognition filters it out at the source.
2309
+
2310
+ ```
2311
+ recognise(query):
2312
+ sites, leaves, splits ← ∅
2313
+ # structural
2314
+ tree ← perceive(query)
2315
+ for each subtree s of tree (bottom-up, with byte offsets):
2316
+ id ← store lookup of s (leaf bytes / branch kid-ids)
2317
+ if id exists and (next(id) ≠ ∅ or halo(id) ≠ ∅):
2318
+ sites += (span(s), id)
2319
+ collect leaves; probe sub-runs within leaf-parents likewise
2320
+ # canonical
2321
+ for each position p in query:
2322
+ chain known leaves from p; for each chained prefix that is a known
2323
+ flat branch, resolve its span and admit it as above
2324
+ splits ← form boundaries that fall inside a perceived leaf
2325
+ return (sites, leaves, splits)
2326
+ ```
2327
+
2328
+ Everything here is a bounded number of O(1) content-addressed probes per byte —
2329
+ never a scan of the corpus.
2330
+
2331
+ ---
2332
+
2333
+ ## 16. Computation: extensions and the ALU
2334
+
2335
+ ### 16.1 Manual rules beside learned ones
2336
+
2337
+ Some knowledge is _rules_, not facts: nobody should teach a memory system
2338
+ arithmetic one sum at a time. Sema accommodates this with **extensions**:
2339
+ `PipelineMechanism`s (§14.1, §14.2) whose distinguishing feature is `parse` —
2340
+ consulted once per query, before the grounding loop, over EVERY mechanism that
2341
+ implements it. An extension's `parse` receives the raw query and returns
2342
+ **computed spans** — byte ranges it recognises as computations, together with
2343
+ the authoritative result bytes for each. The mind lends every extension the same
2344
+ four neutral capabilities it already has (resonant meaning-matching, grounded
2345
+ continuation lookup, geometric segmentation, and the perception window W)
2346
+ through the `ExtensionHost` port; it learns nothing about what the extension
2347
+ computes.
2348
+
2349
+ An extension joins through `Mind`'s `mechanismFactories` option: a factory
2350
+ receiving the `ExtensionHost` and returning a `PipelineMechanism`. Once
2351
+ constructed, it is indistinguishable to the pipeline from CAST, confluence,
2352
+ cover, extraction, or recall — same `floor`/`run` shape, same admissible-floor
2353
+ pruning, same weight ladder. The ONLY place its computed spans get special
2354
+ treatment is cover's masking (§16.3), which is a property of cover's own `run`,
2355
+ not of the pipeline singling out "extensions" as a concept.
2356
+
2357
+ ### 16.2 The ALU
2358
+
2359
+ The built-in extension — wrapped into a `PipelineMechanism` by `aluToMechanism`
2360
+ and appended to the mechanism list at `Mind` construction (`cfg.alu.enabled`) —
2361
+ is a small **arithmetic–logic unit** built the way a mathematician would want
2362
+ it: an irreducible kernel — one logic gate (NAND), the field-and-order
2363
+ primitives (0, 1, add, negate, multiply, reciprocal, sign), one limit operator
2364
+ (converge-to-tolerance), and three structural list operations (construct,
2365
+ length, project) — from which everything else (comparison, powers, roots,
2366
+ calculus, equation solving, map/reduce over nested lists, element-wise
2367
+ broadcasting) is _derived by rewriting_, not separately implemented. It parses
2368
+ infix notation directly, and can also recognise an operation _by meaning_ — a
2369
+ span whose gist resonates with a registered operation's learned anchor — using
2370
+ the host's resonance capability. Results are computed exactly (or to declared
2371
+ tolerance) and deterministically rendered to bytes.
2372
+
2373
+ ### 16.3 Masking: computation always wins
2374
+
2375
+ A computed span enters the cover search (§19) as a recognised completion at the
2376
+ cost of one inference step — an authoritative _derived fact_, on par with
2377
+ following a learned edge. Precedence over memory is enforced not by cost but by
2378
+ **masking**: any recognised site that overlaps a computed span is removed before
2379
+ the search, so within those bytes the computation is the only available
2380
+ completion. A corpus deliberately taught "2+2 → 5" therefore cannot outvote the
2381
+ ALU on "2+2", while remaining free to associate whatever it likes _around_ the
2382
+ computation — a computed result and an unrelated learned rewrite still compose
2383
+ within one answer.
2384
+
2385
+ ---
2386
+
2387
+ ## 17. The consensus climb: points of attention
2388
+
2389
+ ### 17.1 The problem: what is this query about?
2390
+
2391
+ Several mechanisms need to know which learned context(s) a query is _about_ —
2392
+ especially when the query's wording matches nothing directly (scaffolding words
2393
+ dominate), or when it is about _two things at once_. The consensus climb answers
2394
+ this with the machinery already on hand: geometry proposes, structure climbs,
2395
+ and pooled weighted deduction decides.
2396
+
2397
+ ### 17.2 The algorithm
2398
+
2399
+ ```
2400
+ climbAttention(query, k):
2401
+
2402
+ # 1. REGIONS — TWO sources, both structural:
2403
+ # a) PERCEIVED SUBTREES of the query (every branch level of the river
2404
+ # fold). A region that dominates the query (covers more than half)
2405
+ # is admitted only if it is the sole structure.
2406
+ # b) RECOGNISED SITES — content-addressed nodes the query literally
2407
+ # contains (§15). A site IS an exact structural anchor; perceived
2408
+ # sub-regions are approximate (W-byte chunks whose gist must resonate
2409
+ # to find an anchor). Sites fill the gap perception creates: a word
2410
+ # crossing a W-boundary is split into chunks whose partial gists may
2411
+ # not resonate distinctively, but the site names the whole word by
2412
+ # exact content identity. Sites that overlap sub-regions add
2413
+ # corroborating evidence; sites in gaps fill them.
2414
+ # Recognition is memoised per response — adding sites costs zero.
2415
+ regions ← subtrees of perceive(query) ∪ recognise(query).sites
2416
+
2417
+ # 2. VOTE — each region finds its best anchor in the DAG and climbs to
2418
+ # the edge-bearing contexts ("roots") that contain it.
2419
+ # Perceived sub-regions resonate their gist into the content index;
2420
+ # site-regions already carry an exact node id and skip the ANN query
2421
+ # entirely — they climb directly from the resolved node.
2422
+ # The climb uses EXPAND-UNTIL-DECIDED (§17.4): it stops the moment the
2423
+ # answer (saturated vs. voted, with exact contextsReached) is known —
2424
+ # all reads through it are LIMITed at √N, so the cost is bounded by the
2425
+ # hub convention, never by the corpus.
2426
+ for each region r:
2427
+ anchor ← r.nodeId ?? (canonical chunk id ?? contentIndex.nearest(gist(r), k)[0])
2428
+ # contrastive-margin gate: for APPROXIMATE regions only, the gap
2429
+ # between the best and second-best score must exceed the estimator
2430
+ # noise floor (§8.4); otherwise the region abstains
2431
+ if not r.known and (bestScore − secondBestScore) ≤ NOISE: ABSTAIN
2432
+ reach ← expandUntilDecided(anchor):
2433
+ for each level: check prevCount (indexed O(1)) to decide
2434
+ "edge-bearing?" without materialising prev lists; read
2435
+ parentsFirst(√N+1) — getting √N+1 proves "hub" exactly;
2436
+ containment parents page via containersSlice(offset, √N)
2437
+ so a common window's climb stops at the first saturated
2438
+ page; distinct contexts past √N decide "saturated" by an
2439
+ indexed count, never a materialised list.
2440
+ if anchor climbs nowhere and is not saturated:
2441
+ try the remaining hits, nearest first # ranking is
2442
+ # approximate; an
2443
+ # orphan top hit is
2444
+ # an accident
2445
+ if saturated: the region ABSTAINS # §17.4
2446
+ mutual ← mutualExplanation(score, region, anchor) # angle + magnitude
2447
+ w ← mutual · ln(N / contextsReached) / |roots| # IDF weighting
2448
+ the region votes w for each root it reached
2449
+
2450
+ # 2b. CROSS-REGION — voteRegions climbs each region INDEPENDENTLY;
2451
+ # additive pooling can only surface contexts at least one region
2452
+ # already votes for. Two regions whose individual climbs land on
2453
+ # DIFFERENT contexts ("red" → `red square`, "circle" → `circle`)
2454
+ # leave their JOINT context (`red circle`) with no vote — no amount
2455
+ # of pooling can recover it. Cross-region attention recovers it by
2456
+ # the SAME content-addressed junction ascent the bridge uses
2457
+ # (the shared junction ascent):
2458
+ #
2459
+ # Candidate regions: ANY voted region composes — not just recognised
2460
+ # sites (corpus independence). At least one side must be a STRONG
2461
+ # voter (individually discriminative). A known but non-voting region
2462
+ # may serve as the WEAK side (a word never learnt standalone still
2463
+ # binds through its stored byte fragments). Two non-voting regions
2464
+ # never pair (the shared-prefix trap). Only MAXIMAL spans compose
2465
+ # (a span contained in another candidate is not independent of it).
2466
+ # A single known region covering both is skipped — the whole form
2467
+ # already votes directly. Byte-sorted left-to-right.
2468
+ #
2469
+ # junctionSeeds are precomputed ONCE per candidate across all its
2470
+ # pairs (cost hoisting). The junction ascent is ORDER-FREE (a
2471
+ # junction is evidence the forms were learnt together; which one the
2472
+ # query mentioned first is a fact about the query, not the learnt
2473
+ # whole — the byte-containment test probes both orders). All reads
2474
+ # go through the shared per-response walkCache.
2475
+ #
2476
+ # N-ARY SELECTION: the container covering the MOST remaining
2477
+ # candidate forms wins (then tightest interior, then lowest id).
2478
+ # A consumed candidate never re-pairs — its evidence is already
2479
+ # composed at full joint strength.
2480
+ #
2481
+ # SELF-EVIDENCE GUARD: a container whose joined occurrence (left
2482
+ # through right including interior) is a literal substring of the
2483
+ # query is rejected — shards of a contiguous phrase pairing "around"
2484
+ # a gap chunk would merely rediscover the phrase they are shards of.
2485
+ #
2486
+ # EXPLAINING AWAY: when a junction binds, any individual vote whose
2487
+ # bytes the joint container LITERALLY CONTAINS yet whose roots are
2488
+ # FULLY DISJOINT from the junction's is SUPERSEDED — the exact joint
2489
+ # evidence explains those bytes away (grid aliasing). Partial
2490
+ # agreement (shared roots) corroborates and is kept.
2491
+ crossVotes ← []
2492
+ superseded ← ∅
2493
+ seedsOf(ri) ≔ junctionSeeds(ctx, query[regions[ri].start..regions[ri].end])
2494
+ # computed once, reused across all pairs of this candidate
2495
+ consumed ← ∅ # a candidate in one junction never re-pairs
2496
+ for each pair (a, b) of eligible candidates (non-overlapping,
2497
+ at least one strong voter, not both covered by one known region,
2498
+ ≤ k total probes, skipping consumed candidates):
2499
+ left ← query[a.start..a.end]
2500
+ right ← query[b.start..b.end]
2501
+ containers ← junctionContainersFrom(left, right, cap,
2502
+ seedsOf(a), seedsOf(b), undefined, unordered = true)
2503
+ if containers is empty:
2504
+ # Tier 2.5 fallback — same ascent over halo siblings
2505
+ containers ← junctionSynonyms(left, right, maxInterior,
2506
+ unordered = true)
2507
+ if containers not empty:
2508
+ best ← the container covering the MOST remaining candidates
2509
+ (cachedRead + indexOf per extra, never an extra walk);
2510
+ ties → shortest interior → lowest id
2511
+ if best's joined occurrence is a query substring: continue
2512
+ reach ← edgeAncestors(best.id)
2513
+ if reach is discriminative (not saturated, idf > 0):
2514
+ w ← mutual · ln(N / contextsReached) / |roots|
2515
+ crossVotes.push(vote for best.id's roots at weight w,
2516
+ span covering all composed candidates)
2517
+ consumed.add(a); consumed.add(b); consumed.add(all extras)
2518
+ # Explaining away: individual votes whose bytes the container
2519
+ # literally contains and whose roots are fully disjoint from
2520
+ # the junction's are superseded
2521
+ for each individual vote rv:
2522
+ if rv.roots shares any root with reach.roots: keep
2523
+ if containerBytes literally contains rv's query bytes:
2524
+ superseded.add(rv)
2525
+ break # a is consumed — move to next unconsumed candidate
2526
+
2527
+ # 3. POOL — votes (INDEPENDENT, minus any superseded by cross-region
2528
+ # evidence, + CROSS-REGION) accumulate through the
2529
+ # arithmetic semiring (§5.3): each region is an axiom; each
2530
+ # (region → anchor) contribution is a summing rule; a vote for a
2531
+ # TERMINAL answer node redistributes to the ≤ √N contexts that lead
2532
+ # to it (via prevFirst, capped at the store level). Independent
2533
+ # corroboration ADDS.
2534
+ votes ← pooledConclusions
2535
+
2536
+ # 4. COMMIT — rank anchors by vote. The dominant anchor always stands.
2537
+ # A FURTHER (non-overlapping) anchor becomes an independent point of
2538
+ # attention only if its vote clears BOTH:
2539
+ # · the natural break (the steepest ratio drop in the sorted votes —
2540
+ # a scale-free "where does signal end" test), and
2541
+ # · the consensus floor ln N + ½ (§8.6 — more than any single
2542
+ # region could contribute alone).
2543
+ return the surviving points, each with its query span and vote
2544
+ ```
2545
+
2546
+ ### 17.3 Why inverse document frequency
2547
+
2548
+ A region that climbs to few contexts is _specific_ — strong evidence about what
2549
+ the query concerns. A region that climbs to half the corpus (a common phrase)
2550
+ says almost nothing. Weighting a region's vote by ln(N/c) — the classical
2551
+ inverse-document-frequency form (Spärck Jones 1972) — expresses exactly this,
2552
+ with N the store's count of learned contexts, and dividing by the number of
2553
+ roots reached splits a region's voice among the candidates it cannot
2554
+ distinguish.
2555
+
2556
+ The geometric factor is not the raw resonance score but a **mutual-explanation
2557
+ weight** that reads both angle and magnitude (§2.6). Under the linear fold,
2558
+ cosine = shared / (‖region‖ · ‖hit‖), and each unnormalized norm is recoverable
2559
+ as a byte count (the region's own span length; the hit's, read from the store).
2560
+ Splitting the cosine by each side's own magnitude gives two fractions — how much
2561
+ of the _region_ the hit explains, and how much of the _hit_ the region pins down
2562
+ — each capped at 1 (an estimated cosine can imply more shared content than the
2563
+ smaller side even holds; left uncapped, that impossible surplus would let a
2564
+ small region echoing inside a large context, or the reverse, vote above its
2565
+ physical evidence). The product of the two capped fractions is the mutual weight
2566
+ that replaces a bare score: it is exactly the same quantity a plain squared
2567
+ cosine approximated implicitly (score² ≈ (shared/len_region)·(shared/len_hit)
2568
+ when the two sides are close in size), made explicit and safe at every scale.
2569
+ The margin gate that precedes this weighting (§17.2, the noise-floor check
2570
+ before the vote is computed) stays in raw cosine units deliberately: it tests
2571
+ the RaBitQ estimator's own noise floor, which lives in cosine space, not in
2572
+ byte-magnitude space.
2573
+
2574
+ ### 17.4 Saturation: expand-until-decided
2575
+
2576
+ The climb's work is bounded by **expand-until-decided**: the walk stops as soon
2577
+ as it knows whether the reach is saturated (the material is too common to
2578
+ discriminate) or a concrete vote (exact roots and contextsReached). The decision
2579
+ uses only LIMITed store reads:
2580
+
2581
+ - **Is this node edge-bearing?** `prevCount` — an indexed O(1) count — answers
2582
+ "does this node follow at least one learned context?" without materialising
2583
+ the (potentially corpus-sized) predecessor list.
2584
+ - **Is this node a hub?** `parentsFirst(id, √N+1)` — reading √N+1 parents proves
2585
+ "more than √N" exactly; the walk aborts at that step. Below √N, the read IS
2586
+ the full parent list, so the walk is exact.
2587
+ - **Distinct contexts past √N?** If the set of learned contexts visited crosses
2588
+ √N, the region is saturated — decided by a set-size check, never by counting
2589
+ every reachable context.
2590
+ - **Containment paging.** A window's containers are paged in chunks of √N via
2591
+ `containersSlice`. A distinctive window's containers (few, converging on one
2592
+ context) are walked in full — exact. A common window's corpus-sized container
2593
+ list is abandoned at the first page whose climbs push contexts past √N, with
2594
+ O(√N) page-work.
2595
+
2596
+ A region whose climb triggers any of the "decided: saturated" conditions
2597
+ abstains rather than vote noise. Saturation is also _recorded_: a leading
2598
+ saturated stretch of the query (a boilerplate preamble) is treated as
2599
+ scaffolding, and further points of attention are only admitted beyond it. The
2600
+ dual use — abstain from voting, and mark scaffolding — is what keeps long
2601
+ templated queries from diluting their own payload.
2602
+
2603
+ ### 17.5 What consumes the climb
2604
+
2605
+ The climb is computed once per response (memoised) and consumed by five
2606
+ mechanisms: recall's scaffolding tier (§21.3), extraction's search for a skill
2607
+ exemplar (§20), CAST's identification of woven structures (§18), confluence's
2608
+ constraint-stream detection (§18.5), and fusion's grounding of further topics
2609
+ (§23). The cross-region attention pass (§17.6) also runs inside the climb,
2610
+ consuming its region votes and the junction module.
2611
+
2612
+ ### 17.6 Cross-region attention: the binding problem
2613
+
2614
+ Additive pooling has a blind spot. Two regions whose independent climbs land on
2615
+ _different_ contexts leave their **joint context** — the learnt whole that
2616
+ contains both — with zero votes. "red" votes for `red square`, "circle" for
2617
+ `circle`; nothing votes for `red circle`, the only fact holding both. No amount
2618
+ of pooling can recover it: addition can only amplify what at least one region
2619
+ already said. This is the attention counterpart of the binding problem —
2620
+ independent evidence disaggregates what belongs together.
2621
+
2622
+ Cross-region attention recovers the missing joint contexts by the **same
2623
+ content-addressed junction ascent** the bridge uses (§19.5), extracted into the
2624
+ shared junction ascent. "Which learnt whole contains these forms?" is a bounded
2625
+ DAG ascent from the forms' canonical identities — not a resonance guess on a
2626
+ synthesised gist.
2627
+
2628
+ **Why not fold the two vectors?** Folding two region gists cannot even
2629
+ reconstruct the stored joint form. Sema builds a multi-word gist from byte-chunk
2630
+ folds, so isolated word vectors superpose into a different direction and
2631
+ resonate to `red circle` and `red square` indistinguishably. The junction ascent
2632
+ sidesteps this by matching **bytes** (content-addressed identities), not
2633
+ vectors.
2634
+
2635
+ **Candidate selection — corpus independence.** Candidates are ANY region that
2636
+ participated in the independent vote, not only recognised sites. At least one
2637
+ side of each pair must be a **strong** voter (individually discriminative — idf
2638
+
2639
+ > 0, non-saturated). A **known** (content-addressed) region that did _not_ vote
2640
+ > (saturated, or idf ≤ 0) may still serve as the **weak side** of a pair whose
2641
+ > other side voted: saturation is an abstention about where the region _climbs_;
2642
+ > the junction asks a different question — "which whole holds both?" — and the
2643
+ > container's own idf gate still guards the conclusion. This is **corpus
2644
+ > independence**: a word never trained standalone has no site, but its stored
2645
+ > chunks still vote and their bytes still compose — the ascent matches byte
2646
+ > containment, so a fragment pair evidences the same joint container the whole
2647
+ > word would.
2648
+
2649
+ Two disciplines prevent this openness from becoming noise:
2650
+
2651
+ - **Two non-voting regions never pair.** That is exactly the shared-prefix trap:
2652
+ ascending from two non-discriminative fragments can land on an
2653
+ incidentally-unique descendant container, manufacturing confidence the query
2654
+ gave no reason to have. At least one side must be individually discriminative.
2655
+ - **Only maximal spans compose.** A span wholly contained in another candidate
2656
+ is a fragment of that candidate's evidence, never independent of it.
2657
+ Contiguous shards of one word that are both covered by a single known region
2658
+ are skipped — the whole form already votes directly, and re-deriving it from
2659
+ its own pieces would only double-count.
2660
+
2661
+ Pairs are byte-sorted left-to-right; searches are capped at k probes total.
2662
+
2663
+ **Cost hoisting.** Junction seeds are computed once per candidate for the whole
2664
+ pairing loop (a candidate recurs in up to |cand|−1 pairs, and its seeds are a
2665
+ pure function of its bytes). All reads go through a shared per-response walk
2666
+ cache — one response issues many walks whose ancestries overlap heavily, so
2667
+ every identity read is a cache hit instead of a durable read or byte
2668
+ reconstruction.
2669
+
2670
+ **Order-free binding.** The junction ascent is order-free: a junction is
2671
+ evidence the two forms were learnt _together_; which one the query happened to
2672
+ mention first is a fact about the query, not about the learnt whole. The walk is
2673
+ identical (the seed ascent does not depend on order) — only the byte-containment
2674
+ test gains a second probe, so order-freedom costs two `indexOf` calls per
2675
+ visited node, never a second walk. The test also accepts overlapping or abutting
2676
+ occurrences: two grid-aligned fragments of one whole ("red " at 0 and " cir" at
2677
+ 3 in `red circle`) legitimately overlap inside it. A strict-super-form
2678
+ requirement applies (the container must be longer than either side alone):
2679
+ holding both must be more than restating either side.
2680
+
2681
+ **N-ary binding.** Binding is not intrinsically pairwise. When a pair's
2682
+ containers are found, each container is tested against every _remaining_
2683
+ unconsumed candidate form: the container covering the **most** of the query's
2684
+ composable forms (by total byte length of contained extras) wins, with ties
2685
+ going to shortest interior then lowest id. This means three cross-cutting
2686
+ attributes — where every pair is ambiguous across two contexts — resolve to
2687
+ their unique triple, at the cost of one cached byte read + `indexOf` per
2688
+ (container, extra), never an extra walk. A consumed candidate (one whose
2689
+ evidence is already composed in a junction) never re-pairs — its evidence is
2690
+ already at full joint strength, and re-pairing would vote the same container
2691
+ twice.
2692
+
2693
+ **Self-evidence guard.** A container whose joined occurrence (left through right
2694
+ including its interior) is literally a substring of the query is rejected.
2695
+ Binding is only evidence when the query mentions the forms _apart_. Without this
2696
+ guard, shards of a contiguous phrase ("s pa" + "d by" around "inte" in "was
2697
+ painted by") pair to rediscover the phrase they are shards of, then explain away
2698
+ its rivals — breaking multi-candidate tests where the query legitimately weaves
2699
+ several exemplars.
2700
+
2701
+ **Explaining away — the aliasing complement of corpus independence.** A chunk of
2702
+ the query can straddle the byte grid so that it exists verbatim in the _wrong_
2703
+ deposit (" cir" of "red then circle" is a stored chunk of `blue circle`, never
2704
+ of `red circle` — a pure alignment accident) and its independent climb then
2705
+ votes for a context the query gives no reason to believe. When a junction binds,
2706
+ any individual vote whose bytes the joint container **literally contains** AND
2707
+ whose climb roots are **fully disjoint** from the junction's is **superseded**:
2708
+ the exact joint evidence explains those bytes away, so their disagreeing vote is
2709
+ grid aliasing, not signal. A vote sharing even one root with the junction
2710
+ _corroborates_ it (partial agreement — a different slice of the same context)
2711
+ and is kept. Votes whose bytes the container does not hold at all (a genuine
2712
+ second topic) are untouched.
2713
+
2714
+ **Graded ladder.** The junction ascent follows the same evidence discipline as
2715
+ the bridge: Tier 1 (exact containers of the two forms themselves, order-free)
2716
+ first; only when it finds nothing does Tier 2.5 (synonym junctions — the same
2717
+ ascent over halo siblings of one side, sharing one expansion budget across all
2718
+ sibling walks) run as a fallback. Exact evidence outranks distributional
2719
+ approximation; a synonym junction can never override an exact one.
2720
+
2721
+ **Voting.** A joint container is **exact** evidence — it literally holds the
2722
+ composed forms — so it votes at full strength (score = 1, no estimator).
2723
+ Weighting uses the same mutual-explanation and IDF discipline as single-region
2724
+ votes, with the combined byte length of all composed candidates as the region
2725
+ size. The combined pool (independent, minus any superseded by cross-region
2726
+ evidence, + cross-region) means a joint context with no single-region support
2727
+ can still become a point of attention when its combined evidence clears the
2728
+ consensus floor (§8.6).
2729
+
2730
+ ---
2731
+
2732
+ ## 18. Grounding I — counterfactual transfer (CAST)
2733
+
2734
+ ### 18.1 When it applies
2735
+
2736
+ Some queries do not ask about one learned thing; they _weave together several_ —
2737
+ "what if X had Y's property?", "compare X and Y", a sentence that grafts one
2738
+ learned frame onto another's subject. CAST (Counterfactual trAnSfer) detects the
2739
+ weave by **graded alignment** — the same evidence ladder as `locate()`: literal
2740
+ W-gram runs first, then distributional role (halo-matched recognised sites
2741
+ filling gaps where the query has no literal coverage). It transfers structure
2742
+ between the woven parts. CAST is the byte-level, formalized descendant of
2743
+ case-based reasoning's _adaptation_ step (Kolodner 1992).
2744
+
2745
+ Preconditions (all structural, per invariant §14.2): the query is at least two
2746
+ perception windows long; the climb (§17) ranks at least two anchors; aligning
2747
+ the anchors' contexts against the query (below) leaves at least two anchors with
2748
+ _free_ aligned runs; the dominant anchor is a committed point of attention; and
2749
+ at least one aligned run falls **outside every recognised site** — the
2750
+ definition of "woven": material from a learned structure appearing where
2751
+ recognition's normal reading has no account of it. If any of this fails, CAST
2752
+ returns null — the grounding decider considers the remaining candidates.
2753
+
2754
+ ### 18.2 Graded alignment
2755
+
2756
+ Alignment follows the same graded-evidence ladder as `locate()` (§14.4):
2757
+ **literal first, then distributional role.** For each ranked anchor:
2758
+
2759
+ 1. **Literal** — `alignRuns`: W-gram seed-and-extend. Every W-gram of the query
2760
+ is indexed; each W-gram of the context that matches seeds a run, extended
2761
+ greedily in both directions; overlaps resolved longest-first. Weight = 1.0
2762
+ (exact match is full evidence).
2763
+
2764
+ 2. **Halo** — where the query has **no literal coverage** from this anchor,
2765
+ recognised sites with halos are matched to the exemplar context's own
2766
+ recognised sites via `bestHaloMate` (gate: `conceptThreshold`). A query site
2767
+ whose halo resonates with a context site above the bar produces a run with
2768
+ `cs` = the context site's structural byte position and `weight` = the cosine
2769
+ itself (measured evidence, not an invented constant).
2770
+
2771
+ The per-byte depth `depth[i]` is the **sum of weights** of aligned structures
2772
+ covering byte `i` — a byte covered by two literal runs has depth 2.0; a byte
2773
+ covered by one literal (1.0) and one 0.7-halo run has depth 1.7. The `cs`
2774
+ (position in the context) is a real byte offset regardless of run kind, so the
2775
+ substitution and redirection schemas work unchanged on conceptual alignment.
2776
+
2777
+ ### 18.3 Frame gate
2778
+
2779
+ Both components of the frame gate are **derived from the weave itself**, not
2780
+ tuned:
2781
+
2782
+ 1. **MIN_WEAVE** — the minimum number of aligned structures to form a weave: the
2783
+ same `2` that gates CAST entry (`points.length < 2`). Frame requires evidence
2784
+ **beyond** the minimum pair — a third structure agreeing — so the depth gate
2785
+ is `depth[i] > MIN_WEAVE`. One definition, two uses.
2786
+
2787
+ 2. **Half-dominance** — `dominates(part, whole)` (§8.7), the same test
2788
+ `collectRegions`, `liftAnswer`, and confluence's filler gate all use. A byte
2789
+ is frame when `dominates(depth[i], aligned)` — the structures covering it are
2790
+ a majority of all aligned structures. A run is usable when the framed bytes
2791
+ are NOT a majority: `¬dominates(framedCount, runLen)`.
2792
+
2793
+ In full:
2794
+
2795
+ ```
2796
+ frame(i) ⇔ depth[i] > MIN_WEAVE ∧ dominates(depth[i], aligned)
2797
+ usable(qs,qe) ⇔ ¬dominates(framedCount(qs, qe), qe − qs)
2798
+ ```
2799
+
2800
+ The frame gate is the canonical example of **weave-local commonality** (§8.10):
2801
+ `aligned` counts the structures aligned with _this query_, not the corpus. The
2802
+ choice of reference set is load-bearing — replacing the weave-local majority
2803
+ with corpus-global IDF misfires on reordered single-fact queries. See §8.10 for
2804
+ the general theory and the full table of which mechanism uses which measure.
2805
+
2806
+ ### 18.4 Three transfer schemas
2807
+
2808
+ Each schema is tried independently — every one that fires contributes its own
2809
+ candidate to the grounding decider, with its own `accounted` (the runs of
2810
+ exactly the points that schema used, not the whole weave's alignment) and
2811
+ `moves`. The decider's weight comparison replaces CAST's former internal
2812
+ priority order.
2813
+
2814
+ **Substitution** — _the query puts a new subject into a learned structure's
2815
+ seat._ Detected when one structure's run starts mid-context (its head — the seat
2816
+ — is displaced) and another structure, wholly present earlier in the query, fits
2817
+ that seat. The answer is built by projecting the displaced structure onto the
2818
+ new filler: filler + (learned connector if one exists, §19.5) + the displaced
2819
+ structure's tail, then following the projected structure's continuation if it
2820
+ adds something new.
2821
+
2822
+ **Redirection** — _the query names a substitute for the thing the dominant
2823
+ structure is about._ Detected when the latest-positioned structure's run starts
2824
+ at its context's very beginning (it is wholly, freshly named), and none of the
2825
+ dominant anchor's own continuations appears in the query (its usual answer is
2826
+ displaced). The answer is the substitute's own grounded fact: the named thing's
2827
+ knowledge replaces the displaced structure's.
2828
+
2829
+ **Comparison** — _the query juxtaposes two entities of the same kind._ Candidate
2830
+ analogs are the non-dominant points (and their continuation targets, which often
2831
+ name the entity a long exemplar sentence is about). The gate is **analogy
2832
+ strength** (§4.2), a graded three-tier test: (1) the direct halo cosine between
2833
+ dominant and candidate, thresholded at the significance bar; (2) failing that,
2834
+ the strongest _mutual_ halo sibling — two things are analogous if they keep
2835
+ company with the same third things; (3) failing that, **shared-frame strength**
2836
+ — a structural tier that measures what fraction of the shorter side's bytes are
2837
+ covered by a learnt W-window that also occurs in the longer side (two sentences
2838
+ sharing " is " measure as analogs even when their halos never overlapped). If no
2839
+ candidate passes any tier, a deterministic structural fallback picks the
2840
+ best-evidenced genuine hub among the candidates. The answer voices each analog
2841
+ by the context that establishes its role, joined by a learned connector when one
2842
+ exists.
2843
+
2844
+ Whatever CAST produces, the anchors it consumed are marked as such, so the
2845
+ reasoning stage (§22) does not re-walk the same facts. Each schema reports its
2846
+ own `accounted` — the runs of exactly the points that schema transferred
2847
+ between, not the whole weave's alignment. `castFloor` pre-computes the consensus
2848
+ climb (memoised per response, zero extra cost) and returns `null` when the climb
2849
+ cannot possibly support a weave; when non-null, its ranked anchors are reused by
2850
+ the alignment loop, eliminating a redundant second climb call. Every candidate
2851
+ also carries `unexplained` — a human-readable label for the query bytes CAST
2852
+ left unexplained (§14.1).
2853
+
2854
+ ---
2855
+
2856
+ ### 18.5 Confluence Join — the meet of independent constraints
2857
+
2858
+ #### When it applies
2859
+
2860
+ Some queries carry two or more _independent constraints_ — "Which material is
2861
+ translucent and featherlight?" — where each constraint reaches its own set of
2862
+ exemplars, and the entity satisfying both lives exactly where those sets
2863
+ **intersect**. No single-fact mechanism can answer this: any one evidence path
2864
+ grounds only one constraint, and fusion concatenates one fact per constraint
2865
+ from _different_ entities (wrong answer). Confluence detects independent
2866
+ constraint streams and intersects them by content-addressed identity.
2867
+
2868
+ Preconditions (all structural): the query is at least two perception windows
2869
+ long; the consensus climb (§17) ranks at least two anchors; at least two of them
2870
+ hold disjoint discriminative query windows (they are _independent_ constraints,
2871
+ each answering a different part of what was asked); and the intersection of
2872
+ those anchors' content — the **meet** — contains at least one discriminative
2873
+ span (content reaching a corpus minority of contexts, per the half-dominance
2874
+ convention).
2875
+
2876
+ #### The meet is native, not a byte scan
2877
+
2878
+ The store is content-addressed: any content two deposits share IS the same node
2879
+ id, interned once at write time (hash-consing). So "what do these two facts have
2880
+ in common?" is a **set intersection of identities** the write side already
2881
+ computed, asked through the canonical window read — `windowIds` (§9), the same
2882
+ write/read contract recognition runs on. Three identity/structure tests make the
2883
+ whole mechanism:
2884
+
2885
+ 1. **Constraint streams.** The consensus climb's ranked anchors, each bound to
2886
+ the query spans whose _discriminative_ windows it holds by identity (a
2887
+ resonance-voted anchor holding none of the query's discriminative content is
2888
+ no constraint at all). Two streams are independent when the content they bind
2889
+ is disjoint.
2890
+
2891
+ 2. **The meet.** Window ids present in _both_ anchors and _absent from the
2892
+ query_: shared-with-query windows are the constraint being re-named (or its
2893
+ scaffolding), so subtracting the query's own window ids leaves exactly the
2894
+ content the question asks _for_ — the open seat.
2895
+
2896
+ 3. **Filler/scaffolding separation.** The same structural IDF the climb derives
2897
+ (`reachOf`, §9): shared content reaching a corpus minority of contexts is an
2898
+ entity (a filler); content reaching a majority is frame scaffolding. No
2899
+ statistics, no learning — the same global-quantity-from-capped-local-probes
2900
+ reading that makes the climb's IDF work, pointed at a new question.
2901
+
2902
+ The meet thus names content that _byte-literally exists_ in two independently
2903
+ learnt exemplars. An empty intersection yields null and the ordinary pipeline
2904
+ decides — confluence cannot fabricate.
2905
+
2906
+ #### Evidence for the grounding decider
2907
+
2908
+ Confluence reports the query spans whose constraint content the two streams hold
2909
+ by identity (`accounted`), its acts (`moves`: two constraint matches + one meet
2910
+ = 3×STEP), and `unexplained` — a human-readable label for query bytes outside
2911
+ those spans (§14.1).
2912
+
2913
+ ---
2914
+
2915
+ ## 19. Grounding II — cover: the graph search
2916
+
2917
+ The cover is the heart of the system: **one lightest-derivation search (§5) that
2918
+ composes an answer out of the query's own decomposition**. All of "retrieval",
2919
+ "rewriting", "multi-part answers", and "synonym use" are individual _rules_ of
2920
+ this one search.
2921
+
2922
+ ### 19.1 Items
2923
+
2924
+ Three kinds of item populate the deduction system:
2925
+
2926
+ - **Cover(p)** — "the query is covered from position 0 up to p". The axiom is
2927
+ Cover(0); the **goal is Cover(len(query))**.
2928
+ - **Form(i, j, node)** — "the node `node` names the query span [i, j)". A form
2929
+ is a _foothold in the graph_: rules walk out of it.
2930
+ - **Out(i, j, bytes)** — "the span [i, j) will be answered by these bytes". An
2931
+ out may be _recognised_ (a grounded completion — essentially free to cover) or
2932
+ a _literal_ (unrecognised query bytes carried through at PASS cost).
2933
+
2934
+ ### 19.2 Axioms
2935
+
2936
+ - Cover(0), at cost 0.
2937
+ - One Out per perceived leaf of the query (its own bytes, literal) — the
2938
+ covering fabric.
2939
+ - One Form per recognised site (§15) — the graph entry points.
2940
+ - One recognised Out per computed span (§16), at STEP cost.
2941
+
2942
+ ### 19.3 Rules
2943
+
2944
+ Stated abstractly (costs from the ladder, §8.9):
2945
+
2946
+ ```
2947
+ BRIDGE Cover(i) ∧ Out(i, j) → Cover(j)
2948
+ cost: ε if the out is recognised; PASS·(j−i) if literal.
2949
+ The frontier advances; literal connectives (spaces, commas —
2950
+ anything unrecognised) are CARRIED, so the asker's own linking
2951
+ material survives between rewritten parts.
2952
+
2953
+ FOLLOW-EDGE Form(i, j, n) → Form(i, j, n′) cost STEP
2954
+ where n ──▶ n′ is a learned continuation edge. With several
2955
+ continuations, the disambiguator (§25) picks n′.
2956
+
2957
+ CONCEPT-HOP Form(i, j, n), n edge-less → Form(i, j, s′) cost CONCEPT
2958
+ where s is a halo sibling of n above the concept threshold and
2959
+ s ──▶ s′ — answering through a synonym, one order dearer than
2960
+ a literal edge. (Siblings are pre-resolved before the search,
2961
+ since index queries are asynchronous.)
2962
+
2963
+ GROUND Form(i, j, n), n terminal, reached via edges
2964
+ → Out(i, j, bytes(n)) cost 0
2965
+ The chain's endpoint becomes answer bytes for the span. Before
2966
+ emitting, RECOMPLETION (§19.6) may resolve deeper.
2967
+
2968
+ SPLIT Out literal, containing a split position k (§15.2)
2969
+ → the two halves cost 0
2970
+ The query's own chunking is not sacred; a form boundary the
2971
+ store knows can cut a leaf.
2972
+
2973
+ FUSE Out(i, j) ∧ Out(j, k) adjacent → Out(i, k) cost 0
2974
+ The concatenation may name a known node (by content-addressed
2975
+ lookup: as a short leaf; as the branch of the two sides'
2976
+ nodes; or by canonical re-perception when a side is a
2977
+ completed rewrite). If it does, also:
2978
+
2979
+ RECOMPOSE the fused pair → Form(i, k, node) cost 0
2980
+ Two already-rewritten parts fusing into a node that itself
2981
+ continues is a RECOMPOSITION: its onward FOLLOW-EDGE is free,
2982
+ so the consolidated whole strictly beats leaving the parts
2983
+ split. A guard requires the fused node to be halo-bearing —
2984
+ i.e. learned as a meaningful unit, not an accidental interior
2985
+ chunk of some one-shot phrase.
2986
+
2987
+ SPLICE Out(recognised L) ∧ Out(recognised R), a learned connector
2988
+ exists between L's and R's answers → Out(L+connector+R) cost 0
2989
+ (§19.5. Fires only when the gap between them is empty or
2990
+ wholly recognised — never across the asker's own literal
2991
+ separator, which BRIDGE carries instead.)
2992
+ ```
2993
+
2994
+ The A\* heuristic is ε per uncovered byte beyond an item's right edge —
2995
+ admissible because ε is the minimum per-position cost, and what keeps the search
2996
+ output-sensitive (§5.2, §8.9).
2997
+
2998
+ ### 19.4 What the cost ladder buys, concretely
2999
+
3000
+ - Coverage dominates: the search _must_ account for every byte, and prefers
3001
+ recognising to carrying.
3002
+ - A literal continuation beats a synonym's (STEP < CONCEPT); either beats
3003
+ leaving a span unrewritten (≪ PASS).
3004
+ - Free fusion/recomposition means the search always finds the _deepest
3005
+ consolidated reading_: if "D E" recomposes into a learned "DE" that continues
3006
+ to F, the answer is F, not "D′ E′".
3007
+ - Ties resolve by the fixed conventions of §25 — deterministically.
3008
+
3009
+ ### 19.5 Connectors: learned joins (the bridge)
3010
+
3011
+ When an answer has several parts, what belongs _between_ them? Sema asks the
3012
+ store through a **graded junction ladder** — exact evidence before approximate,
3013
+ the same discipline as `locate` (§14.4). The junction search is extracted into
3014
+ one shared procedure so that both the bridge (a connector between answer pieces)
3015
+ and cross-region attention (§17.6, the joint context of query regions) ascend by
3016
+ the same bounded, cached walk:
3017
+
3018
+ 1. **Junction containers, by content-addressed identity.** Hash-consing means
3019
+ "which learned wholes ran L and R together?" is a structural question, not a
3020
+ similarity guess: any deposit containing L's bytes shares L's node (or L's
3021
+ canonical-window identities), so ascending the DAG's parent and containment
3022
+ links from the two sides reaches every containing whole _exactly_, under the
3023
+ one √N fan-out discipline. A container whose bytes literally hold L and then
3024
+ R yields the bytes between them as the **connector** ("and", ", ", " is the
3025
+ opposite of " — whatever the corpus actually joins such things with).
3026
+ 2. **Edge junctions.** A continuation edge _is_ junction information: a learned
3027
+ continuation of L that contains R carries the glue as its prefix; a learned
3028
+ context of R that contains L carries it as its suffix. An _empty_ interior
3029
+ found this way is a confirmed adjacency — returned as such, never confused
3030
+ with a miss.
3031
+ 3. **Synonym junctions.** The content-addressed junction search applied to halo
3032
+ siblings of the two sides: when L or R has no direct junction, one of its
3033
+ distributional siblings may. Container evidence stays exact (same DAG ascent
3034
+ as tier 1, with window-id-enhanced seeds); the relaxation is only in which
3035
+ form occupies one side.
3036
+ 4. **Resonance** (the last resort): the gist of the bare concatenation is
3037
+ resonated into the content index and the nearest containment-passing form
3038
+ supplies the connector — approximate, but it still reaches containers whose
3039
+ identity links are absent or saturated.
3040
+
3041
+ When several junctions qualify, the **response guide** (the query's gist — the
3042
+ same disambiguator every projection uses) picks by resonance; ties prefer the
3043
+ shortest interior (a junction should not insert unnecessary glue), then the
3044
+ lowest node id (deterministic — a property of the corpus, not the seed). No
3045
+ learned evidence at any tier ⇒ no connector invented.
3046
+
3047
+ Connectors are pre-resolved for the query's adjacent site pairs (and for
3048
+ first-to-later pairs of longer groups), then handed to the search, where SPLICE
3049
+ applies them inside the derivation — so multi-part answers are assembled as one
3050
+ globally-coherent whole, not stitched afterwards.
3051
+
3052
+ ### 19.6 Recompletion: answers that resolve deeper
3053
+
3054
+ A followed edge may land on a _composite_ node that leads nowhere as a whole
3055
+ (say, "p1 p2") yet whose parts each continue. Before emitting such a node as
3056
+ terminal, the search **re-covers the node's own bytes** — the very same solve,
3057
+ recursively: recognition, edges, fusion, recomposition. If that inner cover
3058
+ produces something new that itself names a learned node, the deeper completion
3059
+ becomes the answer for the span. Recursion needs no depth cap: a node already
3060
+ being recompleted is not re-entered (cycle guard), node identities are finite,
3061
+ and finished recompletions are memoised — so chains run exactly as deep as the
3062
+ graph licenses, and stop.
3063
+
3064
+ ### 19.7 Reading out the answer
3065
+
3066
+ The finished derivation's chosen spans, left to right, are the cover.
3067
+ **Lifting** extracts the answer from the asker's framing: if one span is
3068
+ recognised, its bytes are the answer (unless it dominates — covers more than
3069
+ half of — the query, in which case the whole cover is kept); with several,
3070
+ everything from the first to the last recognised span (inclusive of carried
3071
+ connectives between them) is kept, and the unrecognised framing outside is
3072
+ dropped.
3073
+
3074
+ ---
3075
+
3076
+ ## 20. Grounding III — extraction by skill
3077
+
3078
+ ### 20.1 Skills are facts with a shape
3079
+
3080
+ Nothing in ingestion marks anything as a "template". But some learned facts
3081
+ _have a shape_: the answer is literally a span of the context (or a few pieces
3082
+ of it), and the context is the frame around it — e.g. ("The Mona Lisa was
3083
+ painted by Leonardo da Vinci.", "Leonardo da Vinci"). Such a fact is a
3084
+ **span-in-context exemplar**, and it can be _applied_ to fresh text: find where
3085
+ the exemplar's frame appears in the query, and read out the analogous span. This
3086
+ is instance-based learning in its purest form (§1.1): the stored episode itself,
3087
+ unmodified, functions as the rule.
3088
+
3089
+ Span-shapedness is read at two deliberately different strengths, and they are
3090
+ not interchangeable:
3091
+
3092
+ - **OPEN reading** (exemplar acceptance): the answer is a sparse subsequence of
3093
+ the context — bytes in order, arbitrary gaps. Permissive, so a multi-piece
3094
+ answer stitched from several context runs validates. Used to _accept_ an
3095
+ exemplar candidate.
3096
+ - **STRONG reading** (answer decomposition): a greedy longest-run decomposition
3097
+ into contiguous pieces. Greedy-longest is strictly stronger than subsequence
3098
+ (a long late match can consume context an earlier shorter choice needed), so
3099
+ an accepted exemplar can still fail to decompose — the mechanism then falls
3100
+ through to recall. Used to _read pieces out_ of the query.
3101
+
3102
+ ### 20.2 The algorithm
3103
+
3104
+ ```
3105
+ extractBySkill(query):
3106
+ ranked ← climbAttention(query).ranked # §17 — every voted anchor
3107
+ exemplar ← the first ranked anchor that is span-shaped:
3108
+ context ← the anchor's bytes (or, for a terminal answer node, the
3109
+ longest span-shaped context among ≤ √N of its
3110
+ predecessors; query-gist resonance breaks length ties)
3111
+ answer ← its continuation
3112
+ span-shaped ⇔ answer is a contiguous span of context, a recognised
3113
+ subtree of it, or an ordered sparse subsequence
3114
+ (a multi-piece answer)
3115
+ if none: return ∅ # no skill applies; decider moves on
3116
+
3117
+ runs ← decompose the exemplar answer into its pieces within the context
3118
+ for each piece:
3119
+ framePre ← up to W bytes of context before the piece
3120
+ framePost ← up to W bytes after (or the next piece's pre-frame)
3121
+ locate framePre / framePost in the QUERY — by exact bytes first,
3122
+ else by halo resonance (the frame's distributional role
3123
+ matches a query form), else by gist resonance against the
3124
+ query's segments # graded matching, §4
3125
+ the query bytes between the located frames are this piece's analog
3126
+ answer ← the concatenated analogs
3127
+ ```
3128
+
3129
+ The demo in the README is this mechanism: three "X was painted by Y" examples
3130
+ make ("…was painted by …", painter) a span-shaped exemplar; the unseen
3131
+ sentence's frames locate; the analogous span — a painter never taught as an
3132
+ answer — is read out of the query itself.
3133
+
3134
+ Extraction reports its **elementary evidence** for the grounding decider: the
3135
+ located frame occurrences in the query (the matched evidence), plus the read
3136
+ span itself when it is **bounded on both sides** — both the pre-border and
3137
+ post-border frames were located, so the read is structurally delimited and its
3138
+ content is a consequence of that match. An open-ended read (the answer reaches
3139
+ the context's end, with no right border located) is NOT accounted — it is the
3140
+ variable being read without a closing delimiter, priced by exclusion through the
3141
+ `unaccounted` bytes the decider charges at PASS each. The act is costed at one
3142
+ CONCEPT (the skill is an analogy) plus one STEP per accounted span.
3143
+
3144
+ This is the mechanism's defining asymmetry. Extraction reads an unknown by
3145
+ structural analogy to a known exemplar: the frames prove "this query has the
3146
+ same shape as this skill" (matched evidence). When both borders are located, the
3147
+ span between them is structurally explained (we know both _where_ and _that_ it
3148
+ should be read); an open-ended read is structurally explained only on one side
3149
+ (we know _where_ to start but not _where_ to stop) — it is the content-novel
3150
+ variable being read, priced the same way the cover's unrecognised literals are:
3151
+ at PASS each. Counting a bounded read as unexplained would let a single-frame
3152
+ extraction tie-weight with mechanisms that genuinely explain more of the query.
3153
+
3154
+ The same discipline applies to recall's consensus tier (§21): the climb's vote
3155
+ explains exactly the query region whose evidence carried the winning point of
3156
+ attention (`Attention.start`–`end`), not the whole query. A consensus vote for
3157
+ "ice" among scaffolding does not explain the word "steel".
3158
+
3159
+ The decider thus prices each mechanism's _actual_ explanatory work — never the
3160
+ bytes it reads out: counting an open-ended read as accounted would let
3161
+ extraction outweigh mechanisms (e.g. CAST on a reworded single-fact question)
3162
+ whose aligned runs are real structural evidence while the extraction's open seat
3163
+ is merely copied. `extractionFloor` pre-computes the consensus climb (zero extra
3164
+ cost) and returns `null` when no anchor exists; when non-null, its ranked
3165
+ anchors are reused by `extractBySkill`, eliminating a redundant climb call.
3166
+ Every extraction candidate carries `unexplained` — a human-readable label for
3167
+ query bytes its frames did not cover (§14.1).
3168
+
3169
+ ---
3170
+
3171
+ ## 21. Grounding IV — recall by resonance
3172
+
3173
+ Recall handles queries whose own decomposition composed nothing: resonate the
3174
+ _whole query's gist_ and ground the nearest learned form. It is the most
3175
+ fallback-like mechanism — its weight carries the full PASS·|query| for most
3176
+ tiers, so it can only win as the sole grounding (the honest price of an
3177
+ ungrounded answer) — but it participates in the same decider as every other
3178
+ mechanism. Four tiers, each gated on structural evidence, orderly degrading from
3179
+ exactness to an honest echo.
3180
+
3181
+ #### The asymmetry of forward and reverse
3182
+
3183
+ The deduction system (§5, §19) is a **forward** engine: its rules (FOLLOW-EDGE,
3184
+ CONCEPT-HOP, FUSE, SPLICE) all move from premises toward conclusions in the
3185
+ direction of the learned edges. There is no backward rule — no inference step
3186
+ that consumes a conclusion to produce a premise. This is not an omission; it is
3187
+ the formalism: a derivation is a directed hyperpath from axioms to a goal, and
3188
+ the cost ladder prices each forward step. A reading against the edge direction —
3189
+ `reverseContext`, which asks "what establishes this?" rather than "what does
3190
+ this lead to?" — produces bytes but no derivation. It explains nothing about the
3191
+ query in the forward direction the search operates in.
3192
+
3193
+ The grounding decider expresses this exactly: reverse readings get
3194
+ `accounted = []` (nothing matched against learnt structure in the forward
3195
+ direction), so their weight is the full PASS·|query| plus a STEP — the most
3196
+ expensive grounding, available when nothing composes forward, impossible to
3197
+ prefer when anything does. The decider _derives_ this from the evidence the
3198
+ formalism itself declares: a backward step carries no explanatory weight.
3199
+
3200
+ Every tier grounds through the shared projections of §14.4: `reverseContext` for
3201
+ the reverse cases, `project` (forward-else-reverse) for the rest — recall owns
3202
+ no grounding machinery of its own.
3203
+
3204
+ **Tier 0 — exact self-match (content-addressed).** If the query _resolves_ — it
3205
+ is literally a stored node — answer with the context that predicts it (the
3206
+ reverse projection; among several predecessors, the query gist picks by
3207
+ resonance). This tier never consults the ANN index: identity is exactly
3208
+ decidable, and an estimated score must never stand in for it (§6.2). The
3209
+ grounding is a pure reverse reading: `accounted = []`, `moves = STEP`. The
3210
+ decider prices this honestly: a reverse reading is the designated last resort,
3211
+ never a peer of forward evidence.
3212
+
3213
+ **Tier 1 — clean resonance.** If the top hit's estimated score clears the merge
3214
+ threshold (§8.1), the query essentially _is_ a learned form: `project` tries
3215
+ forward first (to the continuation fixpoint), then reverse (to the establishing
3216
+ context). A forward grounding accounts for the _whole_ query (identity-grade
3217
+ match, `accounted = [[0, query.length]]`); a reverse reading accounts for
3218
+ nothing. Cost: one STEP either way.
3219
+
3220
+ **Tier 2 — scaffolding-dominated.** If the top score clears only the
3221
+ significance bar (§8.3) — real but diluted, typically because shared boilerplate
3222
+ dominates the gist — run the consensus climb (§17) and ground its dominant
3223
+ anchor, provided the anchor's pooled vote clears the consensus floor (§8.6).
3224
+ Accounts for exactly the query spans whose evidence carried the winning point of
3225
+ attention — not the whole query. Cost: one CONCEPT (the climb is a halo-mediated
3226
+ act).
3227
+
3228
+ **Tier 3 — last resort.** This tier is gated on the **fraction of the query the
3229
+ grounding explains**, not the raw cosine (§2.6). Root gists are unit vectors,
3230
+ but under the linear fold cosine = shared / √(len_query · len_grounding), so the
3231
+ raw cosine of a query fully contained in a much longer grounded answer is
3232
+ √(len_query / len_grounding) — a number that shrinks the longer the honestly-
3233
+ containing answer is, and would refuse a perfectly good containment while
3234
+ letting a same-length answer that shares only scaffolding pass. Converting the
3235
+ cosine into `cos · √(len_grounding / len_query)` — a query-relative fraction —
3236
+ measures exactly what the reach bar is supposed to mean: how much of THE QUERY
3237
+ the store accounts for, regardless of how much longer the matched form is.
3238
+
3239
+ Walk the hits nearest-first and ground the first one whose query-relative
3240
+ fraction clears the reach threshold (§8.2). Failing that: if even the nearest
3241
+ hit's fraction is _below_ the reach threshold, **return nothing** — the store
3242
+ holds nothing related. Otherwise return the nearest form's own bytes verbatim,
3243
+ explicitly flagged as an **echo**: within reach, but not a grounded fact — it
3244
+ accounts for nothing and carries no move cost, so it can only win as the sole
3245
+ grounding (the honest price of an ungrounded answer). The flag travels in the
3246
+ response's provenance (§26) so a confident-looking parrot is always
3247
+ distinguishable from an answer. Each tier also carries `unexplained` — a
3248
+ human-readable label for the query bytes its evidence left on the table (§14.1)
3249
+ — appearing in the rationale trace alongside `accounted` and `moves`.
3250
+
3251
+ ---
3252
+
3253
+ ## 22. Reasoning: the multi-hop chain
3254
+
3255
+ Grounding produces a _first_ answer; reasoning asks what that answer _implies_,
3256
+ iteratively:
3257
+
3258
+ ```
3259
+ reason(query, answer, consumed₀):
3260
+ if the query itself is some context's learned continuation:
3261
+ return answer # echo guard: hopping forward from an
3262
+ # answer-shaped query would chain through
3263
+ # the very fact that produced it
3264
+ consumed ← consumed₀ # everything grounding already spoke for,
3265
+ # expanded through halo siblings (synonyms
3266
+ # of consumed nodes are consumed too)
3267
+ repeat up to K times:
3268
+ 1. FORWARD ABSORB: if the current answer resolves to a node with an
3269
+ unconsumed continuation, follow it (guided, §25) to its fixpoint
3270
+ and absorb — the answer is itself a learned fact; state what it
3271
+ leads to.
3272
+ 2. else PIVOT: find the longest unconsumed learned CONTEXT whose
3273
+ bytes the answer literally CONTAINS (candidates proposed by
3274
+ resonating the answer's subtree gists and by exact recognition;
3275
+ confirmed only by byte containment — resonance alone never
3276
+ hops). Follow the pivot's continuation. If none: stop.
3277
+ mark the followed fact and its neighbours consumed
3278
+ return the fixpoint
3279
+ ```
3280
+
3281
+ The consumed-set discipline is what makes the chain _progress_: each hop must
3282
+ bring in a fact not already spoken for, so the walk cannot circle, and the same
3283
+ content is never restated. This is how "The Weeping Woman was painted by Pablo
3284
+ Picasso" continues onward to what the store knows _about Picasso_: the extracted
3285
+ answer contains the learned context "Pablo Picasso", whose continuation is the
3286
+ Cubism fact.
3287
+
3288
+ ---
3289
+
3290
+ ## 23. Fusion: multi-topic answers
3291
+
3292
+ If the query carries several independent points of attention (§17) — and the
3293
+ grounded answer was _not_ drawn from the query's own text (extraction already
3294
+ spans all its pieces; fusing would add noise) — each further committed point
3295
+ grounds its own answer, and the pieces are joined in query order, with a learned
3296
+ connector (§19.5) between each adjacent pair where one exists. A missing
3297
+ connector joins the pieces bare and records the degradation in the trace. Thus
3298
+ "ice fire" (two topics) becomes "cold hot" — or "cold and hot", if the corpus
3299
+ ever joined such answers with "and".
3300
+
3301
+ ---
3302
+
3303
+ ## 24. Articulation: answering in the asker's words
3304
+
3305
+ The final pass adjusts _voice_, not content. The asker's query is decomposed
3306
+ into its recognised, halo-bearing forms — the asker's _vocabulary_. Each
3307
+ recognised form of the answer whose halo resonates (above the concept threshold,
3308
+ §8.5) with one of the asker's forms is a concept the two express in different
3309
+ words; the answer's wording is substituted with the asker's. The substitutions
3310
+ are spliced by the same cover search (§19), run over the _answer_ with
3311
+ substitute emissions as the only rules — so voicing is a derivation too, subject
3312
+ to the same composition discipline (and traced like everything else).
3313
+ Single-byte forms are excluded on principle: a one-byte form's halo keeps
3314
+ company with everything, so it licenses spurious substitutions. If the revoiced
3315
+ cover does not compose, the answer stands unchanged.
3316
+
3317
+ ---
3318
+
3319
+ ## 25. Disambiguation: choosing among alternatives
3320
+
3321
+ Learned knowledge is plural: a context may have many continuations; a
3322
+ continuation may follow many contexts. Sema's choices among them follow two
3323
+ fixed regimes — and which regime applies is a matter of _direction_:
3324
+
3325
+ - **Forward (which continuation?): structural evidence.** Candidates are often
3326
+ short spans whose gists are dominated by accidental byte correlations, so
3327
+ geometry is _not_ consulted. The winner is the candidate predicted by the most
3328
+ **distinct contexts** (diversity of independent evidence), tie-broken by
3329
+ **halo mass** (sheer episodic repetition), then by insertion order
3330
+ (first-learned). Candidates are capped at the hub bound √N.
3331
+ - **Reverse (which context?): geometric evidence.** Candidate contexts are whole
3332
+ learned experiences — long enough that their gists are semantically meaningful
3333
+ — so the winner is the context whose gist best resonates with the query's gist
3334
+ (again capped at √N). Without a query in flight, the most-corroborated
3335
+ (highest halo mass) context wins.
3336
+
3337
+ Two response-wide conventions (invariant §14.2): every mechanism of one response
3338
+ consults the same query-gist guide and shares one memo of picks, so an ambiguous
3339
+ fact reads the same everywhere in the answer; and all tie-breaks bottom out in
3340
+ fixed, corpus-determined orderings — never in anything nondeterministic.
3341
+
3342
+ Both regimes, and the √N cap they share, are defined exactly once: `corpusN` →
3343
+ `hubBound` (≥ 2, the count of contexts the cap reads from) is the numerical
3344
+ bound passed to the store's LIMITed reads (`nextFirst(id, hubBound)`,
3345
+ `prevFirst(id, hubBound)`); `hubCap` is the list-side reading of the same
3346
+ convention. The forward regime lives inside `chooseNext` (called by the shared
3347
+ `guidedFirst`, which merges guided-pick with the first-inserted fallback into
3348
+ one LIMITed read), and the reverse regime inside `chooseAmong` — both consumed
3349
+ by every mechanism only through the shared projections (`follow`,
3350
+ `reverseContext`, `project`). The store's existence probes (`hasNext`,
3351
+ `hasHalo`) answer "does this lead anywhere?" without materialising edge lists.
3352
+ No mechanism can drift onto a private disambiguation rule, and no per-query read
3353
+ grows with the corpus.
3354
+
3355
+ ---
3356
+
3357
+ ## 26. Auditability: provenance and the rationale
3358
+
3359
+ Sema's answers carry their epistemology with them, at two grains:
3360
+
3361
+ **Provenance** — every response is tagged with the mechanism that grounded it:
3362
+ `cast`, `join`, `cover`, `extract`, `recall`, or `recall-echo`. The `join` tag
3363
+ means the answer was produced by intersecting independent constraint streams
3364
+ (confluence, §18.5) — a conjunctive query where no single fact holds the answer.
3365
+ The `recall-echo` tag is the honesty flag of §21's tier 3: the bytes are a
3366
+ stored form returned verbatim for being _near_, not a derived fact. A consumer
3367
+ can gate on this tag mechanically.
3368
+
3369
+ **Rationale** — on request, the response includes the complete replayable trace:
3370
+ every mechanism's entries and exits, and — at the finest grain — every rule
3371
+ application of the lightest derivation itself (each FOLLOW-EDGE, CONCEPT-HOP,
3372
+ FUSE, RECOMPOSE, SPLICE, BRIDGE, and pooled vote, with its premises, conclusion,
3373
+ local cost, and data-flow edges to the steps that produced its premises). This
3374
+ is a direct serialization of the proof tree of §5.4: the answer _is_ this
3375
+ derivation, so the trace is not instrumentation bolted onto an opaque process —
3376
+ it is the process.
3377
+
3378
+ Together with determinism (same store + same query ⇒ same answer, always), this
3379
+ yields the property regulated and safety-critical settings actually require: any
3380
+ output can be reproduced exactly and attributed to enumerable stored facts and
3381
+ rules.
3382
+
3383
+ ---
3384
+
3385
+ ---
3386
+
3387
+ # Part V — The whole algorithm in pseudocode
3388
+
3389
+ ## 27. End-to-end pseudocode
3390
+
3391
+ This section restates the entire system as one connected program, at a level of
3392
+ detail sufficient to reimplement it. Notation: `≔` binds; `∅` is empty; D, W, N
3393
+ as in §8; thresholds by their §8 names. Store operations (`resolve`, `next`,
3394
+ `prev`, `parents`, `halo`, `resonate`, …) are as defined in Parts I–III.
3395
+
3396
+ ### 27.1 Shared primitives
3397
+
3398
+ ```
3399
+ # ── geometry (VSA, §2) ────────────────────────────────────────────────
3400
+ alphabet[b] ≔ deterministic unit vector for byte b (recursive
3401
+ refinement 16→64→256, seeded)
3402
+ π₀ … π_{S−1} ≔ fixed independent random permutations (the keyring)
3403
+ fold(v₀ … vₖ) ≔ Σᵢ πᵢ·vᵢ # NOT normalized — only
3404
+ # a fold's finished ROOT is
3405
+ # (§2.6): interior gists
3406
+ # keep a byte-proportional
3407
+ # magnitude, ‖·‖ ≈ √len
3408
+ companySignature(id) ≔ seeded random unit vector from `id`
3409
+ # halo pours use identity-based signatures, not gists
3410
+ resonance(a, b) ≔ cosine(a, b)
3411
+ contentLen(id) ≔ the byte length recoverable from a stored gist's own
3412
+ (unnormalized) magnitude, or the store's exact record
3413
+ of it — the linear fold's ‖·‖ ≈ √len read backward
3414
+ fracOfQuery(cos, otherLen, qLen) ≔ min(1, cos · √(otherLen / max(1, qLen)))
3415
+ # converts a raw cosine into a query-relative fraction
3416
+ # of shared content (§2.6, §21)
3417
+
3418
+ # ── perception (§10) ──────────────────────────────────────────────────
3419
+ perceive(bytes):
3420
+ leaves ≔ [ node(bytes = bᵢ, gist = alphabet[bᵢ]) ]
3421
+ p ≔ longest proper prefix already known as a stored leaf-sequence
3422
+ level ≔ leaves
3423
+ while |level| > 1:
3424
+ partition level at the item containing offset p (if p > 0)
3425
+ within each partition, fold complete groups of W;
3426
+ carry incomplete trailing items up unchanged
3427
+ if nothing folded (stall): force-fold in groups of W
3428
+ level ≔ the folded row
3429
+ normalize(level[0].gist) # ONLY the finished root — every
3430
+ # interior gist keeps its raw,
3431
+ # byte-proportional magnitude (§2.6)
3432
+ return level[0] # tree: every node has gist + kids/bytes
3433
+
3434
+ gistOf(bytes) ≔ perceive(bytes).gist
3435
+ resolve(bytes) ≔ intern-lookup of perceive(bytes), bottom-up:
3436
+ leaves by findLeaf, branches by findBranch(kidIds);
3437
+ null the moment any part is unknown
3438
+ read(node) ≔ concatenation of the node's leaf bytes, left to right
3439
+
3440
+ # ── thresholds (§8) ───────────────────────────────────────────────────
3441
+ MERGE ≔ 1 − 1/√D REACH ≔ 1 − 1/(2W) SIG ≔ 3/√D
3442
+ NOISE ≔ 1/√D CONCEPT_BAR ≔ ½ + 1/(2√D)
3443
+ FLOOR(N) ≔ ln N + ½ HUB(N) ≔ ⌈√N⌉
3444
+ DOMINATES(pLen, wLen) ≔ pLen·2 > wLen # half-dominance (§8.7)
3445
+ corpusN ≔ max(2, edgeSourceCount) # N floored at 2 (§8.8)
3446
+ ```
3447
+
3448
+ ### 27.2 Ingestion
3449
+
3450
+ ```
3451
+ ingestPair(context, continuation):
3452
+ (ctxTree, ctxRoot, ctxIds, changed) ≔ deposit(context, tracked)
3453
+ (conTree, conRoot, _, _) ≔ deposit(continuation, untracked)
3454
+ link(ctxRoot → conRoot)
3455
+ for part in changed:
3456
+ pourHalo(ctxIds[part], π₁·companySignature(conRoot)); massOf(part) += 1
3457
+ pourHalo(conRoot, π₀·companySignature(part)); massOf(conRoot) += 1
3458
+ # link/pour lazily admit both subtrees' interiors to the content index
3459
+
3460
+ deposit(input, tracked):
3461
+ tree ≔ perceive(flatten(input))
3462
+ for node in postorder(tree):
3463
+ id(node) ≔ intern(node) # §11.1 ladder:
3464
+ # exact-dedup →
3465
+ # byte-verified near-
3466
+ # dedup → mint (+ the
3467
+ # child→parent edges)
3468
+ intern sliding windows of W and W−1 leaves as flat branches;
3469
+ containment-link each to the chunks it overlaps # §11.3
3470
+ intern the whole stream as one flat branch # §11.3
3471
+ changed ≔ tracked ? maximal-new-subtree(tree, previousDeposit) : [tree]
3472
+ return (tree, rootId, ids, changed)
3473
+ ```
3474
+
3475
+ ### 27.3 respond
3476
+
3477
+ ```
3478
+ respond(input):
3479
+ query ≔ flatten(input)
3480
+ guide ≔ gistOf(query) # the response-wide disambiguation guide
3481
+ thought ≔ think(query)
3482
+ if thought = ∅: return SILENCE
3483
+ answer ≔ articulate(thought.bytes, query) # §24
3484
+ return (answer, thought.provenance [, rationale])
3485
+
3486
+ think(query, mechanisms ≔ defaultMechanisms):
3487
+ rec ≔ recognise(query) # §15
3488
+
3489
+ # Phase 1 — parse: EVERY mechanism that implements it (only computational
3490
+ # ones do — the ALU, any user extension) contributes computed spans
3491
+ # BEFORE any floor()/run() is called. No mechanism-specific branch: the
3492
+ # pipeline just asks "does this one have parse?".
3493
+ computed ≔ ⋃ mech.parse(query) for each mech in mechanisms with parse
3494
+
3495
+ # Phase 2 — the shared precomputation container, response-scoped, read
3496
+ # by every mechanism's floor/run AND by the post-grounding stages:
3497
+ guide ≔ gistOf(query)
3498
+ pre ≔ Precomputed(rec, computed, guide, k) # eager fields only.
3499
+ # Every EXPENSIVE analysis is a lazily-cached method:
3500
+ # pre.attention() — the consensus climb (§17)
3501
+ # pre.weave() — graded alignment over ranked anchors
3502
+ # pre.spanShapedOf(a) — per-anchor skill classification
3503
+ # pre.windowsOf(a) / pre.queryWindows / pre.reachMemo — the
3504
+ # content-addressed identity reads
3505
+ # Computed at most once, shared by every consumer; NEVER computed
3506
+ # if no surviving mechanism asks — a query an extension decided
3507
+ # outright never pays for a climb.
3508
+
3509
+ # ── Phase 3 — Grounding: ONE lightest-derivation choice among UNIFORM
3510
+ # mechanisms. Every mechanism implements the SAME PipelineMechanism
3511
+ # shape (floor, run); think never imports a mechanism-specific type and
3512
+ # never branches on which one it is holding. Each yields a CANDIDATE
3513
+ # weighed in the one cost ladder (§8.9). A candidate's weight is:
3514
+ #
3515
+ # moves + PASS · unaccounted(query, accounted)
3516
+ #
3517
+ # — moves is the ladder cost of the mechanism's acts (STEP per projection/
3518
+ # locate, CONCEPT per halo-mediated act); unaccounted counts query bytes
3519
+ # NOT covered by the union of accounted spans. PASS per unexplained byte
3520
+ # is exactly the cover's own price for a literal connective.
3521
+ #
3522
+ # Weights are compared at STEP resolution (grade ≔ ⌊w/STEP⌋): sub-STEP
3523
+ # costs (MICRO) are non-ordering bookkeeping. Grade TIES keep the
3524
+ # earlier candidate — the mechanism list's own order (cover, cast,
3525
+ # confluence, extract, recall — see defaultMechanisms).
3526
+
3527
+ best ≔ ∅
3528
+ grade(w) ≔ ⌊w / STEP⌋
3529
+ unaccounted(spans) ≔ query.length − total bytes covered by the union of spans
3530
+ weigh(accounted, moves) ≔ moves + PASS · unaccounted(accounted)
3531
+
3532
+ consider(c):
3533
+ if c.bytes.length = 0: return
3534
+ if best = ∅ or grade(c.weight) < grade(best.weight): best ≔ c
3535
+
3536
+ worthRunning(floor) ≔ best = ∅ or grade(floor) < grade(best.weight)
3537
+
3538
+ # `floor` runs for EVERY mechanism, every time, in list order — BEFORE
3539
+ # `run` is even considered. `worthRunning` gates `run`. A mechanism
3540
+ # whose floor itself needs expensive precomputation to refine (CAST's
3541
+ # weave alignment: existence only, the number is always 2·STEP) receives
3542
+ # `worthRunning` too, and checks ITS OWN cheapest possible floor against
3543
+ # the incumbent before paying for that precomputation — the same
3544
+ # admissible-floor economy, applied one level earlier, uniformly. cover
3545
+ # runs FIRST in the list: a computed span masks in at near-zero cost, so
3546
+ # by the time CAST/confluence ask worthRunning, a cheap incumbent may
3547
+ # already have pruned them — not because they know "an extension fired",
3548
+ # only because they know "the incumbent is cheap".
3549
+ for mech in mechanisms:
3550
+ floor ≔ mech.floor(ctx, query, pre, worthRunning)
3551
+ if floor = ∅: continue # structurally can't fire
3552
+ if not worthRunning(floor): continue # can't beat the incumbent
3553
+ for r in mech.run(ctx, query, pre):
3554
+ consider({ bytes: r.bytes, provenance: r.provenance ?? mech.provenance,
3555
+ weight: r.weight ?? weigh(r.accounted, r.moves),
3556
+ used: r.used, accounted: r.accounted,
3557
+ unexplained: r.unexplained })
3558
+
3559
+ if best = ∅: return ∅
3560
+ # ── Diagnostics (observational, never affect the decision) ──────────
3561
+ if |candidates| > 1 and runnerUp exists:
3562
+ margin ≔ grade(runnerUp.weight) − grade(best.weight)
3563
+ if margin ≤ 1: emit narrowDecision trace with both candidates
3564
+ density ≔ |union(best.accounted)| / query.length
3565
+ if density < 1/W: emit thinGrounding trace
3566
+
3567
+ (answer, provenance, consumed) ≔ (best.bytes, best.provenance,
3568
+ best.used ?? ∅)
3569
+
3570
+ # ── Post-grounding ──────────────────────────────────────────────────
3571
+ consumed ≔ per provenance: cast.used | join.used | sites of
3572
+ recognise(answer) | ∅ (recall/recall-echo consume nothing)
3573
+ # cast and join pre-consume their own consumed set for reasoning
3574
+ answer ≔ reason(query, answer, consumed) # §22
3575
+ if provenance ∈ {recall, recall-echo}:
3576
+ answer ≔ fuseAttention(query, answer) # §23
3577
+ return (answer, provenance)
3578
+ ```
3579
+
3580
+ ### 27.4 The cover search (grounding II, §19)
3581
+
3582
+ ```
3583
+ coverMechanism.run(query, pre): # rec, computed read from pre
3584
+ sites ≔ rec.sites minus any site overlapping a computed span # masking §16.3
3585
+ connectors ≔ resolveConnectors(sites) # §19.5, async pre-resolution
3586
+ concepts ≔ resolveConcepts(sites) # halo siblings with edges,
3587
+ # for edge-less sites (§19.3)
3588
+ solved ≔ lightestDerivation( system(query.len, sites, concepts,
3589
+ rec.leaves, rec.splits,
3590
+ connectors, computed) )
3591
+ if solved = ∅: return ∅
3592
+ segs ≔ solved.segs
3593
+ answer ≔ liftAnswer(segs) # §19.7
3594
+ # accounted = RECOGNISED cover spans only (PASS-carried bytes are priced
3595
+ # in `cost`; the diagnostic `unexplained` label reflects the same distinction)
3596
+ accounted ≔ [s.span for each span s in segs where s.rec]
3597
+ return { bytes: answer, cost: solved.cost, accounted,
3598
+ unexplained: unexplainedLabel(query, accounted) }
3599
+
3600
+ system(L, sites, concepts, leaves, splits, connectors, computed):
3601
+ axioms: Cover(0)@0; Out(leaf)@0 ∀ leaves; Form(site)@0 ∀ sites;
3602
+ Out(computed, recognised)@STEP ∀ computed
3603
+ goal: Cover(L)
3604
+ h(item): ε · (L − rightEdge(item)) # admissible (§8.9)
3605
+ rules(item):
3606
+ Cover(p): BRIDGE across every coverable Out starting at p
3607
+ — ε if recognised, PASS·width if literal
3608
+ Form(i,j,n):
3609
+ if substitutionMode: emit the substitute Out@0 (articulation only)
3610
+ elif next(n) ≠ ∅: Form(i,j, chooseNext(n))@(rcmp? 0 : STEP)
3611
+ elif reached-via-edge:
3612
+ deeper ≔ recomplete(n) # §19.6: re-cover n's own
3613
+ # bytes; cycle-guarded, memoised
3614
+ Out(i,j, deeper ?? read(n), recognised)@0
3615
+ elif concepts[n] exists: Form(i,j, concepts[n])@CONCEPT
3616
+ else: (no rule — the form leads nowhere)
3617
+ Out(i,j,b):
3618
+ SPLICE with any finalised partner Out whose (leftNode,rightNode)
3619
+ has a connector, gap empty-or-wholly-recognised @0
3620
+ SPLIT at any split position k ∈ (i,j) if literal @0
3621
+ BRIDGE from Cover(i) if already finalised (symmetric case)
3622
+ FUSE with any adjacent finalised Out:
3623
+ node ≔ findLeaf(bytes) if short | findBranch(nodes)
3624
+ | resolve(bytes) if a side is a completed rewrite
3625
+ require halo(node) ≠ ∅ when a completed rewrite fuses in
3626
+ yield Out(i,k,bytes)@0 [kept only while it could still grow]
3627
+ if node: yield Form(i,k,node, rcmp = both-sides-rewritten
3628
+ ∧ next(node) ≠ ∅)@0
3629
+ ```
3630
+
3631
+ ### 27.5 The consensus climb (§17)
3632
+
3633
+ ```
3634
+ climbAttention(query, k):
3635
+ regions ≔ subtrees of perceive(query), excluding any region that
3636
+ dominates (covers more than half of) the query unless it is
3637
+ the sole structure
3638
+ # sites name content-addressed nodes — exact anchors, no ANN needed
3639
+ regions ∪= recognise(query).sites (as {start, end, gist, nodeId})
3640
+ for each region:
3641
+ anchor ≔ region.nodeId # site: exact, skip resonance
3642
+ ?? canonicalChunkId(region.bytes, HUB(N))
3643
+ ?? contentIndex.nearest(region.gist, k)[0]
3644
+ reach ≔ expandUntilDecided(anchor, HUB(N)):
3645
+ # uses ONLY LIMITed store reads, bounded by √N:
3646
+ # · prevCount(id) — indexed O(1) "edge-bearing?" check
3647
+ # · parentsFirst(id, HUB(N)+1) — hub if |result| > HUB(N)
3648
+ # · containersSlice(anchor, offset, HUB(N)) — paged
3649
+ # · distinct contexts past HUB(N) → saturated
3650
+ # · below √N, every read IS the full set → exact
3651
+ # fall back to lower hits if orphaned and not saturated
3652
+ if reach.saturated: abstain
3653
+ mutual ≔ min(1, score · ratio) · min(1, score / ratio) # §17.3
3654
+ where ratio ≔ √( max(1, contentLen(anchor, region.len·D))
3655
+ / max(1, region.len) )
3656
+ # contentLen capped at region.len·D — beyond that the
3657
+ # mutual weight approaches zero and the full walk is waste
3658
+ w ≔ mutual · ln(N / reach.contexts) / |reach.roots|
3659
+ vote w for each root (a terminal answer root redistributes its
3660
+ vote over prevFirst(root, HUB(N)) of the contexts that lead
3661
+ to it — capped at the store level, never materialised)
3662
+ # cross-region: any two regions (at least one strong voter) pair to
3663
+ # recover joint contexts their independent climbs missed (junction ascent).
3664
+ # Corpus-independent: known but non-voting regions may serve as weak side.
3665
+ # Order-free, n-ary, with self-evidence guard and explaining away.
3666
+ cross ≔ []
3667
+ superseded ≔ ∅
3668
+ seedsOf(ri) ≔ junctionSeeds(ctx, query[regions[ri].start..regions[ri].end])
3669
+ # precomputed once per candidate, reused across all its pairs
3670
+ consumed ≔ ∅
3671
+ for each pair (a, b) of eligible candidates (non-overlapping,
3672
+ at least one strong voter, not both covered by one known region,
3673
+ ≤ k total probes, skipping consumed):
3674
+ containers ≔ junctionContainersFrom(left, right, cap,
3675
+ seedsOf(a), seedsOf(b), undefined, unordered = true)
3676
+ if containers = ∅:
3677
+ containers ≔ junctionSynonyms(left, right, maxInterior,
3678
+ unordered = true)
3679
+ if containers ≠ ∅:
3680
+ best ≔ the container covering the MOST remaining candidates
3681
+ (cached reads, never an extra walk); ties → shortest
3682
+ interior → lowest id
3683
+ if best's joined occurrence is a query substring: continue
3684
+ reach ≔ edgeAncestors(best.id, HUB(N))
3685
+ if not saturated and idf > 0:
3686
+ w ≔ mutual · ln(N / reach.contexts) / |reach.roots|
3687
+ cross.push(vote for best.id's roots at weight w,
3688
+ span covering all composed candidates)
3689
+ consumed.add(a); consumed.add(b); consumed.add(all extras)
3690
+ # Explaining away: supersede individual votes whose bytes the
3691
+ # container literally contains and whose roots are disjoint
3692
+ for each individual vote rv:
3693
+ if rv.roots shares any root with reach.roots: keep
3694
+ if containerBytes literally contains rv's query bytes:
3695
+ superseded.add(rv)
3696
+ break # a is consumed
3697
+ pooled ≔ lightestDerivation in the (+,+) semiring over the union
3698
+ of the independent votes (minus superseded) and cross # §5.3
3699
+ ranked ≔ anchors by pooled vote, descending
3700
+ cut ≔ steepest ratio drop in the sorted focus votes (natural break)
3701
+ roots ≔ [ranked[0]] ∪ { further non-overlapping anchors past any
3702
+ leading saturated stretch whose vote ≥ max(cut, FLOOR(N)) }
3703
+ return (roots, ranked)
3704
+ ```
3705
+
3706
+ ### 27.6 Recall, reasoning, fusion (§21–23)
3707
+
3708
+ ```
3709
+ recallByResonance(query):
3710
+ whole_ ≔ [[0, query.length]] # identity-grade match: the whole query
3711
+ nothing ≔ [] # reverse readings/echoes: nothing matched
3712
+ q ≔ resolve(query)
3713
+ if q ≠ ∅: # tier 0
3714
+ g ≔ reverseContext(q, guide)
3715
+ if g ≠ ∅: return { bytes: g, accounted: nothing, moves: STEP }
3716
+ hits ≔ contentIndex.nearest(gistOf(query), k)
3717
+ if hits = ∅: return ∅
3718
+ if hits[0].score ≥ MERGE: # tier 1
3719
+ for h in hits:
3720
+ g ≔ project(h, guide) # forward first
3721
+ if g ≠ ∅: return { bytes: g, accounted: whole_, moves: STEP }
3722
+ # all reverse — accounted: nothing (no forward rule)
3723
+ g ≔ reverseContext(hits[0], guide)
3724
+ if g ≠ ∅: return { bytes: g, accounted: nothing, moves: STEP }
3725
+ if hits[0].score ≥ SIG: # tier 2
3726
+ forest ≔ climbAttention(query).roots
3727
+ if forest[0].vote ≥ FLOOR(N):
3728
+ g ≔ project(forest[0].anchor, guide)
3729
+ if g ≠ ∅: return { bytes: g,
3730
+ accounted: [[forest[0].start, forest[0].end]],
3731
+ moves: CONCEPT } # the climb
3732
+ for h in hits: # tier 3
3733
+ g ≔ project(h, guide)
3734
+ if g ≠ ∅ and fracOfQuery(resonance(gistOf(query), gistOf(g)),
3735
+ contentLen(g), query.length) ≥ REACH:
3736
+ return { bytes: g, accounted: nothing, moves: STEP }
3737
+ if fracOfQuery(hits[0].score, contentLen(hits[0]), query.length) < REACH:
3738
+ return ∅ # silence
3739
+ return { bytes: read(hits[0]), accounted: nothing,
3740
+ moves: 0, echoed: true } # honest echo
3741
+
3742
+ reason(query, answer, consumed₀): # §22
3743
+ q ≔ resolve(query)
3744
+ if q ≠ ∅ and prevCount(q) > 0: return answer # echo guard
3745
+ consumed ≔ consumed₀
3746
+ # synonym expansion — CAPPED at the hub bound: a common continuation's
3747
+ # reverse fan-in is corpus-sized, and no per-hop operation may grow
3748
+ # with the corpus (the same visibility trade chooseNext documents)
3749
+ for id in consumed₀:
3750
+ for sib in haloSiblings(id): # unified enumeration,
3751
+ consumeNode(sib.id) # above CONCEPT_BAR
3752
+ cur ≔ answer
3753
+ repeat up to K times:
3754
+ c ≔ resolve(cur); consumeNode(c)
3755
+ if c ≠ ∅ and nextFirst(c, hubBound).some(n ∉ consumed):
3756
+ fwd ≔ follow(c, guide) # forward absorb
3757
+ if fwd ≠ ∅ and fwd ≠ cur and resolve(fwd) ∉ consumed:
3758
+ consumeAll(c); cur ≔ fwd; continue
3759
+ consumeAll(c)
3760
+ pivot ≔ pivotInto(cur, consumed) # below
3761
+ if pivot = ∅: break
3762
+ fc ≔ follow(pivot, guide); consumeAll(pivot)
3763
+ if fc = ∅ or fc = cur: break
3764
+ cur ≔ fc
3765
+ return cur
3766
+
3767
+ # consume-set expansion: prevs and nexts are capped at hubBound —
3768
+ # a node suppressed only by a beyond-cap neighbour may still fire,
3769
+ # the same visibility trade the disambiguators make.
3770
+
3771
+ pivotInto(answer, consumed): # §22 — the stepping stone
3772
+ tree ≔ perceive(answer) # ONE perception, shared by the
3773
+ # probe budget and the walk
3774
+ candidates ≔ ∅
3775
+ for each branch node b of tree, breadth-first,
3776
+ at most min(number of branch nodes, k) probes:
3777
+ for hit in contentIndex.nearest(b.gist, k):
3778
+ if hit ∉ consumed and hasNext(hit): candidates += hit
3779
+ for site in recognise(answer).sites: # exact beats
3780
+ if site ∉ consumed and hasNext(site): # approximate
3781
+ candidates += site (full confidence)
3782
+ return the candidate whose bytes `answer` literally CONTAINS,
3783
+ longest such span wins; ∅ if none # resonance proposes,
3784
+ # bytes confirm
3785
+
3786
+ fuseAttention(query, primary): # §23
3787
+ if primary is strictly contained in query: return primary
3788
+ roots ≔ climbAttention(query).roots
3789
+ if |roots| ≤ 1: return primary
3790
+ qv ≔ guide (the response guide, already computed — once, not per root)
3791
+ pieces ≔ [primary] ∪ [ project(r.anchor, qv) for r in roots[1:] ,
3792
+ dropping ∅ and duplicates ]
3793
+ sort pieces by their supporting query span
3794
+ out ≔ pieces[0]
3795
+ for p in pieces[1:]:
3796
+ out ≔ joinWithBridge(out, p) # learned connector when one exists;
3797
+ # bare join + bridgeMiss trace step
3798
+ # otherwise — degradation is never
3799
+ # silent (§19.5, §23)
3800
+ return out
3801
+
3802
+ bridge(left, right): # §19.5 — the graded junction ladder
3803
+ # Tier 1 — junction containers, by content-addressed identity:
3804
+ # ascend parents + containment links from resolve(left)/resolve(right)
3805
+ # (or their canonical-window ids), √N-disciplined; collect ancestors
3806
+ # whose bytes contain left then right.
3807
+ cands ≔ junctionContainers(left, right)
3808
+ # Tier 2 — edge junctions: a continuation of left containing right
3809
+ # (glue = its prefix), or a context of right containing left (glue =
3810
+ # its suffix). An empty interior is a CONFIRMED adjacency, not a miss.
3811
+ if cands = ∅: cands ≔ junctionEdges(left, right)
3812
+ # Tier 2.5 — synonym junctions: tiers 1 + 2 applied to halo siblings
3813
+ if cands = ∅: cands ≔ junctionSynonyms(left, right)
3814
+ if cands ≠ ∅:
3815
+ # guide resonance picks; ties → shortest interior → lowest id
3816
+ return pick(cands, guide).interior
3817
+ # Tier 3 — the resonance fallback (last resort):
3818
+ for hit in contentIndex.nearest(gistOf(left ⧺ right), 2k), nearest first:
3819
+ f ≔ read(hit)
3820
+ if f contains left at position i, and right at position j > i+|left|:
3821
+ return f[i+|left| … j] # the bytes the corpus puts between
3822
+ return ∅
3823
+
3824
+ joinWithBridge(left, right): # the ONE out-of-search assembly step
3825
+ link ≔ bridge(left, right)
3826
+ if link = ∅: emit bridgeMiss trace; return left ⧺ right
3827
+ return left ⧺ link ⧺ right
3828
+
3829
+ # ── the projection family (§14.4) — shared by every mechanism above ──
3830
+
3831
+ follow(node, guide): # FORWARD: the continuation fixpoint
3832
+ nxt ≔ chooseNext(node, guide)
3833
+ if nxt = ∅:
3834
+ nxt ≔ conceptHop(node) # first hop may cross a synonym:
3835
+ # the first halo sibling above
3836
+ # CONCEPT_BAR that has an edge
3837
+ if nxt = ∅: return ∅
3838
+ walk chooseNext from nxt until revisit or dead end (cycle-guarded)
3839
+ return read(final node)
3840
+
3841
+ reverseContext(node, guide, rev?): # REVERSE: the establishing context
3842
+ candidates ≔ rev ?? prevFirst(node, hubBound) # CAPPED at √N: a common
3843
+ # continuation's reverse fan-in
3844
+ # is corpus-sized; prevFirst
3845
+ # reads only the first √N
3846
+ if candidates = ∅: return ∅
3847
+ pick ≔ |candidates| = 1 ? candidates[0] # skip needless gisting
3848
+ : guide ≠ ∅ ? chooseAmong(candidates, guide)
3849
+ : argmax haloMass over candidates (already capped)
3850
+ g ≔ read(pick)
3851
+ return |g| > 0 ? g : ∅ # empty bytes are no grounding
3852
+
3853
+ project(node, guide): # BOTH: the universal grounding step
3854
+ return follow(node, guide) ?? reverseContext(node, guide)
3855
+
3856
+ # ── the disambiguators + the one fan-out convention (§25, §8.8) ──
3857
+
3858
+ corpusN ≔ max(2, edgeSourceCount) # floored at 2 so ln N and √N stay
3859
+ hubBound ≔ ⌈√corpusN⌉ # meaningful on a near-empty store
3860
+
3861
+ hubCap(ids): # THE fan-out cap, defined once
3862
+ return the first hubBound of ids (insertion order); no copy when under
3863
+
3864
+ guidedFirst(node): # guided-or-first, for answer-shaped reads
3865
+ return chooseNext(node, guide) ?? nextFirst(node, 1)[0]
3866
+ # LIMIT 1 read when no guide is in flight
3867
+
3868
+ chooseNext(node, guide): # §25, forward regime
3869
+ nx ≔ nextFirst(node, hubBound) # only the first √N continuations
3870
+ # are ever candidates — a hub context's
3871
+ # full fan-out is never materialised
3872
+ if |nx| ≤ 1 or guide = ∅: return nx[0]
3873
+ among nx (already capped):
3874
+ maximize ( prevCount(candidate) , haloMass(candidate) ) # indexed,
3875
+ ties → first inserted # never materialised
3876
+
3877
+ chooseAmong(candidates, guide): # §25, reverse regime
3878
+ among candidates (already capped by the caller):
3879
+ maximize resonance(guide, gistOf(read(candidate)))
3880
+ return the winner
3881
+ ```
3882
+
3883
+ ### 27.7 Counterfactual transfer (grounding I, §18)
3884
+
3885
+ ```
3886
+ counterfactualTransfer(query, sites, roots, ranked):
3887
+ # castFloor already checked |query|<2W, N=0, |ranked|<2 — these
3888
+ # gates are checked once in the floor, not duplicated here.
3889
+ # If roots/ranked not given (standalone call), compute the climb.
3890
+
3891
+ # ── graded alignment (literal → halo) ────────────────────────────
3892
+ MIN_WEAVE ≔ 2; points ≔ ∅; depth ≔ Float64Array(|query|)
3893
+ for cand in the first 2k of ranked:
3894
+ ... (alignment as before) ...
3895
+ if |points| < 2: return []
3896
+ # frame gate (weave-local): frame(i) ⇔ depth[i] > MIN_WEAVE ∧ dominates(depth[i], |points|)
3897
+ dominant ≔ points[0]; require dominant ∈ roots
3898
+ require some run outside every recognised site
3899
+
3900
+ results ≔ [] # multi-candidate: each schema records independently
3901
+ runSpans(p) ≔ p's free runs as [qs, qe] pairs
3902
+
3903
+ # ── 1. substitution ──────────────────────────────────────────────
3904
+ ... (same detection logic) ...
3905
+ if found:
3906
+ record({ bytes: joinWithBridge(filler, tail) + follow(p.anchor),
3907
+ used: {before, p}, moves: STEP+STEP,
3908
+ accounted: runSpans(before) ∪ runSpans(p),
3909
+ unexplained: query bytes not in those runs })
3910
+
3911
+ # ── 2. redirection ───────────────────────────────────────────────
3912
+ ... (same detection logic) ...
3913
+ if found:
3914
+ record({ bytes: g, used: {dominant, last}, moves: STEP,
3915
+ accounted: runSpans(dominant) ∪ runSpans(last),
3916
+ unexplained: query bytes not in those runs })
3917
+
3918
+ # ── 3. comparison ────────────────────────────────────────────────
3919
+ ... (same detection logic) ...
3920
+ if found:
3921
+ record({ bytes: joinWithBridge(a, b),
3922
+ used: {dominant, bestAnalog},
3923
+ moves: CONCEPT+STEP+STEP,
3924
+ accounted: runSpans(dominant) ∪ (bestAnalog.point ? runSpans(bestAnalog.point) : []),
3925
+ unexplained: query bytes not in those runs })
3926
+
3927
+ return results # possibly empty — the decider weighs whatever fired
3928
+ ```
3929
+
3930
+ ### 27.7a Confluence join (§18.5)
3931
+
3932
+ ```
3933
+ confluenceJoin(query):
3934
+ if |query| < 2W or N = 0: return ∅
3935
+ (roots, ranked) ≔ climbAttention(query, 2k)
3936
+ if |ranked| < 2: return ∅
3937
+
3938
+ queryWin ≔ windowIds(query) # offset → id, canonical W-window read
3939
+ queryIds ≔ set of queryWin values
3940
+ N ≔ corpusN
3941
+
3942
+ # ── constraint streams ────────────────────────────────────────────
3943
+ streams ≔ ∅
3944
+ for cand in ranked (capped at 2k):
3945
+ ids ≔ set of windowIds(read(cand.anchor)).values
3946
+ cover ≔ ∅ # query spans this anchor holds DISCRIMINATIVE windows of
3947
+ held ≔ ∅ # query spans this anchor holds AT ALL (scaffolding included)
3948
+ for (off, wid) in queryWin where ids has wid:
3949
+ merge off into held
3950
+ if not dominates(reachOf(wid, N), N): # scaffolding never binds
3951
+ merge off into cover
3952
+ if cover ≠ ∅: streams += (cand.anchor, cand.vote, ids, cover, held)
3953
+ if |streams| < 2: return ∅
3954
+
3955
+ # ── find the MEET of two independent streams ─────────────────────
3956
+ disjoint(a, b) ≔ a.cover and b.cover share no query byte
3957
+ for each pair (a, b) where disjoint(a, b):
3958
+ wa ≔ windowIds(read(a.anchor))
3959
+ # window ids in BOTH anchors, ABSENT from the query — merged into
3960
+ # maximal contiguous spans (overlapping windows weave one span)
3961
+ for each contiguous span [s, e) of offsets where
3962
+ wa[off] ∈ b.ids and wa[off] ∉ queryIds:
3963
+ # scaffolding gate: the span's most DISCRIMINATIVE window decides
3964
+ reach ≔ min reachOf(wa[off], N) for each window in [s, e)
3965
+ if not isFinite(reach) or dominates(reach, N): continue
3966
+ # feasible — the entity where the constraints meet
3967
+ met ≔ the one with smallest reach, longest span (tie-break)
3968
+ if met = ∅: return ∅
3969
+
3970
+ return { bytes: read(a.anchor).subarray(met.s, met.e),
3971
+ used: {a.anchor, b.anchor},
3972
+ accounted: a.held ∪ b.held, # ALL matched content
3973
+ moves: 3·STEP } # two matches + one meet
3974
+ ```
3975
+
3976
+ ### 27.8 Extraction and articulation (§20, §24)
3977
+
3978
+ ```
3979
+ extractBySkill(query):
3980
+ ranked ≔ climbAttention(query, 2k).ranked
3981
+ for cand in ranked: # first span-shaped wins
3982
+ ex ≔ skillExemplar(cand.anchor):
3983
+ if hasNext(anchor):
3984
+ (context, answer) ≔ (read(anchor), follow(anchor, guide))
3985
+ else:
3986
+ answer ≔ read(anchor)
3987
+ context ≔ the longest span-shaped context among
3988
+ prevFirst(anchor, hubBound); chooseAmong
3989
+ (the reverse-regime disambiguator) breaks
3990
+ length ties via query-gist resonance
3991
+ require answer is a sparse subsequence of context
3992
+ # OPEN reading: in-order, arbitrary gaps (§20.1)
3993
+ # the subsequent DECOMPOSITION step uses a STRONGER
3994
+ # greedy-longest-run reading; an accepted exemplar
3995
+ # can still fail to decompose → extraction returns ∅
3996
+ if ex ≠ ∅: break
3997
+ if no exemplar: return ∅
3998
+ runs ≔ the answer's pieces, decomposed by greedy longest-run
3999
+ matching inside the context (the STRONG reading); contiguous
4000
+ adjacent runs merged
4001
+ accounted ≔ ∅
4002
+ for each run, with isLast flags:
4003
+ pre ≔ up to W context bytes before the run
4004
+ post ≔ up to W after (or the NEXT run's pre)
4005
+ locate pre then post in the query via locate() — the graded
4006
+ matcher ladder of §14.4:
4007
+ 1. exact bytes 2. halo-role match via bestHaloMate above
4008
+ CONCEPT_BAR 3. gist match against query segments above MERGE
4009
+ accounted += the located pre/post frames in the query
4010
+ piece ≔ the query bytes between the located frames
4011
+ # bounded on BOTH sides ⇒ the read span itself is explained
4012
+ if pre-located and post-located: accounted += piece's span
4013
+ return { bytes: concatenation of the pieces,
4014
+ accounted } (∅ if none located)
4015
+ moves ≔ CONCEPT + STEP · |accounted|
4016
+
4017
+ articulate(answer, query):
4018
+ voices ≔ recognised multi-byte forms of the QUERY that bear halos
4019
+ if voices = ∅ or they cover none of the query: return answer
4020
+ subs ≔ ∅
4021
+ for each recognised multi-byte, halo-bearing form f of the ANSWER:
4022
+ v ≔ argmax over voices of cosine(halo(f), halo(v)), ≥ CONCEPT_BAR
4023
+ if v exists and v ≠ f and f is not a fragment of v's own subtree:
4024
+ subs[f] ≔ v.bytes
4025
+ if subs = ∅: return answer
4026
+ solved ≔ the cover search over the ANSWER with subs as the only form
4027
+ rules (each voiced form emits its substitute at cost 0)
4028
+ return solved ? solved.segs composed : answer # unchanged if no cover
4029
+ ```
4030
+
4031
+ ### 27.9 A worked example, end to end
4032
+
4033
+ The README's demo, traced through the pipeline. Deposits:
4034
+
4035
+ ```
4036
+ ("The Mona Lisa was painted by Leonardo da Vinci.", "Leonardo da Vinci")
4037
+ ("The Starry Night was painted by Vincent van Gogh.", "Vincent van Gogh")
4038
+ ("The Night Watch was painted by Rembrandt van Rijn.","Rembrandt van Rijn")
4039
+ ("Pablo Picasso", "Pablo Picasso co-founded the Cubist movement")
4040
+ ```
4041
+
4042
+ Each pair interns both sides (sharing every repeated span: "was painted by" is
4043
+ one set of nodes across all three sentences), records one continuation edge, and
4044
+ pours halos both ways. The three painter names, having each appeared as an
4045
+ answer following a painting-frame, acquire similar halos; "was painted by …"
4046
+ spans become shared, many-parent interior structure.
4047
+
4048
+ Query: `"The Weeping Woman was painted by Pablo Picasso."`
4049
+
4050
+ 1. **Recognise (§15).** Sites include "was painted by" material (shared interior
4051
+ forms), " Pablo Picasso" (a learned context — it has an edge to the Cubism
4052
+ fact), and assorted chunks. "The Weeping Woman" resolves to nothing: never
4053
+ seen.
4054
+ 2. **Compute (§16).** No extension claims any span.
4055
+ 3. **Grounding decider (§14.1).** Every self-gating mechanism weighs in:
4056
+ - **CAST (§18):** The climb ranks the three painting exemplars and the
4057
+ Picasso context; alignment finds runs, but no substitution seat or
4058
+ redirection shape fits — CAST yields no candidate.
4059
+ - **Confluence (§18.5):** Only one constraint stream (the query asks about
4060
+ one painting, not a conjunction of independent properties) — returns null.
4061
+ - **Cover (§19):** The recognised forms do not compose a cover that lifts an
4062
+ answer clear of the framing (the unseen painting title blocks a clean
4063
+ composition) — returns null. _(On other seeds/corpora this query can also
4064
+ ground via cover; the strategies are redundant by design, and provenance
4065
+ records which one fired.)_
4066
+ - **Extraction (§20):** The climb's ranked anchors include the exemplar "The
4067
+ Mona Lisa was painted by Leonardo da Vinci." — span-shaped. Its frames are
4068
+ located in the query; the analogous span reads out **"Pablo Picasso"**. The
4069
+ candidate's weight: CONCEPT (one skill analogy) + STEP per located frame +
4070
+ PASS per unexplained byte. It is the lightest grounding derivation.
4071
+ - **Recall (§21):** Its best candidate carries the full PASS·|query| plus a
4072
+ STEP — heavier.
4073
+ - **Decider:** Extraction wins — lightest grounding derivation.
4074
+ 4. **Reason (§22).** "Pablo Picasso" resolves — and it is a learned context with
4075
+ an unconsumed continuation. Forward absorb follows the edge: **"Pablo Picasso
4076
+ co-founded the Cubist movement"**. The next iteration finds no unconsumed
4077
+ pivot; the chain fixes.
4078
+ 5. **Fuse / articulate (§23–24).** One point of attention grounded from the
4079
+ query's text; no halo-sibling substitutions apply. The answer stands.
4080
+
4081
+ Provenance: `extract`. Every step above is present, with spans, node ids, costs,
4082
+ and data-flow edges, in the rationale when one is requested.
4083
+
4084
+ The second demo query, `"a museum charges 12*4 for a family ticket"`: the ALU
4085
+ claims the span `12*4` with result bytes `48`; recognition's sites overlapping
4086
+ that span are masked; the cover search bridges the literal framing (PASS) and
4087
+ the computed span (recognised, STEP + ε), and lifting drops the framing:
4088
+ **"48"**, provenance `cover`.
4089
+
4090
+ ### 27.10 Determinism, stated as an invariant
4091
+
4092
+ Every function above is deterministic given (seed, store contents): the alphabet
4093
+ and keyring are seeded; perception is positional; interning is
4094
+ content-addressed; the deduction engine breaks ties by fixed conventions;
4095
+ disambiguation bottoms out in corpus-determined orderings; the ANN index is
4096
+ deterministic for a fixed build. Hence: **same seed + same deposits (in order) +
4097
+ same query ⇒ byte-identical answer.** The only approximation in the system — ANN
4098
+ ranking — affects which _candidates are proposed_, never what any accepted
4099
+ answer _asserts_, and it too is deterministic run to run.
4100
+
4101
+ ---
4102
+
4103
+ ---
4104
+
4105
+ # Part VI — Reference
4106
+
4107
+ ## 28. Glossary
4108
+
4109
+ The one-line inventory of §9 doubles as the glossary; this section adds only the
4110
+ terms of art borrowed from the literature.
4111
+
4112
+ - **A\*LD** — A\* Lightest Derivation: A\* generalized from shortest paths to
4113
+ weighted deduction (Felzenszwalb & McAllester 2007). §5.2.
4114
+ - **Binding / superposition** — the two VSA combination operators:
4115
+ association-forming (order-visible) vs. set-forming (similarity- preserving).
4116
+ §2.1.
4117
+ - **Company signature** — a deterministic unit vector derived from a node's
4118
+ identity (seeded by id), used as the halo-pour unit. Decouples content
4119
+ similarity from distributional similarity. §4, §12.2.
4120
+ - **Concentration of measure** — the high-dimensional phenomenon making random
4121
+ vectors quasi-orthogonal; the statistical basis of every threshold. §2.2.
4122
+ - **Content-addressable memory** — retrieval by content, not location. §3.1.
4123
+ - **Distributional hypothesis** — meaning ≈ distribution of use (Harris 1954).
4124
+ §4.
4125
+ - **Hash-consing** — constructing structures modulo equality so equal
4126
+ substructures are shared. §3.2.
4127
+ - **HNSW** — Hierarchical Navigable Small World graph; sub-linear ANN search
4128
+ (Malkov & Yashunin 2018). §6.1.
4129
+ - **Hilbert curve** — the locality-preserving space-filling curve used to
4130
+ linearize grids. §6.3.
4131
+ - **Hyperdimensional computing** — Kanerva's (2009) umbrella term for computing
4132
+ with high-dimensional random vectors; synonym of VSA as used here. §2.
4133
+ - **IDF** — inverse document frequency; the specificity weighting of the
4134
+ consensus climb (Spärck Jones 1972). §17.3.
4135
+ - **Instance-based learning** — generalization at query time from stored
4136
+ instances. §1.1.
4137
+ - **Merkle DAG** — a graph whose node identities derive from content (Merkle
4138
+ 1987). §3.2.
4139
+ - **Non-parametric** — model capacity residing in the data, not a fixed
4140
+ parameter vector. §1.1.
4141
+ - **RaBitQ** — 1-bit quantization with an unbiased similarity estimator (Gao &
4142
+ Long 2024). §6.1.
4143
+ - **Semiring-weighted deduction** — the algebraic generalization of weighted
4144
+ inference (Goodman 1999); Sema uses tropical (min,+) for structure and
4145
+ arithmetic (+,+) for evidence pooling. §5.
4146
+ - **Tropical semiring** — (min, +): the algebra of shortest paths and lightest
4147
+ derivations. §5.3.
4148
+ - **VSA** — Vector Symbolic Architecture (Plate 1995; Gayler 2003). §2.
4149
+
4150
+ ## 29. Complexity summary
4151
+
4152
+ n = input/query length; D = dimension; W = fold window; N = learned contexts; k
4153
+ = retrieval breadth. All store lookups are content-addressed O(1) (amortized);
4154
+ all index queries are sub-linear in the collection (empirically ≈ N^0.32
4155
+ distance computations).
4156
+
4157
+ | Operation | Cost | Where |
4158
+ | :------------------------- | :---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :-------- |
4159
+ | Perceive | O(n·D) vector work; O(n) nodes | §10 |
4160
+ | Deposit (intern + windows) | O(n) content-addressed probes | §11 |
4161
+ | Learn a pair | O(1) edge + O(changed) halo pours | §12 |
4162
+ | Recognise | O(n·W) bounded probes | §15 |
4163
+ | Consensus climb | O(regions · k) index queries + expand-until-decided: work bounded by √N per region regardless of corpus size (LIMITed store reads, indexed existence probes) | §17 |
4164
+ | Cover search | output-sensitive A\*LD: proportional to the lightest derivation, not the corpus (§5.2); the dominant per-query index cost is connector pre-resolution, O(sites) queries | §19 |
4165
+ | Recall | O(k) index probes + graded structural checks | §21 |
4166
+ | Reasoning | ≤ K hops, each bounded by the answer's subtree | §22 |
4167
+ | Storage | O(distinct subtrees); vector index over resonance targets only, 1-bit codes (32× compression) | §3, §12.3 |
4168
+
4169
+ Nothing on any per-query path scans the corpus; every fan-out is capped at the
4170
+ hub bound, and the cap is enforced at the _store level_ through LIMITed reads
4171
+ and indexed existence probes — no per-query read materialises a corpus-sized
4172
+ list. That — not hardware — is why the system runs on a CPU and why inference
4173
+ cost stays decoupled from corpus growth.
4174
+
4175
+ ## 30. Bibliography
4176
+
4177
+ Foundations cited in this document, in alphabetical order:
4178
+
4179
+ - Felzenszwalb, P. F. & McAllester, D. (2007). _The Generalized A\*
4180
+ Architecture._ Journal of Artificial Intelligence Research 29, 153–190.
4181
+ - Gao, J. & Long, C. (2024). _RaBitQ: Quantizing High-Dimensional Vectors with a
4182
+ Theoretical Error Bound for Approximate Nearest Neighbor Search._ Proc. ACM
4183
+ SIGMOD.
4184
+ - Gayler, R. W. (2003). _Vector Symbolic Architectures Answer Jackendoff's
4185
+ Challenges for Cognitive Neuroscience._ Proc. ICCS/ASCS.
4186
+ - Goodman, J. (1999). _Semiring Parsing._ Computational Linguistics 25(4),
4187
+ 573–605.
4188
+ - Harris, Z. S. (1954). _Distributional Structure._ Word 10(2–3), 146–162.
4189
+ - Kanerva, P. (2009). _Hyperdimensional Computing: An Introduction to Computing
4190
+ in Distributed Representation with High-Dimensional Random Vectors._ Cognitive
4191
+ Computation 1, 139–159.
4192
+ - Knuth, D. E. (1977). _A Generalization of Dijkstra's Algorithm._ Information
4193
+ Processing Letters 6(1), 1–5.
4194
+ - Kolodner, J. L. (1992). _An Introduction to Case-Based Reasoning._ Artificial
4195
+ Intelligence Review 6, 3–34.
4196
+ - Malkov, Y. A. & Yashunin, D. A. (2018). _Efficient and Robust Approximate
4197
+ Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs._ IEEE
4198
+ TPAMI 42(4), 824–836.
4199
+ - Merkle, R. C. (1987). _A Digital Signature Based on a Conventional Encryption
4200
+ Function._ Proc. CRYPTO.
4201
+ - Plate, T. A. (1995). _Holographic Reduced Representations._ IEEE Transactions
4202
+ on Neural Networks 6(3), 623–641.
4203
+ - Spärck Jones, K. (1972). _A Statistical Interpretation of Term Specificity and
4204
+ Its Application in Retrieval._ Journal of Documentation 28(1), 11–21.
4205
+
4206
+ ---
4207
+
4208
+ _This document describes concepts and algorithms only. For the codebase —
4209
+ layout, build, tests, invariants, and how to extend the system — see
4210
+ [AGENTS.md](AGENTS.md)._