@hviana/sema 0.7.3 → 0.7.6

package/docs/architecture/bounded-reads.md

@@ -0,0 +1,85 @@
+# Bounded Reads — No Per-Query Read Grows With the Corpus
+
+> **Law:** the cost of one query is proportional to the query, not to how much
+> was learned. No per-query read may grow with corpus size N.
+
+Every fan-out, walk, and disambiguation is capped at `hubBound` —
+`ceil(sqrt(N))` — derived once from `corpusN` and floored at 2 so `sqrt` and
+`ln` stay meaningful on a near-empty store. There is no second convention; do
+not invent one.
+
+## Scale
+
+```
+corpusN(ctx) = max(2, store.edgeSourceCount())   // distinct learnt contexts
+hubBound(ctx) = ceil(sqrt(corpusN(ctx)))          // >= 2, the store cap
+hubCap(ctx, ids) = ids.slice(0, hubBound(ctx))    // list-side reading
+boundFor(n) = ceil(sqrt(max(2, n)))               // ctx-free reading
+```
+
+Defined once in `mind/traverse.ts` (`corpusN`, `hubBound`, `hubCap`,
+`boundFor`). Every consumer imports them; never spell `Math.sqrt` inline.
+
+## Enforcement at the store level
+
+The cap is not advisory — adapters must make bounded reads bounded in SQL.
+
+### 1. LIMITed reads — real `LIMIT ?`
+
+`nextFirst(id, limit)`, `prevFirst(id, limit)`, `parentsFirst(id, limit)`,
+`containersSlice(child, offset, limit)`.
+
+Same statement and `ORDER BY` as the full read, with `LIMIT ?`. Never
+"materialise then slice". Reading `hubBound + 1` parents decides "hub or not"
+exactly without reading the rest. Implemented as thin wrappers in
+`store-sqlite.ts` over `AbstractStore` in `store.ts`.
+
+### 2. Existence probes — indexed point probes
+
+`hasNext(id)`, `hasParents(id)`, `hasContainers(child)`, `hasHalo(id)`,
+`prevCount(id)`.
+
+One indexed `EXISTS` / `COUNT` probe that never decodes vectors or unpacks
+blobs. Use them for every "does this lead anywhere?" question instead of
+`next(id).length > 0` or `prev(id).length`. `prevCount` is the reverse-edge
+support count for `chooseNext`/`chooseAmong`; `hasNext`/`hasHalo` gate the
+`leadsSomewhere` admission predicate in `mind/traverse.ts`.
+
+### 3. Prefix-capped reads — reject without reconstruction
+
+`bytesPrefix(id, cap)` and `contentLen(id, cap)`.
+
+`contentLen` under a cap returns an exact length below the cap and `>= cap`
+otherwise — an indexed memo walk that stops early, never a full subtree walk.
+`bytesPrefix` stops after `cap` bytes. A candidate exceeding the cap is rejected
+on the length probe alone; the weave, junction walks, and bridge all read this
+way. Uncapped reads there cost seconds per query on a large store.
+
+### 4. Transparent scaffolding — one bounded read
+
+`chainRun(id)` climbs a run of transparent nodes (exactly one parent, no edges)
+in a single recursive CTE, cached for the store lifetime and dropped on writes
+that break transparency. The climber hops the whole run where a node-at-a-time
+ascent would pay three probes per node.
+
+## Maintenance only
+
+The full materialising reads — `next(id)`, `prev(id)`, `parents(id)`,
+`containers(child)` — exist for inspection, repair, and compaction only
+(`compactContentIndex`, `repairContentIndex`). Keep them off hot paths.
+
+## Adding a walk
+
+Any new fan-out walk uses `hubBound`/`hubCap`. Do not call `edgeSourceCount()`
+or `Math.ceil(Math.sqrt(...))` inline, and do not invent a per-walk limit. The
+walk's saturation decision (when to stop) is separate from the cap (the safety
+net); a walk with only a cap drifts to the cap.
+
+## Pins
+
+- `test/14` — sublinear inference in corpus size and constant-rate in input
+  length; training throughput floor; exact recall at scale.
+- `test/89` — completion recursion stays output-sensitive (nested searches/pops
+  sublinear); guards the count of reads, not just per-read size.
+- `test/90` — connector probe (`resolveConnectors`) reads by the query length
+  (`QUERY.length + 1`), not by the learnt continuation; per-read size bound.

package/docs/architecture/caches.md

@@ -0,0 +1,89 @@
+# Caches — Every Acceleration Is a BoundedMap
+
+> **Law:** every acceleration is a `BoundedMap` with a byte budget. A miss
+> re-derives from durable state. Degradation order is speed/reach lost, never
+> identity.
+
+No cache may change what is stored, what is resolved, or what tree is folded.
+Eviction costs a re-read, a re-walk, or a narrower reach — never a wrong answer
+or a wrong tree.
+
+## `BoundedMap` — the one cache primitive
+
+`src/store.ts:BoundedMap<K,V>` — LRU with byte accounting (`maxBytes`, `sizeOf`,
+`evict`, `recency`).
+
+- `evict: "lru"` — uniform-cost entries (dedup, vectors, records).
+- `evict: "smallest"` — variable-cost reconstruction (`_bytesCache`): protects
+  expensive large branches over cheap leaves.
+- `recency: "reorder"` (default) — exact LRU via `delete+set`; required when
+  eviction choice is load-bearing (`_depositTrees` — 8 entries, victim changes
+  fold).
+- `recency: "clock"` — bit instead of reorder; only for transparent caches where
+  wrong victim costs a re-read (`_bytesCache`, `_recCache`). Measured: same
+  entries cached, hot-path time 55% → bit.
+
+Persistent cursor over V8 insertion order makes eviction amortised O(1);
+candidate window for `"smallest"` never rescans from front.
+
+## Store caches — budgets in `src/config.ts:StoreConfig`
+
+| Cache               | Field                      | Budget                       | `sizeOf`            | Eviction       |
+| ------------------- | -------------------------- | ---------------------------- | ------------------- | -------------- |
+| dedup leaf/branch   | `_leafKey` / `_branchKey`  | `dedupCacheMax` 1M entries   | 1                   | lru            |
+| reconstructed bytes | `_bytesCache`              | `bytesCacheMax` 20 MB        | `byteLength`        | smallest+clock |
+| content length      | `_lenCache`                | `bytesCacheMax`              | 16                  | lru            |
+| node records        | `_recCache`                | `recCacheBytes` 10 MB        | leaf+4·kids+12      | lru+clock      |
+| pending gists       | `_pendingGist`             | `pendingGistBytes` 16 MB     | `byteLength` (D·4)  | lru            |
+| halo exact / norm   | `_haloExact` / `_haloNorm` | `haloCacheBytes` 16 MB each  | `byteLength`        | lru            |
+| skipped interiors   | `_coveredIds`              | `coveredIdsMax` 100K entries | 1                   | lru            |
+| indexed ids         | `_indexedIds`              | `coveredIdsMax`              | 1                   | lru            |
+| transparent chains  | `_chainMemo`               | `chainCacheBytes` 16 MB      | 4·len+32            | lru            |
+| ingest memo         | `CachedIngest._memo`       | `ingestCacheBytes` 50 MB     | vector+ids+keyBytes | lru            |
+
+ANN read caches (`_resonateCache`, `_resonateHaloCache`) are `Map<string,Hit[]>`
+keyed by `vecKey(v)+":"+k`, dropped on any index mutation.
+`vectorCacheMb`/`sqliteCacheMb` are pure page-cache latency knobs.
+
+`_bytesCache` only caches complete reconstructions — `bytesPrefix(id,cap)` with
+`got < cap`; a truncated prefix is never stored. `_chainMemo` is dropped on any
+write that could break transparency; `_pendingGist` eviction falls back to DAG
+climb; halo eviction re-decodes the durable 2-bit row.
+
+## Mind caches — session and per-response
+
+| Cache               | Location                                         | Budget    | Scope / invalidation                                               |
+| ------------------- | ------------------------------------------------ | --------- | ------------------------------------------------------------------ |
+| `_gistCache`        | `Mind._gistCache`                                | 32 MB     | session-lifetime, never invalidated (perception pure)              |
+| `_depositTrees`     | `Mind._depositTrees`                             | 8 entries | session; `perceiveDeposit` only when `conversational`              |
+| `_depositLens`      | `Mind._depositLens`                              | —         | byte lengths for prefix probes; cleared with map when >64          |
+| `_internIds`        | `Mind._internIds: WeakMap<Sema,number>`          | —         | Mind lifetime; ids permanent                                       |
+| `_resolvedSubtrees` | `Mind._resolvedSubtrees: WeakMap<Sema,{id,len}>` | —         | per-response/conversation; fast path only when `visit===undefined` |
+
+`REACH_MEMO_MAX` / `STRUCT_MEMO_MAX` 100K (`src/mind/traverse.ts`) — whole-climb
+and per-node structural probes (`hasNext`/`prevCount`/`hasParents`); cleared on
+write or when cap reached. `reachMemo`/`structCaches` keyed by `_structMemoKey`,
+bypassed under trace.
+
+## Deposit caches — offset-keyed, caller-discharged
+
+`_depositTrees`/`_depositLens`/`_internIds`/`_resolvedSubtrees` key **offsets**,
+not bytes, for O(1) reuse. Offsets alone cannot witness byte agreement — caller
+must discharge it.
+
+- Correct: conversation append — each turn extends the prior cumulative context
+  by its own bytes; longest cached proper prefix hit (`L < bytes.length`) reuses
+  `contentFoldIncremental` segments bit-identically.
+- Wrong: mismatched `prev` reused by offset produced wrong tree (336 vs 400
+  bytes) — a coincidental prefix length aliased unrelated content.
+- Now: `perceiveDeposit` keys by `latin1Key(bytes.subarray(0,L))` (prefix
+  bytes), probes longest cached proper prefix first; `_depositTrees` populated
+  only for conversational deposits (budget discipline), otherwise cold path
+  always correct.
+
+## Pins
+
+- `test/91` — `chainRun` via capped `_prefix`: bounded transparent-chain hop,
+  not per-node probes.
+- `test/96` — `_bytesCache` is byte-accounted `BoundedMap` that evicts; miss
+  re-derives.

package/docs/architecture/commonality.md

@@ -0,0 +1,45 @@
+# Two Measures of Commonality
+
+Sema needs "what is shared" in two different populations. One is corpus-global
+(how widely a structure is reused), the other is weave-local (what a local
+cohort of overlapping forms agrees on). They use different data and different
+formulas and must not be conflated.
+
+## Corpus-global — `reachOf` + `dominates`
+
+_Defined in `src/mind/traverse.ts` + `src/geometry.ts`; used by climb,
+containment, IDF pooling._
+
+For a node id, `reachOf(id, N)` counts how many learnt contexts contain it
+(ancestor reach via capped graph walks, memoised per response in
+`sharedReachMemo`). `dominates(reach, N)` then asks whether that reach is above
+the corpus-determined majority threshold (derived in `geometry.ts` over `N`).
+Intuition: minority reach discriminates (a filler), majority reach is
+scaffolding. Powers the consensus climb, edge following, and vote pooling.
+
+## Weave-local — `depth[]` + `MIN_WEAVE` + `dominates`
+
+_Defined in `src/mind/match.ts` (`depth[]`, `MIN_WEAVE`, `frame`) and gated in
+`src/mind/match.ts:frame`; used by CAST._
+
+For an alignment weave, `depth[i]` counts how many aligned structures cover byte
+`i` of the query. `MIN_WEAVE = 2` requires agreement beyond a pair (pair columns
+are ambiguous with insertions/deletions), and `dominates(depth[i], aligned)`
+requires agreement by a majority of the aligned cohort:
+
+```
+frame(i) ⇔ depth[i] > MIN_WEAVE ∧ dominates(depth[i], aligned)
+```
+
+This powers CAST's frame gate: what the local cohort shares vs what
+differentiates one member. It never consults corpus reach.
+
+The two measures answer different questions over different populations; CAST's
+frame must not be replaced by a reach check and the climb must not be driven by
+weave depth.
+
+## Pins
+
+- `test/17` — weave-local frame / `MIN_WEAVE` / `dominates` vs corpus-global
+  reach.
+- `test/34` — containment and reach-driven disambiguation.

package/docs/architecture/cost-model.md

@@ -0,0 +1,71 @@
+# Cost Model — One Currency
+
+Every mechanism and every byte competes on one cost ladder defined in
+`src/mind/graph-search.ts`. GraphSearch and `pipeline.ts:think` use the same
+units, so a mechanism-level choice and a byte-level choice are the same kind of
+decision: a lightest derivation.
+
+## Ladder (`src/mind/graph-search.ts`)
+
+| Cost      | Value         | Meaning                                                                                                                                                                           |
+| --------- | ------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `MICRO`   | `1e-3`        | Recognised advance (one `rec` bridge); per-byte unit of the A\* heuristic. A recomposed form's onward edge is also `MICRO`.                                                       |
+| `STEP`    | `1`           | Every edge hop (first or fifth), every computed result, every projection. Charging every hop makes the lightest derivation the shortest chain.                                    |
+| `CONCEPT` | `10`          | Halo-mediated act (synonym hop, consensus climb) and abandoning an edge chain early (`CONCEPT` above chain cost — genuine fixpoint at `+0` always beats giving up at same depth). |
+| `PASS`    | `1000` / byte | Carrying a byte nothing explains. Dominates everything so the search always prefers to recognise.                                                                                 |
+
+Only the **ordering** `MICRO < STEP < CONCEPT < PASS` matters; any constants
+with that order give the same derivations.
+
+## Pipeline weighing (`src/mind/pipeline.ts:think`)
+
+Mechanism candidates are weighed in the same ladder:
+
+```
+weight = moves + PASS * unaccounted_bytes
+grade  = floor(weight / STEP)
+```
+
+`unaccounted` is the query bytes no `accounted` span covers. Comparison is at
+`STEP` resolution: lowest `grade` wins. At equal grade the candidate with fewer
+`scaffolding` bytes (answer bytes lifted from unrecognised spans) wins; only
+then does mechanism list order decide.
+
+## Two semirings
+
+- **(min, +) tropical** — lightest derivation in `GraphSearch` via `src/derive`
+  (`lightestDerivation`). Cost accumulates with `+`, choice selects `min`.
+  Powers `cover`/`form`/`out`, edge following, fusing, and the A\* agenda
+  (`g + h`).
+
+- **(+, +) arithmetic** — evidence pooling in `src/mind/attention.ts:poolVotes`.
+  Each region's vote is an axiom; rules carry `Rule.combine = 'sum'` so costs to
+  the same anchor **add** rather than minimise. Powers IDF-weighted consensus,
+  `votes`/`votesIdf`/`support`, and `regionSupport`/`regionPeak`.
+
+## Admissibility
+
+The A\* heuristic is admissible and consistent:
+
+```
+h(it) = (queryLen - right) * MICRO
+```
+
+`right` is `p` for `cover(p)` or `j` for `form/out [i,j)`. `MICRO` is the
+minimum per-byte cost in the ladder (every real per-byte cost is `>= MICRO`,
+including `PASS`), and only the suffix past `right` is counted, so `h` never
+exceeds the true remaining cost.
+
+## Policy is not cost
+
+"Computation always wins" is **not** priced into the ladder (a computed result
+costs `STEP`, same as a learned edge). It is enforced by masking: `pipeline.ts`
+removes recognised sites overlapped by a `ComputedResult` so the computation is
+the sole completion there. Keep policy in callers; keep the engine neutral.
+
+## Pins
+
+- `test/52` — climb consensus instrumentation
+- `test/53` — cross-region probe instrumentation
+- `test/54` — evidence `k` instrumentation
+- `test/55` — cost meter (`Meter`, `CostReport`, `searchPops`/`searchPushes`)

package/docs/architecture/determinism.md

@@ -0,0 +1,73 @@
+# Determinism — Same Seed + Same Deposits + Same Query ⇒ Same Bytes
+
+## The law
+
+> Same `seed` + same deposit order + same query ⇒ byte-identical answer.
+
+Determinism is the product. Every code path that can reach output must be
+deterministic given `(seed, store contents, query bytes)`.
+
+## Forbidden
+
+No `Math.random` or `Date.now` in behaviour, and no iteration over unordered
+collections where order can reach output. Example-only uses
+(`example/train_base`) are outside the library contract. If a test becomes
+flaky, the contract was broken, not the test.
+
+## All randomness flows from `seed`
+
+`MindConfig.seed` (`src/config.ts:resolveConfig`, `DEFAULT_CONFIG`) is the sole
+entropy root. Subsystems derive deterministically:
+
+- **Alphabet** — `Alphabet` (`src/alphabet.ts`) via `rng` (`src/vec.ts:rng`)
+  seeded as `seed ^ seedMask`; builds 16→64→256 vectors by refinement.
+- **Keyring / Space** — `Space.seats` (`src/sema.ts:Space`) via `makeKeyring`
+  (`src/vec.ts:makeKeyring`) and `rng` seeded from `seed` in `Mind`
+  (`src/mind/mind.ts`); `fold`/`twoEndedSeat`/`companySignature` are pure over
+  `Space`.
+- **Vector indexes** — `VectorDatabase` (`src/rabitq-ivf/src/database.ts`) and
+  `Prng` (`src/rabitq-ivf/src/rabitq.ts`) seeded from config; insertion order is
+  the stored order, not a random choice.
+
+No other PRNG source may affect grounding. Thresholds in `geometry.ts` are
+derived from `D`/`W`/`N`, not sampled.
+
+## Tie-breaks are corpus-determined
+
+Every choice among equals bottoms out in a fixed ordering — insertion order or
+lowest node id. The universal no-evidence fallback is **first-inserted**:
+
+- `guidedFirst` (`src/mind/traverse.ts:guidedFirst`) — guided pick via
+  `chooseNext` else first-inserted edge (`nextFirst` LIMIT 1).
+- `chooseNext` (`src/mind/traverse.ts:chooseNext`) — capped `nextFirst` read
+  (`hubBound`), ranked by `prevCount` then `haloMass`; equal ⇒ first-inserted.
+- `chooseAmong` (`src/mind/traverse.ts:chooseAmong`) — `hubCap` + `argmaxCosine`
+  over `candidateGist`; first-inserted on tie via stable scan.
+- `companySignature` (`src/sema.ts:companySignature`) — `rng(id ^ 0x9e3779b9)`,
+  i.e. seeded by node id, not observation order.
+
+Last-inserted was once used in one place; it was a bug. Never reintroduce it.
+
+## Memoization and trace must not break identity
+
+Per-response memos (`Precomputed`, `perceiveMemo`, `recogniseMemo`, `climbMemo`,
+`_resolvedSubtrees` via `foldTree`, `_edgeChoice`, `_gistCache` in
+`src/mind/mind.ts` / `src/mind/pipeline-mechanism.ts` /
+`src/mind/primitives.ts`) are sound because asking never writes. Only
+`guidedNext`/`sharedReachMemo` are trace-bypassed;
+`perceiveMemo`/`recogniseMemo`/`climbMemo` are always consulted — `foldTree`'s
+subtree fast path skips `visit` (and thus site emission) for cached subtrees, so
+bypassing makes `recognise` non-idempotent.
+
+## Follow it
+
+When you add any choice among equals, name the tie-break explicitly and make it
+corpus-determined. Thread new randomness through `seed`-derived `rng`; never
+call `Math.random`/`Date.now` on a behavioural path.
+
+## Pins
+
+- `test/42` pins recognition idempotence under trace — traced and untraced
+  `recognise` must return the same cached object and site count.
+- Determinism suites — `test/03`, `test/04`, `test/08`, `test/20` and others
+  assert same seed + same training ⇒ byte-identical answers and stores.

package/docs/architecture/exact-vs-approximate.md

@@ -0,0 +1,47 @@
+# Exact vs Approximate — The Law and Its Five Ladders
+
+Vector scores (`resonate` / `resonateHalo`) are RaBitQ **estimates**. They rank
+candidates and gate broad regions; they never decide identity. Identity is
+decided only by content-addressed lookup — `resolve` / `findLeaf` / `findBranch`
+/ `canonResolve` — and by re-folding bytes to verify.
+
+## The law
+
+> Scores propose, bytes dispose.
+
+Even recall's echo decision re-folds the top hit's bytes rather than trusting
+the estimate it already has. No `score >= threshold` path may mint an identity
+claim; thresholds derived in `geometry.ts` gate search breadth, not truth.
+
+## Graded evidence ladders
+
+Five subsystems share one shape — **exact → distributional → geometric** — with
+earlier tiers strictly preferred. Never reorder tiers; never let an approximate
+tier override an exact one.
+
+| # | Site               | Ladder (strong → weak)                                                                                                   | File                                      |
+| - | ------------------ | ------------------------------------------------------------------------------------------------------------------------ | ----------------------------------------- |
+| 1 | `resolve`          | exact content-addressed fold → `canonResolve` (equivalence class, hash-then-verify)                                      | `mind/primitives.ts`                      |
+| 2 | `locate`           | exact bytes → halo role → gist                                                                                           | `mind/match.ts`                           |
+| 3 | `alignGraded`      | literal W-gram runs → halo-matched sites + climb proposals (weave)                                                       | `mind/match.ts` / `pipeline-mechanism.ts` |
+| 4 | `bridge`           | junction containers → edge → synonym → whole-gist                                                                        | `mind/resonance.ts`                       |
+| 5 | `crossRegionVotes` | exact containers → single synonym → double → `structuralResonance` (synthetic gist, gated hardest — no byte containment) | `mind/attention.ts`                       |
+
+## Asymmetries (attention)
+
+Two rules in `attention.ts` encode "exact decides" and must not be flattened:
+
+- Only the **EXACT** tier may explain ordinary votes away.
+- Only **container-backed** evidence may consume its endpoints.
+
+## Pins
+
+- `test/51` pins the cross-region tier ladder and its gating.
+- Recognition idempotence under trace (`test/42`) depends on exact identity
+  remaining byte-determined, not score-determined.
+
+## Adding a matcher
+
+Add a tier to the shared family in `mind/match.ts` with a derived gate
+(`geometry.ts`), never a private `score >= k` check. A new mechanism is a
+`(matcher, direction, gate)` configuration over that family (§2.5).

package/docs/architecture/factored-machinery.md

@@ -0,0 +1,28 @@
+# Factored Machinery — One Definition, Many Consumers
+
+Every shared operation is defined once and imported many times. Duplicating it
+forks the corpus contract; moving it hides who owns the gate.
+
+For the match → project → gate family see `match-project.md`; for the two
+commonality measures see `commonality.md`; for work accounting see `meter.md`.
+
+## Single-definition contracts
+
+| Symbol                                                        | Defined in              | One fact                                                                                                                                                                  |
+| ------------------------------------------------------------- | ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `contentLevels`                                               | `src/geometry.ts`       | Single boundary rule: cuts + levels from one rolling hash pass; every segmentation reads it.                                                                              |
+| `canonicalWindows` / `chainReach` / `leafIdRun` / `windowIds` | `src/mind/canonical.ts` | Write/read contract: training interns `W-1,W` windows, reading chains to `W²` and probes `W`-windows — drift silences recognition.                                        |
+| `junction.ts` + `WalkCache`                                   | `src/mind/junction.ts`  | Shared junction ascent (parents + containers) with bounded `√N·W` walk; `WalkCache` memoizes capped reads/parents/containers per response; bridge and attention share it. |
+| `joinWithBridge`                                              | `src/mind/resonance.ts` | One out-of-search assembly: `bridge(left,right)` or bare concat with `bridgeMiss` trace.                                                                                  |
+| `dismissedKnownContent`                                       | `src/mind/bridge.ts`    | Pure attestation: any unaccounted `W`-window that resolves as known content — shared gap guard for substitution and CAST.                                                 |
+| `sharedReachMemo`                                             | `src/mind/traverse.ts`  | One response-scoped `AncestorReach` memo (cleared on write and for traces); every `reachOf`/`edgeAncestors` consumer shares it.                                           |
+| `guidedFirst`                                                 | `src/mind/traverse.ts`  | Guided-or-first answer bytes: `guidedNext` else first-inserted edge (`LIMIT 1`).                                                                                          |
+| `leadsSomewhere`                                              | `src/mind/traverse.ts`  | Admission predicate: `hasNext` (cached) or `hasHalo`; sites that lead nowhere contribute no derivation.                                                                   |
+| `isChunk`                                                     | `src/sema.ts`           | `kids !== null && kids.every(k=>k.kids===null)` — smallest grouped unit; governs regions, seams, indexing.                                                                |
+| `twoEndedSeat`                                                | `src/sema.ts`           | One seat algebra: first half low seats, second half high seats; shared by perception, `fold`, and canonical folds.                                                        |
+
+## Pins
+
+- `test/47` — frame reading split (matcher vs gate vs inventory).
+- `test/50` — CAST analog / consensus floor (dismissed content, `MIN_WEAVE` /
+  `dominates` frame, `carriesFillers` refusal).

package/docs/architecture/fold-contract.md

@@ -0,0 +1,87 @@
+# Fold Contract — One Tree For The Same Bytes
+
+> **Law:** `perceiveDeposit` ≡ `perceive` — same bytes ⇒ same tree and same node
+> id. Deposit imposes nothing (no boundaries, no turn convention). Geometry
+> never sees conversation metadata.
+
+## The identity
+
+Perception is a pure function of the bytes. The deposit path and the inference
+path compute the same content-defined fold for the same input, so a trained
+context node and `resolve(query)` reach the same node. When the two sides
+disagreed, alignment went quadratic (measured 5.2M cells on a 476-byte context
+vs 0 when they agree) and cumulative contexts stopped resolving to what they
+were trained as.
+
+## Deposit imposes nothing
+
+No boundaries, no turn convention, nothing read out of the bytes. Conversational
+turn offsets are API metadata — they feed `ConversationState`, `answeredSpans`
+and `currentTurnStart`; the geometry never sees them. Passing turn boundaries
+into the fold is a correctness bug, not a tuning choice.
+
+## Boundaries vs reuse — two problems
+
+`contentFoldIncremental` and `stablePrefixFold` solve different problems;
+conflating them is what once put an imposed boundary set on the inference path.
+
+| Mechanism                | What it buys                                              | Cost / shape                                                                                    |
+| ------------------------ | --------------------------------------------------------- | ----------------------------------------------------------------------------------------------- |
+| `contentFoldIncremental` | Transparent segment reuse (cost only)                     | Imposes nothing; tree identical to the cold fold                                                |
+| `stablePrefixFold`       | Caller-supplied cuts left-nested for prefix-ROOT identity | One prefix-ROOT per cut becomes an identical subtree (and same node id) inside the grown stream |
+
+Both carry the same precondition: `prev` must be a fold of a byte-identical
+prefix — reuse is keyed on `[start,end)` offsets, which cannot witness byte
+agreement. A mismatched `prev` produced a wrong tree on 336 of 400 random
+streams. `perceiveDeposit` discharges this via the prefix bytes as cache key; a
+conversation advances only by append. A caller that cannot prove the prefix must
+pass no `prev`.
+
+## Identity must not depend on W or absolute offset
+
+`contentLevels` is the single boundary rule (rolling hash over a bounded
+window). Any grouping by index — stride, tile, fixed-arity row — reintroduces
+the grid's phase bug: `riverFold` groups `W`-ary from byte 0, so the same byte
+run is a different subtree at a different offset. The content-defined hash
+removes this: a change upstream moves only the cut it falls inside; downstream
+cuts and segments are unchanged (99.7% cuts preserved on real deposits after
+shifts of 1..7 bytes vs 14.3% for the grid). The `groupByLevel` above the
+segments splits by content level, not by count.
+
+## `contentLevels` is single source; `contentBoundaries` is projection
+
+`contentBoundaries(space, bytes)` is `contentLevels(space, bytes).cuts`. It once
+carried its own rolling-hash loop, which is how a write side and a read side
+drift without a type error. Levels are read from the hash the cut was accepted
+at — level `L` when `h` vanishes mod `W^(L+1)` — so level-`L` cuts nest inside
+level-`(L-1)` and expected span is `W^(L+1)` bytes.
+
+## Optional canonical capability
+
+`canonAdd`/`canonFind` (`src/store.ts` — `canonCount`/`eachContent`) is an
+optional backend capability. A backend may omit all four; resolution then has no
+equivalence fallback. The store never learns the equivalence — the canonicalizer
+(`Canon` in `src/canon.ts`, e.g. `textCanon`) is injected by the caller and
+every candidate is hash-then-verified (re-canonicalize stored bytes, compare). A
+hash collision costs a read, never a wrong id.
+
+## Cost of changing the cut distribution
+
+The cut rate, which bits are read, `minLen`/`maxLen`, and the forced cut at
+`maxLen` set the segment distribution every downstream mechanism is fitted to.
+Each has been changed experimentally and cost 5–21 tests (rate: 15–18, bits:
+19–21, normalized chunking: 5–6). `W-1` is the minimum (one window minus one);
+`seats.length` is the maximum (one flat node folds exactly one segment). The
+forced cut is load-bearing — relaxing it to reduce the current 32% forced rate
+looked like a tidying but broke the same suites. Re-measure the whole suite for
+any change here.
+
+## Pins
+
+- `test/59` — shift invariance floors (content-defined cuts preserved over
+  random binary and prose).
+- `test/63` — offset/W invariance and `contentLevels` distribution expectations.
+
+See:
+`src/geometry.ts:contentLevels`/`contentBoundaries`/`contentFoldIncremental`/`stablePrefixFold`;
+`AGENTS.md` bootloader invariants.

package/docs/architecture/halo-sketch.md

@@ -0,0 +1,99 @@
+# Halo & Sketch — Distributional Memory
+
+A node's **halo** is its distributional signature: the superposition of
+identity-bound company signatures poured from every episode it participated in.
+Its **gist** is the VSA fold of its own bytes — content, not company.
+
+## Two vectors per node, two indexes
+
+| Vector | Encodes                       | Index         | Query          |
+| ------ | ----------------------------- | ------------- | -------------- |
+| gist   | what the node _is made of_    | content index | `resonate`     |
+| halo   | what _company_ the node keeps | halo index    | `resonateHalo` |
+
+Both indexes are RaBitQ-IVF (`src/rabitq-ivf/`) — 1-bit ANN over the same
+vectors; scores are estimates, never identity. Halos are also persisted durably
+(see below).
+
+## Quantization — 2-bit on disk, float in session
+
+A halo is a superposition of quasi-orthogonal signatures, so coordinates are
+Gaussian. Storage exploits this:
+
+- **In-session accumulator** (`_haloExact` in `src/store.ts`): `Float32Array` —
+  exact, additive, incremented by `pourHalo`.
+- **Durable row** (`_dbUpsertHalo`): 2-bit Lloyd–Max quantizer — decision at
+  ±0.9816σ, levels at ±0.4528σ / ±1.5104σ, σ derived from the stored norm
+  (`norm/√D`). Header is the 4-byte norm; body is 2 bits/coordinate. Keeps ≥0.88
+  correlation with the exact vector.
+- **ANN index**: 1-bit RaBitQ, irreversible — answers only "which halos are near
+  this query?"
+
+Re-indexing is geometric: a halo re-enters the ANN when mass is small or crosses
+a power of two (`geometricMass`), so index writes are O(log mass).
+
+## Bottom-k sketch & company profile
+
+A whole-partner signature alone records tokens, not types — halos of genuine
+synonyms would be quasi-orthogonal. `companyProfile` (`src/mind/learning.ts`)
+superposes:
+
+1. the partner's own identity signature, plus
+2. the bottom-k **constituent sketch** — the
+   `k = profileCapacity(D) = floor(√D)` minimal units of its subtree with
+   smallest `unitPriority`, deduped.
+
+The sketch is composable (bottom-k of a union = bottom-k of children's
+sketches), durable derived state via `sketchGet`/`sketchPut`, and bounded: at
+most `k` constituents are classified, each by one `LIMIT`ed parent read
+(`hubBound`). Beyond `√D` terms a single constituent contributes less than
+`1/√D` — below RaBitQ noise — and extra terms shrink every accepted one; the cap
+is a correctness limit.
+
+## Gist vectors
+
+Folded by the river (`src/geometry.ts`): leaves are alphabet vectors, groups
+bind by two-ended seats, intermediate gists stay unnormalized (magnitude ∝
+√len), only the root is normalized. Gist resonance reads byte-proportional
+overlap; halo resonance reads distributional overlap — the two are independent.
+
+## Thresholds & gating
+
+All bars are derived in `src/geometry.ts`; no tunable constant:
+
+| Symbol             | Formula        | Use                                                                                               |
+| ------------------ | -------------- | ------------------------------------------------------------------------------------------------- |
+| `estimatorNoise`   | `1/√D`         | 1σ RaBitQ noise; contrastive margin must clear it                                                 |
+| `significanceBar`  | `3/√D`         | whole-query relatedness — 3σ above chance; gates consensus climb and `analogyStrength`            |
+| `conceptThreshold` | `0.5 + 0.5/√D` | halo concept sharing — structural midpoint + ½σ; gates `haloSiblings`, concept hops, articulation |
+
+The significance bar gates the whole query; `conceptThreshold` gates per-pair
+halo cosine.
+
+## Probes: `haloMass` and `hasHalo`
+
+- `haloMass(id)` — count of poured episodes; evidence weight, tie-breaker in
+  `chooseAmong`.
+- `hasHalo(id)` — existence probe (indexed point check, no vector decode);
+  mirrors `halo(id) !== null`. One tier of the `leadsSomewhere` admission
+  predicate (with `hasNext`/`hasParents`).
+
+Both are `meter`-counted probes, not full decodes — `halo(id)` is the bounded
+vector read; `resonateHalo` is the IVF ANN query.
+
+## Relation to invariants
+
+- **Derived thresholds** — all bars above live in `geometry.ts`.
+- **Exact decides / approximate proposes** — halo scores rank and gate; identity
+  is content-addressed. The graded ladder is exact → halo → gist
+  (`mind/match.ts`).
+- **Bounded reads** — `hasHalo`/`haloMass` are point probes; `resonateHalo` is
+  capped ANN; constituent classification uses `LIMIT hubBound+1` reads. No
+  per-query scan grows with corpus.
+
+## Pins
+
+- `test/08 storage halo` — halo persistence, 2-bit round-trip,
+  `haloMass`/`hasHalo` contract, index survival across reopen.
+- `test/35 ivf` — RaBitQ-IVF contract (recall vs brute force, sublinear
+  scaling); covers the halo index's own layer.