@hviana/sema 0.7.9 → 0.8.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +22 -1
- package/DATASETS.md +1 -1
- package/dist/example/train_base/config.js +2 -2
- package/dist/example/train_base/corpora/massive.js +1 -1
- package/dist/example/train_base/readers.js +1 -1
- package/dist/src/geometry.d.ts +10 -10
- package/dist/src/geometry.js +25 -24
- package/dist/src/meter.d.ts +4 -12
- package/dist/src/meter.js +14 -14
- package/dist/src/mind/attention.js +12 -12
- package/dist/src/mind/bridge.d.ts +8 -8
- package/dist/src/mind/bridge.js +33 -32
- package/dist/src/mind/graph-search.d.ts +0 -8
- package/dist/src/mind/graph-search.js +38 -25
- package/dist/src/mind/junction.d.ts +1 -1
- package/dist/src/mind/junction.js +8 -8
- package/dist/src/mind/learning.js +36 -35
- package/dist/src/mind/match.js +14 -13
- package/dist/src/mind/mechanisms/cover.js +13 -12
- package/dist/src/mind/mechanisms/prefix-completion.js +24 -24
- package/dist/src/mind/mechanisms/recall.js +38 -40
- package/dist/src/mind/mechanisms/reference.js +16 -16
- package/dist/src/mind/mind.d.ts +6 -7
- package/dist/src/mind/pipeline-mechanism.d.ts +10 -8
- package/dist/src/mind/pipeline-mechanism.js +25 -21
- package/dist/src/mind/pipeline.d.ts +9 -9
- package/dist/src/mind/pipeline.js +24 -23
- package/dist/src/mind/primitives.d.ts +5 -5
- package/dist/src/mind/primitives.js +5 -5
- package/dist/src/mind/recognition.d.ts +14 -13
- package/dist/src/mind/recognition.js +53 -38
- package/dist/src/mind/resonance.js +21 -21
- package/dist/src/mind/traverse.d.ts +54 -52
- package/dist/src/mind/traverse.js +74 -72
- package/dist/src/mind/types.d.ts +4 -4
- package/dist/src/store.d.ts +12 -12
- package/dist/src/store.js +12 -12
- package/docs/INDEX.md +2 -2
- package/docs/architecture/exact-vs-approximate.md +2 -1
- package/docs/architecture/fold-contract.md +1 -1
- package/docs/failures/tempting-but-wrong.md +2 -3
- package/docs/harness/gates.md +7 -7
- package/example/train_base/config.ts +2 -2
- package/example/train_base/corpora/massive.ts +1 -1
- package/example/train_base/readers.ts +1 -1
- package/jsr.json +1 -1
- package/package.json +1 -1
- package/src/geometry.ts +25 -24
- package/src/meter.ts +14 -14
- package/src/mind/attention.ts +12 -12
- package/src/mind/bridge.ts +33 -32
- package/src/mind/graph-search.ts +43 -24
- package/src/mind/junction.ts +8 -8
- package/src/mind/learning.ts +36 -35
- package/src/mind/match.ts +20 -19
- package/src/mind/mechanisms/cover.ts +13 -12
- package/src/mind/mechanisms/prefix-completion.ts +24 -24
- package/src/mind/mechanisms/recall.ts +38 -40
- package/src/mind/mechanisms/reference.ts +16 -16
- package/src/mind/mind.ts +6 -7
- package/src/mind/pipeline-mechanism.ts +25 -21
- package/src/mind/pipeline.ts +33 -32
- package/src/mind/primitives.ts +5 -5
- package/src/mind/recognition.ts +51 -36
- package/src/mind/resonance.ts +21 -21
- package/src/mind/traverse.ts +74 -72
- package/src/mind/types.ts +4 -4
- package/src/store.ts +20 -20
- package/test/08-storage.test.mjs +1 -1
- package/test/35-prefix-edge.test.mjs +1 -1
- package/test/40-choosenext-scale-guard.test.mjs +16 -17
- package/test/46-recognise-multibyte-edge.test.mjs +33 -0
- package/test/56-bridge-identity-admission.test.mjs +6 -6
- package/test/70-prefix-completion.test.mjs +4 -3
- package/test/72-prefix-candidate-supply.test.mjs +3 -3
- package/test/73-scaffolding-only-bridge-abstains.test.mjs +6 -6
- package/test/75-multiturn-context-optimisation.test.mjs +5 -5
- package/test/84-composed-answer-honesty.test.mjs +5 -6
- package/test/88-dependency-footprint.test.mjs +1 -1
- package/test/89-completion-recursion.test.mjs +17 -14
- package/test/90-connector-read-cap.test.mjs +10 -8
- package/test/93-regime-prediction.test.mjs +10 -10
- package/test/94-cross-region-budget.test.mjs +2 -2
- package/test/95-wide-resonance-removed.test.mjs +8 -7
- package/test/96-bytes-walk-termination.test.mjs +3 -3
- package/test/99-fact-join.test.mjs +38 -0
package/AGENTS.md
CHANGED
|
@@ -134,7 +134,28 @@ silence). A simplification that fails an existing test is wrong until the test
|
|
|
134
134
|
is proven wrong. Sublibraries test themselves in
|
|
135
135
|
`src/{alu,derive,rabitq-ivf}/test/` with zero Sema dependency.
|
|
136
136
|
|
|
137
|
-
## 6.
|
|
137
|
+
## 6. Instrumentation — the meter and the rationale ARE the dev surface
|
|
138
|
+
|
|
139
|
+
`src/meter.ts` (what an answer COST) and `inspectRationale` (why it was CHOSEN,
|
|
140
|
+
`src/mind/rationale.ts`) are not debug helpers: they are Sema's development
|
|
141
|
+
instrumentation, and the only ones. Both are read through the public path —
|
|
142
|
+
`new Mind({ profile: true })` → `mind.lastCost` (`sumReports`/`formatReport`),
|
|
143
|
+
and the `inspectRationale` callback on `respond`/`respondText`/`respondTurn`.
|
|
144
|
+
|
|
145
|
+
When a change needs to be seen, measured, or proved, EXTEND THEM: a counter in
|
|
146
|
+
`meter.ts` (the one place a counter name exists — keep its four contracts true),
|
|
147
|
+
a step or note where the mechanism emits it (`src/mind/trace.ts` holds the move
|
|
148
|
+
vocabulary). A gap in instrumentation is a defect IN the instrumentation: close
|
|
149
|
+
it there, once, so the next person sees it too. Never add a parallel channel for
|
|
150
|
+
a single investigation — no ad-hoc logging or timing probes left in `src/`
|
|
151
|
+
(`performance.now()` belongs in `meter.ts`, not at a call site), no private
|
|
152
|
+
per-layer counter where a `meter.ts` field belongs, and no trace channel of your
|
|
153
|
+
own: a callback threaded through a call chain must FEED the rationale, the way
|
|
154
|
+
`GraphSearch`'s `onDerivation` feeds `traceDerivation`. (`store.ts`'s
|
|
155
|
+
`danglingReads`/`compactFailures` and the `console.warn`s that report them
|
|
156
|
+
predate this and stay: session-lifetime HEALTH counters, not per-response work.)
|
|
157
|
+
|
|
158
|
+
## 7. Dependencies and licensing
|
|
138
159
|
|
|
139
160
|
PolyForm Noncommercial 1.0.0 with separate commercial licensing (see
|
|
140
161
|
`LICENSE.md`, `COMMERCIAL-LICENSE.md`, `TRADEMARKS.md`). The library has **no
|
package/DATASETS.md
CHANGED
|
@@ -32,7 +32,7 @@ Two consequences follow, and both are load-bearing:
|
|
|
32
32
|
2. **A corpus under a ShareAlike licence cannot enter the store**, because its
|
|
33
33
|
copyleft would attach to the distributed artifact.
|
|
34
34
|
|
|
35
|
-
Both rules are stated in [AGENTS.md](AGENTS.md) §
|
|
35
|
+
Both rules are stated in [AGENTS.md](AGENTS.md) §7 and must be checked before
|
|
36
36
|
any corpus is added to a trainer.
|
|
37
37
|
|
|
38
38
|
---
|
|
@@ -4,8 +4,8 @@
|
|
|
4
4
|
// the cache ceiling, the read budgets, the caps. A knob that describes ONE
|
|
5
5
|
// CORPUS (which pairs of SmolSent, how many SODA dialogues, how long an Aya
|
|
6
6
|
// field may be) belongs next to that corpus's adapter, together with the
|
|
7
|
-
// evidence that fixed its default —
|
|
8
|
-
// constraint
|
|
7
|
+
// evidence that fixed its default — a comment carries the constraint, and a
|
|
8
|
+
// constraint is only readable beside the code it constrains.
|
|
9
9
|
import { join } from "node:path";
|
|
10
10
|
/** Read an environment variable, or `d` when it is unset. */
|
|
11
11
|
export const env = (k, d) => process.env[k] ?? d;
|
|
@@ -34,7 +34,7 @@ import { convertedParquetUnits } from "./converted-parquet.js";
|
|
|
34
34
|
//
|
|
35
35
|
// So it displaces some wrong answers and manufactures others, INCLUDING turning
|
|
36
36
|
// a correct silence into a wrong answer — and honest silence is a stated
|
|
37
|
-
// property of this engine (
|
|
37
|
+
// property of this engine (INVARIANTS.md). On the mixed-curriculum store the
|
|
38
38
|
// same shape produced the fragment "nus" for "wake me up at nine am".
|
|
39
39
|
//
|
|
40
40
|
// That evidence is four probes on toy stores and is NOT conclusive; it is,
|
|
@@ -13,7 +13,7 @@
|
|
|
13
13
|
//
|
|
14
14
|
// THE ONLY THIRD-PARTY CODE IN THIS REPOSITORY IS BELOW, and it is LAZILY
|
|
15
15
|
// LOADED. Sema itself imports nothing outside `node:` — that is a product
|
|
16
|
-
// property, not an accident (AGENTS.md §
|
|
16
|
+
// property, not an accident (AGENTS.md §7) — and this trainer is an EXAMPLE,
|
|
17
17
|
// not part of the library. hyparquet (+ its Snappy codec) is therefore a dev
|
|
18
18
|
// dependency, and it is loaded by a dynamic import the first time a Parquet
|
|
19
19
|
// corpus is actually read: a curriculum with no Parquet stage (SmolSent,
|
package/dist/src/geometry.d.ts
CHANGED
|
@@ -140,16 +140,16 @@ export declare function knownPrefixLength(bytes: Uint8Array, leafAt: (i: number)
|
|
|
140
140
|
* correct boundary. Pass them through from `perceive`; the geometry
|
|
141
141
|
* computes the stable prefix internally.
|
|
142
142
|
*
|
|
143
|
-
* `boundaries` is the CALLER-computed stable-prefix boundary set
|
|
144
|
-
*
|
|
145
|
-
*
|
|
146
|
-
*
|
|
147
|
-
*
|
|
148
|
-
*
|
|
149
|
-
*
|
|
150
|
-
*
|
|
151
|
-
*
|
|
152
|
-
*
|
|
143
|
+
* `boundaries` is the CALLER-computed stable-prefix boundary set
|
|
144
|
+
* (fold-contract.md): strictly-increasing proper byte offsets, each the length
|
|
145
|
+
* of a prefix that is already a stored whole-stream form. When given, the fold
|
|
146
|
+
* splits into the segments between consecutive boundaries — each folded
|
|
147
|
+
* independently, exactly as it folded when it was learned — and the segment
|
|
148
|
+
* roots join LEFT-NESTED (((s₀·s₁)·s₂)…), so every learnt cumulative-context
|
|
149
|
+
* root reappears as an identical subtree (and, by hash-consing, the very same
|
|
150
|
+
* node) inside the grown stream. This is what lets a conversation's next turn
|
|
151
|
+
* extend perception instead of refolding it: identical prefixes produce
|
|
152
|
+
* identical subtrees regardless of what follows them. */
|
|
153
153
|
export declare function bytesToTree(space: Space, alphabet: Alphabet, bytes: Uint8Array, leafAt?: (i: number) => number | null, lookup?: (leafIds: number[]) => number | null, boundaries?: readonly number[]): Sema;
|
|
154
154
|
/** A plain content fold's reusable state: the level-0 cut edges over the whole
|
|
155
155
|
* stream and each segment's independently-folded root. See
|
package/dist/src/geometry.js
CHANGED
|
@@ -306,19 +306,20 @@ function bytesToLeaves(alphabet, bytes) {
|
|
|
306
306
|
* sentences fall would be importing an assumption the architecture rejects.
|
|
307
307
|
* Random binary must, and does, behave exactly like prose.
|
|
308
308
|
*
|
|
309
|
-
* Every constant is derived (
|
|
310
|
-
*
|
|
311
|
-
*
|
|
312
|
-
*
|
|
313
|
-
*
|
|
314
|
-
*
|
|
315
|
-
*
|
|
316
|
-
*
|
|
317
|
-
*
|
|
318
|
-
*
|
|
319
|
-
*
|
|
320
|
-
*
|
|
321
|
-
*
|
|
309
|
+
* Every constant is derived (thresholds.md): the cut mask is W, so a cut is
|
|
310
|
+
* offered once per quantum of bytes — which, composed with the minimum below,
|
|
311
|
+
* puts the expected segment at minLen + W − 1 ≈ 6 B rather than at W,
|
|
312
|
+
* deliberately (see the refutation recorded at `cutRate` in {@link
|
|
313
|
+
* contentLevels}: a segment is the flat PHRASE-scale unit the W-ary groups are
|
|
314
|
+
* built from, not a group of W children, and forcing E[len] = W costs 15
|
|
315
|
+
* tests). The minimum is W−1, `canonicalWindows`'s straddle neighbour and the
|
|
316
|
+
* write side's own floor for a unit; and the maximum is the KEYRING's seat
|
|
317
|
+
* count, because a segment folds as ONE flat node and `fold` has exactly that
|
|
318
|
+
* many seats to bind children into. Capping there is what keeps the fold light:
|
|
319
|
+
* a segment of 3..seats leaves is a single node, where splitting it into
|
|
320
|
+
* W-groups plus a remainder would cost two or three and the remainders barely
|
|
321
|
+
* share (measured: partial-arity nodes 504 → 3,590, and total distinct nodes
|
|
322
|
+
* 8,142 → 9,712, when segments folded as [W][rest]). */
|
|
322
323
|
/** {@link contentBoundaries} plus, for each cut, its LEVEL — how deep in the
|
|
323
324
|
* tree that cut reaches.
|
|
324
325
|
*
|
|
@@ -568,16 +569,16 @@ export function knownPrefixLength(bytes, leafAt, lookup) {
|
|
|
568
569
|
* correct boundary. Pass them through from `perceive`; the geometry
|
|
569
570
|
* computes the stable prefix internally.
|
|
570
571
|
*
|
|
571
|
-
* `boundaries` is the CALLER-computed stable-prefix boundary set
|
|
572
|
-
*
|
|
573
|
-
*
|
|
574
|
-
*
|
|
575
|
-
*
|
|
576
|
-
*
|
|
577
|
-
*
|
|
578
|
-
*
|
|
579
|
-
*
|
|
580
|
-
*
|
|
572
|
+
* `boundaries` is the CALLER-computed stable-prefix boundary set
|
|
573
|
+
* (fold-contract.md): strictly-increasing proper byte offsets, each the length
|
|
574
|
+
* of a prefix that is already a stored whole-stream form. When given, the fold
|
|
575
|
+
* splits into the segments between consecutive boundaries — each folded
|
|
576
|
+
* independently, exactly as it folded when it was learned — and the segment
|
|
577
|
+
* roots join LEFT-NESTED (((s₀·s₁)·s₂)…), so every learnt cumulative-context
|
|
578
|
+
* root reappears as an identical subtree (and, by hash-consing, the very same
|
|
579
|
+
* node) inside the grown stream. This is what lets a conversation's next turn
|
|
580
|
+
* extend perception instead of refolding it: identical prefixes produce
|
|
581
|
+
* identical subtrees regardless of what follows them. */
|
|
581
582
|
export function bytesToTree(space, alphabet, bytes, leafAt, lookup, boundaries) {
|
|
582
583
|
if (bytes.length === 0) {
|
|
583
584
|
return sema(alphabet.vecs[0], new Uint8Array(0), null);
|
|
@@ -868,7 +869,7 @@ function flatFold(space, alphabet, bytes, from, to) {
|
|
|
868
869
|
}
|
|
869
870
|
return { tree: sema(gist, null, kids), len: n };
|
|
870
871
|
}
|
|
871
|
-
|
|
872
|
+
/* * The stable-prefix segmented fold (fold-contract.md). Each segment between
|
|
872
873
|
* consecutive boundaries folds PLAINLY and independently; segment roots
|
|
873
874
|
* join left-nested, and only the final root is normalized (the linear-fold
|
|
874
875
|
* contract: one normalize per perception). A segment's own inner splits
|
package/dist/src/meter.d.ts
CHANGED
|
@@ -48,9 +48,6 @@ export declare class Meter {
|
|
|
48
48
|
nodeRecords: number;
|
|
49
49
|
/** `store.bytes` / `store.bytesPrefix` — one reconstruction request. */
|
|
50
50
|
byteReads: number;
|
|
51
|
-
/** Bytes actually handed back by those reads — the real I/O volume, and
|
|
52
|
-
* the number that exposes an unbounded read (AGENTS §2.8) that a call
|
|
53
|
-
* count alone hides. */
|
|
54
51
|
bytesRead: number;
|
|
55
52
|
/** `store.contentLen`. */
|
|
56
53
|
lenReads: number;
|
|
@@ -131,10 +128,10 @@ export declare class Meter {
|
|
|
131
128
|
junctionPops: number;
|
|
132
129
|
/** Ascents that ended by EXHAUSTING the expansion budget rather than by
|
|
133
130
|
* deciding — the walk abstained and the caller silently fell through to a
|
|
134
|
-
*
|
|
135
|
-
*
|
|
136
|
-
*
|
|
137
|
-
*
|
|
131
|
+
* lower tier of the ladder — honest degradation, and nothing else reports it
|
|
132
|
+
* (INVARIANTS.md). It rises the moment a SHARED budget is drained by an
|
|
133
|
+
* earlier walk, which is what makes "this tier answered nothing"
|
|
134
|
+
* distinguishable from "this tier never got to look". */
|
|
138
135
|
junctionBudgetExhausted: number;
|
|
139
136
|
/** Arbitrary byte spans whose distributional company was VSA-bundled from
|
|
140
137
|
* existing episode halos. */
|
|
@@ -163,11 +160,6 @@ export declare class Meter {
|
|
|
163
160
|
/** Charge `ms`, one call, and a counter delta to a named phase.
|
|
164
161
|
* Insertion-ordered, so a report reads in execution order. */
|
|
165
162
|
charge(phase: string, ms: number, delta?: Record<string, number>): void;
|
|
166
|
-
/** Time one SYNCHRONOUS phase. The sync/async seam (§2.10) is a real
|
|
167
|
-
* contract — perception, recognition and the graph search are synchronous —
|
|
168
|
-
* so a synchronous layer must not be wrapped in `time`'s promise just to be
|
|
169
|
-
* measured: that would make the profiled path await where the unprofiled
|
|
170
|
-
* one does not, and a meter never changes what a layer computes. */
|
|
171
163
|
timeSync<T>(phase: string, fn: () => T): T;
|
|
172
164
|
/** Time one async phase and attribute the work done inside it. Returns
|
|
173
165
|
* the awaited value untouched — a meter never changes what a layer
|
package/dist/src/meter.js
CHANGED
|
@@ -8,8 +8,8 @@
|
|
|
8
8
|
// Four contracts, all load-bearing:
|
|
9
9
|
//
|
|
10
10
|
// 1. NEVER READ BY INFERENCE. No counter may reach a decision, a threshold,
|
|
11
|
-
// or an ordering. Determinism (
|
|
12
|
-
// meter is write-only from the engine's point of view.
|
|
11
|
+
// or an ordering. Determinism (determinism.md) survives
|
|
12
|
+
// only because the meter is write-only from the engine's point of view.
|
|
13
13
|
// 2. OFF BY DEFAULT, AND FREE WHEN OFF. Every call site is `meter?.x++` on
|
|
14
14
|
// a null field. Nothing allocates, nothing is keyed, nothing is timed
|
|
15
15
|
// unless a Meter is attached (`new Mind({ profile: true })`).
|
|
@@ -33,9 +33,9 @@ export class Meter {
|
|
|
33
33
|
nodeRecords = 0;
|
|
34
34
|
/** `store.bytes` / `store.bytesPrefix` — one reconstruction request. */
|
|
35
35
|
byteReads = 0;
|
|
36
|
-
|
|
37
|
-
*
|
|
38
|
-
*
|
|
36
|
+
/* * Bytes actually handed back by those reads — the real I/O volume, and the
|
|
37
|
+
* number that exposes an unbounded read (bounded-reads.md) that a call count
|
|
38
|
+
* alone hides. */
|
|
39
39
|
bytesRead = 0;
|
|
40
40
|
/** `store.contentLen`. */
|
|
41
41
|
lenReads = 0;
|
|
@@ -124,10 +124,10 @@ export class Meter {
|
|
|
124
124
|
junctionPops = 0;
|
|
125
125
|
/** Ascents that ended by EXHAUSTING the expansion budget rather than by
|
|
126
126
|
* deciding — the walk abstained and the caller silently fell through to a
|
|
127
|
-
*
|
|
128
|
-
*
|
|
129
|
-
*
|
|
130
|
-
*
|
|
127
|
+
* lower tier of the ladder — honest degradation, and nothing else reports it
|
|
128
|
+
* (INVARIANTS.md). It rises the moment a SHARED budget is drained by an
|
|
129
|
+
* earlier walk, which is what makes "this tier answered nothing"
|
|
130
|
+
* distinguishable from "this tier never got to look". */
|
|
131
131
|
junctionBudgetExhausted = 0;
|
|
132
132
|
/** Arbitrary byte spans whose distributional company was VSA-bundled from
|
|
133
133
|
* existing episode halos. */
|
|
@@ -180,11 +180,11 @@ export class Meter {
|
|
|
180
180
|
}
|
|
181
181
|
}
|
|
182
182
|
}
|
|
183
|
-
|
|
184
|
-
*
|
|
185
|
-
*
|
|
186
|
-
*
|
|
187
|
-
*
|
|
183
|
+
/* * Time one SYNCHRONOUS phase. The sync/async seam is a real contract
|
|
184
|
+
* (meter.md) — perception, recognition and the graph search are synchronous —
|
|
185
|
+
* so a synchronous layer must not be wrapped in `time`'s promise just to be
|
|
186
|
+
* measured: that would make the profiled path await where the unprofiled one
|
|
187
|
+
* does not, and a meter never changes what a layer computes. */
|
|
188
188
|
timeSync(phase, fn) {
|
|
189
189
|
const before = this.snapshot();
|
|
190
190
|
const t = performance.now();
|
|
@@ -1364,10 +1364,10 @@ export function canonicalChunkId(ctx, regionBytes, N, reachMemo) {
|
|
|
1364
1364
|
// CAST lost a point of attention it needed (test/29 D1/D2).
|
|
1365
1365
|
//
|
|
1366
1366
|
// So scan every offset and prefer an anchor that still discriminates: not
|
|
1367
|
-
// saturated, and among those the one reaching the FEWEST contexts
|
|
1368
|
-
// corpus-global).
|
|
1369
|
-
// old generalising choice stand — there is then no
|
|
1370
|
-
// find, and abstaining is the honest outcome.
|
|
1367
|
+
// saturated, and among those the one reaching the FEWEST contexts
|
|
1368
|
+
// (commonality.md, corpus-global). Only when every window in the region
|
|
1369
|
+
// saturates does the old generalising choice stand — there is then no
|
|
1370
|
+
// discriminative anchor to find, and abstaining is the honest outcome.
|
|
1371
1371
|
let discId = null;
|
|
1372
1372
|
let discReached = Infinity;
|
|
1373
1373
|
let fallback = null;
|
|
@@ -1849,16 +1849,16 @@ async function crossRegionVotes(ctx, query, regions, rvs, k, N, reachMemo, td) {
|
|
|
1849
1849
|
const consumed = new Set();
|
|
1850
1850
|
let probes = 0;
|
|
1851
1851
|
// When atoms themselves are hubs (atomIsHub — a single byte reaches ≥ √N
|
|
1852
|
-
// contexts,
|
|
1853
|
-
// cross-region junction walks are dominated by the drift through
|
|
1854
|
-
// content's ancestry.
|
|
1855
|
-
// √N·W budget (profiled: 160,210 junction pops, 31% of think at
|
|
1856
|
-
//
|
|
1857
|
-
//
|
|
1852
|
+
// contexts, bounded-reads.md's own predicate), the corpus is large enough
|
|
1853
|
+
// that the cross-region junction walks are dominated by the drift through
|
|
1854
|
+
// common content's ancestry. Each of k candidate pairs otherwise spends its
|
|
1855
|
+
// own √N·W budget (profiled: 160,210 junction pops, 31% of think at N =
|
|
1856
|
+
// 325,608), and a cumulative dialogue multiplies bounded work into tens of
|
|
1857
|
+
// seconds. The structural walk is therefore given ONE k·W allowance per
|
|
1858
1858
|
// evidence tier, shared across every pair — k pairs × W phrase-scale levels,
|
|
1859
1859
|
// the minimal exact check; a pair whose container is not reached within it
|
|
1860
|
-
// falls through to the resonance tier (the ANN proposes what the shallow
|
|
1861
|
-
//
|
|
1860
|
+
// falls through to the resonance tier (the ANN proposes what the shallow walk
|
|
1861
|
+
// no longer exhaustively scans, exact-vs-approximate.md).
|
|
1862
1862
|
//
|
|
1863
1863
|
// Below atomIsHub the store is small and atoms still discriminate, so the
|
|
1864
1864
|
// walks keep exhaustive exact traversal (per-walk √N·W) — the shared budget
|
|
@@ -25,13 +25,13 @@ export declare function dismissedKnownContent(ctx: MindContext, query: Uint8Arra
|
|
|
25
25
|
/** Recall's corroborated-substitution bridge — see the module comment.
|
|
26
26
|
* Returns the best bridged grounding proposal, or null. */
|
|
27
27
|
/** `proposed` is a THUNK, not a list: the bridge's own cheap gates (the
|
|
28
|
-
* two-quantum query floor and the O(|query|) stored-window anchor scan)
|
|
29
|
-
*
|
|
30
|
-
*
|
|
31
|
-
*
|
|
32
|
-
*
|
|
33
|
-
*
|
|
34
|
-
*
|
|
35
|
-
*
|
|
28
|
+
* two-quantum query floor and the O(|query|) stored-window anchor scan) decide
|
|
29
|
+
* whether ANY candidate can be aligned, and they need no proposals to do it.
|
|
30
|
+
* Resolving the caller's proposals eagerly meant recall paid its exhaustive
|
|
31
|
+
* whole-index resonance — the most expensive single act on the refusal path —
|
|
32
|
+
* for every query, including the ones whose windows the store has never seen
|
|
33
|
+
* and which the anchor scan rejects outright. Same investment discipline the
|
|
34
|
+
* mechanism floors follow (mechanism-market.md): never compute a shared
|
|
35
|
+
* analysis just to discard it. */
|
|
36
36
|
export declare function substitutionBridge(ctx: MindContext, query: Uint8Array, proposed?: () => Promise<ReadonlyArray<number>>): Promise<BridgeHit | null>;
|
|
37
37
|
export {};
|
package/dist/src/mind/bridge.js
CHANGED
|
@@ -121,22 +121,22 @@ export function dismissedKnownContent(ctx, query, spans) {
|
|
|
121
121
|
}
|
|
122
122
|
return false;
|
|
123
123
|
}
|
|
124
|
-
// The seeded aligner this file used to own now lives in the shared match
|
|
125
|
-
//
|
|
126
|
-
//
|
|
127
|
-
//
|
|
124
|
+
// The seeded aligner this file used to own now lives in the shared match family
|
|
125
|
+
// as {@link alignAround} — the frame reading (match.ts) reads the same gaps and
|
|
126
|
+
// asks the OPPOSITE question of them (see AlignGap's own doc). Two consumers,
|
|
127
|
+
// one definition (factored-machinery.md); the bridge's reading is unchanged.
|
|
128
128
|
const align = alignAround;
|
|
129
129
|
/** Recall's corroborated-substitution bridge — see the module comment.
|
|
130
130
|
* Returns the best bridged grounding proposal, or null. */
|
|
131
131
|
/** `proposed` is a THUNK, not a list: the bridge's own cheap gates (the
|
|
132
|
-
* two-quantum query floor and the O(|query|) stored-window anchor scan)
|
|
133
|
-
*
|
|
134
|
-
*
|
|
135
|
-
*
|
|
136
|
-
*
|
|
137
|
-
*
|
|
138
|
-
*
|
|
139
|
-
*
|
|
132
|
+
* two-quantum query floor and the O(|query|) stored-window anchor scan) decide
|
|
133
|
+
* whether ANY candidate can be aligned, and they need no proposals to do it.
|
|
134
|
+
* Resolving the caller's proposals eagerly meant recall paid its exhaustive
|
|
135
|
+
* whole-index resonance — the most expensive single act on the refusal path —
|
|
136
|
+
* for every query, including the ones whose windows the store has never seen
|
|
137
|
+
* and which the anchor scan rejects outright. Same investment discipline the
|
|
138
|
+
* mechanism floors follow (mechanism-market.md): never compute a shared
|
|
139
|
+
* analysis just to discard it. */
|
|
140
140
|
export async function substitutionBridge(ctx, query, proposed = async () => []) {
|
|
141
141
|
const meter = ctx.meter;
|
|
142
142
|
return meter
|
|
@@ -238,12 +238,12 @@ async function bridgeImpl(ctx, query, proposed) {
|
|
|
238
238
|
ctx.trace?.step("substitutionBridge", [rItem(query, "query")], [], "no stored query window can anchor a corroborated substitution", undefined, diagnostics);
|
|
239
239
|
return null;
|
|
240
240
|
}
|
|
241
|
-
// NO DISCRIMINATING LITERAL EVIDENCE — abstain (
|
|
242
|
-
// through the literal spans it did NOT substitute; those anchors are
|
|
243
|
-
// whole of its evidence.
|
|
244
|
-
// clamped at the √N hub bound, i.e. the window is corpus-global
|
|
245
|
-
// — the query's unsubstituted part discriminates nothing, and the
|
|
246
|
-
// substituted span is carrying the entire semantic load.
|
|
241
|
+
// NO DISCRIMINATING LITERAL EVIDENCE — abstain (INVARIANTS.md). A bridge
|
|
242
|
+
// grounds through the literal spans it did NOT substitute; those anchors are
|
|
243
|
+
// the whole of its evidence. When every one of them is SATURATED —
|
|
244
|
+
// containment clamped at the √N hub bound, i.e. the window is corpus-global
|
|
245
|
+
// scaffolding — the query's unsubstituted part discriminates nothing, and the
|
|
246
|
+
// single substituted span is carrying the entire semantic load. That is not a
|
|
247
247
|
// corroborated bridge; it is a template match, and it FABRICATES.
|
|
248
248
|
//
|
|
249
249
|
// Measured on the trained store (hubBound 571). "What is the capital of"
|
|
@@ -258,7 +258,8 @@ async function bridgeImpl(ctx, query, proposed) {
|
|
|
258
258
|
// them silent and cannot be credited for them.
|
|
259
259
|
//
|
|
260
260
|
// This introduces NO new threshold: `bound` is the same √N reading of "hub"
|
|
261
|
-
// the anchor scan already clamps its own containment read to (
|
|
261
|
+
// the anchor scan already clamps its own containment read to (thresholds.md,
|
|
262
|
+
// commonality.md).
|
|
262
263
|
if (allWindowsAreScaffolding(ctx, query)) {
|
|
263
264
|
ctx.trace?.step("substitutionBridge", [rItem(query, "query")], [], "every query window that could anchor is corpus-global scaffolding — " +
|
|
264
265
|
"no literal evidence to corroborate a substitution", undefined, diagnostics);
|
|
@@ -297,8 +298,8 @@ async function bridgeImpl(ctx, query, proposed) {
|
|
|
297
298
|
//
|
|
298
299
|
// The question every gap poses is "may the two forms differ HERE without
|
|
299
300
|
// differing in what they SAY?", and that is the discriminative-vs-
|
|
300
|
-
// scaffolding question
|
|
301
|
-
// population.
|
|
301
|
+
// scaffolding question commonality.md names, over the CORPUS-GLOBAL
|
|
302
|
+
// population. It already has one definition — `dominates(reachOf(...), N)`,
|
|
302
303
|
// the same gate confluence's filler test uses ("scaffolding never binds").
|
|
303
304
|
// Nothing new is derived here; the bar is read, not invented.
|
|
304
305
|
//
|
|
@@ -310,17 +311,17 @@ async function bridgeImpl(ctx, query, proposed) {
|
|
|
310
311
|
// climb's own definition of non-discriminative), or it resolves to a
|
|
311
312
|
// majority of the corpus's contexts. "the process of ", " is the ".
|
|
312
313
|
//
|
|
313
|
-
// THE READING MATTERS, not just the population
|
|
314
|
-
// deliberately does NOT go through `reachOf`, which maps BOTH "saturated"
|
|
315
|
-
//
|
|
316
|
-
//
|
|
317
|
-
//
|
|
318
|
-
//
|
|
319
|
-
//
|
|
320
|
-
//
|
|
321
|
-
//
|
|
322
|
-
//
|
|
323
|
-
//
|
|
314
|
+
// THE READING MATTERS, not just the population — see commonality.md. This
|
|
315
|
+
// deliberately does NOT go through `reachOf`, which maps BOTH "saturated" and
|
|
316
|
+
// "reaches nothing" to Infinity. For IDF weighting those are the same thing
|
|
317
|
+
// (no usable identity evidence); for THIS question they are opposites — a
|
|
318
|
+
// window reaching nothing is novel content, the most discriminative material
|
|
319
|
+
// there is, and reading it as Infinity would call it scaffolding. Measured:
|
|
320
|
+
// with `reachOf`, "Is water wet?" was answered with "No, heavy water is not
|
|
321
|
+
// wet." — "heav"/"eavy" occur once, reach no edge-bearing ancestor, and were
|
|
322
|
+
// written off as filler. So an empty-rooted window is NEVER explained, and
|
|
323
|
+
// neither is an untrained one (the same principle attestedQ applies to the
|
|
324
|
+
// query side).
|
|
324
325
|
const reachMemo = sharedReachMemo(ctx);
|
|
325
326
|
const explainedSpan = (bytes, from, to) => {
|
|
326
327
|
if (to - from < W)
|
|
@@ -162,14 +162,6 @@ export declare class GraphSearch {
|
|
|
162
162
|
* recursive completion), and chooseNext (distributional-evidence edge
|
|
163
163
|
* disambiguation when a recognised form has multiple continuations). */
|
|
164
164
|
host: GraphSearchHost);
|
|
165
|
-
/** The hub bound √N (AGENTS §2.8) — the ONE fan-out cap, stated here
|
|
166
|
-
* rather than imported from `traverse.ts` because this module is
|
|
167
|
-
* deliberately host-based (it holds a bare Store, never a MindContext).
|
|
168
|
-
* That is the same write/read-side duplication convention canonical.ts's
|
|
169
|
-
* header documents: if the formula changes it must change in BOTH places.
|
|
170
|
-
* It is stated ONCE per side, though — the expression used to be spelled
|
|
171
|
-
* out at three call sites here, one of them inside a per-item rules
|
|
172
|
-
* generator, and they had already drifted on the `Math.max(2, …)` floor. */
|
|
173
165
|
private hubBound;
|
|
174
166
|
/** Explore the Sema graph for the lightest cover of the query and return its
|
|
175
167
|
* chosen spans left-to-right — WITH the derivation's total weight (the g
|
|
@@ -16,7 +16,7 @@
|
|
|
16
16
|
// So this engine stays decoupled: it knows the graph's shape, never how the
|
|
17
17
|
// graph was learnt.
|
|
18
18
|
import { lightestDerivation, } from "../derive/src/index.js";
|
|
19
|
-
import { bytesEqual, concat2, concatBytes, latin1 } from "../bytes.js";
|
|
19
|
+
import { bytesEqual, concat2, concatBytes, indexOf, latin1 } from "../bytes.js";
|
|
20
20
|
import { ALL } from "./types.js";
|
|
21
21
|
// The cost ladder is a strict ORDERING, not tuned magic:
|
|
22
22
|
// • Coverage dominates everything: leaving one query byte unrecognised (PASS)
|
|
@@ -194,9 +194,10 @@ export class GraphSearch {
|
|
|
194
194
|
this.maxGroup = maxGroup;
|
|
195
195
|
this.host = host;
|
|
196
196
|
}
|
|
197
|
-
|
|
198
|
-
* rather than imported from `traverse.ts` because
|
|
199
|
-
* deliberately host-based (it holds a bare Store, never a
|
|
197
|
+
/* * The hub bound √N (bounded-reads.md) — the ONE
|
|
198
|
+
* fan-out cap, stated here rather than imported from `traverse.ts` because
|
|
199
|
+
* this module is deliberately host-based (it holds a bare Store, never a
|
|
200
|
+
* MindContext).
|
|
200
201
|
* That is the same write/read-side duplication convention canonical.ts's
|
|
201
202
|
* header documents: if the formula changes it must change in BOTH places.
|
|
202
203
|
* It is stated ONCE per side, though — the expression used to be spelled
|
|
@@ -779,12 +780,12 @@ export class GraphSearch {
|
|
|
779
780
|
if (memo.has(node))
|
|
780
781
|
return memo.get(node) ?? null;
|
|
781
782
|
// Re-covering is how a PRODUCED node's bytes enter the search at all: the
|
|
782
|
-
// cover machinery otherwise only ever sees the QUERY's spans.
|
|
783
|
+
// cover machinery otherwise only ever sees the QUERY's spans. The recursion
|
|
783
784
|
// is allowed to nest — a chain IS nested completions — but it is bounded so
|
|
784
|
-
// the work stays the ANSWER's (
|
|
785
|
-
// guard, only ACCEPTED completions recurse, and the nested solve
|
|
786
|
-
// the form by its own shape instead of re-recognising the
|
|
787
|
-
// inside it.
|
|
785
|
+
// the work stays the ANSWER's (bounded-reads.md): the stack below is the
|
|
786
|
+
// cycle guard, only ACCEPTED completions recurse, and the nested solve
|
|
787
|
+
// decomposes the form by its own shape instead of re-recognising the
|
|
788
|
+
// corpus's hub forms inside it.
|
|
788
789
|
//
|
|
789
790
|
// `recompleteOpen` IS the stack of the chain being built, so MEMBERSHIP is
|
|
790
791
|
// the cycle guard: a node already open on this chain cannot re-enter it.
|
|
@@ -894,22 +895,34 @@ export class GraphSearch {
|
|
|
894
895
|
// The entity candidates are the forms the fact's own bytes CONTAIN — the
|
|
895
896
|
// same recogniser the query went through, so the evidence standard is the
|
|
896
897
|
// query's. A byte atom is never a subject; the fact's own node is the span
|
|
897
|
-
// itself, not an entity inside it.
|
|
898
|
-
|
|
899
|
-
|
|
900
|
-
|
|
901
|
-
|
|
902
|
-
|
|
903
|
-
|
|
904
|
-
|
|
905
|
-
|
|
906
|
-
|
|
907
|
-
|
|
908
|
-
|
|
909
|
-
|
|
910
|
-
|
|
911
|
-
|
|
912
|
-
|
|
898
|
+
// itself, not an entity inside it. The admission predicate has ONE
|
|
899
|
+
// definition — `traverse.ts`'s `leadsSomewhere` (edge or halo); the host
|
|
900
|
+
// LENDS it when it can (Mind does, with the response-scoped struct cache),
|
|
901
|
+
// and a bare host falls back to the raw-store probe, so the search stays
|
|
902
|
+
// host-based.
|
|
903
|
+
const leading = this.host.recogniseSpan(fact.bytes).sites.filter((s) => s.payload >= 0 && s.payload !== fact.node &&
|
|
904
|
+
(this.host.leadsSomewhere !== undefined
|
|
905
|
+
? this.host.leadsSomewhere(s.payload)
|
|
906
|
+
: this.store.hasNext(s.payload) || this.store.hasHalo(s.payload)));
|
|
907
|
+
// …then prefer the entity the query did NOT name, and the MAXIMAL one. The
|
|
908
|
+
// join exists to reach the subject the query never wrote, so:
|
|
909
|
+
// • a candidate the query already contains is the query's OWN subject, and
|
|
910
|
+
// following its fact answers about that subject instead of the inferred
|
|
911
|
+
// one (measured: "Gustaf Molander spouse date of death" gave Gustaf's
|
|
912
|
+
// date of death, not his spouse's);
|
|
913
|
+
// • a candidate contained in a longer one is not the entity the fact
|
|
914
|
+
// introduces ("Timur" must not win over "Timur Bekmambetov").
|
|
915
|
+
// Byte work over bytes already read, and the pruning REMOVES the
|
|
916
|
+
// resolve()/nextFirst() probes these candidates would have paid.
|
|
917
|
+
const candidates = leading
|
|
918
|
+
.map((s) => ({
|
|
919
|
+
payload: s.payload,
|
|
920
|
+
bytes: this.store.bytesPrefix(s.payload, ALL),
|
|
921
|
+
}))
|
|
922
|
+
.filter((c) => indexOf(queryBytes, c.bytes, 0) < 0)
|
|
923
|
+
.filter((c, _i, all) => !all.some((o) => o.bytes.length > c.bytes.length && indexOf(o.bytes, c.bytes, 0) >= 0));
|
|
924
|
+
for (const c of candidates) {
|
|
925
|
+
const key = this.host.resolve(concat2(c.bytes, tail));
|
|
913
926
|
if (key === null)
|
|
914
927
|
continue;
|
|
915
928
|
const nx = this.store.nextFirst(key, 1);
|
|
@@ -97,7 +97,7 @@ export declare function cachedRead(ctx: MindContext, cache: WalkCache | null, id
|
|
|
97
97
|
* is legitimately reached across many containing structures. Half the
|
|
98
98
|
* successful junctions would be lost.
|
|
99
99
|
*
|
|
100
|
-
*
|
|
100
|
+
* REFUTED EARLY-STOP (side-cone exhaustion, saturation.md's "real saturation"):
|
|
101
101
|
* stopping the walk the moment ONE side's upward cone is emptied is wrong,
|
|
102
102
|
* in both a hub-guarded form and a hub-flagged form. The junction test is
|
|
103
103
|
* a BYTE containment over the UNION of the two cones, and a junction can be
|
|
@@ -135,7 +135,7 @@ function cachedContainers(ctx, cache, id, limit) {
|
|
|
135
135
|
* is legitimately reached across many containing structures. Half the
|
|
136
136
|
* successful junctions would be lost.
|
|
137
137
|
*
|
|
138
|
-
*
|
|
138
|
+
* REFUTED EARLY-STOP (side-cone exhaustion, saturation.md's "real saturation"):
|
|
139
139
|
* stopping the walk the moment ONE side's upward cone is emptied is wrong,
|
|
140
140
|
* in both a hub-guarded form and a hub-flagged form. The junction test is
|
|
141
141
|
* a BYTE containment over the UNION of the two cones, and a junction can be
|
|
@@ -186,13 +186,13 @@ unordered = false) {
|
|
|
186
186
|
d: 0,
|
|
187
187
|
}));
|
|
188
188
|
while (stack.length > 0 && out.length < bound) {
|
|
189
|
-
// BUDGET EXHAUSTION IS AN ABSTENTION, AND IT MUST BE VISIBLE
|
|
190
|
-
// walk stops with work still on the stack, the caller
|
|
191
|
-
// and falls through to a lower ladder rung —
|
|
192
|
-
// outside, from a walk that looked everywhere
|
|
193
|
-
// SHARED budget (cross-region's one k·W allowance
|
|
194
|
-
// pair can drain it, so a later pair's exact tier may
|
|
195
|
-
// this counter is the only thing that says so.
|
|
189
|
+
// BUDGET EXHAUSTION IS AN ABSTENTION, AND IT MUST BE VISIBLE
|
|
190
|
+
// (INVARIANTS.md). The walk stops with work still on the stack, the caller
|
|
191
|
+
// reads "no container" and falls through to a lower ladder rung —
|
|
192
|
+
// indistinguishable, from the outside, from a walk that looked everywhere
|
|
193
|
+
// and found nothing. With a SHARED budget (cross-region's one k·W allowance
|
|
194
|
+
// per tier) an EARLIER pair can drain it, so a later pair's exact tier may
|
|
195
|
+
// never run at all; this counter is the only thing that says so.
|
|
196
196
|
if (b.n-- <= 0) {
|
|
197
197
|
if (ctx.meter)
|
|
198
198
|
ctx.meter.junctionBudgetExhausted++;
|