@cavelang/store 0.34.0 → 0.36.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +884 -15
- package/dist/src/backup.d.ts.map +1 -1
- package/dist/src/backup.js +143 -70
- package/dist/src/backup.js.map +1 -1
- package/dist/src/capture-claim.d.ts +4 -0
- package/dist/src/capture-claim.d.ts.map +1 -0
- package/dist/src/capture-claim.js +26 -0
- package/dist/src/capture-claim.js.map +1 -0
- package/dist/src/descriptor.d.ts +3 -0
- package/dist/src/descriptor.d.ts.map +1 -0
- package/dist/src/descriptor.js +21 -0
- package/dist/src/descriptor.js.map +1 -0
- package/dist/src/error-message.d.ts +3 -0
- package/dist/src/error-message.d.ts.map +1 -0
- package/dist/src/error-message.js +10 -0
- package/dist/src/error-message.js.map +1 -0
- package/dist/src/index.d.ts +1 -1
- package/dist/src/index.d.ts.map +1 -1
- package/dist/src/locate.d.ts +13 -1
- package/dist/src/locate.d.ts.map +1 -1
- package/dist/src/locate.js +35 -13
- package/dist/src/locate.js.map +1 -1
- package/dist/src/node-adapter.d.ts.map +1 -1
- package/dist/src/node-adapter.js +29 -1
- package/dist/src/node-adapter.js.map +1 -1
- package/dist/src/provenance.d.ts +4 -0
- package/dist/src/provenance.d.ts.map +1 -1
- package/dist/src/provenance.js +36 -1
- package/dist/src/provenance.js.map +1 -1
- package/dist/src/query-sql.d.ts.map +1 -1
- package/dist/src/query-sql.js +4 -2
- package/dist/src/query-sql.js.map +1 -1
- package/dist/src/read-snapshot.d.ts +4 -0
- package/dist/src/read-snapshot.d.ts.map +1 -0
- package/dist/src/read-snapshot.js +22 -0
- package/dist/src/read-snapshot.js.map +1 -0
- package/dist/src/record.d.ts.map +1 -1
- package/dist/src/record.js +102 -25
- package/dist/src/record.js.map +1 -1
- package/dist/src/resolve.d.ts.map +1 -1
- package/dist/src/resolve.js +42 -18
- package/dist/src/resolve.js.map +1 -1
- package/dist/src/row.d.ts.map +1 -1
- package/dist/src/row.js +45 -17
- package/dist/src/row.js.map +1 -1
- package/dist/src/runtime.d.ts.map +1 -1
- package/dist/src/runtime.js +8 -4
- package/dist/src/runtime.js.map +1 -1
- package/dist/src/schema.d.ts +1 -2
- package/dist/src/schema.d.ts.map +1 -1
- package/dist/src/schema.js +116 -15
- package/dist/src/schema.js.map +1 -1
- package/dist/src/store.d.ts +17 -5
- package/dist/src/store.d.ts.map +1 -1
- package/dist/src/store.js +451 -104
- package/dist/src/store.js.map +1 -1
- package/package.json +4 -4
- package/src/backup.ts +141 -65
- package/src/capture-claim.ts +30 -0
- package/src/descriptor.ts +19 -0
- package/src/error-message.ts +8 -0
- package/src/index.ts +1 -1
- package/src/locate.ts +45 -13
- package/src/node-adapter.ts +28 -1
- package/src/provenance.ts +33 -1
- package/src/query-sql.ts +4 -2
- package/src/read-snapshot.ts +21 -0
- package/src/record.ts +102 -26
- package/src/resolve.ts +43 -20
- package/src/row.ts +29 -2
- package/src/runtime.ts +8 -4
- package/src/schema.ts +117 -18
- package/src/store.ts +452 -125
package/README.md
CHANGED
|
@@ -20,6 +20,47 @@ store.exportText({ current: true }) // canonical text through internal
|
|
|
20
20
|
store.exportText({ maxSensitivity: 'restricted' }) // complete portable history
|
|
21
21
|
```
|
|
22
22
|
|
|
23
|
+
The native runtime requires Node 24.16.0+ within 24.x or 26.1.0+ within
|
|
24
|
+
26.x. Older bindings, including Node 22, truncate returned TEXT at embedded
|
|
25
|
+
NUL characters even though SQLite retains all stored bytes. The adapter checks
|
|
26
|
+
text decoding once before opening any caller database and rejects a truncating
|
|
27
|
+
runtime with an upgrade message. It neither strips characters nor rewrites
|
|
28
|
+
stored identities. `node scripts/check-sqlite-text.mjs` isolates the native
|
|
29
|
+
round-trip independently of CAVE.
|
|
30
|
+
Probe verification is cached only after both decoding and probe close succeed.
|
|
31
|
+
A failed attempt leaves caller database paths unopened and runs the probe again
|
|
32
|
+
on the next attempt. Simultaneous probe-read and close failures are retained in
|
|
33
|
+
`AggregateError`, with the read error as cause and first entry; a close-only
|
|
34
|
+
failure propagates directly. Each created probe receives one close attempt.
|
|
35
|
+
|
|
36
|
+
Claim appends require well-formed Unicode strings. `ingest` and `insertResult`
|
|
37
|
+
reject an unpaired UTF-16 surrogate with `TypeError` rather than letting SQLite's
|
|
38
|
+
UTF-8 conversion silently replace it with `�`. The batch preflight checks claim
|
|
39
|
+
columns, raw text, contexts, tags and explicit provenance before allocating or
|
|
40
|
+
observing transaction IDs; rows and vocabulary remain unchanged on rejection.
|
|
41
|
+
This storage check also applies to lenient ingestion and the browser adapter.
|
|
42
|
+
Valid surrogate pairs (including emoji), accented text and embedded NUL remain
|
|
43
|
+
supported. Direct SQL through the exposed database handle is outside this
|
|
44
|
+
claim-append validation path.
|
|
45
|
+
|
|
46
|
+
Direct search and lookup arguments also reject unpaired surrogates with
|
|
47
|
+
`TypeError` before SQLite binding. This covers entity and alias traversal,
|
|
48
|
+
topics, tags, contexts, provenance and belief-key lookups, as well as provenance
|
|
49
|
+
and edge reads by ID. Invalid text cannot silently select a record containing
|
|
50
|
+
`�`; explicitly authored replacement characters remain valid lookup text.
|
|
51
|
+
Search validates malformed Unicode even with a zero result limit or raw full-text
|
|
52
|
+
syntax. Rejected reads leave history unchanged and subsequent valid reads work
|
|
53
|
+
on both native and browser adapters. Direct SQL remains outside this validation.
|
|
54
|
+
|
|
55
|
+
The upstream byte-length conversion is in
|
|
56
|
+
[Node PR #61954](https://github.com/nodejs/node/pull/61954), shipped in
|
|
57
|
+
[Node 24.16.0](https://nodejs.org/en/blog/release/v24.16.0) and
|
|
58
|
+
[Node 26.1.0](https://nodejs.org/en/blog/release/v26.1.0). Boundary-runtime
|
|
59
|
+
checks reproduce truncation on 24.0.0 and 26.0.0 and preserve complete text on
|
|
60
|
+
24.16.0 and 26.1.0. A general SQL rewriting adapter was not introduced: it would
|
|
61
|
+
need to preserve arbitrary result types, duplicate column names and statement
|
|
62
|
+
semantics to avoid creating a second compatibility problem.
|
|
63
|
+
|
|
23
64
|
Runtime selection is explicit at non-Node composition boundaries:
|
|
24
65
|
|
|
25
66
|
```ts
|
|
@@ -29,11 +70,73 @@ declare const adapter: SqliteAdapter
|
|
|
29
70
|
const store = openWith(adapter, ':memory:')
|
|
30
71
|
```
|
|
31
72
|
|
|
73
|
+
Adapter result columns remain own data properties even when their names are
|
|
74
|
+
`__proto__`, `constructor` or `toString`. Repeated output names retain the last
|
|
75
|
+
column value. The shared native/WASM contract checks `get`, `all` and JSON
|
|
76
|
+
serialization so adapter composition does not silently lose named results.
|
|
77
|
+
|
|
32
78
|
`SqliteAdapter` declares the SQL statement surface plus immediate/savepoint
|
|
33
79
|
transactions, FTS4 or FTS5, optional extension loading, and optional exact
|
|
34
80
|
snapshot support. The concrete Node adapter is also available from
|
|
35
81
|
`@cavelang/store/adapter/node`. The shared adapter contract suite runs against
|
|
36
82
|
both Node SQLite and the website's SQL.js/WASM adapter.
|
|
83
|
+
Statement `run()` discards result rows and completes the statement before
|
|
84
|
+
reporting `changes` and `lastInsertRowid`, including writes with a `RETURNING`
|
|
85
|
+
clause. Use `all()` or `get()` when the returned rows themselves are needed.
|
|
86
|
+
Each statement call supplies fresh positional bindings: omitted parameters are
|
|
87
|
+
`NULL`, and excess parameters fail before a write executes. The shared contract
|
|
88
|
+
checks reuse after failure and preserves empty and binary blobs independently
|
|
89
|
+
of later changes to the caller's input buffer.
|
|
90
|
+
|
|
91
|
+
## Resource lifecycle
|
|
92
|
+
|
|
93
|
+
`ingest.strict` must be a boolean when supplied and is captured before parsing;
|
|
94
|
+
malformed flags cannot fall back to a partial, lenient import. The shared append
|
|
95
|
+
`lifecycle` flag on `ingest` and `insertResult` likewise requires a boolean,
|
|
96
|
+
captured before claims are appended. Omitted flags remain false.
|
|
97
|
+
|
|
98
|
+
Append operations capture replay IDs, contexts, source/lifecycle settings and
|
|
99
|
+
provenance fields before preparing a batch, copying their arrays. Changing
|
|
100
|
+
getters cannot substitute an unvalidated replay ID or change metadata between
|
|
101
|
+
claims in that batch.
|
|
102
|
+
Identity allocation and row insertion iterate the prepared claims, so a changing
|
|
103
|
+
claim-collection getter cannot replace or discard the batch after preparation.
|
|
104
|
+
Canonical-result edges also capture their parent index, role and child index
|
|
105
|
+
before insertion, so replay duplicate checks and writes use the same fields.
|
|
106
|
+
Claims capture their declared fields and nested terms, values, contexts and tags
|
|
107
|
+
before storage projections are derived. Columns, emitted text and semantic keys
|
|
108
|
+
therefore use the same values even when structured input contains getters.
|
|
109
|
+
|
|
110
|
+
Combined operation/cleanup errors retain each underlying diagnostic in their
|
|
111
|
+
message as well as in `AggregateError.errors`. This applies to transaction
|
|
112
|
+
rollback, final store cleanup, file descriptors, native text probes and backup
|
|
113
|
+
verification/removal. Backup cleanup errors retain the temporary path and any
|
|
114
|
+
already-published destination status. Existing causes and lone-error identities
|
|
115
|
+
are preserved, so message-only CLI output and programmatic inspection both carry
|
|
116
|
+
the failure details. If a thrown value cannot be converted to text, its message
|
|
117
|
+
uses `[unprintable thrown value]`; the original value remains available in
|
|
118
|
+
`AggregateError.errors`.
|
|
119
|
+
|
|
120
|
+
Call `store.close()` when finished. Components that retain resources derived
|
|
121
|
+
from a store can register synchronous cleanup with `store.onClose(callback)`;
|
|
122
|
+
the returned function unregisters only that registration. Registering the same
|
|
123
|
+
callback more than once creates independent subscriptions; calling one unsubscribe
|
|
124
|
+
function repeatedly is harmless. Cleanup runs before the SQLite connection closes, and reentrant or repeated successful closes do not repeat it.
|
|
125
|
+
Registering cleanup during or after a successful close throws. Returning a
|
|
126
|
+
Promise or thenable reports a synchronous-cleanup error; it is not awaited.
|
|
127
|
+
Native Promise rejection is observed without invoking a custom thenable's work,
|
|
128
|
+
and remaining cleanup callbacks and the database close are still attempted.
|
|
129
|
+
This cannot cancel work already started by the callback; callers must finish
|
|
130
|
+
asynchronous work before closing the store.
|
|
131
|
+
|
|
132
|
+
Every registered callback and the database close are attempted even if cleanup
|
|
133
|
+
throws. A single error is rethrown; multiple errors are reported in an
|
|
134
|
+
`AggregateError`. If the database close itself fails, another `close()` retries
|
|
135
|
+
it without repeating callbacks that were already attempted. Closing the raw
|
|
136
|
+
`store.db` bypasses store-owned cleanup; use `store.close()` for normal ownership.
|
|
137
|
+
Combined callback and database-close failures retain callback errors first and
|
|
138
|
+
the database error last. A failed database close leaves cleanup registration
|
|
139
|
+
available for the retry; newly registered callbacks run once on that retry.
|
|
37
140
|
|
|
38
141
|
## Composable query SQL
|
|
39
142
|
|
|
@@ -53,7 +156,9 @@ retraction; consumers add those predicates for their read mode. `aliasEdges`,
|
|
|
53
156
|
`aliasPairs`, `aliasClosure`, and `aliasSame` implement entity-only current
|
|
54
157
|
positive ALIAS semantics. `transactionBounds` and `asOfBoundary` use
|
|
55
158
|
`Time.parseBoundary` for shared whole-period/whole-second semantics
|
|
56
|
-
(offset-less timestamps are UTC) and exact-transaction rules.
|
|
159
|
+
(offset-less timestamps are UTC) and exact-transaction rules. Transaction periods
|
|
160
|
+
are clipped at the Unix epoch: wholly earlier periods are empty, while periods
|
|
161
|
+
crossing 1970-01-01 retain their overlap with UUID transaction time. Fragments own no
|
|
57
162
|
ordering or outer filtering, so consumers can extend them without copying
|
|
58
163
|
semantic clauses.
|
|
59
164
|
|
|
@@ -63,9 +168,63 @@ Database rows (`Row.t`, snake_case columns) are the storage-oriented API and
|
|
|
63
168
|
may evolve with the schema. Use `store.recordOf(row)` for serialized or
|
|
64
169
|
cross-process data. It returns `cave.claim/v1`: transaction identity (`id`,
|
|
65
170
|
`tx`, `key`), canonical CAVE text, the semantic `Claim.t`, and explicit
|
|
66
|
-
provenance dimensions. `Record.encode`/`Record.decode` own the JSON contract
|
|
67
|
-
|
|
68
|
-
|
|
171
|
+
provenance dimensions. `Record.encode`/`Record.decode` own the JSON contract.
|
|
172
|
+
Record construction and decoding require `id = tx` as one canonical lowercase
|
|
173
|
+
UUIDv7. This matches append and sync identity (§28.1). Both also recompute
|
|
174
|
+
the claim key, including its context set, and reject disagreement with the stored
|
|
175
|
+
or supplied key. Construction captures `id`, `tx` and `claim_key` once. The
|
|
176
|
+
decoder additionally compares emitted canonical text with the supplied text.
|
|
177
|
+
Construction captures and validates provenance before returning a record,
|
|
178
|
+
rejecting empty or malformed entries and unknown dimensions. This also rejects
|
|
179
|
+
empty or binary historical provenance read through `store.recordOf(row)`;
|
|
180
|
+
repairing that data permits structured JSON encode/decode again. Its captured arrays
|
|
181
|
+
preserve valid ordering and duplicates and are independent of the caller's
|
|
182
|
+
provenance arrays. It likewise captures the semantic claim before computing its
|
|
183
|
+
key and canonical text. Construction and decoding share the claim field checks,
|
|
184
|
+
including boolean flags, raw text, terms, payloads and optional metadata. Invalid
|
|
185
|
+
programmatic claims are rejected before a record is returned. Claim getters are
|
|
186
|
+
read during capture and later edits
|
|
187
|
+
to caller-owned terms, values, contexts or tags cannot invalidate the returned
|
|
188
|
+
record. Constructor claim and provenance inputs must be structured-cloneable.
|
|
189
|
+
It operates on the
|
|
190
|
+
record alone, without consulting a store. Use `store.recordOf(row)` on a row
|
|
191
|
+
from the intended store when you need that store's evidence.
|
|
192
|
+
|
|
193
|
+
Object input is structured-cloned before validation, so decoded nested values
|
|
194
|
+
are independent of caller-owned arrays and objects. Enumerable getters are read
|
|
195
|
+
during capture; validation and the returned record use that captured data.
|
|
196
|
+
Object inputs must be structured-cloneable data; functions and proxies are not
|
|
197
|
+
accepted. JSON text is parsed into independently owned data directly. The result
|
|
198
|
+
is not frozen, so callers can still deliberately edit their decoded copy.
|
|
199
|
+
|
|
200
|
+
Construction and decoding apply these field checks before accepting the record:
|
|
201
|
+
|
|
202
|
+
| Field | Required representation |
|
|
203
|
+
|---|---|
|
|
204
|
+
| Format and version | `cave.claim`, version `1`; unknown versions are rejected. |
|
|
205
|
+
| Claim flags and authored text | Boolean `negated` and `importance`; string `raw`. |
|
|
206
|
+
| Subject and relation object | Term kind `entity`, `text` or `code`, with string `text`. |
|
|
207
|
+
| Payload | Kind `relation`, `attribute`, `metric` or `none`; attributes have string names. |
|
|
208
|
+
| Attribute, metric and uncertainty values | Supported value kind, string `raw`, Boolean `approx`; supplied units are strings and numeric scalars/endpoints are finite numbers. |
|
|
209
|
+
| Contexts | Dense arrays of strings; empty arrays are valid. |
|
|
210
|
+
| Provenance | Exactly the four enumerable own dimension fields, containing dense arrays of nonempty strings without unpaired UTF-16 surrogates; empty arrays are valid. |
|
|
211
|
+
| Tags | Objects with string `key` and optional string `value`; flat tags omit `value`. |
|
|
212
|
+
| Optional sigma level | Positive finite number; canonical export preserves tiny and large magnitudes as plain decimals. |
|
|
213
|
+
| Optional comment | String. |
|
|
214
|
+
|
|
215
|
+
Record provenance uses the same validation as `Provenance.parse`, while retaining
|
|
216
|
+
the order and duplicates of valid input arrays. Array holes, empty or non-string
|
|
217
|
+
entries, malformed Unicode and unknown dimensions are rejected for both object
|
|
218
|
+
and JSON input. Matching canonical text cannot compensate for malformed field
|
|
219
|
+
types. Attribute, metric and uncertainty values are reparsed from `raw` with
|
|
220
|
+
the core value parser (or the text/code literal constructor). Their kind,
|
|
221
|
+
approximation, numeric scalar, trajectory endpoints and unit must agree with
|
|
222
|
+
that result. Numeric projections remain absent when parsing cannot represent
|
|
223
|
+
the authored number as a finite JavaScript number. Provenance dimensions are
|
|
224
|
+
validated structurally rather than derived from contexts.
|
|
225
|
+
Uncertainty deltas additionally require a positive finite scalar, matching the
|
|
226
|
+
authored `+/-` rule; zero, negative values, trajectories and literals are rejected.
|
|
227
|
+
The checked-in v1 fixture is the compatibility baseline for
|
|
69
228
|
future decoders.
|
|
70
229
|
|
|
71
230
|
## Text stores
|
|
@@ -73,7 +232,9 @@ future decoders.
|
|
|
73
232
|
`openAt(path, { intent })` opens what a `--db` path names, by content (spec
|
|
74
233
|
§13.7): a SQLite file, `:memory:` fresh, and any other file as CAVE text
|
|
75
234
|
replayed into an in-memory store with `cave import` semantics — no actor
|
|
76
|
-
stamp, so the claims are the ones importing the file would store.
|
|
235
|
+
stamp, so the claims are the ones importing the file would store. Transaction
|
|
236
|
+
IDs and provenance payloads in annotations are ignored under these import
|
|
237
|
+
semantics; use `cave sync` to materialize an annotated replica. `intent:
|
|
77
238
|
'read'` never touches the filesystem: SQLite opens read-only (a
|
|
78
239
|
write-protected store serves), an older schema is reported with the command
|
|
79
240
|
that migrates it instead of being migrated, and a missing path throws instead
|
|
@@ -86,10 +247,40 @@ either. `intent: 'write'` (the default) creates a missing SQLite store,
|
|
|
86
247
|
migrates an older one, and throws on a text file with the materialization
|
|
87
248
|
hint, since nothing appended in memory would outlive the process. A text file
|
|
88
249
|
with a line that fails to parse fails the load, naming every line.
|
|
250
|
+
Text-store files must also be valid UTF-8. Invalid bytes raise `LocateError`
|
|
251
|
+
before an in-memory store is created or the assembler runs, rather than silently
|
|
252
|
+
changing claim text during decoding. The source file stays untouched; correcting
|
|
253
|
+
it allows normal replay, including explicit replacement characters and emoji.
|
|
254
|
+
If replay or source assembly fails after the in-memory store opens, that store
|
|
255
|
+
is closed before the failure is returned. If its cleanup also throws, an
|
|
256
|
+
`AggregateError` retains the load failure as cause and first entry and the
|
|
257
|
+
cleanup failure second. Both appear in the diagnostic, with unprintable values
|
|
258
|
+
retained in the error entries. Correcting assembly permits a fresh load; the
|
|
259
|
+
source text file is never rewritten by this cleanup.
|
|
89
260
|
`kindOf(path)` returns `memory | sqlite | text | missing`, `isStoreFile(path)`
|
|
90
261
|
is the SQLite-header check, `openText(path)` replays a text file directly, and
|
|
91
262
|
`open(path, { access })` exposes the same `read-only | no-migrate | migrate`
|
|
92
263
|
choice to callers that already know the file is SQLite.
|
|
264
|
+
Direct opens require access to be `read-only`, `no-migrate` or `migrate`.
|
|
265
|
+
Unsupported values, including `null`, throw `TypeError` before calling the
|
|
266
|
+
SQLite adapter; they cannot silently open a writable connection. An omitted
|
|
267
|
+
or `undefined` access retains the `migrate` default.
|
|
268
|
+
`openAt` captures its declared options explicitly, including inherited and
|
|
269
|
+
non-enumerable properties. A supplied base registry reaches memory, SQLite,
|
|
270
|
+
text and newly created stores; an assembly callback runs for text stores only.
|
|
271
|
+
Option storage style does not silently remove vocabulary or source assembly.
|
|
272
|
+
An explicitly supplied intent must be `read`, `scratch` or `write`; unsupported
|
|
273
|
+
values throw `TypeError` before file detection, opening or assembly. They are
|
|
274
|
+
not coerced into a mode. An omitted or `undefined` intent retains the `write`
|
|
275
|
+
default. Correct the intent and retry without changing the source store.
|
|
276
|
+
Header detection accumulates short reads until all 16 SQLite signature bytes
|
|
277
|
+
are available or EOF is reached. An incomplete signature is treated as text;
|
|
278
|
+
a complete signature still identifies SQLite regardless of the filename.
|
|
279
|
+
Header detection attempts descriptor close once. If reading and closing both
|
|
280
|
+
fail, an `AggregateError` preserves the read error as cause and first entry and
|
|
281
|
+
the close error second; a close-only failure propagates directly. Detection does
|
|
282
|
+
not classify the file or open a database after either failure. Snapshot hashing,
|
|
283
|
+
file syncing, and header detection share this descriptor ownership contract.
|
|
93
284
|
|
|
94
285
|
```ts
|
|
95
286
|
import { openAt } from '@cavelang/store'
|
|
@@ -101,27 +292,175 @@ notes.currentBeliefs() // the file's claims, assembled in
|
|
|
101
292
|
## Semantics
|
|
102
293
|
|
|
103
294
|
- **Schema upgrades are explicit** (§13.2.1): `PRAGMA user_version` records
|
|
104
|
-
the local format (current version
|
|
295
|
+
the local format (current version 2; version 0 is the unversioned legacy
|
|
105
296
|
baseline). `open()` rejects newer stores, applies every older migration in
|
|
106
297
|
order with its backfill and version update in one `BEGIN IMMEDIATE`
|
|
107
|
-
transaction, then validates required tables, indexes, and columns
|
|
298
|
+
transaction, then validates required tables, indexes, and columns (a view
|
|
299
|
+
with matching columns cannot substitute for a required table, and the search
|
|
300
|
+
index must be a virtual table exposing the hidden FTS search column).
|
|
301
|
+
Validation also requires the `cave_claim(id)` primary key and the ordered
|
|
302
|
+
`cave_provenance(claim_id, dimension, value)` primary key, with binary
|
|
303
|
+
comparison for every key column. Case-insensitive or trailing-space-insensitive
|
|
304
|
+
keys can collapse distinct provenance values; matching column names alone do
|
|
305
|
+
not guarantee claim identity or provenance deduplication. Required secondary
|
|
306
|
+
indexes must belong to the expected table, cover the ordered columns with
|
|
307
|
+
binary collation, and be non-unique and non-partial. A same-named unique
|
|
308
|
+
subject index, for example, would prevent ordinary claims about one subject.
|
|
309
|
+
Required text fields in ordinary tables must have SQLite TEXT affinity;
|
|
310
|
+
equivalent declarations such as `VARCHAR` and `CLOB` are accepted. Numeric
|
|
311
|
+
affinity can turn source identities `001` and `1` into the same integer and
|
|
312
|
+
silently deduplicate them. This check excludes virtual FTS columns and does
|
|
313
|
+
not validate the types of individual historical rows.
|
|
314
|
+
Numeric claim fields must have INTEGER, REAL or NUMERIC affinity. TEXT or
|
|
315
|
+
untyped/BLOB declarations can change range-comparison semantics; for example,
|
|
316
|
+
a text-affinity numeric projection can exclude 10 from `value_num > 3`.
|
|
317
|
+
Compatible numeric declarations retain fractional values and numeric ordering.
|
|
318
|
+
For STRICT tables, fraction-bearing `value_num`, `delta_num`, `sigma_level`
|
|
319
|
+
and `conf` fields must be REAL: STRICT INTEGER fields reject fractional
|
|
320
|
+
writes. STRICT ANY has no numeric coercion and is not a compatible numeric
|
|
321
|
+
declaration. The standard generated schema remains non-STRICT.
|
|
322
|
+
Validation reports the incompatible definition instead of silently replacing
|
|
323
|
+
it. With
|
|
108
324
|
`access: 'read-only'` or `'no-migrate'` it validates only and reports an
|
|
109
325
|
older version instead of migrating it. A crash
|
|
110
326
|
leaves a resumable old or complete new version—never a committed half-step.
|
|
327
|
+
Migration errors identify the version step and retain the original failure
|
|
328
|
+
in `Error.cause`. Unprintable adapter failures use a safe placeholder after
|
|
329
|
+
rollback is attempted, preserving the cause for inspection and retry.
|
|
330
|
+
If rollback also throws, an `AggregateError` retains the migration and rollback
|
|
331
|
+
failures in that order, with the migration failure as its cause. This includes
|
|
332
|
+
SQLite reporting that it already rolled back; a combined error alone does not
|
|
333
|
+
mean the transaction remains active. Direct `Schema.init` callers own recovery
|
|
334
|
+
or closure of their connection; a failed `open()` closes its owned database.
|
|
111
335
|
Migrations are forward-only; make rollback points by closing all users and
|
|
112
336
|
copying the closed SQLite file before upgrade.
|
|
337
|
+
Version 2 adds `idx_cave_tx` on `cave_claim(tx)` for global and bounded
|
|
338
|
+
transaction-head reads. The 1→2 migration preserves claims and metadata;
|
|
339
|
+
read-only and no-migrate opens of version 1 report the required upgrade.
|
|
113
340
|
- **Exact backup is an online SQLite snapshot** (§13.2.2): `backup()` runs
|
|
114
341
|
`VACUUM INTO` to a temporary sibling, verifies integrity, foreign keys and
|
|
115
342
|
schema, fsyncs it, computes SHA-256, and atomically publishes it. WAL-visible
|
|
116
343
|
committed rows and concurrent readers/writers are safe. `restoreBackup()`
|
|
117
344
|
verifies the source and temporary copy before atomic publication and refuses
|
|
118
345
|
destination WAL/SHM/journal sidecars. `verifyBackup()` supports independent checks.
|
|
346
|
+
This verification covers SQLite structure, schema and the supplied checksum.
|
|
347
|
+
A structurally valid snapshot can preserve semantic damage, including an
|
|
348
|
+
inconsistent claim key or empty provenance value. That permits an exact copy
|
|
349
|
+
to be retained before diagnosis and repair. Run `cave doctor` on the restored
|
|
350
|
+
copy to check semantic health before using it for export or sync; repairing
|
|
351
|
+
the copy leaves the original source and snapshot unchanged.
|
|
352
|
+
Expected SHA-256 values must be exactly 64 hexadecimal characters, with either
|
|
353
|
+
letter case accepted. Verification and restore validate them before inspecting
|
|
354
|
+
source or destination paths; whitespace, including a trailing newline, is invalid.
|
|
355
|
+
Verification attempts database close once. If validation and close both fail,
|
|
356
|
+
an `AggregateError` retains validation as its cause and first entry and close
|
|
357
|
+
failure second; a close-only failure propagates directly. Failed verification
|
|
358
|
+
does not return checksum metadata or authorize publication. Hashing and file
|
|
359
|
+
syncing also attempt descriptor close once and preserve simultaneous operation
|
|
360
|
+
and close failures in an `AggregateError`, with the operation error first and
|
|
361
|
+
as cause. A descriptor-close failure alone propagates directly.
|
|
362
|
+
If backup or restore fails and temporary-file removal also fails, both errors
|
|
363
|
+
are retained in the same order, with the operation failure as cause. The error
|
|
364
|
+
message names the temporary path for cleanup after resolving the filesystem
|
|
365
|
+
problem. Failure before publication preserves an existing destination; a
|
|
366
|
+
failed removal can leave the temporary file behind.
|
|
367
|
+
If atomic publication succeeds but subsequent cleanup throws, the error says
|
|
368
|
+
`snapshot published to <destination>` and retains the cleanup error as its
|
|
369
|
+
cause. Its message also includes the underlying failure details, including both
|
|
370
|
+
errors if directory synchronization and descriptor closure fail together.
|
|
371
|
+
The new snapshot remains at that destination; the operation does not
|
|
372
|
+
roll publication back. Verify that destination before deciding whether to
|
|
373
|
+
retry. If the outer temporary-removal attempt also fails, its aggregate error
|
|
374
|
+
includes the publication diagnostic in its message and retains it as the
|
|
375
|
+
operation cause.
|
|
376
|
+
Directory synchronization tolerates unsupported-operation errors (`EINVAL`,
|
|
377
|
+
`ENOTSUP`, `ENOSYS`), Windows directory-open restrictions (`EPERM`, `EACCES`,
|
|
378
|
+
`EISDIR`) and sync `EPERM`, and AIX read-only-directory sync `EBADF`.
|
|
379
|
+
Other directory-open/sync errors, including `EIO` and `ENOSPC`, are reported
|
|
380
|
+
after publication; simultaneous sync and descriptor-close errors are retained.
|
|
381
|
+
This distinction follows the [POSIX fsync error categories](https://pubs.opengroup.org/onlinepubs/009695399/functions/fsync.html)
|
|
382
|
+
and the [documented AIX portability limitation](https://www.gnu.org/software/gnulib/manual/html_node/fsync.html).
|
|
383
|
+
An unsupported directory sync does not establish durability across power loss.
|
|
384
|
+
Backup destinations cannot be the live source's main file or WAL/SHM/journal
|
|
385
|
+
files, even with `force`. Source/destination identity checks include hard links
|
|
386
|
+
and resolved parent directories, so missing sidecars cannot bypass protection
|
|
387
|
+
through alternate directory spellings. Restore also rejects hard links to its
|
|
388
|
+
source snapshot.
|
|
389
|
+
Verification requires a standalone source with no WAL/SHM/journal sidecars.
|
|
390
|
+
Otherwise SQLite could report rows from a WAL that the main-file checksum
|
|
391
|
+
and restored bytes do not contain. Use `backup()` on a live store first;
|
|
392
|
+
verify and restore the snapshot it produces.
|
|
393
|
+
Publishing a backup also requires a destination without SQLite sidecars;
|
|
394
|
+
`force` permits replacing the main file, but does not bypass this check.
|
|
395
|
+
Backup and restore check destination sidecars both before preparation and
|
|
396
|
+
immediately before publication. If sidecars appear during preparation, they
|
|
397
|
+
retain the destination and remove the temporary snapshot. These checks do not
|
|
398
|
+
lock out another process: keep all destination users stopped throughout the
|
|
399
|
+
operation, including publication.
|
|
400
|
+
Backup and restore capture write options at entry, so adapter callbacks or
|
|
401
|
+
changing getters cannot alter the replacement decision before publication.
|
|
402
|
+
Replacement requires the literal boolean `force: true`. Other values, including
|
|
403
|
+
the string `"true"` and boxed booleans passed by JavaScript callers, behave as
|
|
404
|
+
`false`; they are not coerced and cannot enable replacement.
|
|
405
|
+
Without `force`, a destination created during snapshot preparation is retained
|
|
406
|
+
and publication fails instead of replacing it.
|
|
407
|
+
Restore destinations cannot be sidecar paths of the source snapshot.
|
|
408
|
+
Verification and restore accept schema versions 1 through the current version,
|
|
409
|
+
validating each version's structure without migrating either file. Restoring
|
|
410
|
+
an older snapshot preserves its checksum and schema version; a later writable
|
|
411
|
+
open migrates the restored copy. Unversioned and newer snapshots are rejected.
|
|
412
|
+
- **Startup owns connection cleanup**: a failed open closes its database even
|
|
413
|
+
when failure occurs before or during schema setup or during later statement preparation
|
|
414
|
+
and vocabulary loading. A successful close preserves the original exception;
|
|
415
|
+
a failed close produces an `AggregateError` containing both failures and their
|
|
416
|
+
messages, with initialization as its cause. A later open can retry normally.
|
|
119
417
|
- **Append-only** (§9.1): `ingest` only inserts; every row carries a
|
|
120
418
|
monotonic UUIDv7 in `id` and `tx`, so `MAX(tx)` per `claim_key` is the
|
|
121
|
-
current belief. Each ingest call is one SQLite transaction
|
|
419
|
+
current belief. Each ingest call is one SQLite transaction; each newly
|
|
420
|
+
minted row receives its own UUIDv7, with `id = tx` for that row. Outer writes
|
|
122
421
|
reserve SQLite's write lock and re-observe `MAX(tx)` before minting, so
|
|
123
422
|
concurrent processes allocate in commit order; lock contention waits for
|
|
124
|
-
up to five seconds before surfacing `SQLITE_BUSY`.
|
|
423
|
+
up to five seconds before surfacing `SQLITE_BUSY`. Explicit replay IDs must
|
|
424
|
+
be canonical lowercase UUIDv7 strings. `insertResult` validates every used
|
|
425
|
+
ID before insertion or receive-clock observation, so a rejected batch cannot
|
|
426
|
+
change stored history or advance the generator. A UUID-generation failure
|
|
427
|
+
during insertion rolls back the entire batch, including vocabulary changes,
|
|
428
|
+
and the same store can retry after the failure clears. IDs successfully minted
|
|
429
|
+
before a transaction rolls back may leave sequence gaps; transaction rollback
|
|
430
|
+
does not rewind the process-wide generator shared by other stores.
|
|
431
|
+
- **Canonicalization shares the write reservation** (§13.4): `ingest` checks
|
|
432
|
+
SQLite's `data_version` inside the transaction and refreshes the registry
|
|
433
|
+
when another connection has committed. Inverse and lifecycle spellings
|
|
434
|
+
therefore use committed vocabulary even on an already-open store. Local
|
|
435
|
+
appends reuse the registry; rollback restores both the registry and its
|
|
436
|
+
version marker. Callers supplying a pre-canonicalized `insertResult` remain
|
|
437
|
+
responsible for their result's vocabulary snapshot; raw SQL writers must
|
|
438
|
+
call `reloadRegistry` after changing declarations.
|
|
439
|
+
- **Transaction ownership**: `store.transaction(({ outermost }) => …)`
|
|
440
|
+
requires a synchronous callback. Returning a promise or thenable throws and
|
|
441
|
+
rolls back writes made before the callback returned. Promises from the current
|
|
442
|
+
JavaScript realm are recognized even if an own `then` property is non-callable
|
|
443
|
+
or throws when read. Native promise rejection
|
|
444
|
+
is observed without calling a custom thenable’s `then` method, so rejecting
|
|
445
|
+
a thenable does not start its work after rollback. This does not cancel
|
|
446
|
+
already-started asynchronous work or include its later writes. Finish async
|
|
447
|
+
work before calling `transaction`, then perform all transactional writes
|
|
448
|
+
synchronously. The callback context
|
|
449
|
+
reports whether the callback owns the database commit or only a nested
|
|
450
|
+
savepoint. Existing callbacks that take no argument remain valid. Releasing
|
|
451
|
+
a nested savepoint leaves its writes subject to the caller's rollback;
|
|
452
|
+
the outermost scope commits only after its callback returns successfully.
|
|
453
|
+
A commit error still throws and triggers rollback. The shared adapter contract
|
|
454
|
+
exercises a deferred foreign-key failure at commit, verifies that claims and
|
|
455
|
+
vocabulary are restored, and successfully retries on the same connection.
|
|
456
|
+
The context works with both Node SQLite and the browser adapter.
|
|
457
|
+
If rollback cleanup also throws, an `AggregateError` retains the original
|
|
458
|
+
error as its cause and first entry, with the cleanup error second. In-memory
|
|
459
|
+
vocabulary and transaction ownership are restored even in that case. Native
|
|
460
|
+
and browser regressions exercise SQLite automatic rollback from outer and
|
|
461
|
+
nested scopes and verify a later successful write. Other cleanup failures do
|
|
462
|
+
not establish the database's transaction state; close and reopen the store
|
|
463
|
+
before retrying when that state is uncertain.
|
|
125
464
|
- **History is permanent** (§9.6): retraction appends a `0%` row; it never
|
|
126
465
|
erases the earlier row, `raw_line`, metadata, FTS text, export, backup, or
|
|
127
466
|
peer copy. The store has no row-level redact/forget API because local
|
|
@@ -133,6 +472,10 @@ notes.currentBeliefs() // the file's claims, assembled in
|
|
|
133
472
|
`#sensitivity:public|internal|confidential|restricted` labels each immutable
|
|
134
473
|
row; unlabeled means `internal`, while flat, malformed and unknown labels
|
|
135
474
|
fail closed as `restricted`. Export defaults to a maximum of `internal`.
|
|
475
|
+
Export reads `current`, `maxSensitivity` and `tx` once each; changing getters
|
|
476
|
+
cannot switch the selected history, sensitivity ceiling or annotation mode
|
|
477
|
+
during the operation. Shared adapter tests verify the resulting claim content
|
|
478
|
+
and transaction annotations, including grouped canonical output.
|
|
136
479
|
Select `restricted` explicitly for complete portable text history. This is routing metadata,
|
|
137
480
|
not encryption, access control, erasure or a retention boundary.
|
|
138
481
|
- **One row per fact** (§13.3): inverse writes are canonicalized before
|
|
@@ -145,6 +488,30 @@ notes.currentBeliefs() // the file's claims, assembled in
|
|
|
145
488
|
replay predicate mirrors the canonicalizer exactly — qualifier-condition
|
|
146
489
|
rows never declare, and `X IS verb` needs a verb-shaped subject — so the
|
|
147
490
|
registry after reopen equals the registry at close.
|
|
491
|
+
Appending WHEN, VIA or BECAUSE edges to existing declaration rows refreshes
|
|
492
|
+
the live registry in the same transaction, since those rows become qualifier
|
|
493
|
+
children. Rollback restores both the edges and the previous registry. Edges
|
|
494
|
+
to ordinary premise rows do not require declaration replay.
|
|
495
|
+
`appendEdges` captures the batch's parent IDs, roles and child IDs once before
|
|
496
|
+
its transaction. SQL insertion and vocabulary refresh use the same captured
|
|
497
|
+
values, including when callers supply getters.
|
|
498
|
+
Both `appendEdges` and structured `insertResult` reject roles other than `WHEN`,
|
|
499
|
+
`VIA`, `BECAUSE` and `QUALIFIES` with `TypeError`. Structured `append` also
|
|
500
|
+
requires safe integer parent/child indices within its claim batch. These
|
|
501
|
+
checks finish before inserting any rows or observing replay transaction IDs;
|
|
502
|
+
invalid batches leave claims, edges and the live registry unchanged.
|
|
503
|
+
A later edge constraint failure or vocabulary-refresh failure rolls back the
|
|
504
|
+
entire batch and restores the previous registry. The full annotated history
|
|
505
|
+
remains unchanged, and a corrected batch can be retried on the same store.
|
|
506
|
+
Vocabulary rebuilds construct a replacement before updating the live cache.
|
|
507
|
+
An interrupted `reloadRegistry` retains the previous registry and version
|
|
508
|
+
marker; callers can retry without reopening or losing the loaded inverse names.
|
|
509
|
+
Opening captures the configured registry and access mode once. Current
|
|
510
|
+
vocabulary, explicit reload and `registryAsOf` share that same base registry;
|
|
511
|
+
changing option getters cannot give historical reads a different configuration.
|
|
512
|
+
Replay filters ordinary `IS` facts in SQL: only `IS verb` can declare a
|
|
513
|
+
verb. This avoids materializing unrelated type facts during frequent action
|
|
514
|
+
refreshes while retaining transaction ordering and qualifier exclusions.
|
|
148
515
|
- **Verb renames preserve history** (§5.8): after `OLD RENAMED-TO NEW`,
|
|
149
516
|
either spelling writes the stable `OLD` storage verb and therefore the same
|
|
150
517
|
claim key. `NEW` is preferred for authors while `OLD` remains compatible;
|
|
@@ -154,7 +521,9 @@ notes.currentBeliefs() // the file's claims, assembled in
|
|
|
154
521
|
read *current beliefs* and skip negated (`VERB NOT`) and retracted
|
|
155
522
|
(`@ 0%`) rows; opt back in with `{ negated: true, retracted: true }`.
|
|
156
523
|
Contradictions still coexist as rows (§9.4) — resolution belongs to the
|
|
157
|
-
query layer.
|
|
524
|
+
query layer. The traversal flags `negated`, `retracted`, `aliases` and
|
|
525
|
+
`resolve` must be booleans when supplied; each is captured once and validated
|
|
526
|
+
before query construction. Omission retains the false default.
|
|
158
527
|
- **Alias closure is opt-in** (§13.6): `{ aliases: true }` on traversal
|
|
159
528
|
(and on `claimsAbout`) matches the entity through every name linked by
|
|
160
529
|
current positive `ALIAS` claims, read as undirected edges;
|
|
@@ -162,6 +531,12 @@ notes.currentBeliefs() // the file's claims, assembled in
|
|
|
162
531
|
matching widens, stored names come back untouched, and disagreeing
|
|
163
532
|
belief series surface side by side rather than merging silently. Unmerge
|
|
164
533
|
is retraction — `dupe ALIAS canonical @ 0%`.
|
|
534
|
+
`claimsAbout` captures its `aliases` option once and requires a boolean when
|
|
535
|
+
supplied; strings, numbers and null are errors rather than disabled alias
|
|
536
|
+
matching. It still returns full history, newest first, including prior belief
|
|
537
|
+
revisions. It is not a current-belief lookup. The alias flags on
|
|
538
|
+
`resolvedBeliefs` and `contested` likewise require booleans and are captured
|
|
539
|
+
once before resolution reads.
|
|
165
540
|
- **Actor provenance is caller-supplied** (§9.5): `ingest`/`insertResult`
|
|
166
541
|
accept `{ source }` and stamp `@src:<source>` on every appended claim
|
|
167
542
|
that carries no `src:` context — *before* the claim key is computed, so
|
|
@@ -175,6 +550,20 @@ notes.currentBeliefs() // the file's claims, assembled in
|
|
|
175
550
|
representation. Lifecycle systems use `run` lookup rather than authored
|
|
176
551
|
`src:` strings; resolution reads actor/source. Opening old stores backfills
|
|
177
552
|
only established actor/run prefixes, decoded sources, and explicit scopes.
|
|
553
|
+
Backfill collects contexts once per claim without repeatedly copying prefixes;
|
|
554
|
+
the native and browser migration contract checks 5,000 source contexts,
|
|
555
|
+
context-free claims, retained row identities and repeat initialization.
|
|
556
|
+
Annotated export preserves dimensions that compact contexts cannot encode:
|
|
557
|
+
an optional JSON payload on `;@ <tx>` carries all four dimension sets.
|
|
558
|
+
`Provenance.normalize` provides a stable set representation;
|
|
559
|
+
`Provenance.parse` requires exactly the four named enumerable own fields
|
|
560
|
+
(`actors`, `sources`, `runs`, `domains`); a hidden required field cannot mask
|
|
561
|
+
an unknown enumerable dimension. It validates the complete object for replay, rejecting
|
|
562
|
+
unpaired UTF-16 surrogates in every dimension before UTF-8 storage can replace
|
|
563
|
+
them. Valid astral characters, embedded NUL and literal replacement characters
|
|
564
|
+
remain valid provenance text.
|
|
565
|
+
Parsing captures each dimension and its entries once, so accessors and later
|
|
566
|
+
changes to caller-owned arrays cannot replace the values being validated.
|
|
178
567
|
- **Source spans retain source identity** (§9.8):
|
|
179
568
|
`@src:docs/design%20notes.md#L10-L20` carries a one-based inclusive range;
|
|
180
569
|
`SourceSpan` owns escaping and parsing. The exact context remains in the
|
|
@@ -190,10 +579,47 @@ notes.currentBeliefs() // the file's claims, assembled in
|
|
|
190
579
|
`rule` 1) with in-band `source/<name> HAS precedence:` /
|
|
191
580
|
`HAS reliability:` claims, matched to `src:` contexts by longest
|
|
192
581
|
segment prefix; the declarations themselves resolve under the built-ins
|
|
193
|
-
alone, so ingested text cannot self-elevate.
|
|
582
|
+
alone, so ingested text cannot self-elevate. Declaration-source precedence
|
|
583
|
+
is accumulated iteratively, so a large provenance list does not exceed
|
|
584
|
+
JavaScript's function-argument limit. Contexts still reside in memory during
|
|
585
|
+
policy lookup; this is not a provenance-size budget. Policy identities preserve
|
|
586
|
+
embedded NUL characters: declaration keys use structured tuples, SQL literals
|
|
587
|
+
retain complete UTF-8 text, and segment matching uses byte lengths so SQLite
|
|
588
|
+
text functions cannot truncate a prefix. Native and WASM regressions check
|
|
589
|
+
exact, descendant and non-matching sibling sources. Physical source contexts
|
|
590
|
+
use the canonical escaping provided by `SourceSpan.context`. Actor compatibility
|
|
591
|
+
stamps use `Context.source`, retaining the actor name verbatim. Resolution
|
|
592
|
+
includes an actor only when that exact stamp is present; an authored source
|
|
593
|
+
can suppress an ordinary actor stamp while actor provenance remains recorded.
|
|
594
|
+
Up to 16 policy entries use SQL `VALUES`; larger policies use one materialized
|
|
595
|
+
JSON table to reduce SQL preparation cost and avoid repeated JSON decoding
|
|
596
|
+
during source matching. Native tests cover 33,000 entries and the strategy
|
|
597
|
+
boundary; SQLite WASM tests cover more than 1,000 in-band declarations. The
|
|
598
|
+
complete policy still resides in memory, and matching still scans candidate
|
|
599
|
+
prefixes; this is not a general policy-size or execution-time budget.
|
|
600
|
+
Ranked precedence is projected as a floating-point value, matching the
|
|
601
|
+
JavaScript numeric API. Finite values outside the safe-integer range, such as
|
|
602
|
+
`10000000000000000`, remain readable without a SQLite integer-conversion error;
|
|
603
|
+
positive and negative values retain their ordering.
|
|
604
|
+
Winners come back
|
|
194
605
|
verbatim — resolution filters, it never rewrites — and it composes with
|
|
195
606
|
`aliases`, which widens groups through the closure (the §13.6
|
|
196
|
-
pick-a-winner story).
|
|
607
|
+
pick-a-winner story). Alias-aware grouping preserves complete subject and
|
|
608
|
+
object names, including embedded NUL and Unicode characters; sharing the text
|
|
609
|
+
before a NUL does not make two names aliases. Only the declared alias closure
|
|
610
|
+
combines their resolution groups.
|
|
611
|
+
`resolvedBeliefs`, `contested` and traversals with `resolve: true` hold one
|
|
612
|
+
deferred read snapshot across policy lookup and claim selection. Concurrent
|
|
613
|
+
policy and claim commits become visible together on the next call. These
|
|
614
|
+
reads work on read-only connections and nest inside caller transactions.
|
|
615
|
+
Traversals capture their four boolean options once before reading; traversals
|
|
616
|
+
without resolution do not add this policy snapshot. Reverse reads always hold
|
|
617
|
+
a snapshot across their vocabulary check and fact selection, so a concurrent
|
|
618
|
+
inverse rename cannot label new facts with an old inverse name.
|
|
619
|
+
If both a read and snapshot release fail, the aggregate message includes both
|
|
620
|
+
diagnostics, with the original errors retained and the read failure as cause.
|
|
621
|
+
Canonical export uses the same cleanup and reporting behavior. A release-only
|
|
622
|
+
failure preserves that error itself.
|
|
197
623
|
|
|
198
624
|
## API
|
|
199
625
|
|
|
@@ -201,7 +627,7 @@ notes.currentBeliefs() // the file's claims, assembled in
|
|
|
201
627
|
|---|---|---|
|
|
202
628
|
| `ingest(text, {strict, source})` | §13.4 | parse → canonicalize → append; lenient by default |
|
|
203
629
|
| `insertResult(result, {source})` | | append a pre-canonicalized `@cavelang/canonical` result |
|
|
204
|
-
| `currentBeliefs({minConf})` | §13.5 | latest row per key |
|
|
630
|
+
| `currentBeliefs({minConf})` | §13.5 | latest row per key; optional inclusive threshold must be a finite number in 0..1, captured once and validated before querying |
|
|
205
631
|
| `currentBelief(key)` / `history(key)` | §9.1 | one fact's belief series |
|
|
206
632
|
| `provenanceOf(rowOrId)` / `byProvenance(dimension, value)` | §9.5.1 | inspect or filter actor/source/run/domain |
|
|
207
633
|
| `resolvedBeliefs({aliases})` | §26 | one winner per resolution group |
|
|
@@ -213,7 +639,7 @@ notes.currentBeliefs() // the file's claims, assembled in
|
|
|
213
639
|
| `byTag(key, value?)` | §13.5 | flat (`value` omitted → `IS NULL`) or scoped |
|
|
214
640
|
| `byContext(ctx)` | §13.5 | context filter |
|
|
215
641
|
| `topicMembers(t)` / `topicsOf(e)` | §11.2 | topic layer over `CONTAINS` |
|
|
216
|
-
| `search(q, {raw, limit, maxSensitivity})` | §13.2 | adapter full-text search; literal phrase by default, `limit` caps in the query;
|
|
642
|
+
| `search(q, {raw, limit, maxSensitivity, currentOnly})` | §13.2 | adapter full-text search; literal phrase by default, `limit` caps in the query; optional current-only and sensitivity filters |
|
|
217
643
|
| `edgesOf(id)` | §13.2 | qualifier/grouping edges with roles |
|
|
218
644
|
| `toClaim(row)` | | reconstruct the canonical claim + side tables |
|
|
219
645
|
| `registry()` / `baseRegistry()` | §5.5 | current registry and its configured pre-declaration base |
|
|
@@ -223,8 +649,49 @@ notes.currentBeliefs() // the file's claims, assembled in
|
|
|
223
649
|
| `openAt(path, {intent, registry})` / `openText(path)` / `kindOf(path)` / `isStoreFile(path)` | §13.7 | open a `--db` path by content: SQLite (read-only for `read`, unmigrated for `scratch`), CAVE text replayed into memory; only `write` creates or migrates |
|
|
224
650
|
| `adapter` / `db` | | selected adapter capabilities and its raw structural database handle |
|
|
225
651
|
|
|
652
|
+
### Search history or current knowledge
|
|
653
|
+
|
|
654
|
+
Use historical search to find evidence across revisions. Set `currentOnly` when
|
|
655
|
+
the result should contain only the latest positive-confidence rows. Retraction
|
|
656
|
+
removes a fact from current-only results while keeping its history searchable:
|
|
657
|
+
|
|
658
|
+
```ts
|
|
659
|
+
import { open } from '@cavelang/store'
|
|
660
|
+
|
|
661
|
+
const store = open()
|
|
662
|
+
try {
|
|
663
|
+
store.ingest('auth HAS provider: legacy')
|
|
664
|
+
store.ingest('auth HAS provider: modern')
|
|
665
|
+
|
|
666
|
+
console.log(store.search('auth').map(row => row.value_text))
|
|
667
|
+
// ['modern', 'legacy']
|
|
668
|
+
console.log(store.search('auth', { currentOnly: true }).map(row => row.value_text))
|
|
669
|
+
// ['modern']
|
|
670
|
+
|
|
671
|
+
store.ingest('auth HAS provider: modern @ 0%')
|
|
672
|
+
console.log(store.search('auth', { currentOnly: true }).length) // 0
|
|
673
|
+
console.log(store.search('auth').length) // 3
|
|
674
|
+
} finally {
|
|
675
|
+
store.close()
|
|
676
|
+
}
|
|
677
|
+
```
|
|
678
|
+
|
|
679
|
+
Both modes return newest matches first. A `limit` applies after the selected
|
|
680
|
+
history scope; `maxSensitivity` independently sets the audience ceiling when
|
|
681
|
+
supplied. CLI, MCP and viewer searches retain their historical behavior.
|
|
682
|
+
|
|
226
683
|
## Storage decisions
|
|
227
684
|
|
|
685
|
+
The exported `Resolve.rankedSql(entries, currentSql)` and
|
|
686
|
+
`Resolve.resolvedSql(entries, currentSql)` helpers accept an empty explicit
|
|
687
|
+
policy. With no entries, every source has precedence `0` and reliability `1`,
|
|
688
|
+
so confidence and then recency choose the winner. Store methods obtain the
|
|
689
|
+
built-in ladder plus declarations through `resolutionPolicy()`.
|
|
690
|
+
Partial explicit policies apply those defaults to each unmatched source
|
|
691
|
+
before taking maximum precedence and minimum reliability. For example, a
|
|
692
|
+
source assigned precedence `-1` and an unmatched source together give a claim
|
|
693
|
+
precedence `0`.
|
|
694
|
+
|
|
228
695
|
- **Terms are stored formatted**: entities as plain text (so the spec's
|
|
229
696
|
`WHERE subject = 'auth/middleware'` queries work verbatim), literals with
|
|
230
697
|
their delimiters (`` `<=` ``, `"…"`) so they never collide with
|
|
@@ -232,6 +699,11 @@ notes.currentBeliefs() // the file's claims, assembled in
|
|
|
232
699
|
- **`value_text` is the value as written** (including `~`, multiplier
|
|
233
700
|
letters and literal delimiters); `value_num`/`value_unit` hold the
|
|
234
701
|
normalized §13.4 forms. Same for `delta_*`.
|
|
702
|
+
- **Confidence keeps its stored numeric value** through canonical export and
|
|
703
|
+
import, including positive subnormal values and computed probabilities.
|
|
704
|
+
A zero-confidence retraction remains distinct from a later positive
|
|
705
|
+
reassertion, even at `Number.MIN_VALUE`. Import assigns fresh transaction
|
|
706
|
+
identities; annotated text sync preserves identities and skips repeated rows.
|
|
235
707
|
- **`id` doubles as `tx`** by default: a UUIDv7 is both unique and
|
|
236
708
|
time-ordered, and per-row tx ids keep same-document belief updates
|
|
237
709
|
ordered by line. Allocation is database-serialized across processes;
|
|
@@ -239,6 +711,29 @@ notes.currentBeliefs() // the file's claims, assembled in
|
|
|
239
711
|
- **`search()` phrase-quotes by default** — SQLite full-text syntax would parse
|
|
240
712
|
`token-expiry` as a column filter; `{ raw: true }` opts into full MATCH
|
|
241
713
|
syntax.
|
|
714
|
+
A supplied `limit` must be a non-negative safe integer; zero returns no rows,
|
|
715
|
+
and omission leaves search uncapped. Invalid limits throw `TypeError` before
|
|
716
|
+
querying, including on an empty store. Search includes historical beliefs by
|
|
717
|
+
default. `{ currentOnly: true }` selects positive-confidence current rows
|
|
718
|
+
before applying the limit, so revisions and retractions cannot crowd out
|
|
719
|
+
current matches. The option must be a boolean when supplied.
|
|
720
|
+
`currentOnly` is a programmatic store option used by ingestion context.
|
|
721
|
+
CLI `search`, MCP `cave_search` and viewer search retain historical results;
|
|
722
|
+
their public inputs do not expose this option.
|
|
723
|
+
The returned-row limit does not cap SQL work; see the
|
|
724
|
+
[current-search trial](#current-search-trial) for measurements and query-plan tradeoffs.
|
|
725
|
+
Search captures `raw`, `limit`, `currentOnly` and `maxSensitivity` once before validation and
|
|
726
|
+
SQL construction. Getters cannot change the cap or sensitivity ceiling between
|
|
727
|
+
validation and execution.
|
|
728
|
+
Supplied `raw` and `currentOnly` values must be booleans; malformed switches
|
|
729
|
+
raise `TypeError` instead of silently choosing another search mode.
|
|
730
|
+
Literal phrase quoting accounts for FTS4 versus FTS5, including embedded
|
|
731
|
+
quotation marks. `{ raw: true }` passes syntax directly to the selected
|
|
732
|
+
engine; it does not translate between their query languages.
|
|
733
|
+
Both modes reject NUL characters in query text with `TypeError` before database
|
|
734
|
+
access, including when `limit` is zero. SQLite's MATCH parser can otherwise
|
|
735
|
+
silently truncate a raw query at NUL or report an unterminated literal phrase.
|
|
736
|
+
Empty or punctuation-only literal phrases return no matches.
|
|
242
737
|
- **Current-only export remaps edge endpoints** to the current row of each
|
|
243
738
|
endpoint's claim key: a superseded qualified parent keeps its `WHEN`
|
|
244
739
|
attached to the surviving belief, and orphaned condition claims are never
|
|
@@ -248,6 +743,189 @@ notes.currentBeliefs() // the file's claims, assembled in
|
|
|
248
743
|
revives an older visible row. Full-history export checks each row. Edges with
|
|
249
744
|
either endpoint hidden are omitted.
|
|
250
745
|
|
|
746
|
+
`registryAsOf` bounds qualifier parents as well as declaration rows. A parent
|
|
747
|
+
recorded later cannot exclude an older verb, inverse, or rename from a past
|
|
748
|
+
snapshot; date and timestamp intervals use the same boundary on both sides.
|
|
749
|
+
|
|
750
|
+
Live connections refresh vocabulary after another SQLite connection commits.
|
|
751
|
+
`store.registry()`, query compilation, and inverse reads therefore see new
|
|
752
|
+
verb and lifecycle declarations without reopening or writing. The read path
|
|
753
|
+
works on read-only connections and reuses its registry cache while SQLite's
|
|
754
|
+
data-version counter is unchanged. Raw SQL writes on the same connection still
|
|
755
|
+
require `reloadRegistry`; precompiled SQL retains their supplied vocabulary.
|
|
756
|
+
|
|
757
|
+
Canonical export may include `@claim` for qualifier claims with verb `NOT` or
|
|
758
|
+
to distinguish reserved entity subjects
|
|
759
|
+
from qualifiers or continuations. The marker changes no stored identity or
|
|
760
|
+
context, including in annotated exports. Readers of those exports must support
|
|
761
|
+
the explicit-claim syntax (§8.6).
|
|
762
|
+
|
|
763
|
+
## Structured appends
|
|
764
|
+
|
|
765
|
+
A supplied `source` must be a string. The value is captured once and validated
|
|
766
|
+
before stamping or deriving actor/run attribution, even for empty batches.
|
|
767
|
+
Omit it (or use `undefined`) to preserve interchange claim identity without a
|
|
768
|
+
source stamp.
|
|
769
|
+
|
|
770
|
+
The optional `provenance` container must be an object when supplied; null,
|
|
771
|
+
primitives, functions and arrays are rejected before any fields are read.
|
|
772
|
+
An empty object is supported, and omission or `undefined` keeps inferred
|
|
773
|
+
provenance behavior. Supplied `actor` and `run` must be strings; every entry
|
|
774
|
+
in `sources` and `domains` must also be a string. Validation happens before
|
|
775
|
+
row insertion, including for empty batches. Empty strings retain their existing
|
|
776
|
+
meaning of no explicit value.
|
|
777
|
+
|
|
778
|
+
Append option collections (`ids`, `contexts`, `provenance.sources`, and
|
|
779
|
+
`provenance.domains`) must be arrays when supplied. They are captured once and
|
|
780
|
+
validated before copying, including for empty batches; strings, sets, null and
|
|
781
|
+
array-like objects are rejected. Omitted or `undefined` collections keep their
|
|
782
|
+
existing defaults.
|
|
783
|
+
|
|
784
|
+
`insertResult` accepts a canonicalization result and prepares every claim before
|
|
785
|
+
observing replay IDs or inserting rows. It validates canonical emission even
|
|
786
|
+
when the caller supplies raw text. If a claim fails that check, the batch
|
|
787
|
+
rejects without appending earlier valid claims. Context and tag collections must
|
|
788
|
+
be arrays before capture: strings, sets and array-like objects are rejected
|
|
789
|
+
rather than being converted into a different collection.
|
|
790
|
+
|
|
791
|
+
The [emitter's validation rules](../canonical/README.md#structured-claim-validation)
|
|
792
|
+
reject non-string term/value text, unsupported term kinds, non-boolean negation or importance, malformed verb tokens, embedded LF in claim fields, text/code literals containing their own
|
|
793
|
+
closing delimiter, and entity subjects that do not represent one non-metadata
|
|
794
|
+
atom. Contexts and tags must retain their metadata tokens, and tags must keep
|
|
795
|
+
the same key/value split. The rendered body must not open a comment or leave
|
|
796
|
+
a literal delimiter unmatched. Literal subjects and multiline comments remain
|
|
797
|
+
supported.
|
|
798
|
+
|
|
799
|
+
This validation prevents the listed malformed fields from entering through new
|
|
800
|
+
structured appends. It does not validate every possible programmatic claim,
|
|
801
|
+
rewrite historical rows, or govern low-level SQL writes. Existing malformed
|
|
802
|
+
rows may still fail canonical export. When an included row cannot be decoded or
|
|
803
|
+
emitted, export reports its claim ID and retains the validation error as `cause`.
|
|
804
|
+
The diagnostic applies to full/current and annotated/plain exports. It does not
|
|
805
|
+
modify rows or return a partial export. Row validation is repeated only after an
|
|
806
|
+
emission failure, so successful exports avoid an extra validation pass.
|
|
807
|
+
Export reads claims, metadata, lineage and provenance within one deferred read
|
|
808
|
+
snapshot, including on read-only connections. A peer commit cannot add its edges
|
|
809
|
+
to an export that omitted its claims; the next export sees the committed state.
|
|
810
|
+
If an included claim has an edge whose other endpoint is missing from storage,
|
|
811
|
+
export fails instead of silently dropping that relationship. Claims excluded by
|
|
812
|
+
sensitivity remain valid endpoints and retain the existing filtering behavior;
|
|
813
|
+
edges with neither endpoint included are outside this export's diagnostic scope.
|
|
814
|
+
The error contains no claim text, export leaves storage unchanged, and corrected
|
|
815
|
+
relationships are checked again on retry. This applies to plain/annotated and
|
|
816
|
+
historical/current exports, including current revision remapping.
|
|
817
|
+
The snapshot nests inside caller-owned transactions without committing them.
|
|
818
|
+
Release is attempted on success and failure. If both the read and release fail,
|
|
819
|
+
an `AggregateError` retains the read error first and as `cause`; a lone release
|
|
820
|
+
error propagates unchanged. A failed SQLite release may require caller recovery
|
|
821
|
+
before reusing the connection.
|
|
822
|
+
|
|
823
|
+
Export captures `current`, `maxSensitivity` and `tx` at entry, before database
|
|
824
|
+
reads and row decoding. Adapter callbacks that change the caller's options cannot
|
|
825
|
+
remove requested transaction annotations midway through an export. The shared
|
|
826
|
+
adapter contract checks this with native SQLite and browser SQLite.
|
|
827
|
+
With `tx: true`, every included row must have a canonical lowercase UUIDv7
|
|
828
|
+
identity with `id = tx`; a mismatch fails with the affected claim ID before
|
|
829
|
+
annotation emission. Plain semantic export does not carry row identities.
|
|
830
|
+
Annotated export also recomputes each included claim's key, including its
|
|
831
|
+
contexts, and rejects disagreement with the stored key. This prevents replay
|
|
832
|
+
from silently assigning the same transaction to a different belief series.
|
|
833
|
+
The error identifies the affected claim and preserves its cause; export leaves
|
|
834
|
+
the database unchanged for repair and retry.
|
|
835
|
+
Annotated export also validates explicit stored provenance against the replay
|
|
836
|
+
contract: each value must be a nonempty Unicode string. Historical empty or
|
|
837
|
+
binary values fail with the affected claim ID and an underlying cause instead
|
|
838
|
+
of producing annotations that sync would reject. Validation occurs only for
|
|
839
|
+
included rows; narrower sensitivity exports can still succeed. Export does not
|
|
840
|
+
repair stored data. After the malformed provenance is corrected, annotated
|
|
841
|
+
export and identity-preserving replay can proceed normally.
|
|
842
|
+
`cave doctor` checks for empty and non-text provenance values under `store.rows`
|
|
843
|
+
without printing the values or claim identities and without changing the database.
|
|
844
|
+
The same check recomputes historical claim keys from their semantic claims and
|
|
845
|
+
stored contexts, detecting disagreement before an annotated export is attempted.
|
|
846
|
+
It also checks canonical emission with the stored tags, diagnosing historical
|
|
847
|
+
claim text that export cannot emit while keeping the data unchanged for repair.
|
|
848
|
+
Stored edge roles outside `WHEN`, `VIA`, `BECAUSE` and `QUALIFIES` also fail
|
|
849
|
+
`store.rows`. Diagnosis leaves those relationships unchanged, redacts their
|
|
850
|
+
values and checks them again after repair.
|
|
851
|
+
|
|
852
|
+
The optional `current` and `tx` switches must be booleans when supplied. Export
|
|
853
|
+
captures them once and rejects malformed values with `TypeError` before querying,
|
|
854
|
+
including on empty stores. A string such as `"true"` cannot silently select
|
|
855
|
+
historical output or omit transaction annotations.
|
|
856
|
+
|
|
857
|
+
`search` and `exportText` also reject invalid supplied `maxSensitivity` values
|
|
858
|
+
with `TypeError`, including `null`, unknown strings and non-string values.
|
|
859
|
+
Omitting the option retains each method's existing scope: search includes all
|
|
860
|
+
levels, while export defaults to `internal`. Stored malformed sensitivity tags
|
|
861
|
+
continue to be treated as `restricted`; they are distinct from invalid API options.
|
|
862
|
+
|
|
863
|
+
Sensitivity filtering precedes row decoding and emission: malformed restricted
|
|
864
|
+
rows do not break a public-only export or disclose their claim IDs in its output.
|
|
865
|
+
When the requested sensitivity includes those rows, the same row-specific error
|
|
866
|
+
applies. Regressions cover newline-corrupted historical subjects, objects,
|
|
867
|
+
contexts and tag values across full/current and plain/annotated exports, with
|
|
868
|
+
stored claims, metadata and edges unchanged after failure.
|
|
869
|
+
|
|
870
|
+
Export also rejects stored edge roles other than `WHEN`, `VIA`, `BECAUSE` and
|
|
871
|
+
`QUALIFIES` when both endpoints are included.
|
|
872
|
+
The error identifies both endpoint IDs. Edges omitted by sensitivity filtering
|
|
873
|
+
do not expose their endpoints through this diagnostic. Invalid roles previously
|
|
874
|
+
emitted text that could re-import without its original edge; export now fails
|
|
875
|
+
without changing the store, and repairing the role restores export.
|
|
876
|
+
|
|
877
|
+
Self-referential edges are retained, including edges whose historical endpoints
|
|
878
|
+
resolve to the same current row. The canonical emitter re-states the claim once
|
|
879
|
+
under its own edge; annotated export gives both appearances the same transaction
|
|
880
|
+
ID. This preserves the graph relation without recursively expanding the cycle.
|
|
881
|
+
Syncing that annotated export into a fresh store restores one row and one
|
|
882
|
+
self-edge for each of the four supported roles; repeating the sync adds neither
|
|
883
|
+
another row nor another edge. Current export uses the current row's identity
|
|
884
|
+
for both endpoints when it remaps an older revision.
|
|
885
|
+
|
|
886
|
+
Row-to-claim conversion also rejects conflicting payload columns: an object
|
|
887
|
+
cannot coexist with an attribute or value, and an attribute requires a value.
|
|
888
|
+
These malformed historical or low-level SQL rows previously lost fields during
|
|
889
|
+
conversion. Export now reports their claim ID when the sensitivity scope includes
|
|
890
|
+
them, without changing stored data; correcting the row permits export again.
|
|
891
|
+
The stored `negated`, `importance` and `value_approx` flags must each be exactly
|
|
892
|
+
numeric 0 or 1.
|
|
893
|
+
Other values are diagnosed during claim conversion instead of being coerced
|
|
894
|
+
into a negation or importance marker, or silently ignoring an invalid
|
|
895
|
+
approximation flag. Native and browser SQLite share this check.
|
|
896
|
+
|
|
897
|
+
Claim-specific export failures retain the affected claim ID and the original
|
|
898
|
+
thrown value in `Error.cause`, even if an adapter throws a value whose message
|
|
899
|
+
cannot be read or converted to text. Such diagnostics use
|
|
900
|
+
`[unprintable thrown value]`; the export read snapshot is still released and a
|
|
901
|
+
corrected read can be retried without changing stored history.
|
|
902
|
+
|
|
903
|
+
Conversion also checks `value_num`, `value_unit`, `value_approx`, `delta_num` and
|
|
904
|
+
`delta_unit` against the parsed authored value and uncertainty text. A mismatch
|
|
905
|
+
fails with the field name; export adds the claim ID and leaves the database
|
|
906
|
+
unchanged. Correcting the cached columns permits conversion and export again.
|
|
907
|
+
This conversion check does not validate every row before SQL filtering.
|
|
908
|
+
`cave doctor` separately streams all stored history through the row decoder to
|
|
909
|
+
check authored values and their cached projections before export is attempted.
|
|
910
|
+
That scan also requires a canonical lowercase UUIDv7 identity with `id = tx`
|
|
911
|
+
for each row; SQLite integrity and matching search contents alone do not prove
|
|
912
|
+
that identity invariant.
|
|
913
|
+
Doctor holds a read transaction across its SQLite checks, so concurrent commits
|
|
914
|
+
do not mix database states within a report; a later invocation sees those commits.
|
|
915
|
+
In rollback-journal mode, the read snapshot can temporarily block a competing
|
|
916
|
+
writer's commit. Closing doctor's connection releases the read lock, including
|
|
917
|
+
after an invalid-row report. WAL and rollback-journal regressions exercise these
|
|
918
|
+
different concurrency behaviors using a second live connection.
|
|
919
|
+
If closing the owned connection fails, doctor retains the database checks and
|
|
920
|
+
adds a redacted `store.cleanup` failure instead of exposing the thrown value.
|
|
921
|
+
For an existing current-schema SQLite file, `cave doctor` reports these payload
|
|
922
|
+
and flag problems through `store.rows` before export is attempted. The same check
|
|
923
|
+
rejects nonnumeric or out-of-range confidence and nonnumeric, nonpositive or
|
|
924
|
+
infinite sigma levels; a null sigma level retains the legacy default of 2. It checks
|
|
925
|
+
stored history without changing it and omits claim contents and identifiers
|
|
926
|
+
from the diagnostic. This targeted check is not a complete semantic audit of
|
|
927
|
+
every historical field.
|
|
928
|
+
|
|
251
929
|
## Tests
|
|
252
930
|
|
|
253
931
|
```
|
|
@@ -263,6 +941,197 @@ cross-actor retraction, stamped round-trips), the §26 resolution policy
|
|
|
263
941
|
(human-over-ingest precedence, reliability weighting, longest-prefix
|
|
264
942
|
specificity, polarity contests, no self-elevation, alias-widened
|
|
265
943
|
groups, contested ranking), the §11.2 topic reads, edge persistence,
|
|
266
|
-
registry and lifecycle rebuild across reopen, transactional strict ingest and export
|
|
944
|
+
registry and lifecycle rebuild across reopen and historical qualifier-parent boundaries, transactional strict ingest and export
|
|
267
945
|
round-trips, ordered schema migration, newer-version rejection, transactional
|
|
268
946
|
rollback/retry, and closed-file backup recovery.
|
|
947
|
+
|
|
948
|
+
## Empty scoped-export trial
|
|
949
|
+
|
|
950
|
+
When sensitivity filtering selects no claims, export returns empty text before
|
|
951
|
+
reading lineage or building historical-to-current identity maps. The scoped claim
|
|
952
|
+
query still runs inside the read snapshot, validates the sensitivity ceiling and
|
|
953
|
+
determines that nothing is visible. This does not make empty exports constant-time.
|
|
954
|
+
|
|
955
|
+
Run `node scripts/empty-export-bench.mjs` from the repository root. The fixture
|
|
956
|
+
contains restricted claims with qualifier edges and exports at the public ceiling,
|
|
957
|
+
with transaction annotations enabled. Setup and output checks are outside timing;
|
|
958
|
+
each result is the median of five calls after one warm-up. A local macOS ARM64
|
|
959
|
+
trial with 10,000 claims and 5,000 edges measured:
|
|
960
|
+
|
|
961
|
+
| Export mode | Node 24.16.0 before → after | Node 26.5.0 before → after |
|
|
962
|
+
|---|---|---|
|
|
963
|
+
| Full history | 6.89 → 3.46 ms | 6.86 → 3.45 ms |
|
|
964
|
+
| Current only | 21.09 → 10.62 ms | 20.96 → 10.69 ms |
|
|
965
|
+
|
|
966
|
+
These figures describe this empty-output fixture, not exports containing visible
|
|
967
|
+
claims. Existing adapter tests cover empty sensitivity scopes and snapshot cleanup.
|
|
968
|
+
|
|
969
|
+
## Current-export history trial
|
|
970
|
+
|
|
971
|
+
Current export resolves edge endpoints directly when they are selected current
|
|
972
|
+
rows. It builds the historical ID/key map only when an endpoint needs remapping,
|
|
973
|
+
restricts it to IDs referenced by stored edges, and shares it across the remaining
|
|
974
|
+
endpoints within the same read snapshot.
|
|
975
|
+
An export without edges does not build it. Historical qualifier edges still
|
|
976
|
+
remap to the current rows, preserving their conditions.
|
|
977
|
+
Plain and annotated current exports first validate the semantic keys of all
|
|
978
|
+
selected current rows. This also prevents a corrupted newer claim from taking
|
|
979
|
+
another claim’s key and receiving its historical relationships.
|
|
980
|
+
Before a historical endpoint is mapped to a selected current row, export
|
|
981
|
+
reconstructs that endpoint's semantic key from its claim and contexts and checks
|
|
982
|
+
it against the stored key. A mismatch fails plain and annotated current exports
|
|
983
|
+
with the historical claim ID, instead of attaching the edge to an unrelated
|
|
984
|
+
current claim. Successful verification is cached per endpoint within the read
|
|
985
|
+
snapshot. This checks historical rows used for remapping, not every omitted row;
|
|
986
|
+
`doctor` remains the broader historical integrity check. For example, a malformed
|
|
987
|
+
superseded attribute row with no relationships does not block a current export
|
|
988
|
+
of its valid newer revision. Full-history export includes and rejects that row.
|
|
989
|
+
Adding an edge through the old row makes current export validate it too; repairing
|
|
990
|
+
the old row permits the edge to remap to the newer value. Both SQLite adapter
|
|
991
|
+
contracts verify this boundary and unchanged database contents on rejection. The measurements below
|
|
992
|
+
predate this additional verification.
|
|
993
|
+
|
|
994
|
+
Run `node scripts/current-export-history-bench.mjs` from the repository root.
|
|
995
|
+
It measures annotated current exports after 100, 1,000 and 10,000 revisions of
|
|
996
|
+
one attribute, with no edge, a current-row edge, or a historical-row edge.
|
|
997
|
+
Setup and output checks are outside timing; medians use five calls after one
|
|
998
|
+
warm-up. A local macOS ARM64 trial at 10,000 revisions measured:
|
|
999
|
+
|
|
1000
|
+
| Edge case | Node 24.16.0 before → after | Node 26.5.0 before → after |
|
|
1001
|
+
|---|---|---|
|
|
1002
|
+
| None | 6.67 → 1.17 ms | 6.43 → 1.16 ms |
|
|
1003
|
+
| Current row | 6.71 → 1.19 ms | 6.96 → 1.18 ms |
|
|
1004
|
+
| Historical row | 6.67 → 1.20 ms | 6.99 → 1.19 ms |
|
|
1005
|
+
|
|
1006
|
+
The before column predates both lazy map construction and limiting the map to
|
|
1007
|
+
edge endpoints. The fallback still scans stored edges and resolves their IDs;
|
|
1008
|
+
unrelated historical revisions no longer enter the map.
|
|
1009
|
+
|
|
1010
|
+
`node scripts/dense-current-export-bench.mjs` is the control with every exported
|
|
1011
|
+
parent requiring remapping. With 3,000 historical rows, 2,000 current rows and
|
|
1012
|
+
1,000 edges, narrowing the map changed local medians from 52.41 to 50.00 ms on
|
|
1013
|
+
Node 24 and 48.18 to 47.35 ms on Node 26. It uses the same warm-up, five-sample
|
|
1014
|
+
method and output checks outside timing. These are fixture measurements, not a
|
|
1015
|
+
general export work bound.
|
|
1016
|
+
|
|
1017
|
+
A refreshed checkpoint includes historical endpoint semantic-key verification.
|
|
1018
|
+
The same scripts, run sequentially on Node 24.21.0 and Node 26.8.1, retain their
|
|
1019
|
+
output checks and one warm-up/five-call medians:
|
|
1020
|
+
|
|
1021
|
+
| Current workload | Node 24.21.0 median | Node 26.8.1 median |
|
|
1022
|
+
|---|---:|---:|
|
|
1023
|
+
| 10,000 revisions, no edge | 1.214 ms | 1.201 ms |
|
|
1024
|
+
| 10,000 revisions, current-row edge | 1.225 ms | 1.220 ms |
|
|
1025
|
+
| 10,000 revisions, historical-row edge | 1.241 ms | 1.252 ms |
|
|
1026
|
+
| 3,000 history rows, 2,000 current rows, 1,000 historical edges | 74.736 ms | 67.876 ms |
|
|
1027
|
+
|
|
1028
|
+
[Current samples and source hashes](../../benchmarks/verified-historical-export-review.json)
|
|
1029
|
+
include all three sizes in each workload. These current-state measurements do
|
|
1030
|
+
not isolate endpoint verification's cost from runtime and other implementation
|
|
1031
|
+
changes since the older trials. The dense case exercises verification for every
|
|
1032
|
+
remapped parent; the sparse case checks that long unrelated history remains
|
|
1033
|
+
outside the endpoint map. They establish neither a process-memory limit nor a
|
|
1034
|
+
general elapsed-time bound.
|
|
1035
|
+
|
|
1036
|
+
Historical verification now prepares its context and tag queries once per export,
|
|
1037
|
+
reusing them for each checked endpoint within the same read snapshot. It reads
|
|
1038
|
+
the same data and retains full row reconstruction and semantic-key checks.
|
|
1039
|
+
Repeating the dense trial after this change records 69.659 ms on Node 24.21.0
|
|
1040
|
+
and 59.309 ms on Node 26.8.1 for 1,000 edges, compared with 74.736 and 67.876 ms
|
|
1041
|
+
in the preceding checkpoint. [Statement-reuse samples and hashes](../../benchmarks/historical-export-statements-review.json)
|
|
1042
|
+
retain all three sizes and the same warm-up/five-call controls. The measurements
|
|
1043
|
+
support this dense workload; they do not establish a universal speedup.
|
|
1044
|
+
|
|
1045
|
+
## Transaction-index trial
|
|
1046
|
+
|
|
1047
|
+
An isolated Node 26.5.0 / SQLite 3.53.3 macOS arm64 trial compared the version-1
|
|
1048
|
+
schema with a temporary `CREATE INDEX trial_tx ON cave_claim(tx)`. Each fixture
|
|
1049
|
+
contained one distinct attribute claim per entity. Five samples each executed
|
|
1050
|
+
100 prepared head queries, verifying identical results before and after adding
|
|
1051
|
+
the index. A bounded query used the fixture's middle transaction as `tx <= ?`.
|
|
1052
|
+
|
|
1053
|
+
| Claims | Head reads, existing / indexed | Bounded reads, existing / indexed | Added database pages in bytes |
|
|
1054
|
+
|---|---:|---:|---:|
|
|
1055
|
+
| 1,000 | 4.52 / 0.075 ms | 4.96 / 0.085 ms | 49,152 |
|
|
1056
|
+
| 10,000 | 44.38 / 0.071 ms | 47.79 / 0.084 ms | 454,656 |
|
|
1057
|
+
| 50,000 | 218.68 / 0.072 ms | 235.89 / 0.086 ms | 2,252,800 |
|
|
1058
|
+
|
|
1059
|
+
Query plans used the existing `(claim_key, tx)` covering index before the trial
|
|
1060
|
+
and the transaction-leading index afterward. In separate five-sample fixtures
|
|
1061
|
+
starting with 10,000 claims, appending 100 claims in individual transactions
|
|
1062
|
+
took a median 55.66 ms without the index and 10.38 ms with it; row counts and
|
|
1063
|
+
SQLite integrity checks passed. The receive-clock head read benefits each append.
|
|
1064
|
+
These in-memory measurements do not characterize disk durability or all write
|
|
1065
|
+
workloads. They motivated schema version 2, which installs `idx_cave_tx` through
|
|
1066
|
+
an atomic migration and validates its table, column, binary collation and non-partial coverage.
|
|
1067
|
+
A same-named index with another collation is incompatible: it cannot provide
|
|
1068
|
+
the binary transaction range search used by bounded head reads.
|
|
1069
|
+
|
|
1070
|
+
A follow-up using the production schema-2 migration on the same runtime and
|
|
1071
|
+
five fresh 10,000-row fixtures measured 0.079 ms for 100 prepared head reads
|
|
1072
|
+
and 10.83 ms for 100 individual appends (medians). Every fixture retained
|
|
1073
|
+
10,100 rows and passed SQLite integrity checks. Native and WASM regression
|
|
1074
|
+
tests also verify indexed global and bounded query plans, migration rollback
|
|
1075
|
+
and retry, and unchanged annotated exports.
|
|
1076
|
+
|
|
1077
|
+
|
|
1078
|
+
## Claim capture cost trial
|
|
1079
|
+
|
|
1080
|
+
Run `node --disable-warning=ExperimentalWarning scripts/claim-capture-bench.mjs`
|
|
1081
|
+
alone. It appends pre-canonicalized batches of 100 and 1,000 claims to fresh
|
|
1082
|
+
in-memory stores: simple relations, or numeric attributes with uncertainty,
|
|
1083
|
+
confidence, source/scope contexts, a tag and a comment. Canonicalization, store
|
|
1084
|
+
creation and result validation are outside timing; each median uses five fresh
|
|
1085
|
+
stores after one warm-up store. Checks verify inserted counts and decode the
|
|
1086
|
+
first and last claim records, including numeric metadata and source provenance.
|
|
1087
|
+
|
|
1088
|
+
On macOS arm64, Node 24.16.0 and 26.5.0 produced these medians. The baseline
|
|
1089
|
+
bypassed only the new per-claim capture helper; option and edge capture remained
|
|
1090
|
+
in place. The final source was restored before the after runs. The benchmark
|
|
1091
|
+
itself measures the checked-out code and exposes no bypass option.
|
|
1092
|
+
|
|
1093
|
+
| Claims | Payload | Node 24 before → after | Node 26 before → after |
|
|
1094
|
+
|---|---|---|---|
|
|
1095
|
+
| 100 | Relation | 1.57 → 1.60 ms | 1.65 → 1.65 ms |
|
|
1096
|
+
| 100 | Numeric with metadata | 4.27 → 4.33 ms | 4.12 → 4.50 ms |
|
|
1097
|
+
| 1,000 | Relation | 11.84 → 11.69 ms | 11.85 → 12.37 ms |
|
|
1098
|
+
| 1,000 | Numeric with metadata | 38.47 → 37.33 ms | 35.00 → 35.41 ms |
|
|
1099
|
+
|
|
1100
|
+
These measurements describe complete append calls, not isolated copy costs.
|
|
1101
|
+
Baseline runs preceded final runs; cache and execution-order effects remain
|
|
1102
|
+
possible, and the varying differences do not establish a speedup or universal
|
|
1103
|
+
overhead bound. File durability, contention, large batches and WASM are outside
|
|
1104
|
+
this trial. Separate getter regressions establish projection consistency.
|
|
1105
|
+
|
|
1106
|
+
## Current-search trial
|
|
1107
|
+
|
|
1108
|
+
This trial uses native Node SQLite in memory; it does not measure SQL.js/WASM
|
|
1109
|
+
or browser rendering. The shared adapter tests check search semantics on both
|
|
1110
|
+
native SQLite and SQL.js separately.
|
|
1111
|
+
|
|
1112
|
+
The returned-row limit does not cap SQL work over matching history. Run
|
|
1113
|
+
`node scripts/current-search-bench.mjs` alone from the repository root to
|
|
1114
|
+
compare historical and current-only search at 100, 1,000 and 10,000 revisions
|
|
1115
|
+
of one fact, alongside an older current fact. The benchmark requests five
|
|
1116
|
+
results and checks that current-only mode retains both current facts. It
|
|
1117
|
+
reports eleven-call medians after two warmups, including row decoding and
|
|
1118
|
+
excluding fixture construction and assertions. These are local measurements,
|
|
1119
|
+
not CI thresholds or general latency guarantees.
|
|
1120
|
+
|
|
1121
|
+
An additional workload combines 10,000 revisions with 10,000 unrelated claim
|
|
1122
|
+
keys to expose current-row resolution costs beyond matching history. On the
|
|
1123
|
+
audit machine, historical/current-only medians were 11.75/19.56 ms on Node
|
|
1124
|
+
24.21.0 and 11.36/19.07 ms on Node 26.8.1. The two matching current facts
|
|
1125
|
+
remain the same; unrelated store size can still affect query execution cost.
|
|
1126
|
+
|
|
1127
|
+
Checking for a newer revision through the existing `(claim_key, tx)` index
|
|
1128
|
+
replaced whole-store grouping in current-only search. In the same workload,
|
|
1129
|
+
current-only medians fell from 19.56 to 11.91 ms on Node 24.21.0 and from
|
|
1130
|
+
19.07 to 12.35 ms on Node 26.8.1. The check considers newer rows regardless of
|
|
1131
|
+
sensitivity or confidence before applying visibility and result limits;
|
|
1132
|
+
a hidden replacement or retraction cannot revive an older match.
|
|
1133
|
+
|
|
1134
|
+
The 10,000-revision fixture without unrelated keys measured about 11.8 ms
|
|
1135
|
+
with the indexed check, compared with about 10.9 ms in the earlier grouped
|
|
1136
|
+
run. This is a workload tradeoff, not a universal speedup; broad match sets
|
|
1137
|
+
still require searches and ordering work.
|