@cavelang/store 0.34.0 → 0.36.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (73) hide show
  1. package/README.md +884 -15
  2. package/dist/src/backup.d.ts.map +1 -1
  3. package/dist/src/backup.js +143 -70
  4. package/dist/src/backup.js.map +1 -1
  5. package/dist/src/capture-claim.d.ts +4 -0
  6. package/dist/src/capture-claim.d.ts.map +1 -0
  7. package/dist/src/capture-claim.js +26 -0
  8. package/dist/src/capture-claim.js.map +1 -0
  9. package/dist/src/descriptor.d.ts +3 -0
  10. package/dist/src/descriptor.d.ts.map +1 -0
  11. package/dist/src/descriptor.js +21 -0
  12. package/dist/src/descriptor.js.map +1 -0
  13. package/dist/src/error-message.d.ts +3 -0
  14. package/dist/src/error-message.d.ts.map +1 -0
  15. package/dist/src/error-message.js +10 -0
  16. package/dist/src/error-message.js.map +1 -0
  17. package/dist/src/index.d.ts +1 -1
  18. package/dist/src/index.d.ts.map +1 -1
  19. package/dist/src/locate.d.ts +13 -1
  20. package/dist/src/locate.d.ts.map +1 -1
  21. package/dist/src/locate.js +35 -13
  22. package/dist/src/locate.js.map +1 -1
  23. package/dist/src/node-adapter.d.ts.map +1 -1
  24. package/dist/src/node-adapter.js +29 -1
  25. package/dist/src/node-adapter.js.map +1 -1
  26. package/dist/src/provenance.d.ts +4 -0
  27. package/dist/src/provenance.d.ts.map +1 -1
  28. package/dist/src/provenance.js +36 -1
  29. package/dist/src/provenance.js.map +1 -1
  30. package/dist/src/query-sql.d.ts.map +1 -1
  31. package/dist/src/query-sql.js +4 -2
  32. package/dist/src/query-sql.js.map +1 -1
  33. package/dist/src/read-snapshot.d.ts +4 -0
  34. package/dist/src/read-snapshot.d.ts.map +1 -0
  35. package/dist/src/read-snapshot.js +22 -0
  36. package/dist/src/read-snapshot.js.map +1 -0
  37. package/dist/src/record.d.ts.map +1 -1
  38. package/dist/src/record.js +102 -25
  39. package/dist/src/record.js.map +1 -1
  40. package/dist/src/resolve.d.ts.map +1 -1
  41. package/dist/src/resolve.js +42 -18
  42. package/dist/src/resolve.js.map +1 -1
  43. package/dist/src/row.d.ts.map +1 -1
  44. package/dist/src/row.js +45 -17
  45. package/dist/src/row.js.map +1 -1
  46. package/dist/src/runtime.d.ts.map +1 -1
  47. package/dist/src/runtime.js +8 -4
  48. package/dist/src/runtime.js.map +1 -1
  49. package/dist/src/schema.d.ts +1 -2
  50. package/dist/src/schema.d.ts.map +1 -1
  51. package/dist/src/schema.js +116 -15
  52. package/dist/src/schema.js.map +1 -1
  53. package/dist/src/store.d.ts +17 -5
  54. package/dist/src/store.d.ts.map +1 -1
  55. package/dist/src/store.js +451 -104
  56. package/dist/src/store.js.map +1 -1
  57. package/package.json +4 -4
  58. package/src/backup.ts +141 -65
  59. package/src/capture-claim.ts +30 -0
  60. package/src/descriptor.ts +19 -0
  61. package/src/error-message.ts +8 -0
  62. package/src/index.ts +1 -1
  63. package/src/locate.ts +45 -13
  64. package/src/node-adapter.ts +28 -1
  65. package/src/provenance.ts +33 -1
  66. package/src/query-sql.ts +4 -2
  67. package/src/read-snapshot.ts +21 -0
  68. package/src/record.ts +102 -26
  69. package/src/resolve.ts +43 -20
  70. package/src/row.ts +29 -2
  71. package/src/runtime.ts +8 -4
  72. package/src/schema.ts +117 -18
  73. package/src/store.ts +452 -125
package/README.md CHANGED
@@ -20,6 +20,47 @@ store.exportText({ current: true }) // canonical text through internal
20
20
  store.exportText({ maxSensitivity: 'restricted' }) // complete portable history
21
21
  ```
22
22
 
23
+ The native runtime requires Node 24.16.0+ within 24.x or 26.1.0+ within
24
+ 26.x. Older bindings, including Node 22, truncate returned TEXT at embedded
25
+ NUL characters even though SQLite retains all stored bytes. The adapter checks
26
+ text decoding once before opening any caller database and rejects a truncating
27
+ runtime with an upgrade message. It neither strips characters nor rewrites
28
+ stored identities. `node scripts/check-sqlite-text.mjs` isolates the native
29
+ round-trip independently of CAVE.
30
+ Probe verification is cached only after both decoding and probe close succeed.
31
+ A failed attempt leaves caller database paths unopened and runs the probe again
32
+ on the next attempt. Simultaneous probe-read and close failures are retained in
33
+ `AggregateError`, with the read error as cause and first entry; a close-only
34
+ failure propagates directly. Each created probe receives one close attempt.
35
+
36
+ Claim appends require well-formed Unicode strings. `ingest` and `insertResult`
37
+ reject an unpaired UTF-16 surrogate with `TypeError` rather than letting SQLite's
38
+ UTF-8 conversion silently replace it with `�`. The batch preflight checks claim
39
+ columns, raw text, contexts, tags and explicit provenance before allocating or
40
+ observing transaction IDs; rows and vocabulary remain unchanged on rejection.
41
+ This storage check also applies to lenient ingestion and the browser adapter.
42
+ Valid surrogate pairs (including emoji), accented text and embedded NUL remain
43
+ supported. Direct SQL through the exposed database handle is outside this
44
+ claim-append validation path.
45
+
46
+ Direct search and lookup arguments also reject unpaired surrogates with
47
+ `TypeError` before SQLite binding. This covers entity and alias traversal,
48
+ topics, tags, contexts, provenance and belief-key lookups, as well as provenance
49
+ and edge reads by ID. Invalid text cannot silently select a record containing
50
+ `�`; explicitly authored replacement characters remain valid lookup text.
51
+ Search validates malformed Unicode even with a zero result limit or raw full-text
52
+ syntax. Rejected reads leave history unchanged and subsequent valid reads work
53
+ on both native and browser adapters. Direct SQL remains outside this validation.
54
+
55
+ The upstream byte-length conversion is in
56
+ [Node PR #61954](https://github.com/nodejs/node/pull/61954), shipped in
57
+ [Node 24.16.0](https://nodejs.org/en/blog/release/v24.16.0) and
58
+ [Node 26.1.0](https://nodejs.org/en/blog/release/v26.1.0). Boundary-runtime
59
+ checks reproduce truncation on 24.0.0 and 26.0.0 and preserve complete text on
60
+ 24.16.0 and 26.1.0. A general SQL rewriting adapter was not introduced: it would
61
+ need to preserve arbitrary result types, duplicate column names and statement
62
+ semantics to avoid creating a second compatibility problem.
63
+
23
64
  Runtime selection is explicit at non-Node composition boundaries:
24
65
 
25
66
  ```ts
@@ -29,11 +70,73 @@ declare const adapter: SqliteAdapter
29
70
  const store = openWith(adapter, ':memory:')
30
71
  ```
31
72
 
73
+ Adapter result columns remain own data properties even when their names are
74
+ `__proto__`, `constructor` or `toString`. Repeated output names retain the last
75
+ column value. The shared native/WASM contract checks `get`, `all` and JSON
76
+ serialization so adapter composition does not silently lose named results.
77
+
32
78
  `SqliteAdapter` declares the SQL statement surface plus immediate/savepoint
33
79
  transactions, FTS4 or FTS5, optional extension loading, and optional exact
34
80
  snapshot support. The concrete Node adapter is also available from
35
81
  `@cavelang/store/adapter/node`. The shared adapter contract suite runs against
36
82
  both Node SQLite and the website's SQL.js/WASM adapter.
83
+ Statement `run()` discards result rows and completes the statement before
84
+ reporting `changes` and `lastInsertRowid`, including writes with a `RETURNING`
85
+ clause. Use `all()` or `get()` when the returned rows themselves are needed.
86
+ Each statement call supplies fresh positional bindings: omitted parameters are
87
+ `NULL`, and excess parameters fail before a write executes. The shared contract
88
+ checks reuse after failure and preserves empty and binary blobs independently
89
+ of later changes to the caller's input buffer.
90
+
91
+ ## Resource lifecycle
92
+
93
+ `ingest.strict` must be a boolean when supplied and is captured before parsing;
94
+ malformed flags cannot fall back to a partial, lenient import. The shared append
95
+ `lifecycle` flag on `ingest` and `insertResult` likewise requires a boolean,
96
+ captured before claims are appended. Omitted flags remain false.
97
+
98
+ Append operations capture replay IDs, contexts, source/lifecycle settings and
99
+ provenance fields before preparing a batch, copying their arrays. Changing
100
+ getters cannot substitute an unvalidated replay ID or change metadata between
101
+ claims in that batch.
102
+ Identity allocation and row insertion iterate the prepared claims, so a changing
103
+ claim-collection getter cannot replace or discard the batch after preparation.
104
+ Canonical-result edges also capture their parent index, role and child index
105
+ before insertion, so replay duplicate checks and writes use the same fields.
106
+ Claims capture their declared fields and nested terms, values, contexts and tags
107
+ before storage projections are derived. Columns, emitted text and semantic keys
108
+ therefore use the same values even when structured input contains getters.
109
+
110
+ Combined operation/cleanup errors retain each underlying diagnostic in their
111
+ message as well as in `AggregateError.errors`. This applies to transaction
112
+ rollback, final store cleanup, file descriptors, native text probes and backup
113
+ verification/removal. Backup cleanup errors retain the temporary path and any
114
+ already-published destination status. Existing causes and lone-error identities
115
+ are preserved, so message-only CLI output and programmatic inspection both carry
116
+ the failure details. If a thrown value cannot be converted to text, its message
117
+ uses `[unprintable thrown value]`; the original value remains available in
118
+ `AggregateError.errors`.
119
+
120
+ Call `store.close()` when finished. Components that retain resources derived
121
+ from a store can register synchronous cleanup with `store.onClose(callback)`;
122
+ the returned function unregisters only that registration. Registering the same
123
+ callback more than once creates independent subscriptions; calling one unsubscribe
124
+ function repeatedly is harmless. Cleanup runs before the SQLite connection closes, and reentrant or repeated successful closes do not repeat it.
125
+ Registering cleanup during or after a successful close throws. Returning a
126
+ Promise or thenable reports a synchronous-cleanup error; it is not awaited.
127
+ Native Promise rejection is observed without invoking a custom thenable's work,
128
+ and remaining cleanup callbacks and the database close are still attempted.
129
+ This cannot cancel work already started by the callback; callers must finish
130
+ asynchronous work before closing the store.
131
+
132
+ Every registered callback and the database close are attempted even if cleanup
133
+ throws. A single error is rethrown; multiple errors are reported in an
134
+ `AggregateError`. If the database close itself fails, another `close()` retries
135
+ it without repeating callbacks that were already attempted. Closing the raw
136
+ `store.db` bypasses store-owned cleanup; use `store.close()` for normal ownership.
137
+ Combined callback and database-close failures retain callback errors first and
138
+ the database error last. A failed database close leaves cleanup registration
139
+ available for the retry; newly registered callbacks run once on that retry.
37
140
 
38
141
  ## Composable query SQL
39
142
 
@@ -53,7 +156,9 @@ retraction; consumers add those predicates for their read mode. `aliasEdges`,
53
156
  `aliasPairs`, `aliasClosure`, and `aliasSame` implement entity-only current
54
157
  positive ALIAS semantics. `transactionBounds` and `asOfBoundary` use
55
158
  `Time.parseBoundary` for shared whole-period/whole-second semantics
56
- (offset-less timestamps are UTC) and exact-transaction rules. Fragments own no
159
+ (offset-less timestamps are UTC) and exact-transaction rules. Transaction periods
160
+ are clipped at the Unix epoch: wholly earlier periods are empty, while periods
161
+ crossing 1970-01-01 retain their overlap with UUID transaction time. Fragments own no
57
162
  ordering or outer filtering, so consumers can extend them without copying
58
163
  semantic clauses.
59
164
 
@@ -63,9 +168,63 @@ Database rows (`Row.t`, snake_case columns) are the storage-oriented API and
63
168
  may evolve with the schema. Use `store.recordOf(row)` for serialized or
64
169
  cross-process data. It returns `cave.claim/v1`: transaction identity (`id`,
65
170
  `tx`, `key`), canonical CAVE text, the semantic `Claim.t`, and explicit
66
- provenance dimensions. `Record.encode`/`Record.decode` own the JSON contract;
67
- decoding rejects unknown versions and verifies transaction, key, and canonical
68
- text identity. The checked-in v1 fixture is the compatibility baseline for
171
+ provenance dimensions. `Record.encode`/`Record.decode` own the JSON contract.
172
+ Record construction and decoding require `id = tx` as one canonical lowercase
173
+ UUIDv7. This matches append and sync identity (§28.1). Both also recompute
174
+ the claim key, including its context set, and reject disagreement with the stored
175
+ or supplied key. Construction captures `id`, `tx` and `claim_key` once. The
176
+ decoder additionally compares emitted canonical text with the supplied text.
177
+ Construction captures and validates provenance before returning a record,
178
+ rejecting empty or malformed entries and unknown dimensions. This also rejects
179
+ empty or binary historical provenance read through `store.recordOf(row)`;
180
+ repairing that data permits structured JSON encode/decode again. Its captured arrays
181
+ preserve valid ordering and duplicates and are independent of the caller's
182
+ provenance arrays. It likewise captures the semantic claim before computing its
183
+ key and canonical text. Construction and decoding share the claim field checks,
184
+ including boolean flags, raw text, terms, payloads and optional metadata. Invalid
185
+ programmatic claims are rejected before a record is returned. Claim getters are
186
+ read during capture and later edits
187
+ to caller-owned terms, values, contexts or tags cannot invalidate the returned
188
+ record. Constructor claim and provenance inputs must be structured-cloneable.
189
+ It operates on the
190
+ record alone, without consulting a store. Use `store.recordOf(row)` on a row
191
+ from the intended store when you need that store's evidence.
192
+
193
+ Object input is structured-cloned before validation, so decoded nested values
194
+ are independent of caller-owned arrays and objects. Enumerable getters are read
195
+ during capture; validation and the returned record use that captured data.
196
+ Object inputs must be structured-cloneable data; functions and proxies are not
197
+ accepted. JSON text is parsed into independently owned data directly. The result
198
+ is not frozen, so callers can still deliberately edit their decoded copy.
199
+
200
+ Construction and decoding apply these field checks before accepting the record:
201
+
202
+ | Field | Required representation |
203
+ |---|---|
204
+ | Format and version | `cave.claim`, version `1`; unknown versions are rejected. |
205
+ | Claim flags and authored text | Boolean `negated` and `importance`; string `raw`. |
206
+ | Subject and relation object | Term kind `entity`, `text` or `code`, with string `text`. |
207
+ | Payload | Kind `relation`, `attribute`, `metric` or `none`; attributes have string names. |
208
+ | Attribute, metric and uncertainty values | Supported value kind, string `raw`, Boolean `approx`; supplied units are strings and numeric scalars/endpoints are finite numbers. |
209
+ | Contexts | Dense arrays of strings; empty arrays are valid. |
210
+ | Provenance | Exactly the four enumerable own dimension fields, containing dense arrays of nonempty strings without unpaired UTF-16 surrogates; empty arrays are valid. |
211
+ | Tags | Objects with string `key` and optional string `value`; flat tags omit `value`. |
212
+ | Optional sigma level | Positive finite number; canonical export preserves tiny and large magnitudes as plain decimals. |
213
+ | Optional comment | String. |
214
+
215
+ Record provenance uses the same validation as `Provenance.parse`, while retaining
216
+ the order and duplicates of valid input arrays. Array holes, empty or non-string
217
+ entries, malformed Unicode and unknown dimensions are rejected for both object
218
+ and JSON input. Matching canonical text cannot compensate for malformed field
219
+ types. Attribute, metric and uncertainty values are reparsed from `raw` with
220
+ the core value parser (or the text/code literal constructor). Their kind,
221
+ approximation, numeric scalar, trajectory endpoints and unit must agree with
222
+ that result. Numeric projections remain absent when parsing cannot represent
223
+ the authored number as a finite JavaScript number. Provenance dimensions are
224
+ validated structurally rather than derived from contexts.
225
+ Uncertainty deltas additionally require a positive finite scalar, matching the
226
+ authored `+/-` rule; zero, negative values, trajectories and literals are rejected.
227
+ The checked-in v1 fixture is the compatibility baseline for
69
228
  future decoders.
70
229
 
71
230
  ## Text stores
@@ -73,7 +232,9 @@ future decoders.
73
232
  `openAt(path, { intent })` opens what a `--db` path names, by content (spec
74
233
  §13.7): a SQLite file, `:memory:` fresh, and any other file as CAVE text
75
234
  replayed into an in-memory store with `cave import` semantics — no actor
76
- stamp, so the claims are the ones importing the file would store. `intent:
235
+ stamp, so the claims are the ones importing the file would store. Transaction
236
+ IDs and provenance payloads in annotations are ignored under these import
237
+ semantics; use `cave sync` to materialize an annotated replica. `intent:
77
238
  'read'` never touches the filesystem: SQLite opens read-only (a
78
239
  write-protected store serves), an older schema is reported with the command
79
240
  that migrates it instead of being migrated, and a missing path throws instead
@@ -86,10 +247,40 @@ either. `intent: 'write'` (the default) creates a missing SQLite store,
86
247
  migrates an older one, and throws on a text file with the materialization
87
248
  hint, since nothing appended in memory would outlive the process. A text file
88
249
  with a line that fails to parse fails the load, naming every line.
250
+ Text-store files must also be valid UTF-8. Invalid bytes raise `LocateError`
251
+ before an in-memory store is created or the assembler runs, rather than silently
252
+ changing claim text during decoding. The source file stays untouched; correcting
253
+ it allows normal replay, including explicit replacement characters and emoji.
254
+ If replay or source assembly fails after the in-memory store opens, that store
255
+ is closed before the failure is returned. If its cleanup also throws, an
256
+ `AggregateError` retains the load failure as cause and first entry and the
257
+ cleanup failure second. Both appear in the diagnostic, with unprintable values
258
+ retained in the error entries. Correcting assembly permits a fresh load; the
259
+ source text file is never rewritten by this cleanup.
89
260
  `kindOf(path)` returns `memory | sqlite | text | missing`, `isStoreFile(path)`
90
261
  is the SQLite-header check, `openText(path)` replays a text file directly, and
91
262
  `open(path, { access })` exposes the same `read-only | no-migrate | migrate`
92
263
  choice to callers that already know the file is SQLite.
264
+ Direct opens require access to be `read-only`, `no-migrate` or `migrate`.
265
+ Unsupported values, including `null`, throw `TypeError` before calling the
266
+ SQLite adapter; they cannot silently open a writable connection. An omitted
267
+ or `undefined` access retains the `migrate` default.
268
+ `openAt` captures its declared options explicitly, including inherited and
269
+ non-enumerable properties. A supplied base registry reaches memory, SQLite,
270
+ text and newly created stores; an assembly callback runs for text stores only.
271
+ Option storage style does not silently remove vocabulary or source assembly.
272
+ An explicitly supplied intent must be `read`, `scratch` or `write`; unsupported
273
+ values throw `TypeError` before file detection, opening or assembly. They are
274
+ not coerced into a mode. An omitted or `undefined` intent retains the `write`
275
+ default. Correct the intent and retry without changing the source store.
276
+ Header detection accumulates short reads until all 16 SQLite signature bytes
277
+ are available or EOF is reached. An incomplete signature is treated as text;
278
+ a complete signature still identifies SQLite regardless of the filename.
279
+ Header detection attempts descriptor close once. If reading and closing both
280
+ fail, an `AggregateError` preserves the read error as cause and first entry and
281
+ the close error second; a close-only failure propagates directly. Detection does
282
+ not classify the file or open a database after either failure. Snapshot hashing,
283
+ file syncing, and header detection share this descriptor ownership contract.
93
284
 
94
285
  ```ts
95
286
  import { openAt } from '@cavelang/store'
@@ -101,27 +292,175 @@ notes.currentBeliefs() // the file's claims, assembled in
101
292
  ## Semantics
102
293
 
103
294
  - **Schema upgrades are explicit** (§13.2.1): `PRAGMA user_version` records
104
- the local format (current version 1; version 0 is the unversioned legacy
295
+ the local format (current version 2; version 0 is the unversioned legacy
105
296
  baseline). `open()` rejects newer stores, applies every older migration in
106
297
  order with its backfill and version update in one `BEGIN IMMEDIATE`
107
- transaction, then validates required tables, indexes, and columns; with
298
+ transaction, then validates required tables, indexes, and columns (a view
299
+ with matching columns cannot substitute for a required table, and the search
300
+ index must be a virtual table exposing the hidden FTS search column).
301
+ Validation also requires the `cave_claim(id)` primary key and the ordered
302
+ `cave_provenance(claim_id, dimension, value)` primary key, with binary
303
+ comparison for every key column. Case-insensitive or trailing-space-insensitive
304
+ keys can collapse distinct provenance values; matching column names alone do
305
+ not guarantee claim identity or provenance deduplication. Required secondary
306
+ indexes must belong to the expected table, cover the ordered columns with
307
+ binary collation, and be non-unique and non-partial. A same-named unique
308
+ subject index, for example, would prevent ordinary claims about one subject.
309
+ Required text fields in ordinary tables must have SQLite TEXT affinity;
310
+ equivalent declarations such as `VARCHAR` and `CLOB` are accepted. Numeric
311
+ affinity can turn source identities `001` and `1` into the same integer and
312
+ silently deduplicate them. This check excludes virtual FTS columns and does
313
+ not validate the types of individual historical rows.
314
+ Numeric claim fields must have INTEGER, REAL or NUMERIC affinity. TEXT or
315
+ untyped/BLOB declarations can change range-comparison semantics; for example,
316
+ a text-affinity numeric projection can exclude 10 from `value_num > 3`.
317
+ Compatible numeric declarations retain fractional values and numeric ordering.
318
+ For STRICT tables, fraction-bearing `value_num`, `delta_num`, `sigma_level`
319
+ and `conf` fields must be REAL: STRICT INTEGER fields reject fractional
320
+ writes. STRICT ANY has no numeric coercion and is not a compatible numeric
321
+ declaration. The standard generated schema remains non-STRICT.
322
+ Validation reports the incompatible definition instead of silently replacing
323
+ it. With
108
324
  `access: 'read-only'` or `'no-migrate'` it validates only and reports an
109
325
  older version instead of migrating it. A crash
110
326
  leaves a resumable old or complete new version—never a committed half-step.
327
+ Migration errors identify the version step and retain the original failure
328
+ in `Error.cause`. Unprintable adapter failures use a safe placeholder after
329
+ rollback is attempted, preserving the cause for inspection and retry.
330
+ If rollback also throws, an `AggregateError` retains the migration and rollback
331
+ failures in that order, with the migration failure as its cause. This includes
332
+ SQLite reporting that it already rolled back; a combined error alone does not
333
+ mean the transaction remains active. Direct `Schema.init` callers own recovery
334
+ or closure of their connection; a failed `open()` closes its owned database.
111
335
  Migrations are forward-only; make rollback points by closing all users and
112
336
  copying the closed SQLite file before upgrade.
337
+ Version 2 adds `idx_cave_tx` on `cave_claim(tx)` for global and bounded
338
+ transaction-head reads. The 1→2 migration preserves claims and metadata;
339
+ read-only and no-migrate opens of version 1 report the required upgrade.
113
340
  - **Exact backup is an online SQLite snapshot** (§13.2.2): `backup()` runs
114
341
  `VACUUM INTO` to a temporary sibling, verifies integrity, foreign keys and
115
342
  schema, fsyncs it, computes SHA-256, and atomically publishes it. WAL-visible
116
343
  committed rows and concurrent readers/writers are safe. `restoreBackup()`
117
344
  verifies the source and temporary copy before atomic publication and refuses
118
345
  destination WAL/SHM/journal sidecars. `verifyBackup()` supports independent checks.
346
+ This verification covers SQLite structure, schema and the supplied checksum.
347
+ A structurally valid snapshot can preserve semantic damage, including an
348
+ inconsistent claim key or empty provenance value. That permits an exact copy
349
+ to be retained before diagnosis and repair. Run `cave doctor` on the restored
350
+ copy to check semantic health before using it for export or sync; repairing
351
+ the copy leaves the original source and snapshot unchanged.
352
+ Expected SHA-256 values must be exactly 64 hexadecimal characters, with either
353
+ letter case accepted. Verification and restore validate them before inspecting
354
+ source or destination paths; whitespace, including a trailing newline, is invalid.
355
+ Verification attempts database close once. If validation and close both fail,
356
+ an `AggregateError` retains validation as its cause and first entry and close
357
+ failure second; a close-only failure propagates directly. Failed verification
358
+ does not return checksum metadata or authorize publication. Hashing and file
359
+ syncing also attempt descriptor close once and preserve simultaneous operation
360
+ and close failures in an `AggregateError`, with the operation error first and
361
+ as cause. A descriptor-close failure alone propagates directly.
362
+ If backup or restore fails and temporary-file removal also fails, both errors
363
+ are retained in the same order, with the operation failure as cause. The error
364
+ message names the temporary path for cleanup after resolving the filesystem
365
+ problem. Failure before publication preserves an existing destination; a
366
+ failed removal can leave the temporary file behind.
367
+ If atomic publication succeeds but subsequent cleanup throws, the error says
368
+ `snapshot published to <destination>` and retains the cleanup error as its
369
+ cause. Its message also includes the underlying failure details, including both
370
+ errors if directory synchronization and descriptor closure fail together.
371
+ The new snapshot remains at that destination; the operation does not
372
+ roll publication back. Verify that destination before deciding whether to
373
+ retry. If the outer temporary-removal attempt also fails, its aggregate error
374
+ includes the publication diagnostic in its message and retains it as the
375
+ operation cause.
376
+ Directory synchronization tolerates unsupported-operation errors (`EINVAL`,
377
+ `ENOTSUP`, `ENOSYS`), Windows directory-open restrictions (`EPERM`, `EACCES`,
378
+ `EISDIR`) and sync `EPERM`, and AIX read-only-directory sync `EBADF`.
379
+ Other directory-open/sync errors, including `EIO` and `ENOSPC`, are reported
380
+ after publication; simultaneous sync and descriptor-close errors are retained.
381
+ This distinction follows the [POSIX fsync error categories](https://pubs.opengroup.org/onlinepubs/009695399/functions/fsync.html)
382
+ and the [documented AIX portability limitation](https://www.gnu.org/software/gnulib/manual/html_node/fsync.html).
383
+ An unsupported directory sync does not establish durability across power loss.
384
+ Backup destinations cannot be the live source's main file or WAL/SHM/journal
385
+ files, even with `force`. Source/destination identity checks include hard links
386
+ and resolved parent directories, so missing sidecars cannot bypass protection
387
+ through alternate directory spellings. Restore also rejects hard links to its
388
+ source snapshot.
389
+ Verification requires a standalone source with no WAL/SHM/journal sidecars.
390
+ Otherwise SQLite could report rows from a WAL that the main-file checksum
391
+ and restored bytes do not contain. Use `backup()` on a live store first;
392
+ verify and restore the snapshot it produces.
393
+ Publishing a backup also requires a destination without SQLite sidecars;
394
+ `force` permits replacing the main file, but does not bypass this check.
395
+ Backup and restore check destination sidecars both before preparation and
396
+ immediately before publication. If sidecars appear during preparation, they
397
+ retain the destination and remove the temporary snapshot. These checks do not
398
+ lock out another process: keep all destination users stopped throughout the
399
+ operation, including publication.
400
+ Backup and restore capture write options at entry, so adapter callbacks or
401
+ changing getters cannot alter the replacement decision before publication.
402
+ Replacement requires the literal boolean `force: true`. Other values, including
403
+ the string `"true"` and boxed booleans passed by JavaScript callers, behave as
404
+ `false`; they are not coerced and cannot enable replacement.
405
+ Without `force`, a destination created during snapshot preparation is retained
406
+ and publication fails instead of replacing it.
407
+ Restore destinations cannot be sidecar paths of the source snapshot.
408
+ Verification and restore accept schema versions 1 through the current version,
409
+ validating each version's structure without migrating either file. Restoring
410
+ an older snapshot preserves its checksum and schema version; a later writable
411
+ open migrates the restored copy. Unversioned and newer snapshots are rejected.
412
+ - **Startup owns connection cleanup**: a failed open closes its database even
413
+ when failure occurs before or during schema setup or during later statement preparation
414
+ and vocabulary loading. A successful close preserves the original exception;
415
+ a failed close produces an `AggregateError` containing both failures and their
416
+ messages, with initialization as its cause. A later open can retry normally.
119
417
  - **Append-only** (§9.1): `ingest` only inserts; every row carries a
120
418
  monotonic UUIDv7 in `id` and `tx`, so `MAX(tx)` per `claim_key` is the
121
- current belief. Each ingest call is one SQLite transaction. Outer writes
419
+ current belief. Each ingest call is one SQLite transaction; each newly
420
+ minted row receives its own UUIDv7, with `id = tx` for that row. Outer writes
122
421
  reserve SQLite's write lock and re-observe `MAX(tx)` before minting, so
123
422
  concurrent processes allocate in commit order; lock contention waits for
124
- up to five seconds before surfacing `SQLITE_BUSY`.
423
+ up to five seconds before surfacing `SQLITE_BUSY`. Explicit replay IDs must
424
+ be canonical lowercase UUIDv7 strings. `insertResult` validates every used
425
+ ID before insertion or receive-clock observation, so a rejected batch cannot
426
+ change stored history or advance the generator. A UUID-generation failure
427
+ during insertion rolls back the entire batch, including vocabulary changes,
428
+ and the same store can retry after the failure clears. IDs successfully minted
429
+ before a transaction rolls back may leave sequence gaps; transaction rollback
430
+ does not rewind the process-wide generator shared by other stores.
431
+ - **Canonicalization shares the write reservation** (§13.4): `ingest` checks
432
+ SQLite's `data_version` inside the transaction and refreshes the registry
433
+ when another connection has committed. Inverse and lifecycle spellings
434
+ therefore use committed vocabulary even on an already-open store. Local
435
+ appends reuse the registry; rollback restores both the registry and its
436
+ version marker. Callers supplying a pre-canonicalized `insertResult` remain
437
+ responsible for their result's vocabulary snapshot; raw SQL writers must
438
+ call `reloadRegistry` after changing declarations.
439
+ - **Transaction ownership**: `store.transaction(({ outermost }) => …)`
440
+ requires a synchronous callback. Returning a promise or thenable throws and
441
+ rolls back writes made before the callback returned. Promises from the current
442
+ JavaScript realm are recognized even if an own `then` property is non-callable
443
+ or throws when read. Native promise rejection
444
+ is observed without calling a custom thenable’s `then` method, so rejecting
445
+ a thenable does not start its work after rollback. This does not cancel
446
+ already-started asynchronous work or include its later writes. Finish async
447
+ work before calling `transaction`, then perform all transactional writes
448
+ synchronously. The callback context
449
+ reports whether the callback owns the database commit or only a nested
450
+ savepoint. Existing callbacks that take no argument remain valid. Releasing
451
+ a nested savepoint leaves its writes subject to the caller's rollback;
452
+ the outermost scope commits only after its callback returns successfully.
453
+ A commit error still throws and triggers rollback. The shared adapter contract
454
+ exercises a deferred foreign-key failure at commit, verifies that claims and
455
+ vocabulary are restored, and successfully retries on the same connection.
456
+ The context works with both Node SQLite and the browser adapter.
457
+ If rollback cleanup also throws, an `AggregateError` retains the original
458
+ error as its cause and first entry, with the cleanup error second. In-memory
459
+ vocabulary and transaction ownership are restored even in that case. Native
460
+ and browser regressions exercise SQLite automatic rollback from outer and
461
+ nested scopes and verify a later successful write. Other cleanup failures do
462
+ not establish the database's transaction state; close and reopen the store
463
+ before retrying when that state is uncertain.
125
464
  - **History is permanent** (§9.6): retraction appends a `0%` row; it never
126
465
  erases the earlier row, `raw_line`, metadata, FTS text, export, backup, or
127
466
  peer copy. The store has no row-level redact/forget API because local
@@ -133,6 +472,10 @@ notes.currentBeliefs() // the file's claims, assembled in
133
472
  `#sensitivity:public|internal|confidential|restricted` labels each immutable
134
473
  row; unlabeled means `internal`, while flat, malformed and unknown labels
135
474
  fail closed as `restricted`. Export defaults to a maximum of `internal`.
475
+ Export reads `current`, `maxSensitivity` and `tx` once each; changing getters
476
+ cannot switch the selected history, sensitivity ceiling or annotation mode
477
+ during the operation. Shared adapter tests verify the resulting claim content
478
+ and transaction annotations, including grouped canonical output.
136
479
  Select `restricted` explicitly for complete portable text history. This is routing metadata,
137
480
  not encryption, access control, erasure or a retention boundary.
138
481
  - **One row per fact** (§13.3): inverse writes are canonicalized before
@@ -145,6 +488,30 @@ notes.currentBeliefs() // the file's claims, assembled in
145
488
  replay predicate mirrors the canonicalizer exactly — qualifier-condition
146
489
  rows never declare, and `X IS verb` needs a verb-shaped subject — so the
147
490
  registry after reopen equals the registry at close.
491
+ Appending WHEN, VIA or BECAUSE edges to existing declaration rows refreshes
492
+ the live registry in the same transaction, since those rows become qualifier
493
+ children. Rollback restores both the edges and the previous registry. Edges
494
+ to ordinary premise rows do not require declaration replay.
495
+ `appendEdges` captures the batch's parent IDs, roles and child IDs once before
496
+ its transaction. SQL insertion and vocabulary refresh use the same captured
497
+ values, including when callers supply getters.
498
+ Both `appendEdges` and structured `insertResult` reject roles other than `WHEN`,
499
+ `VIA`, `BECAUSE` and `QUALIFIES` with `TypeError`. Structured `append` also
500
+ requires safe integer parent/child indices within its claim batch. These
501
+ checks finish before inserting any rows or observing replay transaction IDs;
502
+ invalid batches leave claims, edges and the live registry unchanged.
503
+ A later edge constraint failure or vocabulary-refresh failure rolls back the
504
+ entire batch and restores the previous registry. The full annotated history
505
+ remains unchanged, and a corrected batch can be retried on the same store.
506
+ Vocabulary rebuilds construct a replacement before updating the live cache.
507
+ An interrupted `reloadRegistry` retains the previous registry and version
508
+ marker; callers can retry without reopening or losing the loaded inverse names.
509
+ Opening captures the configured registry and access mode once. Current
510
+ vocabulary, explicit reload and `registryAsOf` share that same base registry;
511
+ changing option getters cannot give historical reads a different configuration.
512
+ Replay filters ordinary `IS` facts in SQL: only `IS verb` can declare a
513
+ verb. This avoids materializing unrelated type facts during frequent action
514
+ refreshes while retaining transaction ordering and qualifier exclusions.
148
515
  - **Verb renames preserve history** (§5.8): after `OLD RENAMED-TO NEW`,
149
516
  either spelling writes the stable `OLD` storage verb and therefore the same
150
517
  claim key. `NEW` is preferred for authors while `OLD` remains compatible;
@@ -154,7 +521,9 @@ notes.currentBeliefs() // the file's claims, assembled in
154
521
  read *current beliefs* and skip negated (`VERB NOT`) and retracted
155
522
  (`@ 0%`) rows; opt back in with `{ negated: true, retracted: true }`.
156
523
  Contradictions still coexist as rows (§9.4) — resolution belongs to the
157
- query layer.
524
+ query layer. The traversal flags `negated`, `retracted`, `aliases` and
525
+ `resolve` must be booleans when supplied; each is captured once and validated
526
+ before query construction. Omission retains the false default.
158
527
  - **Alias closure is opt-in** (§13.6): `{ aliases: true }` on traversal
159
528
  (and on `claimsAbout`) matches the entity through every name linked by
160
529
  current positive `ALIAS` claims, read as undirected edges;
@@ -162,6 +531,12 @@ notes.currentBeliefs() // the file's claims, assembled in
162
531
  matching widens, stored names come back untouched, and disagreeing
163
532
  belief series surface side by side rather than merging silently. Unmerge
164
533
  is retraction — `dupe ALIAS canonical @ 0%`.
534
+ `claimsAbout` captures its `aliases` option once and requires a boolean when
535
+ supplied; strings, numbers and null are errors rather than disabled alias
536
+ matching. It still returns full history, newest first, including prior belief
537
+ revisions. It is not a current-belief lookup. The alias flags on
538
+ `resolvedBeliefs` and `contested` likewise require booleans and are captured
539
+ once before resolution reads.
165
540
  - **Actor provenance is caller-supplied** (§9.5): `ingest`/`insertResult`
166
541
  accept `{ source }` and stamp `@src:<source>` on every appended claim
167
542
  that carries no `src:` context — *before* the claim key is computed, so
@@ -175,6 +550,20 @@ notes.currentBeliefs() // the file's claims, assembled in
175
550
  representation. Lifecycle systems use `run` lookup rather than authored
176
551
  `src:` strings; resolution reads actor/source. Opening old stores backfills
177
552
  only established actor/run prefixes, decoded sources, and explicit scopes.
553
+ Backfill collects contexts once per claim without repeatedly copying prefixes;
554
+ the native and browser migration contract checks 5,000 source contexts,
555
+ context-free claims, retained row identities and repeat initialization.
556
+ Annotated export preserves dimensions that compact contexts cannot encode:
557
+ an optional JSON payload on `;@ <tx>` carries all four dimension sets.
558
+ `Provenance.normalize` provides a stable set representation;
559
+ `Provenance.parse` requires exactly the four named enumerable own fields
560
+ (`actors`, `sources`, `runs`, `domains`); a hidden required field cannot mask
561
+ an unknown enumerable dimension. It validates the complete object for replay, rejecting
562
+ unpaired UTF-16 surrogates in every dimension before UTF-8 storage can replace
563
+ them. Valid astral characters, embedded NUL and literal replacement characters
564
+ remain valid provenance text.
565
+ Parsing captures each dimension and its entries once, so accessors and later
566
+ changes to caller-owned arrays cannot replace the values being validated.
178
567
  - **Source spans retain source identity** (§9.8):
179
568
  `@src:docs/design%20notes.md#L10-L20` carries a one-based inclusive range;
180
569
  `SourceSpan` owns escaping and parsing. The exact context remains in the
@@ -190,10 +579,47 @@ notes.currentBeliefs() // the file's claims, assembled in
190
579
  `rule` 1) with in-band `source/<name> HAS precedence:` /
191
580
  `HAS reliability:` claims, matched to `src:` contexts by longest
192
581
  segment prefix; the declarations themselves resolve under the built-ins
193
- alone, so ingested text cannot self-elevate. Winners come back
582
+ alone, so ingested text cannot self-elevate. Declaration-source precedence
583
+ is accumulated iteratively, so a large provenance list does not exceed
584
+ JavaScript's function-argument limit. Contexts still reside in memory during
585
+ policy lookup; this is not a provenance-size budget. Policy identities preserve
586
+ embedded NUL characters: declaration keys use structured tuples, SQL literals
587
+ retain complete UTF-8 text, and segment matching uses byte lengths so SQLite
588
+ text functions cannot truncate a prefix. Native and WASM regressions check
589
+ exact, descendant and non-matching sibling sources. Physical source contexts
590
+ use the canonical escaping provided by `SourceSpan.context`. Actor compatibility
591
+ stamps use `Context.source`, retaining the actor name verbatim. Resolution
592
+ includes an actor only when that exact stamp is present; an authored source
593
+ can suppress an ordinary actor stamp while actor provenance remains recorded.
594
+ Up to 16 policy entries use SQL `VALUES`; larger policies use one materialized
595
+ JSON table to reduce SQL preparation cost and avoid repeated JSON decoding
596
+ during source matching. Native tests cover 33,000 entries and the strategy
597
+ boundary; SQLite WASM tests cover more than 1,000 in-band declarations. The
598
+ complete policy still resides in memory, and matching still scans candidate
599
+ prefixes; this is not a general policy-size or execution-time budget.
600
+ Ranked precedence is projected as a floating-point value, matching the
601
+ JavaScript numeric API. Finite values outside the safe-integer range, such as
602
+ `10000000000000000`, remain readable without a SQLite integer-conversion error;
603
+ positive and negative values retain their ordering.
604
+ Winners come back
194
605
  verbatim — resolution filters, it never rewrites — and it composes with
195
606
  `aliases`, which widens groups through the closure (the §13.6
196
- pick-a-winner story).
607
+ pick-a-winner story). Alias-aware grouping preserves complete subject and
608
+ object names, including embedded NUL and Unicode characters; sharing the text
609
+ before a NUL does not make two names aliases. Only the declared alias closure
610
+ combines their resolution groups.
611
+ `resolvedBeliefs`, `contested` and traversals with `resolve: true` hold one
612
+ deferred read snapshot across policy lookup and claim selection. Concurrent
613
+ policy and claim commits become visible together on the next call. These
614
+ reads work on read-only connections and nest inside caller transactions.
615
+ Traversals capture their four boolean options once before reading; traversals
616
+ without resolution do not add this policy snapshot. Reverse reads always hold
617
+ a snapshot across their vocabulary check and fact selection, so a concurrent
618
+ inverse rename cannot label new facts with an old inverse name.
619
+ If both a read and snapshot release fail, the aggregate message includes both
620
+ diagnostics, with the original errors retained and the read failure as cause.
621
+ Canonical export uses the same cleanup and reporting behavior. A release-only
622
+ failure preserves that error itself.
197
623
 
198
624
  ## API
199
625
 
@@ -201,7 +627,7 @@ notes.currentBeliefs() // the file's claims, assembled in
201
627
  |---|---|---|
202
628
  | `ingest(text, {strict, source})` | §13.4 | parse → canonicalize → append; lenient by default |
203
629
  | `insertResult(result, {source})` | | append a pre-canonicalized `@cavelang/canonical` result |
204
- | `currentBeliefs({minConf})` | §13.5 | latest row per key |
630
+ | `currentBeliefs({minConf})` | §13.5 | latest row per key; optional inclusive threshold must be a finite number in 0..1, captured once and validated before querying |
205
631
  | `currentBelief(key)` / `history(key)` | §9.1 | one fact's belief series |
206
632
  | `provenanceOf(rowOrId)` / `byProvenance(dimension, value)` | §9.5.1 | inspect or filter actor/source/run/domain |
207
633
  | `resolvedBeliefs({aliases})` | §26 | one winner per resolution group |
@@ -213,7 +639,7 @@ notes.currentBeliefs() // the file's claims, assembled in
213
639
  | `byTag(key, value?)` | §13.5 | flat (`value` omitted → `IS NULL`) or scoped |
214
640
  | `byContext(ctx)` | §13.5 | context filter |
215
641
  | `topicMembers(t)` / `topicsOf(e)` | §11.2 | topic layer over `CONTAINS` |
216
- | `search(q, {raw, limit, maxSensitivity})` | §13.2 | adapter full-text search; literal phrase by default, `limit` caps in the query; sensitivity is opt-in for enclosing publication surfaces |
642
+ | `search(q, {raw, limit, maxSensitivity, currentOnly})` | §13.2 | adapter full-text search; literal phrase by default, `limit` caps in the query; optional current-only and sensitivity filters |
217
643
  | `edgesOf(id)` | §13.2 | qualifier/grouping edges with roles |
218
644
  | `toClaim(row)` | | reconstruct the canonical claim + side tables |
219
645
  | `registry()` / `baseRegistry()` | §5.5 | current registry and its configured pre-declaration base |
@@ -223,8 +649,49 @@ notes.currentBeliefs() // the file's claims, assembled in
223
649
  | `openAt(path, {intent, registry})` / `openText(path)` / `kindOf(path)` / `isStoreFile(path)` | §13.7 | open a `--db` path by content: SQLite (read-only for `read`, unmigrated for `scratch`), CAVE text replayed into memory; only `write` creates or migrates |
224
650
  | `adapter` / `db` | | selected adapter capabilities and its raw structural database handle |
225
651
 
652
+ ### Search history or current knowledge
653
+
654
+ Use historical search to find evidence across revisions. Set `currentOnly` when
655
+ the result should contain only the latest positive-confidence rows. Retraction
656
+ removes a fact from current-only results while keeping its history searchable:
657
+
658
+ ```ts
659
+ import { open } from '@cavelang/store'
660
+
661
+ const store = open()
662
+ try {
663
+ store.ingest('auth HAS provider: legacy')
664
+ store.ingest('auth HAS provider: modern')
665
+
666
+ console.log(store.search('auth').map(row => row.value_text))
667
+ // ['modern', 'legacy']
668
+ console.log(store.search('auth', { currentOnly: true }).map(row => row.value_text))
669
+ // ['modern']
670
+
671
+ store.ingest('auth HAS provider: modern @ 0%')
672
+ console.log(store.search('auth', { currentOnly: true }).length) // 0
673
+ console.log(store.search('auth').length) // 3
674
+ } finally {
675
+ store.close()
676
+ }
677
+ ```
678
+
679
+ Both modes return newest matches first. A `limit` applies after the selected
680
+ history scope; `maxSensitivity` independently sets the audience ceiling when
681
+ supplied. CLI, MCP and viewer searches retain their historical behavior.
682
+
226
683
  ## Storage decisions
227
684
 
685
+ The exported `Resolve.rankedSql(entries, currentSql)` and
686
+ `Resolve.resolvedSql(entries, currentSql)` helpers accept an empty explicit
687
+ policy. With no entries, every source has precedence `0` and reliability `1`,
688
+ so confidence and then recency choose the winner. Store methods obtain the
689
+ built-in ladder plus declarations through `resolutionPolicy()`.
690
+ Partial explicit policies apply those defaults to each unmatched source
691
+ before taking maximum precedence and minimum reliability. For example, a
692
+ source assigned precedence `-1` and an unmatched source together give a claim
693
+ precedence `0`.
694
+
228
695
  - **Terms are stored formatted**: entities as plain text (so the spec's
229
696
  `WHERE subject = 'auth/middleware'` queries work verbatim), literals with
230
697
  their delimiters (`` `<=` ``, `"…"`) so they never collide with
@@ -232,6 +699,11 @@ notes.currentBeliefs() // the file's claims, assembled in
232
699
  - **`value_text` is the value as written** (including `~`, multiplier
233
700
  letters and literal delimiters); `value_num`/`value_unit` hold the
234
701
  normalized §13.4 forms. Same for `delta_*`.
702
+ - **Confidence keeps its stored numeric value** through canonical export and
703
+ import, including positive subnormal values and computed probabilities.
704
+ A zero-confidence retraction remains distinct from a later positive
705
+ reassertion, even at `Number.MIN_VALUE`. Import assigns fresh transaction
706
+ identities; annotated text sync preserves identities and skips repeated rows.
235
707
  - **`id` doubles as `tx`** by default: a UUIDv7 is both unique and
236
708
  time-ordered, and per-row tx ids keep same-document belief updates
237
709
  ordered by line. Allocation is database-serialized across processes;
@@ -239,6 +711,29 @@ notes.currentBeliefs() // the file's claims, assembled in
239
711
  - **`search()` phrase-quotes by default** — SQLite full-text syntax would parse
240
712
  `token-expiry` as a column filter; `{ raw: true }` opts into full MATCH
241
713
  syntax.
714
+ A supplied `limit` must be a non-negative safe integer; zero returns no rows,
715
+ and omission leaves search uncapped. Invalid limits throw `TypeError` before
716
+ querying, including on an empty store. Search includes historical beliefs by
717
+ default. `{ currentOnly: true }` selects positive-confidence current rows
718
+ before applying the limit, so revisions and retractions cannot crowd out
719
+ current matches. The option must be a boolean when supplied.
720
+ `currentOnly` is a programmatic store option used by ingestion context.
721
+ CLI `search`, MCP `cave_search` and viewer search retain historical results;
722
+ their public inputs do not expose this option.
723
+ The returned-row limit does not cap SQL work; see the
724
+ [current-search trial](#current-search-trial) for measurements and query-plan tradeoffs.
725
+ Search captures `raw`, `limit`, `currentOnly` and `maxSensitivity` once before validation and
726
+ SQL construction. Getters cannot change the cap or sensitivity ceiling between
727
+ validation and execution.
728
+ Supplied `raw` and `currentOnly` values must be booleans; malformed switches
729
+ raise `TypeError` instead of silently choosing another search mode.
730
+ Literal phrase quoting accounts for FTS4 versus FTS5, including embedded
731
+ quotation marks. `{ raw: true }` passes syntax directly to the selected
732
+ engine; it does not translate between their query languages.
733
+ Both modes reject NUL characters in query text with `TypeError` before database
734
+ access, including when `limit` is zero. SQLite's MATCH parser can otherwise
735
+ silently truncate a raw query at NUL or report an unterminated literal phrase.
736
+ Empty or punctuation-only literal phrases return no matches.
242
737
  - **Current-only export remaps edge endpoints** to the current row of each
243
738
  endpoint's claim key: a superseded qualified parent keeps its `WHEN`
244
739
  attached to the surviving belief, and orphaned condition claims are never
@@ -248,6 +743,189 @@ notes.currentBeliefs() // the file's claims, assembled in
248
743
  revives an older visible row. Full-history export checks each row. Edges with
249
744
  either endpoint hidden are omitted.
250
745
 
746
+ `registryAsOf` bounds qualifier parents as well as declaration rows. A parent
747
+ recorded later cannot exclude an older verb, inverse, or rename from a past
748
+ snapshot; date and timestamp intervals use the same boundary on both sides.
749
+
750
+ Live connections refresh vocabulary after another SQLite connection commits.
751
+ `store.registry()`, query compilation, and inverse reads therefore see new
752
+ verb and lifecycle declarations without reopening or writing. The read path
753
+ works on read-only connections and reuses its registry cache while SQLite's
754
+ data-version counter is unchanged. Raw SQL writes on the same connection still
755
+ require `reloadRegistry`; precompiled SQL retains their supplied vocabulary.
756
+
757
+ Canonical export may include `@claim` for qualifier claims with verb `NOT` or
758
+ to distinguish reserved entity subjects
759
+ from qualifiers or continuations. The marker changes no stored identity or
760
+ context, including in annotated exports. Readers of those exports must support
761
+ the explicit-claim syntax (§8.6).
762
+
763
+ ## Structured appends
764
+
765
+ A supplied `source` must be a string. The value is captured once and validated
766
+ before stamping or deriving actor/run attribution, even for empty batches.
767
+ Omit it (or use `undefined`) to preserve interchange claim identity without a
768
+ source stamp.
769
+
770
+ The optional `provenance` container must be an object when supplied; null,
771
+ primitives, functions and arrays are rejected before any fields are read.
772
+ An empty object is supported, and omission or `undefined` keeps inferred
773
+ provenance behavior. Supplied `actor` and `run` must be strings; every entry
774
+ in `sources` and `domains` must also be a string. Validation happens before
775
+ row insertion, including for empty batches. Empty strings retain their existing
776
+ meaning of no explicit value.
777
+
778
+ Append option collections (`ids`, `contexts`, `provenance.sources`, and
779
+ `provenance.domains`) must be arrays when supplied. They are captured once and
780
+ validated before copying, including for empty batches; strings, sets, null and
781
+ array-like objects are rejected. Omitted or `undefined` collections keep their
782
+ existing defaults.
783
+
784
+ `insertResult` accepts a canonicalization result and prepares every claim before
785
+ observing replay IDs or inserting rows. It validates canonical emission even
786
+ when the caller supplies raw text. If a claim fails that check, the batch
787
+ rejects without appending earlier valid claims. Context and tag collections must
788
+ be arrays before capture: strings, sets and array-like objects are rejected
789
+ rather than being converted into a different collection.
790
+
791
+ The [emitter's validation rules](../canonical/README.md#structured-claim-validation)
792
+ reject non-string term/value text, unsupported term kinds, non-boolean negation or importance, malformed verb tokens, embedded LF in claim fields, text/code literals containing their own
793
+ closing delimiter, and entity subjects that do not represent one non-metadata
794
+ atom. Contexts and tags must retain their metadata tokens, and tags must keep
795
+ the same key/value split. The rendered body must not open a comment or leave
796
+ a literal delimiter unmatched. Literal subjects and multiline comments remain
797
+ supported.
798
+
799
+ This validation prevents the listed malformed fields from entering through new
800
+ structured appends. It does not validate every possible programmatic claim,
801
+ rewrite historical rows, or govern low-level SQL writes. Existing malformed
802
+ rows may still fail canonical export. When an included row cannot be decoded or
803
+ emitted, export reports its claim ID and retains the validation error as `cause`.
804
+ The diagnostic applies to full/current and annotated/plain exports. It does not
805
+ modify rows or return a partial export. Row validation is repeated only after an
806
+ emission failure, so successful exports avoid an extra validation pass.
807
+ Export reads claims, metadata, lineage and provenance within one deferred read
808
+ snapshot, including on read-only connections. A peer commit cannot add its edges
809
+ to an export that omitted its claims; the next export sees the committed state.
810
+ If an included claim has an edge whose other endpoint is missing from storage,
811
+ export fails instead of silently dropping that relationship. Claims excluded by
812
+ sensitivity remain valid endpoints and retain the existing filtering behavior;
813
+ edges with neither endpoint included are outside this export's diagnostic scope.
814
+ The error contains no claim text, export leaves storage unchanged, and corrected
815
+ relationships are checked again on retry. This applies to plain/annotated and
816
+ historical/current exports, including current revision remapping.
817
+ The snapshot nests inside caller-owned transactions without committing them.
818
+ Release is attempted on success and failure. If both the read and release fail,
819
+ an `AggregateError` retains the read error first and as `cause`; a lone release
820
+ error propagates unchanged. A failed SQLite release may require caller recovery
821
+ before reusing the connection.
822
+
823
+ Export captures `current`, `maxSensitivity` and `tx` at entry, before database
824
+ reads and row decoding. Adapter callbacks that change the caller's options cannot
825
+ remove requested transaction annotations midway through an export. The shared
826
+ adapter contract checks this with native SQLite and browser SQLite.
827
+ With `tx: true`, every included row must have a canonical lowercase UUIDv7
828
+ identity with `id = tx`; a mismatch fails with the affected claim ID before
829
+ annotation emission. Plain semantic export does not carry row identities.
830
+ Annotated export also recomputes each included claim's key, including its
831
+ contexts, and rejects disagreement with the stored key. This prevents replay
832
+ from silently assigning the same transaction to a different belief series.
833
+ The error identifies the affected claim and preserves its cause; export leaves
834
+ the database unchanged for repair and retry.
835
+ Annotated export also validates explicit stored provenance against the replay
836
+ contract: each value must be a nonempty Unicode string. Historical empty or
837
+ binary values fail with the affected claim ID and an underlying cause instead
838
+ of producing annotations that sync would reject. Validation occurs only for
839
+ included rows; narrower sensitivity exports can still succeed. Export does not
840
+ repair stored data. After the malformed provenance is corrected, annotated
841
+ export and identity-preserving replay can proceed normally.
842
+ `cave doctor` checks for empty and non-text provenance values under `store.rows`
843
+ without printing the values or claim identities and without changing the database.
844
+ The same check recomputes historical claim keys from their semantic claims and
845
+ stored contexts, detecting disagreement before an annotated export is attempted.
846
+ It also checks canonical emission with the stored tags, diagnosing historical
847
+ claim text that export cannot emit while keeping the data unchanged for repair.
848
+ Stored edge roles outside `WHEN`, `VIA`, `BECAUSE` and `QUALIFIES` also fail
849
+ `store.rows`. Diagnosis leaves those relationships unchanged, redacts their
850
+ values and checks them again after repair.
851
+
852
+ The optional `current` and `tx` switches must be booleans when supplied. Export
853
+ captures them once and rejects malformed values with `TypeError` before querying,
854
+ including on empty stores. A string such as `"true"` cannot silently select
855
+ historical output or omit transaction annotations.
856
+
857
+ `search` and `exportText` also reject invalid supplied `maxSensitivity` values
858
+ with `TypeError`, including `null`, unknown strings and non-string values.
859
+ Omitting the option retains each method's existing scope: search includes all
860
+ levels, while export defaults to `internal`. Stored malformed sensitivity tags
861
+ continue to be treated as `restricted`; they are distinct from invalid API options.
862
+
863
+ Sensitivity filtering precedes row decoding and emission: malformed restricted
864
+ rows do not break a public-only export or disclose their claim IDs in its output.
865
+ When the requested sensitivity includes those rows, the same row-specific error
866
+ applies. Regressions cover newline-corrupted historical subjects, objects,
867
+ contexts and tag values across full/current and plain/annotated exports, with
868
+ stored claims, metadata and edges unchanged after failure.
869
+
870
+ Export also rejects stored edge roles other than `WHEN`, `VIA`, `BECAUSE` and
871
+ `QUALIFIES` when both endpoints are included.
872
+ The error identifies both endpoint IDs. Edges omitted by sensitivity filtering
873
+ do not expose their endpoints through this diagnostic. Invalid roles previously
874
+ emitted text that could re-import without its original edge; export now fails
875
+ without changing the store, and repairing the role restores export.
876
+
877
+ Self-referential edges are retained, including edges whose historical endpoints
878
+ resolve to the same current row. The canonical emitter re-states the claim once
879
+ under its own edge; annotated export gives both appearances the same transaction
880
+ ID. This preserves the graph relation without recursively expanding the cycle.
881
+ Syncing that annotated export into a fresh store restores one row and one
882
+ self-edge for each of the four supported roles; repeating the sync adds neither
883
+ another row nor another edge. Current export uses the current row's identity
884
+ for both endpoints when it remaps an older revision.
885
+
886
+ Row-to-claim conversion also rejects conflicting payload columns: an object
887
+ cannot coexist with an attribute or value, and an attribute requires a value.
888
+ These malformed historical or low-level SQL rows previously lost fields during
889
+ conversion. Export now reports their claim ID when the sensitivity scope includes
890
+ them, without changing stored data; correcting the row permits export again.
891
+ The stored `negated`, `importance` and `value_approx` flags must each be exactly
892
+ numeric 0 or 1.
893
+ Other values are diagnosed during claim conversion instead of being coerced
894
+ into a negation or importance marker, or silently ignoring an invalid
895
+ approximation flag. Native and browser SQLite share this check.
896
+
897
+ Claim-specific export failures retain the affected claim ID and the original
898
+ thrown value in `Error.cause`, even if an adapter throws a value whose message
899
+ cannot be read or converted to text. Such diagnostics use
900
+ `[unprintable thrown value]`; the export read snapshot is still released and a
901
+ corrected read can be retried without changing stored history.
902
+
903
+ Conversion also checks `value_num`, `value_unit`, `value_approx`, `delta_num` and
904
+ `delta_unit` against the parsed authored value and uncertainty text. A mismatch
905
+ fails with the field name; export adds the claim ID and leaves the database
906
+ unchanged. Correcting the cached columns permits conversion and export again.
907
+ This conversion check does not validate every row before SQL filtering.
908
+ `cave doctor` separately streams all stored history through the row decoder to
909
+ check authored values and their cached projections before export is attempted.
910
+ That scan also requires a canonical lowercase UUIDv7 identity with `id = tx`
911
+ for each row; SQLite integrity and matching search contents alone do not prove
912
+ that identity invariant.
913
+ Doctor holds a read transaction across its SQLite checks, so concurrent commits
914
+ do not mix database states within a report; a later invocation sees those commits.
915
+ In rollback-journal mode, the read snapshot can temporarily block a competing
916
+ writer's commit. Closing doctor's connection releases the read lock, including
917
+ after an invalid-row report. WAL and rollback-journal regressions exercise these
918
+ different concurrency behaviors using a second live connection.
919
+ If closing the owned connection fails, doctor retains the database checks and
920
+ adds a redacted `store.cleanup` failure instead of exposing the thrown value.
921
+ For an existing current-schema SQLite file, `cave doctor` reports these payload
922
+ and flag problems through `store.rows` before export is attempted. The same check
923
+ rejects nonnumeric or out-of-range confidence and nonnumeric, nonpositive or
924
+ infinite sigma levels; a null sigma level retains the legacy default of 2. It checks
925
+ stored history without changing it and omits claim contents and identifiers
926
+ from the diagnostic. This targeted check is not a complete semantic audit of
927
+ every historical field.
928
+
251
929
  ## Tests
252
930
 
253
931
  ```
@@ -263,6 +941,197 @@ cross-actor retraction, stamped round-trips), the §26 resolution policy
263
941
  (human-over-ingest precedence, reliability weighting, longest-prefix
264
942
  specificity, polarity contests, no self-elevation, alias-widened
265
943
  groups, contested ranking), the §11.2 topic reads, edge persistence,
266
- registry and lifecycle rebuild across reopen, transactional strict ingest and export
944
+ registry and lifecycle rebuild across reopen and historical qualifier-parent boundaries, transactional strict ingest and export
267
945
  round-trips, ordered schema migration, newer-version rejection, transactional
268
946
  rollback/retry, and closed-file backup recovery.
947
+
948
+ ## Empty scoped-export trial
949
+
950
+ When sensitivity filtering selects no claims, export returns empty text before
951
+ reading lineage or building historical-to-current identity maps. The scoped claim
952
+ query still runs inside the read snapshot, validates the sensitivity ceiling and
953
+ determines that nothing is visible. This does not make empty exports constant-time.
954
+
955
+ Run `node scripts/empty-export-bench.mjs` from the repository root. The fixture
956
+ contains restricted claims with qualifier edges and exports at the public ceiling,
957
+ with transaction annotations enabled. Setup and output checks are outside timing;
958
+ each result is the median of five calls after one warm-up. A local macOS ARM64
959
+ trial with 10,000 claims and 5,000 edges measured:
960
+
961
+ | Export mode | Node 24.16.0 before → after | Node 26.5.0 before → after |
962
+ |---|---|---|
963
+ | Full history | 6.89 → 3.46 ms | 6.86 → 3.45 ms |
964
+ | Current only | 21.09 → 10.62 ms | 20.96 → 10.69 ms |
965
+
966
+ These figures describe this empty-output fixture, not exports containing visible
967
+ claims. Existing adapter tests cover empty sensitivity scopes and snapshot cleanup.
968
+
969
+ ## Current-export history trial
970
+
971
+ Current export resolves edge endpoints directly when they are selected current
972
+ rows. It builds the historical ID/key map only when an endpoint needs remapping,
973
+ restricts it to IDs referenced by stored edges, and shares it across the remaining
974
+ endpoints within the same read snapshot.
975
+ An export without edges does not build it. Historical qualifier edges still
976
+ remap to the current rows, preserving their conditions.
977
+ Plain and annotated current exports first validate the semantic keys of all
978
+ selected current rows. This also prevents a corrupted newer claim from taking
979
+ another claim’s key and receiving its historical relationships.
980
+ Before a historical endpoint is mapped to a selected current row, export
981
+ reconstructs that endpoint's semantic key from its claim and contexts and checks
982
+ it against the stored key. A mismatch fails plain and annotated current exports
983
+ with the historical claim ID, instead of attaching the edge to an unrelated
984
+ current claim. Successful verification is cached per endpoint within the read
985
+ snapshot. This checks historical rows used for remapping, not every omitted row;
986
+ `doctor` remains the broader historical integrity check. For example, a malformed
987
+ superseded attribute row with no relationships does not block a current export
988
+ of its valid newer revision. Full-history export includes and rejects that row.
989
+ Adding an edge through the old row makes current export validate it too; repairing
990
+ the old row permits the edge to remap to the newer value. Both SQLite adapter
991
+ contracts verify this boundary and unchanged database contents on rejection. The measurements below
992
+ predate this additional verification.
993
+
994
+ Run `node scripts/current-export-history-bench.mjs` from the repository root.
995
+ It measures annotated current exports after 100, 1,000 and 10,000 revisions of
996
+ one attribute, with no edge, a current-row edge, or a historical-row edge.
997
+ Setup and output checks are outside timing; medians use five calls after one
998
+ warm-up. A local macOS ARM64 trial at 10,000 revisions measured:
999
+
1000
+ | Edge case | Node 24.16.0 before → after | Node 26.5.0 before → after |
1001
+ |---|---|---|
1002
+ | None | 6.67 → 1.17 ms | 6.43 → 1.16 ms |
1003
+ | Current row | 6.71 → 1.19 ms | 6.96 → 1.18 ms |
1004
+ | Historical row | 6.67 → 1.20 ms | 6.99 → 1.19 ms |
1005
+
1006
+ The before column predates both lazy map construction and limiting the map to
1007
+ edge endpoints. The fallback still scans stored edges and resolves their IDs;
1008
+ unrelated historical revisions no longer enter the map.
1009
+
1010
+ `node scripts/dense-current-export-bench.mjs` is the control with every exported
1011
+ parent requiring remapping. With 3,000 historical rows, 2,000 current rows and
1012
+ 1,000 edges, narrowing the map changed local medians from 52.41 to 50.00 ms on
1013
+ Node 24 and 48.18 to 47.35 ms on Node 26. It uses the same warm-up, five-sample
1014
+ method and output checks outside timing. These are fixture measurements, not a
1015
+ general export work bound.
1016
+
1017
+ A refreshed checkpoint includes historical endpoint semantic-key verification.
1018
+ The same scripts, run sequentially on Node 24.21.0 and Node 26.8.1, retain their
1019
+ output checks and one warm-up/five-call medians:
1020
+
1021
+ | Current workload | Node 24.21.0 median | Node 26.8.1 median |
1022
+ |---|---:|---:|
1023
+ | 10,000 revisions, no edge | 1.214 ms | 1.201 ms |
1024
+ | 10,000 revisions, current-row edge | 1.225 ms | 1.220 ms |
1025
+ | 10,000 revisions, historical-row edge | 1.241 ms | 1.252 ms |
1026
+ | 3,000 history rows, 2,000 current rows, 1,000 historical edges | 74.736 ms | 67.876 ms |
1027
+
1028
+ [Current samples and source hashes](../../benchmarks/verified-historical-export-review.json)
1029
+ include all three sizes in each workload. These current-state measurements do
1030
+ not isolate endpoint verification's cost from runtime and other implementation
1031
+ changes since the older trials. The dense case exercises verification for every
1032
+ remapped parent; the sparse case checks that long unrelated history remains
1033
+ outside the endpoint map. They establish neither a process-memory limit nor a
1034
+ general elapsed-time bound.
1035
+
1036
+ Historical verification now prepares its context and tag queries once per export,
1037
+ reusing them for each checked endpoint within the same read snapshot. It reads
1038
+ the same data and retains full row reconstruction and semantic-key checks.
1039
+ Repeating the dense trial after this change records 69.659 ms on Node 24.21.0
1040
+ and 59.309 ms on Node 26.8.1 for 1,000 edges, compared with 74.736 and 67.876 ms
1041
+ in the preceding checkpoint. [Statement-reuse samples and hashes](../../benchmarks/historical-export-statements-review.json)
1042
+ retain all three sizes and the same warm-up/five-call controls. The measurements
1043
+ support this dense workload; they do not establish a universal speedup.
1044
+
1045
+ ## Transaction-index trial
1046
+
1047
+ An isolated Node 26.5.0 / SQLite 3.53.3 macOS arm64 trial compared the version-1
1048
+ schema with a temporary `CREATE INDEX trial_tx ON cave_claim(tx)`. Each fixture
1049
+ contained one distinct attribute claim per entity. Five samples each executed
1050
+ 100 prepared head queries, verifying identical results before and after adding
1051
+ the index. A bounded query used the fixture's middle transaction as `tx <= ?`.
1052
+
1053
+ | Claims | Head reads, existing / indexed | Bounded reads, existing / indexed | Added database pages in bytes |
1054
+ |---|---:|---:|---:|
1055
+ | 1,000 | 4.52 / 0.075 ms | 4.96 / 0.085 ms | 49,152 |
1056
+ | 10,000 | 44.38 / 0.071 ms | 47.79 / 0.084 ms | 454,656 |
1057
+ | 50,000 | 218.68 / 0.072 ms | 235.89 / 0.086 ms | 2,252,800 |
1058
+
1059
+ Query plans used the existing `(claim_key, tx)` covering index before the trial
1060
+ and the transaction-leading index afterward. In separate five-sample fixtures
1061
+ starting with 10,000 claims, appending 100 claims in individual transactions
1062
+ took a median 55.66 ms without the index and 10.38 ms with it; row counts and
1063
+ SQLite integrity checks passed. The receive-clock head read benefits each append.
1064
+ These in-memory measurements do not characterize disk durability or all write
1065
+ workloads. They motivated schema version 2, which installs `idx_cave_tx` through
1066
+ an atomic migration and validates its table, column, binary collation and non-partial coverage.
1067
+ A same-named index with another collation is incompatible: it cannot provide
1068
+ the binary transaction range search used by bounded head reads.
1069
+
1070
+ A follow-up using the production schema-2 migration on the same runtime and
1071
+ five fresh 10,000-row fixtures measured 0.079 ms for 100 prepared head reads
1072
+ and 10.83 ms for 100 individual appends (medians). Every fixture retained
1073
+ 10,100 rows and passed SQLite integrity checks. Native and WASM regression
1074
+ tests also verify indexed global and bounded query plans, migration rollback
1075
+ and retry, and unchanged annotated exports.
1076
+
1077
+
1078
+ ## Claim capture cost trial
1079
+
1080
+ Run `node --disable-warning=ExperimentalWarning scripts/claim-capture-bench.mjs`
1081
+ alone. It appends pre-canonicalized batches of 100 and 1,000 claims to fresh
1082
+ in-memory stores: simple relations, or numeric attributes with uncertainty,
1083
+ confidence, source/scope contexts, a tag and a comment. Canonicalization, store
1084
+ creation and result validation are outside timing; each median uses five fresh
1085
+ stores after one warm-up store. Checks verify inserted counts and decode the
1086
+ first and last claim records, including numeric metadata and source provenance.
1087
+
1088
+ On macOS arm64, Node 24.16.0 and 26.5.0 produced these medians. The baseline
1089
+ bypassed only the new per-claim capture helper; option and edge capture remained
1090
+ in place. The final source was restored before the after runs. The benchmark
1091
+ itself measures the checked-out code and exposes no bypass option.
1092
+
1093
+ | Claims | Payload | Node 24 before → after | Node 26 before → after |
1094
+ |---|---|---|---|
1095
+ | 100 | Relation | 1.57 → 1.60 ms | 1.65 → 1.65 ms |
1096
+ | 100 | Numeric with metadata | 4.27 → 4.33 ms | 4.12 → 4.50 ms |
1097
+ | 1,000 | Relation | 11.84 → 11.69 ms | 11.85 → 12.37 ms |
1098
+ | 1,000 | Numeric with metadata | 38.47 → 37.33 ms | 35.00 → 35.41 ms |
1099
+
1100
+ These measurements describe complete append calls, not isolated copy costs.
1101
+ Baseline runs preceded final runs; cache and execution-order effects remain
1102
+ possible, and the varying differences do not establish a speedup or universal
1103
+ overhead bound. File durability, contention, large batches and WASM are outside
1104
+ this trial. Separate getter regressions establish projection consistency.
1105
+
1106
+ ## Current-search trial
1107
+
1108
+ This trial uses native Node SQLite in memory; it does not measure SQL.js/WASM
1109
+ or browser rendering. The shared adapter tests check search semantics on both
1110
+ native SQLite and SQL.js separately.
1111
+
1112
+ The returned-row limit does not cap SQL work over matching history. Run
1113
+ `node scripts/current-search-bench.mjs` alone from the repository root to
1114
+ compare historical and current-only search at 100, 1,000 and 10,000 revisions
1115
+ of one fact, alongside an older current fact. The benchmark requests five
1116
+ results and checks that current-only mode retains both current facts. It
1117
+ reports eleven-call medians after two warmups, including row decoding and
1118
+ excluding fixture construction and assertions. These are local measurements,
1119
+ not CI thresholds or general latency guarantees.
1120
+
1121
+ An additional workload combines 10,000 revisions with 10,000 unrelated claim
1122
+ keys to expose current-row resolution costs beyond matching history. On the
1123
+ audit machine, historical/current-only medians were 11.75/19.56 ms on Node
1124
+ 24.21.0 and 11.36/19.07 ms on Node 26.8.1. The two matching current facts
1125
+ remain the same; unrelated store size can still affect query execution cost.
1126
+
1127
+ Checking for a newer revision through the existing `(claim_key, tx)` index
1128
+ replaced whole-store grouping in current-only search. In the same workload,
1129
+ current-only medians fell from 19.56 to 11.91 ms on Node 24.21.0 and from
1130
+ 19.07 to 12.35 ms on Node 26.8.1. The check considers newer rows regardless of
1131
+ sensitivity or confidence before applying visibility and result limits;
1132
+ a hidden replacement or retraction cannot revive an older match.
1133
+
1134
+ The 10,000-revision fixture without unrelated keys measured about 11.8 ms
1135
+ with the indexed check, compared with about 10.9 ms in the earlier grouped
1136
+ run. This is a workload tradeoff, not a universal speedup; broad match sets
1137
+ still require searches and ordering work.