@jarenjs/db 0.66.1 → 0.67.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/ARCHITECTURE.md +19 -0
- package/README.md +15 -4
- package/docs/LIVE-FORMAT.md +47 -6
- package/docs/MODEL-FORMAT.md +8 -0
- package/docs/REPLICATION-FORMAT.md +208 -0
- package/package.json +4 -4
- package/schemas/jaren-replication-snapshot.draft-07.schema.json +83 -0
- package/schemas/jaren-replication-snapshot.schema.json +83 -0
- package/schemas/jaren-replication.draft-07.schema.json +82 -0
- package/schemas/jaren-replication.schema.json +82 -0
- package/src/capture.js +13 -3
- package/src/cursor.js +11 -5
- package/src/dialect.js +1 -1
- package/src/dialects/sqlite.js +1 -0
- package/src/errors.js +8 -0
- package/src/index.js +2 -0
- package/src/live-join.js +250 -0
- package/src/live-nested.js +120 -0
- package/src/live.js +18 -4
- package/src/logical-rows.js +90 -0
- package/src/replication-format.js +115 -0
- package/src/replication.js +332 -0
- package/src/store.js +67 -14
- package/types/index.d.ts +62 -3
- package/types/typed.d.ts +3 -2
package/ARCHITECTURE.md
CHANGED
|
@@ -1095,3 +1095,22 @@ processes creating one fresh file never meet the deferred-upgrade
|
|
|
1095
1095
|
`SQLITE_BUSY` the busy handler cannot retry. The job queue's
|
|
1096
1096
|
administration (`page`, `cancel`, `requeue`, `sweep`) lives in
|
|
1097
1097
|
`src/jobs.js` beside the fence it authorises through (JOBS-FORMAT §10).
|
|
1098
|
+
|
|
1099
|
+
|
|
1100
|
+
## Replication and bounded dependency maintenance
|
|
1101
|
+
|
|
1102
|
+
`replication-format.js` owns JSON normalization; `replication.js` persists replica
|
|
1103
|
+
identity, frontiers, canonical receipts, before-images, outbox and conflict
|
|
1104
|
+
evidence through the capture transaction's `beforeCommit` seam. `logical-rows.js`
|
|
1105
|
+
shares mapped, validated row access between replication and live dependencies.
|
|
1106
|
+
Remote application suppresses only local envelope allocation: its data still
|
|
1107
|
+
invalidates capture subscribers after commit. Snapshot reset carries complete
|
|
1108
|
+
receipt evidence and refuses histories beyond its explicit credits.
|
|
1109
|
+
|
|
1110
|
+
`live-join.js` compiles indexed dependency descriptors from the entity planner
|
|
1111
|
+
and mapping. The same cache strategy maintains inner/left joins and eligible
|
|
1112
|
+
nested graph projections. `live-nested.js` maintains explicit two-level groups
|
|
1113
|
+
by refreshing affected parents from bounded leaves. Source and result payloads
|
|
1114
|
+
consume row and byte credits; unsupported shapes keep named reruns. See
|
|
1115
|
+
[the replication contract](docs/REPLICATION-FORMAT.md) and
|
|
1116
|
+
[the strategy matrix](docs/LIVE-FORMAT.md).
|
package/README.md
CHANGED
|
@@ -990,11 +990,17 @@ shutdown limits, the browser persistence matrix and measured latency/memory loss
|
|
|
990
990
|
`sessionReason`. One diff
|
|
991
991
|
format runs store → patch → live query → O(k) render. Capture is
|
|
992
992
|
opt-in; the overhead is published, not waved away.
|
|
993
|
+
- **Portable replication** ([REPLICATION-FORMAT](docs/REPLICATION-FORMAT.md)):
|
|
994
|
+
opt-in replica identities, causal frontiers, bounded logical envelopes,
|
|
995
|
+
durable replay receipts, explicit conflict evidence and snapshot resets.
|
|
996
|
+
Data and acknowledgements commit together. Hosts supply transport and any
|
|
997
|
+
conflict resolver; the default preserves both contenders and rejects the write.
|
|
993
998
|
- **Live queries** (LIVE-FORMAT §§7–13): `collection.live(document)`
|
|
994
999
|
maintains a result as writes arrive and emits patches — incremental
|
|
995
1000
|
for `where`/`select`/`orderBy`+`limit`/aggregates/single-level
|
|
996
1001
|
`groupBy` and a spatial `where` over a derived index (the geofence;
|
|
997
|
-
the normative maintenance table),
|
|
1002
|
+
the normative maintenance table), indexed inner/left entity joins, bounded
|
|
1003
|
+
graph projections and explicit two-level groups; re-run for other shapes,
|
|
998
1004
|
**declared, never silent** (`live.mode` names the reason).
|
|
999
1005
|
Unaffected rows stay reference-identical; a seeded oracle holds the
|
|
1000
1006
|
maintained result equal to a fresh re-query after every mutation.
|
|
@@ -1276,7 +1282,8 @@ its side-effect-free status read are in
|
|
|
1276
1282
|
there. Same-host processes over WAL are the supported topology. No
|
|
1277
1283
|
priority classes, no cron, no workflow compensation.
|
|
1278
1284
|
- **Live-query maintenance is limited to the declared table** (§7);
|
|
1279
|
-
joins
|
|
1285
|
+
indexed joins and graph projections require bounded dependencies. Offset
|
|
1286
|
+
windows, unindexed joins, load-spec graphs and non-canonical shapes re-run, reported.
|
|
1280
1287
|
- **`eventTime.retention` bounds repair work, not memory.** It is the
|
|
1281
1288
|
horizon a view claims and is checked against the window it maintains;
|
|
1282
1289
|
the maintained state is still bounded by `live.maxMaintained`, and no
|
|
@@ -1287,8 +1294,8 @@ its side-effect-free status read are in
|
|
|
1287
1294
|
durability differences; see [execution hosts](docs/HOSTS.md).
|
|
1288
1295
|
- **Named future work, not silent gaps**: `$groupby` pushdown beyond
|
|
1289
1296
|
the `$time-bucket` ladder, a many-to-many hop on the chain, membership
|
|
1290
|
-
on an auto-keyed pending insert,
|
|
1291
|
-
|
|
1297
|
+
on an auto-keyed pending insert, additional join/group shapes, other SQL dialects,
|
|
1298
|
+
transport policy, database introspection (MODEL-FORMAT §10.6, the roadmap).
|
|
1292
1299
|
|
|
1293
1300
|
The normative formats are
|
|
1294
1301
|
[docs/MODEL-FORMAT.md](docs/MODEL-FORMAT.md) (storage §§1–7, safe
|
|
@@ -1320,6 +1327,10 @@ Every subpath a consumer can import, derived from the manifest by
|
|
|
1320
1327
|
| `@jarenjs/db/schemas/jaren-migration.schema.json` | schema | — |
|
|
1321
1328
|
| `@jarenjs/db/schemas/jaren-model.draft-07.schema.json` | schema | — |
|
|
1322
1329
|
| `@jarenjs/db/schemas/jaren-model.schema.json` | schema | — |
|
|
1330
|
+
| `@jarenjs/db/schemas/jaren-replication-snapshot.draft-07.schema.json` | schema | — |
|
|
1331
|
+
| `@jarenjs/db/schemas/jaren-replication-snapshot.schema.json` | schema | — |
|
|
1332
|
+
| `@jarenjs/db/schemas/jaren-replication.draft-07.schema.json` | schema | — |
|
|
1333
|
+
| `@jarenjs/db/schemas/jaren-replication.schema.json` | schema | — |
|
|
1323
1334
|
| `@jarenjs/db/package.json` | metadata | — |
|
|
1324
1335
|
| `@jarenjs/db/node-worker` | JavaScript | declared |
|
|
1325
1336
|
| `@jarenjs/db/node-pool` | JavaScript | declared |
|
package/docs/LIVE-FORMAT.md
CHANGED
|
@@ -273,7 +273,10 @@ what the pushdown planner already means by it.
|
|
|
273
273
|
| a spatial predicate the planner **refused** (no `derive` index on the member, an untyped member, an unbounded probe) | **re-run on invalidation**, the refusal named — it never translated, so nothing narrows the fetch | the previous result, for diffing |
|
|
274
274
|
| a `$resample` or `$rolling` document over the collection, with an explicit `eventTime` and a fixed width (§13) | **event-time bucket / rolling state**: rows kept by bucket, or in instant order; only what a write can reach is folded again, through `@jarenjs/core/series` itself | the contributing rows, plus one fold per bucket |
|
|
275
275
|
| the same document with no `eventTime`, a calendar width, a named zone, a `locf`/`linear` fill, a `first`/`last` aggregate, or a retention that does not cover the window | **re-run on invalidation**, the member that stopped it named (§13.2) | the previous result, for diffing |
|
|
276
|
-
| joins
|
|
276
|
+
| indexed inner equi-joins and canonical allowing-empty left joins over mapped entity roots | **join dependency maintenance**; point-read changed keys and reevaluate their bounded outer owners | source rows, key indexes and projected tuples, bounded by `maxMaintained` and `maxBytes` |
|
|
277
|
+
| nested entity graph projections with indexed equality edges and unique binding names | **graph dependency maintenance**; a child change refreshes its bounded owners | source rows, reverse key indexes and graph outputs |
|
|
278
|
+
| explicit two-level collection groups with a singular parent key and bounded nested input | **nested-group maintenance**; recompute affected parents through the query engine | source leaves and parent outputs |
|
|
279
|
+
| unindexed/non-equi joins, self joins, explicit entity ordering/windows, object-root documents and load-spec graphs | **re-run on invalidation**, with the dependency or planner reason | previous result for diffing |
|
|
277
280
|
| anything else: non-translatable predicates, `limit` without `orderBy`, `offset` > 0, windowed aggregates, `@jarenjs/linq`'s nested two-level `groupBy` emission, non-canonical group returns | **re-run on invalidation**, the reason named | the previous result, for diffing |
|
|
278
281
|
|
|
279
282
|
The **physical mapping** of a `derive: 'bbox'` index (MODEL-FORMAT §2.1,
|
|
@@ -310,8 +313,8 @@ refuses at registration (`JD0051`), the same shape as capture's
|
|
|
310
313
|
demanded session — an application that needs the property can refuse
|
|
311
314
|
to start.
|
|
312
315
|
|
|
313
|
-
The **canonical group form** the classifier recognises (
|
|
314
|
-
|
|
316
|
+
The **canonical group form** the classifier recognises (the linq chain's group-of-groups emission still re-runs; explicit
|
|
317
|
+
nested groups have a separate bounded strategy below):
|
|
315
318
|
|
|
316
319
|
```json
|
|
317
320
|
{ "$for": { "it": "$[*]" },
|
|
@@ -502,13 +505,14 @@ ERRORING rather than degrading (the D14 rule — the bound is printed):
|
|
|
502
505
|
delete-correctness, and a count over a table larger than the bound
|
|
503
506
|
is a conscious `maxMaintained` raise, not a silent one.
|
|
504
507
|
|
|
505
|
-
Non-claims, in one place: no
|
|
506
|
-
|
|
508
|
+
Non-claims, in one place: no maintenance of unindexed or non-equality joins,
|
|
509
|
+
no cross-connection invalidation (§6's `data_version` is
|
|
507
510
|
the signal), no maintenance over asynchronous connections —
|
|
508
511
|
`capabilities.live` is `false` there and a registration is `JD0051`
|
|
509
512
|
naming the reason, because maintenance point-reads rows synchronously
|
|
510
513
|
inside delivery (the wasm driver's oo1 API is synchronous, which is
|
|
511
|
-
why the browser has live queries at all)
|
|
514
|
+
why the browser has live queries at all). Replication is specified separately in
|
|
515
|
+
[REPLICATION-FORMAT](REPLICATION-FORMAT.md). There is no ordering guarantee for
|
|
512
516
|
unordered queries beyond §9's determinism.
|
|
513
517
|
|
|
514
518
|
## 13. Event time
|
|
@@ -616,3 +620,40 @@ holds a shuffled stream of inserts, in-place updates, instant moves and
|
|
|
616
620
|
deletes against `resampleSeries` / `rollingSeries` over the whole
|
|
617
621
|
collection after each one, which is the only oracle that cannot drift
|
|
618
622
|
with the implementation.
|
|
623
|
+
|
|
624
|
+
|
|
625
|
+
## Bounded joins, graph projections and nested groups
|
|
626
|
+
|
|
627
|
+
`join`, `graph` and `nested-group` strategies charge their source rows and
|
|
628
|
+
result entries to `live.maxMaintained`, and serialized input/output payloads to
|
|
629
|
+
`live.maxBytes` (default 4 MiB). These credits bound cached payloads rather than
|
|
630
|
+
claiming to measure JavaScript heap overhead. Initialization uses a limited
|
|
631
|
+
source read; updates read changed keys, then visit cached indexed dependencies.
|
|
632
|
+
Bounds are checked while caches grow. Overflow is `JD2060`, emits one error,
|
|
633
|
+
closes the subscription and releases dependency caches. It never relabels an
|
|
634
|
+
unbounded query as incremental.
|
|
635
|
+
|
|
636
|
+
Entity equality columns need a primary-key prefix, declared index or mapped
|
|
637
|
+
foreign-key index. Every binding needs a key and a distinct root. The projected
|
|
638
|
+
identity is the tuple of source identities, so duplicate projected values remain
|
|
639
|
+
distinct. Default entity result order follows physical row insertion order;
|
|
640
|
+
point reads preserve that order even when a key is deleted and reinserted. A
|
|
641
|
+
left join uses a canonical `$allowing-empty` binding over an equality-filtered
|
|
642
|
+
inner subquery. Its absent child can be defaulted to null. Graph projections
|
|
643
|
+
embed equality-filtered child queries in a single outer row's return object.
|
|
644
|
+
Global-root reads outside those bindings re-run because changing one row can
|
|
645
|
+
change every projected graph.
|
|
646
|
+
|
|
647
|
+
`dependencyReads`, `refreshedRoots` and `refreshedGroups` expose the work done.
|
|
648
|
+
`dependencyReads` counts logical changed-row reads; a row-position lookup is an
|
|
649
|
+
additional statement. No full query reruns occur under these strategy labels.
|
|
650
|
+
Materializing and diffing the final bounded output still costs work proportional
|
|
651
|
+
to its size. See the equal-correctness [measurements](REPLICATION-FORMAT.md#measurements)
|
|
652
|
+
for startup and high-fan-out losses beside selective wins.
|
|
653
|
+
|
|
654
|
+
Two-level grouping currently accepts an explicit parent `$groupby` over a
|
|
655
|
+
singular member, with one nested group over that parent's bound row sequence.
|
|
656
|
+
Count, sum, average, minimum and maximum recompute from only the affected
|
|
657
|
+
parent's bounded leaves. An offset, an unsupported operator, a global input to
|
|
658
|
+
the nested group, or a group-of-groups LINQ emission remains a named rerun.
|
|
659
|
+
Replicated writes enter the same committed capture stream as local writes.
|
package/docs/MODEL-FORMAT.md
CHANGED
|
@@ -1078,6 +1078,14 @@ error.
|
|
|
1078
1078
|
| `JD2092` | a worker row, compatibility result or remote identity count exceeds its declared bound |
|
|
1079
1079
|
| `JD2093` | malformed worker protocol request |
|
|
1080
1080
|
| `JD2094` | invalid or uncommitted durable snapshot; reopen the last committed version |
|
|
1081
|
+
| `JD0060` | a replication envelope or snapshot is invalid |
|
|
1082
|
+
| `JD2100` | a replica sequence or causal dependency has a gap |
|
|
1083
|
+
| `JD2101` | an envelope identity names different content or an unknown local origin |
|
|
1084
|
+
| `JD2102` | the replica identity or model revision disagrees |
|
|
1085
|
+
| `JD2103` | a resolver fails its synchronous decision contract |
|
|
1086
|
+
| `JD2104` | a logical row or snapshot disagrees with its causal history |
|
|
1087
|
+
| `JD2105` | an explicit snapshot reset is required, or reset would discard acknowledged history |
|
|
1088
|
+
| `JD2106` | a replication operation or snapshot exceeds its configured bound |
|
|
1081
1089
|
|
|
1082
1090
|
The table above is proven in sync with the runtime `DB_CODES` table by
|
|
1083
1091
|
a test.
|
|
@@ -0,0 +1,208 @@
|
|
|
1
|
+
# Replication format and runtime
|
|
2
|
+
|
|
3
|
+
`jaren-replication` 0.1 is a transport-neutral logical transaction. Open an empty
|
|
4
|
+
store with `replication: { replica: 'host-issued-id' }`; capture is enabled when
|
|
5
|
+
omitted. Existing replication stores must reopen with the same identity and
|
|
6
|
+
model. Every writer of that file must enable replication. External SQL writes
|
|
7
|
+
and opening a replicated file without replication are outside the history
|
|
8
|
+
contract; a detected before-image disagreement refuses with `JD2104`.
|
|
9
|
+
|
|
10
|
+
The host owns network transport, authentication, replica identity allocation and
|
|
11
|
+
resolver policy. A replica id must remain unique to its writer lineage. The
|
|
12
|
+
runtime works on SQLite Node, worker and wasm hosts through session or journal
|
|
13
|
+
capture. Drivers without `changeCapture` refuse at open; PostgreSQL capture is
|
|
14
|
+
not implemented. Journal replication refuses models with cascading or set-null
|
|
15
|
+
child foreign keys (`JD0051`), because the journal cannot observe those effects.
|
|
16
|
+
Session capture includes them. Both modes support membership cascades and
|
|
17
|
+
roll back an envelope whose side effects are absent from its logical operations.
|
|
18
|
+
|
|
19
|
+
## Envelope
|
|
20
|
+
|
|
21
|
+
```json
|
|
22
|
+
{
|
|
23
|
+
"$replication": "0.1",
|
|
24
|
+
"replica": "site-a",
|
|
25
|
+
"seq": 1,
|
|
26
|
+
"frontier": {},
|
|
27
|
+
"model": "application-model-revision",
|
|
28
|
+
"operations": [
|
|
29
|
+
{ "table": "notes", "key": "note/one", "before": null,
|
|
30
|
+
"after": { "id": "note/one", "body": "hello" } }
|
|
31
|
+
]
|
|
32
|
+
}
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
`model` is the store's `shapeHash(model)` revision. `seq` is a positive safe
|
|
36
|
+
integer, allocated from the replica's durable frontier in the data transaction.
|
|
37
|
+
The identity is `replicationIdentity(replica, seq)`, the JSON encoding of their
|
|
38
|
+
pair, so separators inside host ids cannot collide. A frontier maps host ids to
|
|
39
|
+
non-negative safe sequences. The sender's frontier entry must equal `seq - 1`,
|
|
40
|
+
with an absent entry meaning zero. Property names such as `__proto__` and
|
|
41
|
+
`constructor` remain ordinary replica ids.
|
|
42
|
+
|
|
43
|
+
Each operation is one net row transition. `before: null` inserts, `after: null`
|
|
44
|
+
deletes, and two objects replace the complete logical row. A table/key occurs
|
|
45
|
+
once per envelope. Composite keys use the capture format's JSON-array token;
|
|
46
|
+
single keys use their scalar text. Entity documents include mapped scalar and
|
|
47
|
+
foreign-key fields, omit relation projections and retain authored version and
|
|
48
|
+
timestamp values. Membership tables carry their two keys. Application performs
|
|
49
|
+
schema validation through the store's configured validator and reuses physical
|
|
50
|
+
column plans; it does not generate fresh defaults or version stamps.
|
|
51
|
+
|
|
52
|
+
`normalizeReplication` validates and detaches the document. `encodeReplication`
|
|
53
|
+
produces deterministic JSON. Object members are canonically ordered; operation
|
|
54
|
+
array order remains part of identity. Capture sorts net rows by table/key, making
|
|
55
|
+
physical session and journal output equal. Envelope order preserves transaction
|
|
56
|
+
order. Physical insertion order inside a netted transaction or snapshot is not a
|
|
57
|
+
portable query ordering: use an explicit query order when comparing replicas.
|
|
58
|
+
Receipts retain the entire canonical payload, avoiding hash-collision ambiguity.
|
|
59
|
+
|
|
60
|
+
Unknown members and versions, empty ids, unsafe sequences, non-JSON values,
|
|
61
|
+
cycles, duplicate net rows and unchanged operations are `JD0060`. Nesting is
|
|
62
|
+
limited to 128 object/array ancestors. JSON Schema artifacts describe structural
|
|
63
|
+
validation; cross-field sequence, uniqueness and causal checks are runtime
|
|
64
|
+
constraints. Current and draft-07 artifacts ship for envelopes and snapshots:
|
|
65
|
+
|
|
66
|
+
- `@jarenjs/db/schemas/jaren-replication.schema.json`
|
|
67
|
+
- `@jarenjs/db/schemas/jaren-replication.draft-07.schema.json`
|
|
68
|
+
- `@jarenjs/db/schemas/jaren-replication-snapshot.schema.json`
|
|
69
|
+
- `@jarenjs/db/schemas/jaren-replication-snapshot.draft-07.schema.json`
|
|
70
|
+
|
|
71
|
+
## Applying and exporting
|
|
72
|
+
|
|
73
|
+
```js
|
|
74
|
+
const source = await openStore(model, {
|
|
75
|
+
driver, replication: { replica: 'site-a', retention: 1000 }
|
|
76
|
+
});
|
|
77
|
+
const page = await source.replication.page({ after: 0, limit: 100, maxBytes: 1048576 });
|
|
78
|
+
for (const envelope of page.items) {
|
|
79
|
+
const result = await target.replication.apply(envelope);
|
|
80
|
+
if (result.status === 'conflict') hostReport(result.conflicts);
|
|
81
|
+
}
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
`apply` returns `applied`, `duplicate` or `conflict`, together with the committed
|
|
85
|
+
frontier and conflict records. Successful duplicate delivery performs no row
|
|
86
|
+
write or live emission. An acknowledged identity with different bytes is
|
|
87
|
+
`JD2101`. A sequence or causal gap is `JD2100`: no suffix is applied. Deliver
|
|
88
|
+
missing envelopes first, then retry, or perform an explicit reset. Model mismatch
|
|
89
|
+
is `JD2102`. Imported envelopes keep their original identity and produce no local
|
|
90
|
+
outbox entry, preventing echoes. A host forwarding between peers must preserve
|
|
91
|
+
the original envelopes; `page()` exports this replica's own commits only.
|
|
92
|
+
|
|
93
|
+
Data, receipt, per-row causal metadata, capture record and frontier settle in
|
|
94
|
+
the same transaction. Replication is available only on the root Store; transaction
|
|
95
|
+
views expose no replication API. Failed validation, cancellation, constraint checks or
|
|
96
|
+
commit roll everything back. Subscribers run through the existing committed
|
|
97
|
+
capture delivery path. Tests kill a process immediately before and after commit
|
|
98
|
+
and verify both data and frontier after reopening.
|
|
99
|
+
|
|
100
|
+
Pages use the shared cursor and byte-credit drain. `limit` counts whole envelopes;
|
|
101
|
+
`maxBytes` counts complete canonical envelope bytes. An indivisible oversize
|
|
102
|
+
envelope is `JD2074`. Retention loss returns `resetRequired: true`, no items and
|
|
103
|
+
no continuation. `earliestAvailable`, `highWatermark`, `next` and `hasMore` describe
|
|
104
|
+
the local outbox. Pages and snapshots take a consistent store transaction.
|
|
105
|
+
`signal` and `deadline` are honored through existing cancellation contracts:
|
|
106
|
+
queue cancellation is `JD2064`, operation-boundary cancellation `JD2072`, and an
|
|
107
|
+
expired deadline `JD2075`.
|
|
108
|
+
|
|
109
|
+
Defaults are `retention: 1000`, `maxOperations: 10000`, `maxBytes: 4194304`.
|
|
110
|
+
Operation overflow is `JD2106`; all credits are positive safe integers. Retention
|
|
111
|
+
prunes only outgoing envelope history. Replay receipts, row tombstones and
|
|
112
|
+
conflict evidence remain durable. They are deliberately not claimed to be a
|
|
113
|
+
bounded total database size. Internal tables use the reserved `_jaren_replica`
|
|
114
|
+
prefix. No schema migration or revision rewrite of this protocol is implicit.
|
|
115
|
+
|
|
116
|
+
## Conflicts
|
|
117
|
+
|
|
118
|
+
Concurrent incompatible whole-row writes reject by default. A `conflict` result
|
|
119
|
+
does not acknowledge the remote envelope and leaves the local row intact.
|
|
120
|
+
`replication.conflicts({ limit, maxBytes, signal, deadline })` reads persisted evidence: the common base,
|
|
121
|
+
local value and frontier, remote value and envelope position, resolver identity
|
|
122
|
+
and decision. A rejected envelope reserves its canonical identity too, so retries
|
|
123
|
+
cannot replace its contender with different content.
|
|
124
|
+
|
|
125
|
+
An optional `replication.resolver` has `{ id, resolve }`. Its synchronous pure
|
|
126
|
+
function receives frozen evidence and returns `{ action: 'local' }`,
|
|
127
|
+
`{ action: 'remote' }`, or `{ action: 'merged', value }`; a merged value may be
|
|
128
|
+
null for deletion. Invalid decisions refuse with `JD2103`. Both contenders and
|
|
129
|
+
the resolver id are retained for every resolution. No resolver is bundled.
|
|
130
|
+
Determinism and convergence of a custom policy are the host's responsibility:
|
|
131
|
+
choosing the receiver's local value on each replica is deterministic but does
|
|
132
|
+
not make replicas agree. Independent non-conflicting deliveries converge without
|
|
133
|
+
resolver policy. Concurrent changes to different fields of the same row are
|
|
134
|
+
conservatively whole-row conflicts in this version.
|
|
135
|
+
|
|
136
|
+
## Reset handshake
|
|
137
|
+
|
|
138
|
+
`source.replication.snapshot()` returns a `jaren-replication-snapshot` 0.1
|
|
139
|
+
document containing `model`, `frontier`, logical `rows` with their causal
|
|
140
|
+
frontiers, and complete acknowledged `receipts`.
|
|
141
|
+
`target.replication.reset(snapshot)` validates and atomically installs it while
|
|
142
|
+
emitting ordinary live invalidations. The snapshot must dominate all target
|
|
143
|
+
acknowledgements and preserve every known receipt's bytes. It cannot discard
|
|
144
|
+
unsynchronized local history (`JD2105`) or rewrite identities (`JD2101`).
|
|
145
|
+
Snapshot receipt sequences must completely cover their frontier, and row
|
|
146
|
+
frontiers cannot be ahead of it. Subsequent duplicate deliveries remain
|
|
147
|
+
verifiable because the receipts travel with the snapshot.
|
|
148
|
+
|
|
149
|
+
Resets are intentionally bounded single documents: rows plus receipts must fit
|
|
150
|
+
`maxOperations`, and the complete encoding must fit `maxBytes`. Snapshot and
|
|
151
|
+
conflict readers pull through the shared cursor and stop before accumulating
|
|
152
|
+
beyond the configured credits. Resolver outputs and conflict evidence also
|
|
153
|
+
obey `maxBytes`. Very large
|
|
154
|
+
histories require a separately designed paged snapshot/receipt compaction
|
|
155
|
+
protocol; this implementation refuses them rather than splicing partial state.
|
|
156
|
+
A reset preserves the receiver's own replica identity and clears its outgoing
|
|
157
|
+
history. Future writes continue its sequence from the imported frontier.
|
|
158
|
+
|
|
159
|
+
## Authoring
|
|
160
|
+
|
|
161
|
+
`@jarenjs/linq/db` exports a deterministic pen using the same validator:
|
|
162
|
+
|
|
163
|
+
```js
|
|
164
|
+
const envelope = defineReplication({ replica: 'site-a', seq: 1,
|
|
165
|
+
frontier: {}, model: revision })
|
|
166
|
+
.change('notes', 'one', null, { id: 'one', body: 'hello' })
|
|
167
|
+
.toDocument();
|
|
168
|
+
```
|
|
169
|
+
|
|
170
|
+
The pen snapshots its input and emits the same canonical bytes as hand-authored
|
|
171
|
+
JSON. Normal applications export committed Store envelopes; constructing an
|
|
172
|
+
envelope does not allocate or acknowledge a Store sequence.
|
|
173
|
+
|
|
174
|
+
## Measurements
|
|
175
|
+
|
|
176
|
+
Run `npm run benchmark:changeflow`. Every timed mutation is followed by a fresh
|
|
177
|
+
query and a patch-only consumer comparison; every measured mutation changes its
|
|
178
|
+
visible result. Graph measurements mutate a child of one owner. Replication deliveries compare
|
|
179
|
+
complete database state, including relations. Conflict timings retry one
|
|
180
|
+
rejected contender and assert unchanged data/frontier plus durable evidence. Timing includes capture and live
|
|
181
|
+
delivery; oracle time is reported separately. Heap deltas are uncollected
|
|
182
|
+
allocations observed after initialization, not precise retained-heap sizes.
|
|
183
|
+
|
|
184
|
+
<!--fact:db.changeflow-->
|
|
185
|
+
|
|
186
|
+
Measured 2026-09-08, v24.19.0, AMD Ryzen 9 5900HX with Radeon Graphics; 15 mutations per case.
|
|
187
|
+
|
|
188
|
+
| Shape | Strategy | Initialize ms | Mutation p50 ms | Mutation p95 ms | Initialization heap bytes |
|
|
189
|
+
|---|---|---:|---:|---:|---:|
|
|
190
|
+
| selective join | join | 16.663 | 0.323 | 2.119 | 4878848 |
|
|
191
|
+
| selective join | rerun | 0.899 | 0.746 | 1.622 | 347160 |
|
|
192
|
+
| high fan-out join | join | 7.314 | 1.901 | 3.179 | 4851232 |
|
|
193
|
+
| high fan-out join | rerun | 0.860 | 0.850 | 1.188 | 340328 |
|
|
194
|
+
| graph | graph | 7.462 | 0.459 | 0.750 | 4811712 |
|
|
195
|
+
| graph | rerun | 4.539 | 4.042 | 4.979 | 1417928 |
|
|
196
|
+
| nested groups | nested-group | 6.775 | 0.252 | 0.795 | 2057832 |
|
|
197
|
+
| nested groups | rerun | 0.897 | 0.472 | 0.685 | 295808 |
|
|
198
|
+
| offset groups | rerun | 1.519 | 0.459 | 0.547 | 383648 |
|
|
199
|
+
| offset groups | rerun | 0.884 | 0.426 | 0.585 | 282280 |
|
|
200
|
+
|
|
201
|
+
| Capture | Envelopes | Operations | Bytes | Apply p50 ms | Replay p50 ms | Conflict p50 ms |
|
|
202
|
+
|---|---:|---:|---:|---:|---:|---:|---:|
|
|
203
|
+
| session | 15 | 15 | 3169 | 0.458 | 0.118 | 0.204 |
|
|
204
|
+
| journal | 15 | 15 | 3169 | 0.294 | 0.092 | 0.161 |
|
|
205
|
+
|
|
206
|
+
Selective maintenance avoids repeated full SQL evaluation. Initialization and high-fan-out maintenance can cost more than rerunning; the table includes both. Offset groups remain rerun in both requested modes.
|
|
207
|
+
|
|
208
|
+
<!--/fact-->
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@jarenjs/db",
|
|
3
3
|
"private": false,
|
|
4
|
-
"version": "0.
|
|
4
|
+
"version": "0.67.0",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "./src/index.js",
|
|
7
7
|
"types": "./types/index.d.ts",
|
|
@@ -84,9 +84,9 @@
|
|
|
84
84
|
"prepack": "npm run build:types"
|
|
85
85
|
},
|
|
86
86
|
"dependencies": {
|
|
87
|
-
"@jarenjs/core": "^0.
|
|
88
|
-
"@jarenjs/json": "^0.
|
|
89
|
-
"@jarenjs/validate": "^0.
|
|
87
|
+
"@jarenjs/core": "^0.67.0",
|
|
88
|
+
"@jarenjs/json": "^0.67.0",
|
|
89
|
+
"@jarenjs/validate": "^0.67.0"
|
|
90
90
|
},
|
|
91
91
|
"bin": {
|
|
92
92
|
"jaren-db": "./src/cli.js"
|
|
@@ -0,0 +1,83 @@
|
|
|
1
|
+
{
|
|
2
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
3
|
+
"$id": "https://jarenjs.dev/schemas/jaren-replication-snapshot/draft-07",
|
|
4
|
+
"title": "Bounded replication snapshot and frontier handshake",
|
|
5
|
+
"type": "object",
|
|
6
|
+
"additionalProperties": false,
|
|
7
|
+
"required": [
|
|
8
|
+
"$replicationSnapshot",
|
|
9
|
+
"model",
|
|
10
|
+
"frontier",
|
|
11
|
+
"rows",
|
|
12
|
+
"receipts"
|
|
13
|
+
],
|
|
14
|
+
"properties": {
|
|
15
|
+
"$replicationSnapshot": {
|
|
16
|
+
"const": "0.1"
|
|
17
|
+
},
|
|
18
|
+
"model": {
|
|
19
|
+
"type": "string",
|
|
20
|
+
"minLength": 1
|
|
21
|
+
},
|
|
22
|
+
"frontier": {
|
|
23
|
+
"type": "object",
|
|
24
|
+
"propertyNames": {
|
|
25
|
+
"type": "string",
|
|
26
|
+
"minLength": 1,
|
|
27
|
+
"maxLength": 128
|
|
28
|
+
},
|
|
29
|
+
"additionalProperties": {
|
|
30
|
+
"type": "integer",
|
|
31
|
+
"minimum": 0,
|
|
32
|
+
"maximum": 9007199254740991
|
|
33
|
+
}
|
|
34
|
+
},
|
|
35
|
+
"rows": {
|
|
36
|
+
"type": "array",
|
|
37
|
+
"items": {
|
|
38
|
+
"type": "object",
|
|
39
|
+
"additionalProperties": false,
|
|
40
|
+
"required": [
|
|
41
|
+
"table",
|
|
42
|
+
"key",
|
|
43
|
+
"value",
|
|
44
|
+
"frontier"
|
|
45
|
+
],
|
|
46
|
+
"properties": {
|
|
47
|
+
"table": {
|
|
48
|
+
"type": "string",
|
|
49
|
+
"pattern": "^[A-Za-z_][A-Za-z0-9_]*$"
|
|
50
|
+
},
|
|
51
|
+
"key": {
|
|
52
|
+
"type": "string"
|
|
53
|
+
},
|
|
54
|
+
"value": {
|
|
55
|
+
"type": [
|
|
56
|
+
"object",
|
|
57
|
+
"null"
|
|
58
|
+
]
|
|
59
|
+
},
|
|
60
|
+
"frontier": {
|
|
61
|
+
"type": "object",
|
|
62
|
+
"propertyNames": {
|
|
63
|
+
"type": "string",
|
|
64
|
+
"minLength": 1,
|
|
65
|
+
"maxLength": 128
|
|
66
|
+
},
|
|
67
|
+
"additionalProperties": {
|
|
68
|
+
"type": "integer",
|
|
69
|
+
"minimum": 0,
|
|
70
|
+
"maximum": 9007199254740991
|
|
71
|
+
}
|
|
72
|
+
}
|
|
73
|
+
}
|
|
74
|
+
}
|
|
75
|
+
},
|
|
76
|
+
"receipts": {
|
|
77
|
+
"type": "array",
|
|
78
|
+
"items": {
|
|
79
|
+
"$ref": "https://jarenjs.dev/schemas/jaren-replication/draft-07"
|
|
80
|
+
}
|
|
81
|
+
}
|
|
82
|
+
}
|
|
83
|
+
}
|
|
@@ -0,0 +1,83 @@
|
|
|
1
|
+
{
|
|
2
|
+
"$schema": "https://json-schema.org/draft/2020-12/schema",
|
|
3
|
+
"$id": "https://jarenjs.dev/schemas/jaren-replication-snapshot",
|
|
4
|
+
"title": "Bounded replication snapshot and frontier handshake",
|
|
5
|
+
"type": "object",
|
|
6
|
+
"additionalProperties": false,
|
|
7
|
+
"required": [
|
|
8
|
+
"$replicationSnapshot",
|
|
9
|
+
"model",
|
|
10
|
+
"frontier",
|
|
11
|
+
"rows",
|
|
12
|
+
"receipts"
|
|
13
|
+
],
|
|
14
|
+
"properties": {
|
|
15
|
+
"$replicationSnapshot": {
|
|
16
|
+
"const": "0.1"
|
|
17
|
+
},
|
|
18
|
+
"model": {
|
|
19
|
+
"type": "string",
|
|
20
|
+
"minLength": 1
|
|
21
|
+
},
|
|
22
|
+
"frontier": {
|
|
23
|
+
"type": "object",
|
|
24
|
+
"propertyNames": {
|
|
25
|
+
"type": "string",
|
|
26
|
+
"minLength": 1,
|
|
27
|
+
"maxLength": 128
|
|
28
|
+
},
|
|
29
|
+
"additionalProperties": {
|
|
30
|
+
"type": "integer",
|
|
31
|
+
"minimum": 0,
|
|
32
|
+
"maximum": 9007199254740991
|
|
33
|
+
}
|
|
34
|
+
},
|
|
35
|
+
"rows": {
|
|
36
|
+
"type": "array",
|
|
37
|
+
"items": {
|
|
38
|
+
"type": "object",
|
|
39
|
+
"additionalProperties": false,
|
|
40
|
+
"required": [
|
|
41
|
+
"table",
|
|
42
|
+
"key",
|
|
43
|
+
"value",
|
|
44
|
+
"frontier"
|
|
45
|
+
],
|
|
46
|
+
"properties": {
|
|
47
|
+
"table": {
|
|
48
|
+
"type": "string",
|
|
49
|
+
"pattern": "^[A-Za-z_][A-Za-z0-9_]*$"
|
|
50
|
+
},
|
|
51
|
+
"key": {
|
|
52
|
+
"type": "string"
|
|
53
|
+
},
|
|
54
|
+
"value": {
|
|
55
|
+
"type": [
|
|
56
|
+
"object",
|
|
57
|
+
"null"
|
|
58
|
+
]
|
|
59
|
+
},
|
|
60
|
+
"frontier": {
|
|
61
|
+
"type": "object",
|
|
62
|
+
"propertyNames": {
|
|
63
|
+
"type": "string",
|
|
64
|
+
"minLength": 1,
|
|
65
|
+
"maxLength": 128
|
|
66
|
+
},
|
|
67
|
+
"additionalProperties": {
|
|
68
|
+
"type": "integer",
|
|
69
|
+
"minimum": 0,
|
|
70
|
+
"maximum": 9007199254740991
|
|
71
|
+
}
|
|
72
|
+
}
|
|
73
|
+
}
|
|
74
|
+
}
|
|
75
|
+
},
|
|
76
|
+
"receipts": {
|
|
77
|
+
"type": "array",
|
|
78
|
+
"items": {
|
|
79
|
+
"$ref": "https://jarenjs.dev/schemas/jaren-replication"
|
|
80
|
+
}
|
|
81
|
+
}
|
|
82
|
+
}
|
|
83
|
+
}
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
{
|
|
2
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
|
3
|
+
"$id": "https://jarenjs.dev/schemas/jaren-replication/draft-07",
|
|
4
|
+
"title": "Portable logical replication transaction",
|
|
5
|
+
"type": "object",
|
|
6
|
+
"additionalProperties": false,
|
|
7
|
+
"required": [
|
|
8
|
+
"$replication",
|
|
9
|
+
"replica",
|
|
10
|
+
"seq",
|
|
11
|
+
"frontier",
|
|
12
|
+
"model",
|
|
13
|
+
"operations"
|
|
14
|
+
],
|
|
15
|
+
"properties": {
|
|
16
|
+
"$replication": {
|
|
17
|
+
"const": "0.1"
|
|
18
|
+
},
|
|
19
|
+
"replica": {
|
|
20
|
+
"type": "string",
|
|
21
|
+
"minLength": 1,
|
|
22
|
+
"maxLength": 128
|
|
23
|
+
},
|
|
24
|
+
"seq": {
|
|
25
|
+
"type": "integer",
|
|
26
|
+
"minimum": 1,
|
|
27
|
+
"maximum": 9007199254740991
|
|
28
|
+
},
|
|
29
|
+
"frontier": {
|
|
30
|
+
"type": "object",
|
|
31
|
+
"propertyNames": {
|
|
32
|
+
"type": "string",
|
|
33
|
+
"minLength": 1,
|
|
34
|
+
"maxLength": 128
|
|
35
|
+
},
|
|
36
|
+
"additionalProperties": {
|
|
37
|
+
"type": "integer",
|
|
38
|
+
"minimum": 0,
|
|
39
|
+
"maximum": 9007199254740991
|
|
40
|
+
}
|
|
41
|
+
},
|
|
42
|
+
"model": {
|
|
43
|
+
"type": "string",
|
|
44
|
+
"minLength": 1
|
|
45
|
+
},
|
|
46
|
+
"operations": {
|
|
47
|
+
"type": "array",
|
|
48
|
+
"minItems": 1,
|
|
49
|
+
"items": {
|
|
50
|
+
"type": "object",
|
|
51
|
+
"additionalProperties": false,
|
|
52
|
+
"required": [
|
|
53
|
+
"table",
|
|
54
|
+
"key",
|
|
55
|
+
"before",
|
|
56
|
+
"after"
|
|
57
|
+
],
|
|
58
|
+
"properties": {
|
|
59
|
+
"table": {
|
|
60
|
+
"type": "string",
|
|
61
|
+
"pattern": "^[A-Za-z_][A-Za-z0-9_]*$"
|
|
62
|
+
},
|
|
63
|
+
"key": {
|
|
64
|
+
"type": "string"
|
|
65
|
+
},
|
|
66
|
+
"before": {
|
|
67
|
+
"type": [
|
|
68
|
+
"object",
|
|
69
|
+
"null"
|
|
70
|
+
]
|
|
71
|
+
},
|
|
72
|
+
"after": {
|
|
73
|
+
"type": [
|
|
74
|
+
"object",
|
|
75
|
+
"null"
|
|
76
|
+
]
|
|
77
|
+
}
|
|
78
|
+
}
|
|
79
|
+
}
|
|
80
|
+
}
|
|
81
|
+
}
|
|
82
|
+
}
|