@jarenjs/db 0.46.5 → 0.56.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/ARCHITECTURE.md +133 -17
- package/README.md +270 -36
- package/docs/JOBS-FORMAT.md +24 -8
- package/docs/LIVE-FORMAT.md +139 -7
- package/docs/MIGRATION-FORMAT.md +118 -36
- package/docs/MODEL-FORMAT.md +251 -30
- package/package.json +4 -5
- package/schemas/jaren-migration.draft-07.schema.json +73 -0
- package/schemas/jaren-migration.schema.json +73 -0
- package/src/algebra.js +22 -3
- package/src/capture.js +66 -28
- package/src/cli.js +225 -44
- package/src/ddl.js +23 -3
- package/src/dialect.js +13 -0
- package/src/dialects/sqlite.js +21 -1
- package/src/driver.js +63 -16
- package/src/drivers/wasm.js +1 -0
- package/src/emit-model.js +14 -0
- package/src/emit.js +42 -9
- package/src/entity.js +92 -47
- package/src/errors.js +28 -0
- package/src/index.js +2 -2
- package/src/jobs.js +40 -5
- package/src/live-time.js +605 -0
- package/src/live.js +52 -9
- package/src/migrate.js +397 -191
- package/src/model.js +173 -8
- package/src/plan.js +834 -47
- package/src/query.js +296 -22
- package/src/residual.js +15 -6
- package/src/series.js +349 -0
- package/src/store.js +243 -69
- package/src/tracker.js +173 -48
- package/types/index.d.ts +206 -12
- package/types/node.d.ts +3 -1
- package/types/typed.d.ts +58 -2
- package/types/wasm.d.ts +7 -0
- package/dist/types/algebra.d.ts +0 -199
- package/dist/types/app.d.ts +0 -49
- package/dist/types/capture.d.ts +0 -85
- package/dist/types/cli.d.ts +0 -2
- package/dist/types/dag-job.d.ts +0 -40
- package/dist/types/ddl.d.ts +0 -229
- package/dist/types/derive.d.ts +0 -250
- package/dist/types/dialect.d.ts +0 -149
- package/dist/types/dialects/sqlite.d.ts +0 -9
- package/dist/types/driver.d.ts +0 -110
- package/dist/types/drivers/bun.d.ts +0 -47
- package/dist/types/drivers/node.d.ts +0 -37
- package/dist/types/drivers/wasm.d.ts +0 -65
- package/dist/types/emit-model.d.ts +0 -44
- package/dist/types/emit.d.ts +0 -75
- package/dist/types/entity.d.ts +0 -23
- package/dist/types/errors.d.ts +0 -167
- package/dist/types/graph.d.ts +0 -28
- package/dist/types/index.d.ts +0 -37
- package/dist/types/jobs.d.ts +0 -140
- package/dist/types/knn.d.ts +0 -69
- package/dist/types/live.d.ts +0 -62
- package/dist/types/migrate.d.ts +0 -170
- package/dist/types/model.d.ts +0 -36
- package/dist/types/patch-sql.d.ts +0 -37
- package/dist/types/plan.d.ts +0 -140
- package/dist/types/profile.d.ts +0 -80
- package/dist/types/query.d.ts +0 -111
- package/dist/types/residual.d.ts +0 -61
- package/dist/types/store.d.ts +0 -53
- package/dist/types/tracker.d.ts +0 -43
- package/dist/types/typed.d.ts +0 -15
- package/dist/types/types.d.ts +0 -26
- package/dist/types/udf.d.ts +0 -75
- package/dist/types/window.d.ts +0 -52
package/docs/LIVE-FORMAT.md
CHANGED
|
@@ -92,7 +92,10 @@ them).
|
|
|
92
92
|
already knows what it wrote. **Less complete, stated plainly**: it
|
|
93
93
|
cannot see writes made through raw SQL, triggers, or another
|
|
94
94
|
connection; a journal-mode delete of a row the store never read
|
|
95
|
-
emits its `remove` without having seen the old document;
|
|
95
|
+
emits its `remove` without having seen the old document; a keyed
|
|
96
|
+
`put` reads the stored document first, so it emits the `replace`
|
|
97
|
+
session mode emits and a `put` that changes nothing emits nothing;
|
|
98
|
+
and of the
|
|
96
99
|
database's own `ON DELETE` side effects it reconstructs exactly ONE
|
|
97
100
|
— join-table membership dying with its entity (read before the
|
|
98
101
|
delete) — while cascades into CHILD rows (`onDelete: 'cascade'` /
|
|
@@ -114,8 +117,14 @@ reads forward:
|
|
|
114
117
|
const records = await store.changesSince(lastSeq); // JD2051 when no log
|
|
115
118
|
```
|
|
116
119
|
|
|
117
|
-
`seq` is monotonic; with the log enabled
|
|
118
|
-
|
|
120
|
+
`seq` is monotonic; with the log enabled the DATABASE allocates it —
|
|
121
|
+
each record's `seq` is `MAX(seq) + 1` computed inside the insert
|
|
122
|
+
statement and read back through `RETURNING` — so two stores over one
|
|
123
|
+
file never collide on the log's key and each sees the other's
|
|
124
|
+
sequence continue; without the log it is per-process. `changesSince`
|
|
125
|
+
answers records in the shape observers receive, `collections`
|
|
126
|
+
included; a cursor that is not a number is a `TypeError`, as is a
|
|
127
|
+
`retention` that is not a positive integer. Retention is
|
|
119
128
|
a bounded count (`retention`, default 1000): older rows are pruned in
|
|
120
129
|
the same transaction. The log is an ordered, replayable stream —
|
|
121
130
|
which is what makes a late-joining consumer possible. **Replication
|
|
@@ -148,6 +157,7 @@ against its own result document (§9). Registration:
|
|
|
148
157
|
const live = await store.collection('users').live(document, {
|
|
149
158
|
externals: {}, // fixed at registration (§8)
|
|
150
159
|
mode: 'auto', // 'auto' | 'incremental' | 'rerun'
|
|
160
|
+
eventTime: undefined, // a temporal view's watermark (§13)
|
|
151
161
|
});
|
|
152
162
|
live.result; // the maintained result document
|
|
153
163
|
live.mode; // { strategy, mode: 'incremental'|'rerun', reason }
|
|
@@ -160,6 +170,14 @@ multi-entity shape of MODEL-FORMAT §10) the same way. Live queries
|
|
|
160
170
|
REQUIRE change capture — the patch stream is the invalidation source —
|
|
161
171
|
and registering on a store opened without `capture` is `JD0050`.
|
|
162
172
|
|
|
173
|
+
A producer may hand a registration a CHAIN instead of a document: the
|
|
174
|
+
`@jarenjs/linq/db` client's `live(chain, options)` passes the chain's
|
|
175
|
+
`toDocument()` and its `explain().bindings` as the externals to exactly
|
|
176
|
+
these two registrations (`store.live` for an entity-root chain,
|
|
177
|
+
`collection.live` for a collection's), so the strategy, the reason and
|
|
178
|
+
the maintenance are this table's — an entity chain re-runs, declared —
|
|
179
|
+
and this document stays the only place they are decided.
|
|
180
|
+
|
|
163
181
|
**This table is normative.** Every row is implemented and tested;
|
|
164
182
|
nothing outside it is attempted. Classification reads the compiled
|
|
165
183
|
PLAN (never the raw document), so "extractable" below means exactly
|
|
@@ -177,6 +195,8 @@ what the pushdown planner already means by it.
|
|
|
177
195
|
| `orderBy` beside a refined spatial predicate — over `$distance` (not a path) or over a member (the set residual drops the planner's order terms) | **re-run on invalidation**, the ordering named as the reason | the previous result, for diffing |
|
|
178
196
|
| a whole-query aggregate or a `groupBy` whose `where` is a refined spatial predicate | **re-run on invalidation** — the accumulator needs a fully translated selection and a refinement is not one; the reason says so | the previous result, for diffing |
|
|
179
197
|
| a spatial predicate the planner **refused** (no `derive` index on the member, an untyped member, an unbounded probe) | **re-run on invalidation**, the refusal named — it never translated, so nothing narrows the fetch | the previous result, for diffing |
|
|
198
|
+
| a `$resample` or `$rolling` document over the collection, with an explicit `eventTime` and a fixed width (§13) | **event-time bucket / rolling state**: rows kept by bucket, or in instant order; only what a write can reach is folded again, through `@jarenjs/core/series` itself | the contributing rows, plus one fold per bucket |
|
|
199
|
+
| the same document with no `eventTime`, a calendar width, a named zone, a `locf`/`linear` fill, a `first`/`last` aggregate, or a retention that does not cover the window | **re-run on invalidation**, the member that stopped it named (§13.2) | the previous result, for diffing |
|
|
180
200
|
| joins, multi-entity roots, graph loads, every entity query | **re-run on invalidation — declared, not attempted** in this version | the previous result, for diffing |
|
|
181
201
|
| anything else: non-translatable predicates, `limit` without `orderBy`, `offset` > 0, windowed aggregates, `@jarenjs/linq`'s nested two-level `groupBy` emission, non-canonical group returns | **re-run on invalidation**, the reason named | the previous result, for diffing |
|
|
182
202
|
|
|
@@ -257,7 +277,11 @@ state. `externals` are fixed at registration — a query whose inputs
|
|
|
257
277
|
change is a new registration.
|
|
258
278
|
|
|
259
279
|
Maintenance runs synchronously inside patch delivery, in commit
|
|
260
|
-
order, on the store's own connection.
|
|
280
|
+
order, on the store's own connection. Delivery is never re-entered: a
|
|
281
|
+
write made from inside an observer or a subscriber commits at once,
|
|
282
|
+
but its record is queued and delivered after the current record has
|
|
283
|
+
reached every consumer, so sibling live views see commits in commit
|
|
284
|
+
order rather than in call-stack order. Writes from ANOTHER connection
|
|
261
285
|
are invisible to capture (§6) and therefore to live queries; the
|
|
262
286
|
coarse `dataVersion()` signal and the §11 topology are the honest
|
|
263
287
|
answers, and re-registering re-reads.
|
|
@@ -372,7 +396,115 @@ ERRORING rather than degrading (the D14 rule — the bound is printed):
|
|
|
372
396
|
|
|
373
397
|
Non-claims, in one place: no incremental joins (re-run is the declared
|
|
374
398
|
strategy), no cross-connection invalidation (§6's `data_version` is
|
|
375
|
-
the signal), no maintenance over asynchronous connections
|
|
376
|
-
|
|
377
|
-
|
|
399
|
+
the signal), no maintenance over asynchronous connections —
|
|
400
|
+
`capabilities.live` is `false` there and a registration is `JD0051`
|
|
401
|
+
naming the reason, because maintenance point-reads rows synchronously
|
|
402
|
+
inside delivery (the wasm driver's oo1 API is synchronous, which is
|
|
403
|
+
why the browser has live queries at all) — no replication, and no ordering guarantee for
|
|
378
404
|
unordered queries beyond §9's determinism.
|
|
405
|
+
|
|
406
|
+
## 13. Event time
|
|
407
|
+
|
|
408
|
+
A live view over time needs to know what "now" is — which reading counts
|
|
409
|
+
as late — and the machine's clock is a different quantity from the
|
|
410
|
+
instant a reading carries. So there is no clock in this layer and none
|
|
411
|
+
under it: the **watermark arrives**.
|
|
412
|
+
|
|
413
|
+
```js
|
|
414
|
+
const live = await store.collection('readings').live(
|
|
415
|
+
[{ $resample: ['$[*]', { every: 60_000, aggregate: 'mean' }] }],
|
|
416
|
+
{ eventTime: {
|
|
417
|
+
path: '$.at', // the instant member, as a row selector
|
|
418
|
+
watermark: 1767225600000, // a finite epoch the HOST supplies
|
|
419
|
+
allowedLateness: 300_000, // how late a reading may still be
|
|
420
|
+
retention: 900_000, // the horizon this view claims
|
|
421
|
+
} });
|
|
422
|
+
|
|
423
|
+
live.advance(1767225660000); // the only way a watermark moves
|
|
424
|
+
live.stats().watermark; // what it is now
|
|
425
|
+
```
|
|
426
|
+
|
|
427
|
+
`eventTime` is a **closed** member set: `path`, `watermark`,
|
|
428
|
+
`allowedLateness` (default 0) and `retention`. Anything else — a
|
|
429
|
+
misspelling, a non-finite epoch, a negative lateness, a `path` that is
|
|
430
|
+
not a singular row selector — is `JD0053` at registration (its
|
|
431
|
+
`docPath` names the collection, `/collections/<name>`), not a member
|
|
432
|
+
quietly ignored. `advance()` refuses a value that is not finite or that
|
|
433
|
+
goes backwards (a `TypeError`), and it is absent on every view
|
|
434
|
+
registered without an `eventTime`. An entity document has no collection
|
|
435
|
+
to place rows in and re-runs, so an `eventTime` on `store.live` is
|
|
436
|
+
`JD0053` too.
|
|
437
|
+
|
|
438
|
+
### 13.1 What is maintained
|
|
439
|
+
|
|
440
|
+
Two documents, and only these two shapes: `$resample` and `$rolling`
|
|
441
|
+
whose series operand is the collection (`"$[*]"`, or a FLWOR over it
|
|
442
|
+
whose `$where` narrows and whose `$return` is the bare binding). A
|
|
443
|
+
spec that spells `at` and `value` explicitly — the spelling the query
|
|
444
|
+
language accepts — is maintained: the view folds with the kernel
|
|
445
|
+
reading the declared instant member and the `value` member the spec
|
|
446
|
+
names, and a `value` selector the view cannot follow re-runs with the
|
|
447
|
+
reason named, never a maintained view that dies on its first fold.
|
|
448
|
+
|
|
449
|
+
- **A bucket view keeps its rows by bucket.** A write touches one bucket
|
|
450
|
+
— two, when it moves a reading across a boundary — and exactly those
|
|
451
|
+
are folded again by calling `resampleSeries` over that bucket's own
|
|
452
|
+
rows. The aggregate is therefore the kernel's, and cannot drift from
|
|
453
|
+
what a fresh query would answer.
|
|
454
|
+
- **A rolling view keeps its rows in instant order.** A write at `t` can
|
|
455
|
+
only change the windows ending in `[t, t + width)`, so exactly that
|
|
456
|
+
stretch is recomputed — again by the kernel, over the slice those
|
|
457
|
+
windows can see.
|
|
458
|
+
|
|
459
|
+
`retention` is the horizon the view claims, and it is checked rather
|
|
460
|
+
than assumed: it must cover the width plus `allowedLateness`, which is
|
|
461
|
+
the span a single repair can read. A shorter one is a re-run with the
|
|
462
|
+
numbers printed. It is **not** a compaction policy — the maintained
|
|
463
|
+
state is bounded by `live.maxMaintained` exactly as every other
|
|
464
|
+
strategy's is (§12), and nothing an answer still depends on is dropped.
|
|
465
|
+
That is the honest statement of what this version buys: bounded repair
|
|
466
|
+
work and a visible lateness contract, not a smaller heap.
|
|
467
|
+
|
|
468
|
+
### 13.2 What re-runs, and why
|
|
469
|
+
|
|
470
|
+
| Refused | Because |
|
|
471
|
+
|---|---|
|
|
472
|
+
| no `eventTime` | a temporal view maintains event time, and a hidden clock is the one source this suite will not use |
|
|
473
|
+
| a calendar `every`/`width` (`P1M`, `P1D` on a zone) | a calendar ladder walks a wall clock and a month has no width, so its boundaries move with the data rather than with arithmetic |
|
|
474
|
+
| a named `zone` | it resolves through the injected provider, which maintenance would have to consult per boundary |
|
|
475
|
+
| `fill: 'locf'` / `'linear'` | they fill an empty bucket from its neighbours, so one late reading moves buckets it never belonged to |
|
|
476
|
+
| `aggregate: 'first'` / `'last'` | they name a row by its position in the series, which a per-key state does not preserve — the same refusal the pushdown planner makes |
|
|
477
|
+
| a `retention` under `width + allowedLateness` | a repair could read outside the horizon the view claims |
|
|
478
|
+
| a `$subsequence` window, a projecting operand, a collection with no document key | there is no row a key can be tracked through |
|
|
479
|
+
| a spec whose `at` selector is not `eventTime.path` | the state would place a row by one instant and aggregate it by another |
|
|
480
|
+
|
|
481
|
+
Each of those is `live.mode.reason`, and `mode: 'incremental'` still
|
|
482
|
+
refuses them at registration with `JD0051`.
|
|
483
|
+
|
|
484
|
+
### 13.3 Late data is visible, never lost
|
|
485
|
+
|
|
486
|
+
A reading is **late** when its instant — before the write, after it, or
|
|
487
|
+
both — is behind `watermark - allowedLateness`. Late readings are not
|
|
488
|
+
dropped and not quietly folded in. The view **re-reads** from the store,
|
|
489
|
+
so the answer still equals what a fresh query would give, and the
|
|
490
|
+
emission carries the reason:
|
|
491
|
+
|
|
492
|
+
```js
|
|
493
|
+
live.subscribe(({ patch, seq, lateData }) => {
|
|
494
|
+
if (lateData !== undefined) {
|
|
495
|
+
// { reason: 'late-data', at, key, watermark, allowedLateness, boundary }
|
|
496
|
+
}
|
|
497
|
+
});
|
|
498
|
+
```
|
|
499
|
+
|
|
500
|
+
`stats().lateData` counts them and `stats().reruns` counts the re-reads
|
|
501
|
+
they forced. A reading outside the view's own `start`/`end` window is
|
|
502
|
+
not late data: it belongs to no bucket this view maintains, so there is
|
|
503
|
+
nothing to be late for.
|
|
504
|
+
|
|
505
|
+
The maintained answer is **equal to a full recomputation after every
|
|
506
|
+
mutation** — not approximately, and not eventually. `test/db/live-time.test.js`
|
|
507
|
+
holds a shuffled stream of inserts, in-place updates, instant moves and
|
|
508
|
+
deletes against `resampleSeries` / `rollingSeries` over the whole
|
|
509
|
+
collection after each one, which is the only oracle that cannot drift
|
|
510
|
+
with the implementation.
|
package/docs/MIGRATION-FORMAT.md
CHANGED
|
@@ -45,18 +45,29 @@ shape change is a **transformation of values**, not a table rebuild.
|
|
|
45
45
|
- `kind: "jslt"` rewrites every document of a collection through a
|
|
46
46
|
compiled JSLT stylesheet, in batches, inside the migration's
|
|
47
47
|
transaction. The empty stylesheet (`[]`) is the identity transform.
|
|
48
|
-
|
|
49
|
-
|
|
48
|
+
Over an ENTITY table the stylesheet sees the whole row — the mapped
|
|
49
|
+
columns merged into the document under the TARGET model's mapping —
|
|
50
|
+
and what it returns is split back into columns and document by that
|
|
51
|
+
mapping; the key member is kept from the row (a stylesheet that
|
|
52
|
+
omits it loses nothing) and a stylesheet that changes it is
|
|
53
|
+
`JD0023`. A step carrying `"draft": true` is a planner placeholder
|
|
54
|
+
and MUST refuse to run (`JD0021`) until the author fills it in.
|
|
50
55
|
- `kind: "query"` is an assertion: the query runs over the
|
|
51
56
|
collection's documents and MUST answer an empty sequence (`expect:
|
|
52
57
|
"empty"`, the default) or an EBV-true value (`expect: "ebv"`) for
|
|
53
58
|
the migration to proceed. This is how a migration states its own
|
|
54
59
|
precondition — "no user has a null email before the NOT NULL
|
|
55
|
-
index" — and it is checked on the shadow first.
|
|
60
|
+
index" — and it is checked on the shadow first. Over an entity table
|
|
61
|
+
the assertion reads the same merged rows a `jslt` step sees.
|
|
56
62
|
- `kind: "derive"` recomputes named STORED derived index columns from
|
|
57
63
|
the documents already in a collection — the backfill described in
|
|
58
64
|
§2.1. It is idempotent: a derived value is a pure function of the
|
|
59
65
|
document, so a replay writes what the first run wrote.
|
|
66
|
+
- `kind: "sql"` executes one rendered DATA statement — a fold of a
|
|
67
|
+
column into the document, a backfill, an `INSERT … SELECT` — §9.4; a
|
|
68
|
+
dry run always prints it with its note.
|
|
69
|
+
- `kind: "rebuild"` is the entity restructure of §10, self-contained:
|
|
70
|
+
the `CREATE` of the new shape, the copy and the index DDL.
|
|
60
71
|
- Steps are ordered, and the order is the contract.
|
|
61
72
|
|
|
62
73
|
### 2.1 Derived spatial columns and the backfill
|
|
@@ -97,7 +108,11 @@ and the column entry carries the width the value is packed to:
|
|
|
97
108
|
## 3. Planning and the widening/narrowing rule
|
|
98
109
|
|
|
99
110
|
`planMigration(fromModel, toModel, { dialect, id, derived })` produces
|
|
100
|
-
`{ migration, report }` by diffing the two models' PHYSICAL plans
|
|
111
|
+
`{ migration, report }` by diffing the two models' PHYSICAL plans. The
|
|
112
|
+
from-model is the previous model — the previous model FILE, or, under
|
|
113
|
+
the CLI's snapshot discipline (§11), the committed `model.snapshot.json`
|
|
114
|
+
the last `plan` advanced: a database stores shape hashes, never models,
|
|
115
|
+
so the previous shape lives beside the code, where a diff can read it.
|
|
101
116
|
|
|
102
117
|
- An added collection becomes its full CREATE DDL; a removed
|
|
103
118
|
collection becomes a `DROP TABLE` step whose note says
|
|
@@ -107,7 +122,16 @@ and the column entry carries the width the value is packed to:
|
|
|
107
122
|
`"x-rename": "oldName"` on the target collection; the planner emits
|
|
108
123
|
the rename first and rebuilds the indexes (a renamed SQLite table
|
|
109
124
|
keeps its old index names — probed). Without the hint, a rename is
|
|
110
|
-
a drop plus a create and the report says so.
|
|
125
|
+
a drop plus a create and the report says so. The hint is not part of
|
|
126
|
+
the shape: `x-rename` is stripped before the shape hash is computed,
|
|
127
|
+
so a model that keeps carrying a satisfied hint hashes the same as
|
|
128
|
+
one without it and plans nothing — a rename is idempotent across
|
|
129
|
+
`plan` runs. An entity rename carries its join tables with it, by
|
|
130
|
+
the endpoints the mapping records rather than by splitting the
|
|
131
|
+
table's name (an entity name may itself contain `_`); where the
|
|
132
|
+
rename flips the sorted endpoint order the join table is rebuilt
|
|
133
|
+
(create, `INSERT … SELECT`, drop) under its new name and no
|
|
134
|
+
membership is lost.
|
|
111
135
|
- Added, removed and changed indexes become index DDL — reusing the
|
|
112
136
|
store's own DDL generator, never a second implementation. A changed
|
|
113
137
|
generated column (type or path) is a drop plus an add, with its
|
|
@@ -123,7 +147,10 @@ and the column entry carries the width the value is packed to:
|
|
|
123
147
|
schema through the injected `compileSchema` hook. A document that no
|
|
124
148
|
longer validates is `JD0021` and the whole migration rolls back — a
|
|
125
149
|
narrowing without an adequate transform cannot land. A widening
|
|
126
|
-
needs no transform, and passes this check by fact.
|
|
150
|
+
needs no transform, and passes this check by fact. For an entity
|
|
151
|
+
the validated document is the whole row — columns merged back under
|
|
152
|
+
the target mapping — so a pure widening of a column-mapped member
|
|
153
|
+
passes and a narrowing of one is caught.
|
|
127
154
|
- Changing a collection's key declaration is not planned (a rebuild);
|
|
128
155
|
the planner refuses with a `TypeError` naming the non-goal.
|
|
129
156
|
|
|
@@ -139,10 +166,11 @@ store untouched.
|
|
|
139
166
|
|
|
140
167
|
The shadow runs over an empty data set; the real-data facts (the
|
|
141
168
|
widening check, key consistency, the assertions over real rows) run on
|
|
142
|
-
the real store inside its transaction. The
|
|
143
|
-
|
|
144
|
-
the shadow
|
|
145
|
-
|
|
169
|
+
the real store inside its transaction. The shadow registers the same
|
|
170
|
+
functions as the real run: `migrate(…, { registerFunctions })` runs on
|
|
171
|
+
the shadow, the real and the reference connections before any DDL
|
|
172
|
+
(§10), so a hand-created index over a registered deterministic
|
|
173
|
+
function neither fails the shadow nor is silently dropped by it.
|
|
146
174
|
|
|
147
175
|
## 5. History and checksums
|
|
148
176
|
|
|
@@ -171,8 +199,16 @@ hash of the `baseline` model when no migration has run.
|
|
|
171
199
|
otherwise), the physical end shape is verified, and the real-data
|
|
172
200
|
validation of §3 runs.
|
|
173
201
|
- `dryRun: true` prints every statement and the affected document
|
|
174
|
-
counts, validates the chain on the shadow, and writes NOTHING
|
|
175
|
-
|
|
202
|
+
counts, validates the chain on the shadow, and writes NOTHING — not
|
|
203
|
+
even the history table: it PROBES for one and reads an absent one as
|
|
204
|
+
an empty history, so a dry run may be pointed at a production
|
|
205
|
+
database and leave its file byte-identical. The API default is to
|
|
206
|
+
run; a CLI SHOULD default to the dry run.
|
|
207
|
+
- `migrationStatus` (and the CLI's `status`/`check`) create the empty
|
|
208
|
+
history table on a database that has none — the one write a reading
|
|
209
|
+
command makes, so a fresh file answers `applied: (none)` rather than
|
|
210
|
+
a missing-table error. This is the one place the two differ: a dry
|
|
211
|
+
run reports the same state and writes nothing at all.
|
|
176
212
|
- Each pending migration runs in ONE exclusive transaction
|
|
177
213
|
(`BEGIN IMMEDIATE` on SQLite — concurrent writers wait or time out
|
|
178
214
|
under the busy timeout) with a savepoint per step; any failure rolls
|
|
@@ -225,13 +261,13 @@ every row has a shadow-verified test that migrates seeded data:
|
|
|
225
261
|
| Change | Strategy |
|
|
226
262
|
|---|---|
|
|
227
263
|
| add mapped column (property added, or moved out of the document) | `ALTER TABLE ADD COLUMN` — always nullable (absent reads back absent, MODEL-FORMAT §9.3) — plus a `sql` data step when the property's values already live in the document |
|
|
228
|
-
| drop mapped column (property removed, or moved into the document) | fold the column back into the document first (`sql` step) when the property survives; drop its index, then `DROP COLUMN` where SQLite's conditions hold, else rebuild |
|
|
264
|
+
| drop mapped column (property removed, or moved into the document) | fold the column back into the document first (`sql` step) when the property survives — a `NULL` column folds to ABSENT, never to JSON `null`, so §9.3's rule survives the fold, in the rebuild copy too; drop its index, then `DROP COLUMN` where SQLite's conditions hold, else rebuild |
|
|
229
265
|
| change type / enum CHECK / key / epoch flavor | **rebuild** (§10) |
|
|
230
266
|
| add or drop an index (`unique`/`index`/version) | plain DDL |
|
|
231
|
-
| add or drop a relation (foreign-key column, join table) | foreign keys **rebuild** the holder; join tables create/drop directly |
|
|
232
|
-
| entity added / dropped | create / `DROP TABLE` (destructive, named) |
|
|
267
|
+
| add or drop a relation (foreign-key column, join table) | foreign keys **rebuild** the holder — an inferred foreign-key column the target model no longer declares is folded into the document when the target declares the property, else named in `report.lost` and the plan is destructive; join tables create/drop directly |
|
|
268
|
+
| entity added / dropped | create / `DROP TABLE` (destructive, named), children before parents so no foreign key dangles mid-migration |
|
|
233
269
|
| entity renamed | declared with `x-rename` on the target entity — never inferred; join tables renamed mechanically with their endpoints |
|
|
234
|
-
| scalar ⇄ JSONB move (`column: "json"` toggled,
|
|
270
|
+
| scalar ⇄ JSONB move (`column: "json"` toggled) | the first two rows: `ADD COLUMN` plus a `sql` lift out of the document, or a `sql` fold plus `DROP COLUMN` — a rebuild only where SQLite cannot drop the column in place |
|
|
235
271
|
|
|
236
272
|
Two rules keep the diff honest:
|
|
237
273
|
|
|
@@ -279,12 +315,13 @@ the transaction** — a broken reference fails the migration rather
|
|
|
279
315
|
than shipping.
|
|
280
316
|
|
|
281
317
|
Two deviations from the cited twelve steps, recorded: (1) the
|
|
282
|
-
procedure
|
|
283
|
-
|
|
284
|
-
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
|
|
318
|
+
procedure's `PRAGMA foreign_keys=OFF/ON` bracket is honoured
|
|
319
|
+
literally, OUTSIDE the transaction (inside one the pragma is a no-op):
|
|
320
|
+
`node:sqlite` enables enforcement by default, and with it on a parent
|
|
321
|
+
table could not even be dropped, so a migration holding a rebuild
|
|
322
|
+
step turns enforcement off before `BEGIN IMMEDIATE` and back on after
|
|
323
|
+
it settles, and `foreign_key_check` inside the transaction provides
|
|
324
|
+
the guarantee the bracket suspended; (2) triggers and views are not re-created because this
|
|
288
325
|
store creates none — a hand-added trigger is outside the model and
|
|
289
326
|
outside the diff, which drift (§12) will name.
|
|
290
327
|
|
|
@@ -307,23 +344,68 @@ drop the index) on a schema the store accepts.
|
|
|
307
344
|
`jaren-db` drives the workflow (mirroring `jaren-emit`):
|
|
308
345
|
|
|
309
346
|
```
|
|
310
|
-
jaren-db plan
|
|
311
|
-
jaren-db
|
|
312
|
-
jaren-db
|
|
313
|
-
jaren-db
|
|
314
|
-
jaren-db
|
|
347
|
+
jaren-db plan --from <model> --to <model> [--store <db>] [--id x] [--out file]
|
|
348
|
+
jaren-db plan --model <model> [--snapshot <file>] [--store <db>] [--id x] --out file
|
|
349
|
+
jaren-db snapshot --model <model> [--snapshot <file>] [--types <file>]
|
|
350
|
+
jaren-db status --model <model> --store <db> --baseline <model> [--migrations <dir>] [--snapshot <file>]
|
|
351
|
+
jaren-db apply --store <db> --baseline <model> --migrations <dir> [--model <m>] [--dry-run] [--yes]
|
|
352
|
+
jaren-db check --model <model> --store <db> --baseline <model> [--migrations <dir>] [--snapshot <file>]
|
|
353
|
+
jaren-db shape --model <model>
|
|
315
354
|
```
|
|
316
355
|
|
|
356
|
+
- **A model or a migration is a `.json` file or a MODULE.** `--model`,
|
|
357
|
+
`--from`, `--to` and `--baseline` accept a `.json` file or a module
|
|
358
|
+
(`.js`, `.mjs`, `.cjs` — and `.ts` where the host strips types: Node
|
|
359
|
+
≥ 24 does by default, and `--no-strip-types` is refused by name)
|
|
360
|
+
loaded with `import()` and read as its `default` export or its `model`
|
|
361
|
+
export — the model pen's document, or any object whose `toJSON()`
|
|
362
|
+
emits one; `--migrations <dir>` reads `.json` files and modules
|
|
363
|
+
(`default` or `migration` — the migration pen's builder), sorted by
|
|
364
|
+
file name. A module whose export is not a document, or whose emission
|
|
365
|
+
is not JSON, fails with the module named. **Modules are pure:** the
|
|
366
|
+
CLI loads every module TWICE (two `import()`s under distinct
|
|
367
|
+
cache-busting queries) and refuses one whose two emissions differ —
|
|
368
|
+
no clock, no env, no randomness — because a migration that hashes
|
|
369
|
+
differently per load can never match its own history.
|
|
317
370
|
- `plan` diffs two model FILES (a database stores shape hashes, not
|
|
318
|
-
models — the from-model is the previous model file)
|
|
319
|
-
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
|
|
323
|
-
|
|
324
|
-
|
|
325
|
-
|
|
326
|
-
|
|
371
|
+
models — the from-model is the previous model file), or, with
|
|
372
|
+
`--model`, the committed SNAPSHOT against the model: `--snapshot`
|
|
373
|
+
names it and defaults to `model.snapshot.json` beside the model; a
|
|
374
|
+
model whose shape equals the snapshot's plans nothing and exits 0;
|
|
375
|
+
otherwise the migration is written (`--out`) and the snapshot is
|
|
376
|
+
advanced to the model — without `--out` the plan is printed and the
|
|
377
|
+
snapshot stays, and the CLI says so. With `--store` it first compares
|
|
378
|
+
the from-model's physical shape with the database itself — never
|
|
379
|
+
with the history, which would refuse every database that has applied
|
|
380
|
+
a migration.
|
|
381
|
+
- `snapshot` writes the model's snapshot (`--snapshot`, the same
|
|
382
|
+
default) — from the model the store was created with, before the
|
|
383
|
+
first `plan --model`; with `--types <file>` it also writes emit's
|
|
384
|
+
TypeScript declaration for the model (`entityEmitModel` rendered by
|
|
385
|
+
`@jarenjs/emit`, loaded lazily — `@jarenjs/db` does not depend on emit,
|
|
386
|
+
and a host without it is told exactly what `--types` needs), so a
|
|
387
|
+
transform over a JSON snapshot can be typed by annotation. Two runs on
|
|
388
|
+
one input write nothing the second time.
|
|
389
|
+
- `check` is the CI command: exit 1 on an UNPLANNED MODEL CHANGE (a
|
|
390
|
+
snapshot in use whose shape is not the model's — the model moved and
|
|
391
|
+
nobody planned; named as such, never as the database's drift), when
|
|
392
|
+
migrations are pending, OR when the database drifted; 0 in sync.
|
|
393
|
+
`--model` is required — without it drift cannot be measured, and
|
|
394
|
+
`check` refuses rather than print `in sync`. `status` reports the same
|
|
395
|
+
verdict on its `model:` line.
|
|
396
|
+
- `apply` prints every statement, then asks; destructive steps (drop
|
|
397
|
+
table/column, rebuild) print what is lost and ask for that
|
|
398
|
+
separately. `--yes` answers both, `--dry-run` stops after the
|
|
399
|
+
printout. Without an interactive terminal there is nobody to ask, so
|
|
400
|
+
`apply` without `--yes` exits 1 after the printout with nothing
|
|
401
|
+
applied — a CI job passes `--yes` deliberately, never by default.
|
|
402
|
+
`apply --dry-run` is the CLI's printout, not §6's `dryRun: true`: it
|
|
403
|
+
reads the history the way `status` does — creating the empty table on
|
|
404
|
+
a database that has none — and does NOT replay the chain on the
|
|
405
|
+
shadow, so a draft step still prints instead of refusing. The
|
|
406
|
+
shadow's verdict comes with the real `apply`.
|
|
407
|
+
- `status` lists applied/pending and reports drift (§12); on a
|
|
408
|
+
database without a history table it creates the empty one (§6).
|
|
327
409
|
- `shape` prints the physical mapping a model produces.
|
|
328
410
|
|
|
329
411
|
## 12. Drift
|