@jarenjs/db 0.46.5 → 0.56.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (72) hide show
  1. package/ARCHITECTURE.md +133 -17
  2. package/README.md +270 -36
  3. package/docs/JOBS-FORMAT.md +24 -8
  4. package/docs/LIVE-FORMAT.md +139 -7
  5. package/docs/MIGRATION-FORMAT.md +118 -36
  6. package/docs/MODEL-FORMAT.md +251 -30
  7. package/package.json +4 -5
  8. package/schemas/jaren-migration.draft-07.schema.json +73 -0
  9. package/schemas/jaren-migration.schema.json +73 -0
  10. package/src/algebra.js +22 -3
  11. package/src/capture.js +66 -28
  12. package/src/cli.js +225 -44
  13. package/src/ddl.js +23 -3
  14. package/src/dialect.js +13 -0
  15. package/src/dialects/sqlite.js +21 -1
  16. package/src/driver.js +63 -16
  17. package/src/drivers/wasm.js +1 -0
  18. package/src/emit-model.js +14 -0
  19. package/src/emit.js +42 -9
  20. package/src/entity.js +92 -47
  21. package/src/errors.js +28 -0
  22. package/src/index.js +2 -2
  23. package/src/jobs.js +40 -5
  24. package/src/live-time.js +605 -0
  25. package/src/live.js +52 -9
  26. package/src/migrate.js +397 -191
  27. package/src/model.js +173 -8
  28. package/src/plan.js +834 -47
  29. package/src/query.js +296 -22
  30. package/src/residual.js +15 -6
  31. package/src/series.js +349 -0
  32. package/src/store.js +243 -69
  33. package/src/tracker.js +173 -48
  34. package/types/index.d.ts +206 -12
  35. package/types/node.d.ts +3 -1
  36. package/types/typed.d.ts +58 -2
  37. package/types/wasm.d.ts +7 -0
  38. package/dist/types/algebra.d.ts +0 -199
  39. package/dist/types/app.d.ts +0 -49
  40. package/dist/types/capture.d.ts +0 -85
  41. package/dist/types/cli.d.ts +0 -2
  42. package/dist/types/dag-job.d.ts +0 -40
  43. package/dist/types/ddl.d.ts +0 -229
  44. package/dist/types/derive.d.ts +0 -250
  45. package/dist/types/dialect.d.ts +0 -149
  46. package/dist/types/dialects/sqlite.d.ts +0 -9
  47. package/dist/types/driver.d.ts +0 -110
  48. package/dist/types/drivers/bun.d.ts +0 -47
  49. package/dist/types/drivers/node.d.ts +0 -37
  50. package/dist/types/drivers/wasm.d.ts +0 -65
  51. package/dist/types/emit-model.d.ts +0 -44
  52. package/dist/types/emit.d.ts +0 -75
  53. package/dist/types/entity.d.ts +0 -23
  54. package/dist/types/errors.d.ts +0 -167
  55. package/dist/types/graph.d.ts +0 -28
  56. package/dist/types/index.d.ts +0 -37
  57. package/dist/types/jobs.d.ts +0 -140
  58. package/dist/types/knn.d.ts +0 -69
  59. package/dist/types/live.d.ts +0 -62
  60. package/dist/types/migrate.d.ts +0 -170
  61. package/dist/types/model.d.ts +0 -36
  62. package/dist/types/patch-sql.d.ts +0 -37
  63. package/dist/types/plan.d.ts +0 -140
  64. package/dist/types/profile.d.ts +0 -80
  65. package/dist/types/query.d.ts +0 -111
  66. package/dist/types/residual.d.ts +0 -61
  67. package/dist/types/store.d.ts +0 -53
  68. package/dist/types/tracker.d.ts +0 -43
  69. package/dist/types/typed.d.ts +0 -15
  70. package/dist/types/types.d.ts +0 -26
  71. package/dist/types/udf.d.ts +0 -75
  72. package/dist/types/window.d.ts +0 -52
@@ -92,7 +92,10 @@ them).
92
92
  already knows what it wrote. **Less complete, stated plainly**: it
93
93
  cannot see writes made through raw SQL, triggers, or another
94
94
  connection; a journal-mode delete of a row the store never read
95
- emits its `remove` without having seen the old document; and of the
95
+ emits its `remove` without having seen the old document; a keyed
96
+ `put` reads the stored document first, so it emits the `replace`
97
+ session mode emits and a `put` that changes nothing emits nothing;
98
+ and of the
96
99
  database's own `ON DELETE` side effects it reconstructs exactly ONE
97
100
  — join-table membership dying with its entity (read before the
98
101
  delete) — while cascades into CHILD rows (`onDelete: 'cascade'` /
@@ -114,8 +117,14 @@ reads forward:
114
117
  const records = await store.changesSince(lastSeq); // JD2051 when no log
115
118
  ```
116
119
 
117
- `seq` is monotonic; with the log enabled it continues across reopens
118
- (seeded from `MAX(seq)`), without it it is per-process. Retention is
120
+ `seq` is monotonic; with the log enabled the DATABASE allocates it —
121
+ each record's `seq` is `MAX(seq) + 1` computed inside the insert
122
+ statement and read back through `RETURNING` — so two stores over one
123
+ file never collide on the log's key and each sees the other's
124
+ sequence continue; without the log it is per-process. `changesSince`
125
+ answers records in the shape observers receive, `collections`
126
+ included; a cursor that is not a number is a `TypeError`, as is a
127
+ `retention` that is not a positive integer. Retention is
119
128
  a bounded count (`retention`, default 1000): older rows are pruned in
120
129
  the same transaction. The log is an ordered, replayable stream —
121
130
  which is what makes a late-joining consumer possible. **Replication
@@ -148,6 +157,7 @@ against its own result document (§9). Registration:
148
157
  const live = await store.collection('users').live(document, {
149
158
  externals: {}, // fixed at registration (§8)
150
159
  mode: 'auto', // 'auto' | 'incremental' | 'rerun'
160
+ eventTime: undefined, // a temporal view's watermark (§13)
151
161
  });
152
162
  live.result; // the maintained result document
153
163
  live.mode; // { strategy, mode: 'incremental'|'rerun', reason }
@@ -160,6 +170,14 @@ multi-entity shape of MODEL-FORMAT §10) the same way. Live queries
160
170
  REQUIRE change capture — the patch stream is the invalidation source —
161
171
  and registering on a store opened without `capture` is `JD0050`.
162
172
 
173
+ A producer may hand a registration a CHAIN instead of a document: the
174
+ `@jarenjs/linq/db` client's `live(chain, options)` passes the chain's
175
+ `toDocument()` and its `explain().bindings` as the externals to exactly
176
+ these two registrations (`store.live` for an entity-root chain,
177
+ `collection.live` for a collection's), so the strategy, the reason and
178
+ the maintenance are this table's — an entity chain re-runs, declared —
179
+ and this document stays the only place they are decided.
180
+
163
181
  **This table is normative.** Every row is implemented and tested;
164
182
  nothing outside it is attempted. Classification reads the compiled
165
183
  PLAN (never the raw document), so "extractable" below means exactly
@@ -177,6 +195,8 @@ what the pushdown planner already means by it.
177
195
  | `orderBy` beside a refined spatial predicate — over `$distance` (not a path) or over a member (the set residual drops the planner's order terms) | **re-run on invalidation**, the ordering named as the reason | the previous result, for diffing |
178
196
  | a whole-query aggregate or a `groupBy` whose `where` is a refined spatial predicate | **re-run on invalidation** — the accumulator needs a fully translated selection and a refinement is not one; the reason says so | the previous result, for diffing |
179
197
  | a spatial predicate the planner **refused** (no `derive` index on the member, an untyped member, an unbounded probe) | **re-run on invalidation**, the refusal named — it never translated, so nothing narrows the fetch | the previous result, for diffing |
198
+ | a `$resample` or `$rolling` document over the collection, with an explicit `eventTime` and a fixed width (§13) | **event-time bucket / rolling state**: rows kept by bucket, or in instant order; only what a write can reach is folded again, through `@jarenjs/core/series` itself | the contributing rows, plus one fold per bucket |
199
+ | the same document with no `eventTime`, a calendar width, a named zone, a `locf`/`linear` fill, a `first`/`last` aggregate, or a retention that does not cover the window | **re-run on invalidation**, the member that stopped it named (§13.2) | the previous result, for diffing |
180
200
  | joins, multi-entity roots, graph loads, every entity query | **re-run on invalidation — declared, not attempted** in this version | the previous result, for diffing |
181
201
  | anything else: non-translatable predicates, `limit` without `orderBy`, `offset` > 0, windowed aggregates, `@jarenjs/linq`'s nested two-level `groupBy` emission, non-canonical group returns | **re-run on invalidation**, the reason named | the previous result, for diffing |
182
202
 
@@ -257,7 +277,11 @@ state. `externals` are fixed at registration — a query whose inputs
257
277
  change is a new registration.
258
278
 
259
279
  Maintenance runs synchronously inside patch delivery, in commit
260
- order, on the store's own connection. Writes from ANOTHER connection
280
+ order, on the store's own connection. Delivery is never re-entered: a
281
+ write made from inside an observer or a subscriber commits at once,
282
+ but its record is queued and delivered after the current record has
283
+ reached every consumer, so sibling live views see commits in commit
284
+ order rather than in call-stack order. Writes from ANOTHER connection
261
285
  are invisible to capture (§6) and therefore to live queries; the
262
286
  coarse `dataVersion()` signal and the §11 topology are the honest
263
287
  answers, and re-registering re-reads.
@@ -372,7 +396,115 @@ ERRORING rather than degrading (the D14 rule — the bound is printed):
372
396
 
373
397
  Non-claims, in one place: no incremental joins (re-run is the declared
374
398
  strategy), no cross-connection invalidation (§6's `data_version` is
375
- the signal), no maintenance over asynchronous connections in this
376
- version (every current driver is synchronous; the browser driver's
377
- order owns that story), no replication, and no ordering guarantee for
399
+ the signal), no maintenance over asynchronous connections
400
+ `capabilities.live` is `false` there and a registration is `JD0051`
401
+ naming the reason, because maintenance point-reads rows synchronously
402
+ inside delivery (the wasm driver's oo1 API is synchronous, which is
403
+ why the browser has live queries at all) — no replication, and no ordering guarantee for
378
404
  unordered queries beyond §9's determinism.
405
+
406
+ ## 13. Event time
407
+
408
+ A live view over time needs to know what "now" is — which reading counts
409
+ as late — and the machine's clock is a different quantity from the
410
+ instant a reading carries. So there is no clock in this layer and none
411
+ under it: the **watermark arrives**.
412
+
413
+ ```js
414
+ const live = await store.collection('readings').live(
415
+ [{ $resample: ['$[*]', { every: 60_000, aggregate: 'mean' }] }],
416
+ { eventTime: {
417
+ path: '$.at', // the instant member, as a row selector
418
+ watermark: 1767225600000, // a finite epoch the HOST supplies
419
+ allowedLateness: 300_000, // how late a reading may still be
420
+ retention: 900_000, // the horizon this view claims
421
+ } });
422
+
423
+ live.advance(1767225660000); // the only way a watermark moves
424
+ live.stats().watermark; // what it is now
425
+ ```
426
+
427
+ `eventTime` is a **closed** member set: `path`, `watermark`,
428
+ `allowedLateness` (default 0) and `retention`. Anything else — a
429
+ misspelling, a non-finite epoch, a negative lateness, a `path` that is
430
+ not a singular row selector — is `JD0053` at registration (its
431
+ `docPath` names the collection, `/collections/<name>`), not a member
432
+ quietly ignored. `advance()` refuses a value that is not finite or that
433
+ goes backwards (a `TypeError`), and it is absent on every view
434
+ registered without an `eventTime`. An entity document has no collection
435
+ to place rows in and re-runs, so an `eventTime` on `store.live` is
436
+ `JD0053` too.
437
+
438
+ ### 13.1 What is maintained
439
+
440
+ Two documents, and only these two shapes: `$resample` and `$rolling`
441
+ whose series operand is the collection (`"$[*]"`, or a FLWOR over it
442
+ whose `$where` narrows and whose `$return` is the bare binding). A
443
+ spec that spells `at` and `value` explicitly — the spelling the query
444
+ language accepts — is maintained: the view folds with the kernel
445
+ reading the declared instant member and the `value` member the spec
446
+ names, and a `value` selector the view cannot follow re-runs with the
447
+ reason named, never a maintained view that dies on its first fold.
448
+
449
+ - **A bucket view keeps its rows by bucket.** A write touches one bucket
450
+ — two, when it moves a reading across a boundary — and exactly those
451
+ are folded again by calling `resampleSeries` over that bucket's own
452
+ rows. The aggregate is therefore the kernel's, and cannot drift from
453
+ what a fresh query would answer.
454
+ - **A rolling view keeps its rows in instant order.** A write at `t` can
455
+ only change the windows ending in `[t, t + width)`, so exactly that
456
+ stretch is recomputed — again by the kernel, over the slice those
457
+ windows can see.
458
+
459
+ `retention` is the horizon the view claims, and it is checked rather
460
+ than assumed: it must cover the width plus `allowedLateness`, which is
461
+ the span a single repair can read. A shorter one is a re-run with the
462
+ numbers printed. It is **not** a compaction policy — the maintained
463
+ state is bounded by `live.maxMaintained` exactly as every other
464
+ strategy's is (§12), and nothing an answer still depends on is dropped.
465
+ That is the honest statement of what this version buys: bounded repair
466
+ work and a visible lateness contract, not a smaller heap.
467
+
468
+ ### 13.2 What re-runs, and why
469
+
470
+ | Refused | Because |
471
+ |---|---|
472
+ | no `eventTime` | a temporal view maintains event time, and a hidden clock is the one source this suite will not use |
473
+ | a calendar `every`/`width` (`P1M`, `P1D` on a zone) | a calendar ladder walks a wall clock and a month has no width, so its boundaries move with the data rather than with arithmetic |
474
+ | a named `zone` | it resolves through the injected provider, which maintenance would have to consult per boundary |
475
+ | `fill: 'locf'` / `'linear'` | they fill an empty bucket from its neighbours, so one late reading moves buckets it never belonged to |
476
+ | `aggregate: 'first'` / `'last'` | they name a row by its position in the series, which a per-key state does not preserve — the same refusal the pushdown planner makes |
477
+ | a `retention` under `width + allowedLateness` | a repair could read outside the horizon the view claims |
478
+ | a `$subsequence` window, a projecting operand, a collection with no document key | there is no row a key can be tracked through |
479
+ | a spec whose `at` selector is not `eventTime.path` | the state would place a row by one instant and aggregate it by another |
480
+
481
+ Each of those is `live.mode.reason`, and `mode: 'incremental'` still
482
+ refuses them at registration with `JD0051`.
483
+
484
+ ### 13.3 Late data is visible, never lost
485
+
486
+ A reading is **late** when its instant — before the write, after it, or
487
+ both — is behind `watermark - allowedLateness`. Late readings are not
488
+ dropped and not quietly folded in. The view **re-reads** from the store,
489
+ so the answer still equals what a fresh query would give, and the
490
+ emission carries the reason:
491
+
492
+ ```js
493
+ live.subscribe(({ patch, seq, lateData }) => {
494
+ if (lateData !== undefined) {
495
+ // { reason: 'late-data', at, key, watermark, allowedLateness, boundary }
496
+ }
497
+ });
498
+ ```
499
+
500
+ `stats().lateData` counts them and `stats().reruns` counts the re-reads
501
+ they forced. A reading outside the view's own `start`/`end` window is
502
+ not late data: it belongs to no bucket this view maintains, so there is
503
+ nothing to be late for.
504
+
505
+ The maintained answer is **equal to a full recomputation after every
506
+ mutation** — not approximately, and not eventually. `test/db/live-time.test.js`
507
+ holds a shuffled stream of inserts, in-place updates, instant moves and
508
+ deletes against `resampleSeries` / `rollingSeries` over the whole
509
+ collection after each one, which is the only oracle that cannot drift
510
+ with the implementation.
@@ -45,18 +45,29 @@ shape change is a **transformation of values**, not a table rebuild.
45
45
  - `kind: "jslt"` rewrites every document of a collection through a
46
46
  compiled JSLT stylesheet, in batches, inside the migration's
47
47
  transaction. The empty stylesheet (`[]`) is the identity transform.
48
- A step carrying `"draft": true` is a planner placeholder and MUST
49
- refuse to run (`JD0021`) until the author fills it in.
48
+ Over an ENTITY table the stylesheet sees the whole row — the mapped
49
+ columns merged into the document under the TARGET model's mapping
50
+ and what it returns is split back into columns and document by that
51
+ mapping; the key member is kept from the row (a stylesheet that
52
+ omits it loses nothing) and a stylesheet that changes it is
53
+ `JD0023`. A step carrying `"draft": true` is a planner placeholder
54
+ and MUST refuse to run (`JD0021`) until the author fills it in.
50
55
  - `kind: "query"` is an assertion: the query runs over the
51
56
  collection's documents and MUST answer an empty sequence (`expect:
52
57
  "empty"`, the default) or an EBV-true value (`expect: "ebv"`) for
53
58
  the migration to proceed. This is how a migration states its own
54
59
  precondition — "no user has a null email before the NOT NULL
55
- index" — and it is checked on the shadow first.
60
+ index" — and it is checked on the shadow first. Over an entity table
61
+ the assertion reads the same merged rows a `jslt` step sees.
56
62
  - `kind: "derive"` recomputes named STORED derived index columns from
57
63
  the documents already in a collection — the backfill described in
58
64
  §2.1. It is idempotent: a derived value is a pure function of the
59
65
  document, so a replay writes what the first run wrote.
66
+ - `kind: "sql"` executes one rendered DATA statement — a fold of a
67
+ column into the document, a backfill, an `INSERT … SELECT` — §9.4; a
68
+ dry run always prints it with its note.
69
+ - `kind: "rebuild"` is the entity restructure of §10, self-contained:
70
+ the `CREATE` of the new shape, the copy and the index DDL.
60
71
  - Steps are ordered, and the order is the contract.
61
72
 
62
73
  ### 2.1 Derived spatial columns and the backfill
@@ -97,7 +108,11 @@ and the column entry carries the width the value is packed to:
97
108
  ## 3. Planning and the widening/narrowing rule
98
109
 
99
110
  `planMigration(fromModel, toModel, { dialect, id, derived })` produces
100
- `{ migration, report }` by diffing the two models' PHYSICAL plans:
111
+ `{ migration, report }` by diffing the two models' PHYSICAL plans. The
112
+ from-model is the previous model — the previous model FILE, or, under
113
+ the CLI's snapshot discipline (§11), the committed `model.snapshot.json`
114
+ the last `plan` advanced: a database stores shape hashes, never models,
115
+ so the previous shape lives beside the code, where a diff can read it.
101
116
 
102
117
  - An added collection becomes its full CREATE DDL; a removed
103
118
  collection becomes a `DROP TABLE` step whose note says
@@ -107,7 +122,16 @@ and the column entry carries the width the value is packed to:
107
122
  `"x-rename": "oldName"` on the target collection; the planner emits
108
123
  the rename first and rebuilds the indexes (a renamed SQLite table
109
124
  keeps its old index names — probed). Without the hint, a rename is
110
- a drop plus a create and the report says so.
125
+ a drop plus a create and the report says so. The hint is not part of
126
+ the shape: `x-rename` is stripped before the shape hash is computed,
127
+ so a model that keeps carrying a satisfied hint hashes the same as
128
+ one without it and plans nothing — a rename is idempotent across
129
+ `plan` runs. An entity rename carries its join tables with it, by
130
+ the endpoints the mapping records rather than by splitting the
131
+ table's name (an entity name may itself contain `_`); where the
132
+ rename flips the sorted endpoint order the join table is rebuilt
133
+ (create, `INSERT … SELECT`, drop) under its new name and no
134
+ membership is lost.
111
135
  - Added, removed and changed indexes become index DDL — reusing the
112
136
  store's own DDL generator, never a second implementation. A changed
113
137
  generated column (type or path) is a drop plus an add, with its
@@ -123,7 +147,10 @@ and the column entry carries the width the value is packed to:
123
147
  schema through the injected `compileSchema` hook. A document that no
124
148
  longer validates is `JD0021` and the whole migration rolls back — a
125
149
  narrowing without an adequate transform cannot land. A widening
126
- needs no transform, and passes this check by fact.
150
+ needs no transform, and passes this check by fact. For an entity
151
+ the validated document is the whole row — columns merged back under
152
+ the target mapping — so a pure widening of a column-mapped member
153
+ passes and a narrowing of one is caught.
127
154
  - Changing a collection's key declaration is not planned (a rebuild);
128
155
  the planner refuses with a `TypeError` naming the non-goal.
129
156
 
@@ -139,10 +166,11 @@ store untouched.
139
166
 
140
167
  The shadow runs over an empty data set; the real-data facts (the
141
168
  widening check, key consistency, the assertions over real rows) run on
142
- the real store inside its transaction. The model format declares no
143
- UDF-expression indexes, so there is no function set to re-register on
144
- the shadow stated here because a dialect that allowed such indexes
145
- would make the shadow fail on a schema the real store accepts.
169
+ the real store inside its transaction. The shadow registers the same
170
+ functions as the real run: `migrate(…, { registerFunctions })` runs on
171
+ the shadow, the real and the reference connections before any DDL
172
+ (§10), so a hand-created index over a registered deterministic
173
+ function neither fails the shadow nor is silently dropped by it.
146
174
 
147
175
  ## 5. History and checksums
148
176
 
@@ -171,8 +199,16 @@ hash of the `baseline` model when no migration has run.
171
199
  otherwise), the physical end shape is verified, and the real-data
172
200
  validation of §3 runs.
173
201
  - `dryRun: true` prints every statement and the affected document
174
- counts, validates the chain on the shadow, and writes NOTHING. The
175
- API default is to run; a CLI SHOULD default to the dry run.
202
+ counts, validates the chain on the shadow, and writes NOTHING — not
203
+ even the history table: it PROBES for one and reads an absent one as
204
+ an empty history, so a dry run may be pointed at a production
205
+ database and leave its file byte-identical. The API default is to
206
+ run; a CLI SHOULD default to the dry run.
207
+ - `migrationStatus` (and the CLI's `status`/`check`) create the empty
208
+ history table on a database that has none — the one write a reading
209
+ command makes, so a fresh file answers `applied: (none)` rather than
210
+ a missing-table error. This is the one place the two differ: a dry
211
+ run reports the same state and writes nothing at all.
176
212
  - Each pending migration runs in ONE exclusive transaction
177
213
  (`BEGIN IMMEDIATE` on SQLite — concurrent writers wait or time out
178
214
  under the busy timeout) with a savepoint per step; any failure rolls
@@ -225,13 +261,13 @@ every row has a shadow-verified test that migrates seeded data:
225
261
  | Change | Strategy |
226
262
  |---|---|
227
263
  | add mapped column (property added, or moved out of the document) | `ALTER TABLE ADD COLUMN` — always nullable (absent reads back absent, MODEL-FORMAT §9.3) — plus a `sql` data step when the property's values already live in the document |
228
- | drop mapped column (property removed, or moved into the document) | fold the column back into the document first (`sql` step) when the property survives; drop its index, then `DROP COLUMN` where SQLite's conditions hold, else rebuild |
264
+ | drop mapped column (property removed, or moved into the document) | fold the column back into the document first (`sql` step) when the property survives — a `NULL` column folds to ABSENT, never to JSON `null`, so §9.3's rule survives the fold, in the rebuild copy too; drop its index, then `DROP COLUMN` where SQLite's conditions hold, else rebuild |
229
265
  | change type / enum CHECK / key / epoch flavor | **rebuild** (§10) |
230
266
  | add or drop an index (`unique`/`index`/version) | plain DDL |
231
- | add or drop a relation (foreign-key column, join table) | foreign keys **rebuild** the holder; join tables create/drop directly |
232
- | entity added / dropped | create / `DROP TABLE` (destructive, named) |
267
+ | add or drop a relation (foreign-key column, join table) | foreign keys **rebuild** the holder — an inferred foreign-key column the target model no longer declares is folded into the document when the target declares the property, else named in `report.lost` and the plan is destructive; join tables create/drop directly |
268
+ | entity added / dropped | create / `DROP TABLE` (destructive, named), children before parents so no foreign key dangles mid-migration |
233
269
  | entity renamed | declared with `x-rename` on the target entity — never inferred; join tables renamed mechanically with their endpoints |
234
- | scalar ⇄ JSONB move (`column: "json"` toggled, shape change) | rebuild + a data step |
270
+ | scalar ⇄ JSONB move (`column: "json"` toggled) | the first two rows: `ADD COLUMN` plus a `sql` lift out of the document, or a `sql` fold plus `DROP COLUMN` — a rebuild only where SQLite cannot drop the column in place |
235
271
 
236
272
  Two rules keep the diff honest:
237
273
 
@@ -279,12 +315,13 @@ the transaction** — a broken reference fails the migration rather
279
315
  than shipping.
280
316
 
281
317
  Two deviations from the cited twelve steps, recorded: (1) the
282
- procedure brackets itself with `PRAGMA foreign_keys=OFF/ON`, which is
283
- a no-op inside a transaction the migration connection never enables
284
- the pragma (SQLite's default is off; `openStore` enables AND verifies
285
- it per connection), so enforcement during the rebuild is off exactly
286
- as the procedure wants, and `foreign_key_check` provides the
287
- guarantee; (2) triggers and views are not re-created because this
318
+ procedure's `PRAGMA foreign_keys=OFF/ON` bracket is honoured
319
+ literally, OUTSIDE the transaction (inside one the pragma is a no-op):
320
+ `node:sqlite` enables enforcement by default, and with it on a parent
321
+ table could not even be dropped, so a migration holding a rebuild
322
+ step turns enforcement off before `BEGIN IMMEDIATE` and back on after
323
+ it settles, and `foreign_key_check` inside the transaction provides
324
+ the guarantee the bracket suspended; (2) triggers and views are not re-created because this
288
325
  store creates none — a hand-added trigger is outside the model and
289
326
  outside the diff, which drift (§12) will name.
290
327
 
@@ -307,23 +344,68 @@ drop the index) on a schema the store accepts.
307
344
  `jaren-db` drives the workflow (mirroring `jaren-emit`):
308
345
 
309
346
  ```
310
- jaren-db plan --from <model> --to <model> [--store <db>] [--id x] [--out file]
311
- jaren-db status --model <model> --store <db> --baseline <model> [--migrations <dir>]
312
- jaren-db apply --store <db> --baseline <model> --migrations <dir> [--model <m>] [--dry-run] [--yes]
313
- jaren-db check --model <model> --store <db> --baseline <model> [--migrations <dir>]
314
- jaren-db shape --model <model>
347
+ jaren-db plan --from <model> --to <model> [--store <db>] [--id x] [--out file]
348
+ jaren-db plan --model <model> [--snapshot <file>] [--store <db>] [--id x] --out file
349
+ jaren-db snapshot --model <model> [--snapshot <file>] [--types <file>]
350
+ jaren-db status --model <model> --store <db> --baseline <model> [--migrations <dir>] [--snapshot <file>]
351
+ jaren-db apply --store <db> --baseline <model> --migrations <dir> [--model <m>] [--dry-run] [--yes]
352
+ jaren-db check --model <model> --store <db> --baseline <model> [--migrations <dir>] [--snapshot <file>]
353
+ jaren-db shape --model <model>
315
354
  ```
316
355
 
356
+ - **A model or a migration is a `.json` file or a MODULE.** `--model`,
357
+ `--from`, `--to` and `--baseline` accept a `.json` file or a module
358
+ (`.js`, `.mjs`, `.cjs` — and `.ts` where the host strips types: Node
359
+ ≥ 24 does by default, and `--no-strip-types` is refused by name)
360
+ loaded with `import()` and read as its `default` export or its `model`
361
+ export — the model pen's document, or any object whose `toJSON()`
362
+ emits one; `--migrations <dir>` reads `.json` files and modules
363
+ (`default` or `migration` — the migration pen's builder), sorted by
364
+ file name. A module whose export is not a document, or whose emission
365
+ is not JSON, fails with the module named. **Modules are pure:** the
366
+ CLI loads every module TWICE (two `import()`s under distinct
367
+ cache-busting queries) and refuses one whose two emissions differ —
368
+ no clock, no env, no randomness — because a migration that hashes
369
+ differently per load can never match its own history.
317
370
  - `plan` diffs two model FILES (a database stores shape hashes, not
318
- models — the from-model is the previous model file); with `--store`
319
- it first verifies the from-model's hash matches the database's
320
- recorded shape.
321
- - `check` is the CI command: exit 1 when migrations are pending OR the
322
- database drifted; 0 in sync.
323
- - `apply` prints every statement before running; destructive steps
324
- (drop table/column, rebuild) require `--yes` or an interactive
325
- confirmation that NAMES what is lost. Default is dry-run + ask.
326
- - `status` lists applied/pending and reports drift (§12).
371
+ models — the from-model is the previous model file), or, with
372
+ `--model`, the committed SNAPSHOT against the model: `--snapshot`
373
+ names it and defaults to `model.snapshot.json` beside the model; a
374
+ model whose shape equals the snapshot's plans nothing and exits 0;
375
+ otherwise the migration is written (`--out`) and the snapshot is
376
+ advanced to the model — without `--out` the plan is printed and the
377
+ snapshot stays, and the CLI says so. With `--store` it first compares
378
+ the from-model's physical shape with the database itself never
379
+ with the history, which would refuse every database that has applied
380
+ a migration.
381
+ - `snapshot` writes the model's snapshot (`--snapshot`, the same
382
+ default) — from the model the store was created with, before the
383
+ first `plan --model`; with `--types <file>` it also writes emit's
384
+ TypeScript declaration for the model (`entityEmitModel` rendered by
385
+ `@jarenjs/emit`, loaded lazily — `@jarenjs/db` does not depend on emit,
386
+ and a host without it is told exactly what `--types` needs), so a
387
+ transform over a JSON snapshot can be typed by annotation. Two runs on
388
+ one input write nothing the second time.
389
+ - `check` is the CI command: exit 1 on an UNPLANNED MODEL CHANGE (a
390
+ snapshot in use whose shape is not the model's — the model moved and
391
+ nobody planned; named as such, never as the database's drift), when
392
+ migrations are pending, OR when the database drifted; 0 in sync.
393
+ `--model` is required — without it drift cannot be measured, and
394
+ `check` refuses rather than print `in sync`. `status` reports the same
395
+ verdict on its `model:` line.
396
+ - `apply` prints every statement, then asks; destructive steps (drop
397
+ table/column, rebuild) print what is lost and ask for that
398
+ separately. `--yes` answers both, `--dry-run` stops after the
399
+ printout. Without an interactive terminal there is nobody to ask, so
400
+ `apply` without `--yes` exits 1 after the printout with nothing
401
+ applied — a CI job passes `--yes` deliberately, never by default.
402
+ `apply --dry-run` is the CLI's printout, not §6's `dryRun: true`: it
403
+ reads the history the way `status` does — creating the empty table on
404
+ a database that has none — and does NOT replay the chain on the
405
+ shadow, so a draft step still prints instead of refusing. The
406
+ shadow's verdict comes with the real `apply`.
407
+ - `status` lists applied/pending and reports drift (§12); on a
408
+ database without a history table it creates the empty one (§6).
327
409
  - `shape` prints the physical mapping a model produces.
328
410
 
329
411
  ## 12. Drift