@jarenjs/db 0.46.5 → 0.56.0

package/ARCHITECTURE.md

@@ -153,6 +153,17 @@ member the schema types as an array or an object. Those are exact; the
 spatial predicates that only NARROW are rows in the residual table, and
 the implied-conjunct table below carries every proof.
 
+Plus one GROUPING a composite instant index makes exact: the
+fixed-width temporal bucket. A `$groupby` whose single key is
+`$time-bucket` over a schema-typed integer epoch path with a literal
+width and origin, projected to the group key plus `$count` of the
+binding and `$sum`/`$avg`/`$min`/`$max` over singular schema-typed
+numeric paths, becomes a `GROUP BY` over integer arithmetic. So does a
+`$resample` whose frozen spec asks for nothing that ladder cannot do —
+a fixed width, `fill: 'omit'`, and one of `sum|mean|min|max|count`.
+Everything else about time is a NAMED refinement ("The temporal plan"
+below).
+
 Plus one ORDERING a **`derive: 'vector'`** column makes cheap without
 making it native: the k-nearest composition — `$orderby` on a
 `$similarity` key, descending, `$empty: 'least'`, under a `$subsequence`
@@ -181,6 +192,7 @@ mode, `knn`, beside native, row and set.
 | the k-nearest ordering over a `derive: 'vector'` column | the column cuts the candidates; the engine orders them (mode `knn`, "The k-nearest plan" below) — engine work, so `strict: true` refuses it |
 | `$similarity` anywhere else — a threshold in `$where`, a score in `$return`, a second ordering key | no native spelling; runs in the residual over whatever the rest of the document pushed |
 | the k-nearest shape with no finite window, ascending, `$empty: 'greatest'`, a probe that is neither a literal vector nor an external, a literal probe of another width, or a selection not pushed whole (a residual conjunct, an implied one, a `$let` before the where) | nothing is proven; the whole document runs in the engine, and `explain()` names which precondition failed — a k-nearest query never falls to the full scan silently |
+| `$resample` with a fill policy other than `omit`, `$rolling`, `$asof`, `first`/`last`, a calendar width, a named zone, a `$time-bucket` whose width or origin is an EXPRESSION, or an instant the schema does not type as an integer | the index bounds the fetch and `@jarenjs/core/series` decides over what comes back; `explain().series.reasons` carries the CODE — `fill-policy`, `rolling-refinement`, `asof-refinement`, `unsupported-aggregate`, `calendar-width`, `named-zone`, `nonliteral-spec`, `instant-not-integer` — and `strict: true` refuses every one of them |
 
 ### The type truth table
 
@@ -235,14 +247,21 @@ index-usable spelling and read every row of the collection.
 
 The guards make the forms sound for typed AND untyped paths alike —
 the schema type's job is choosing the generated COLUMN (the index),
-never weakening the guard. Two documented preconditions: string
-operators, aggregates AND ordering are only promoted on schema-typed
-paths because the engine ERRORS on non-conforming operands where SQL
-would coerce or sort — a stored `null` under an ordered key, a
-non-string under a string operator — so on an unvalidated store, rows
-violating the collection schema can make the engine throw where the
-database answers; keep `compileSchema` injected if that distinction
-matters to you.
+never weakening the guard. Two documented preconditions. String
+operators, aggregates and ordering are only promoted on schema-typed
+paths that cannot hold `null` and are not boolean, because the engine
+ERRORS on non-conforming operands where SQL would coerce or sort — a
+`null` under an ordered key, a non-string under a string operator —
+so a path whose schema admits `null` is a named residual for those
+forms and the two engines keep answering alike over conforming data.
+And the guards protect the JSON TYPE, not the VALUE: a typed
+generated column carries SQLite affinity, so on an unvalidated store a
+row that violates the schema — the string `'020'` under an `integer`
+path — reads as the integer `20` in the column while `json_type` still
+says text, and a pushed comparison answers rows the engine (comparing
+the JSON value) does not. Keep `compileSchema` injected: with the hook
+no such row is ever stored, and without it the pushdown is exact only
+over documents that happen to conform.
 
 **Bind-time diversion.** SQLite cannot bind a boolean, and a
 `null`-valued external needs Jaren's null semantics, not SQL's. At
@@ -418,6 +437,79 @@ engine's business. With `strict: true` the shape is `JD0010` naming the
 rank: the order is engine work, the same honesty as a spatial
 refinement.
 
+### The temporal plan (series)
+
+Time needs no storage kind here. The physical declaration is the
+composite JSONPath index a model already has —
+`{ "name": "by_series_at", "path": ["$.series", "$.at"] }` — over a
+numeric epoch member, and the whole of `src/series.js` is deciding
+which questions that index answers. There is no `derive: 'series'`, no
+column type, no host function and no extension.
+
+A B-tree seeks exactly as far as its leading columns are decided: a run
+of equalities, then at most one range. So the index that a temporal
+plan reports is the declared one with the longest leading run of
+columns an equality PINNED whose next column is the instant the query
+ranges over. Three shapes follow from that, and they are closed:
+
+| shape | the document | the plan |
+|---|---|---|
+| **range** | every leading column pinned, a half-open range on the instant, ordered by it | native — `SEARCH … (series=? AND at>? AND at<?)` |
+| **as-of** | the same prefix with ONE instant bound, ordered by the instant, under a finite window | native — the index read backwards, one row |
+| **bucket** | a `$groupby` over `$time-bucket`, or a `$resample` whose spec asks for nothing more | native — `GROUP BY` over `at - (((at - origin) % every + every) % every)` |
+
+The ladder is integer arithmetic all the way down, and the non-negative
+remainder is why: a truncating division puts an instant before 1970 in
+the bucket AFTER its own. The width and the anchor are read through the
+kernel's own `compileBuckets`, so `'PT1H'` and `3600000` are the same
+ladder and the default anchor is the kernel's rather than a second
+guess at it; an `{ offset }` calendar context folds into the anchor,
+because a constant number of minutes east is arithmetic. A named zone
+never does: its clock is host code the database does not have.
+
+Everything else is a **named refinement** — the implied-conjunct
+pattern again, applied to a whole operator. The planner narrows the
+fetch by whatever the frozen spec makes provable and the residual, which
+is the ENGINE running the caller's own document, decides:
+
+| operator | what bounds the fetch |
+|---|---|
+| `$resample` (fill, calendar, `first`/`last`) | the spec's own `start`/`end`, plus the operand's `$where` |
+| `$rolling` | the operand's `$where` alone — a window measured in time answers once per input instant |
+| `$asof` with the collection on the RIGHT | the probes' own span (`at <= max` backward, `at >= min` forward, both sides under a `tolerance`) and a membership test over the probes' `by` keys |
+| `$asof` with the collection on the LEFT | nothing — a join answers once per LEFT row, so every left row is needed |
+
+The as-of bound is one STATEMENT, whatever the probes number, which is
+what `test/db/statement-count.test.js` pins: the failure mode a batch
+exists to refuse is one seek per left row. Without a `tolerance` a
+backward join can only be bounded ABOVE, so that one statement can read
+most of a long history — `benchmark/series.js` publishes the candidate
+count beside the timing rather than netting it out.
+
+**A group with no instant.** A row whose instant member is missing or
+is not a number groups under SQL `NULL`; the kernel REFUSES such a row
+(`JQ2001`). SQL cannot refuse, so a native bucket that meets one hands
+the whole question back — the full-collection residual, where the engine
+answers what it answers everywhere — and the fallback is COUNTED as
+`collection.stats().series.diverted`, the same honesty the k-nearest
+divert has.
+
+**`explain().series`** is `null` unless the document asked a temporal
+question, which it does by naming a §8.16 operator or by being the
+closed range/as-of shape over a COMPOSITE instant index. (A singular
+index over an ordinary member cannot make a query temporal: nothing in
+a column says "instant", and inventing one would make every `age > 21`
+a temporal plan.) It carries `{ mode, operation, index, prefix, range,
+ladder, aggregates, refinement, reasons, counts }`. `mode` is
+`'native'` (the statement alone answers), `'hybrid'` (the database
+narrows and a kernel decides) or `'engine'` (the database narrowed
+nothing). `reasons` is `{ code, reason }` where the code is the first
+word of the sentence, so the machine-readable code and the sentence
+`strict: true` prints cannot drift apart. `counts` is the LAST ACTUAL
+execution's `{ statements, candidates, results }` and is `null` before
+the document has run once — an estimate mislabelled as a count is
+exactly what an honest explain may not print.
+
 ### The two residual modes
 
 - **Row residual** — only the projection is untranslated: predicates,
@@ -465,7 +557,9 @@ and this is where it says so (MODEL-FORMAT §4). Under `'rtree'` the
 names the virtual table. `rank` is `null` or the k-nearest stage —
 `{ column, dims, probe, limit, offset, margin, decides: 'engine' }` —
 what the fetch reads, the window the cut serves, the margin it keeps,
-and who decides the order (always the engine). `estimatedRows` is ABSENT on SQLite drivers — the capability slot
+and who decides the order (always the engine). `series` is `null` or
+the temporal record ("The temporal plan" above), whose `counts` are the
+last ACTUAL run's rather than an estimate. `estimatedRows` is ABSENT on SQLite drivers — the capability slot
 is empty and no number is fabricated. `residual` is `null` or
 `{ mode: 'row' | 'set' | 'knn', reasons: [{ construct, reason }] }` with
 reasons drawn from the deliberate-residual table. With
@@ -484,8 +578,9 @@ collection binding, so the planner passes that binding's **name** in:
 wrapped under any other name every reference would read as an external,
 the determinism rule would reject the fragment, and the hatch would
 silently not engage — no error and no reason in `explain()`.
-Registration is keyed by `contentKey(fragment)`
-so identical fragments share one registration, and the planner MUST
+Registration is keyed by `semanticKey(fragment)` (the
+order-insensitive identity from `@jarenjs/core/object`) so identical
+fragments share one registration, and the planner MUST
 produce a correct plan with the capability disabled (tested that way).
 Preference order: native SQL → deterministic function → residual, and
 `explain()` names the choice. **Index-form UDFs are deliberately not
@@ -498,11 +593,15 @@ UDF-expression indexes.
 ### The statement cache
 
 A caller of the core primitives, not an eighth implementation:
-`createBoundedCache` keyed by `contentKey(document)` plus collection,
-dialect and strictness. `contentKey` is the memo-grade key
-(`hashContent(stableStringify(x) ?? '')` — drops `undefined` members,
-no cycle guard; both properties acceptable for a cache key), never
-`canonicalizeJson` (signature-grade, throws on `undefined`).
+`createSemanticCache` keyed by the whole discriminating tuple — the
+document plus collection, dialect, strictness, the pushdown switch and
+the profile — where the identity is the tuple's COMPLETE
+serialization, never a fingerprint of it: a 32-bit content hash
+collides after tens of thousands of documents, and a collision here
+answers one query with another query's plan and rows (the
+cache-identity test exists to keep that key abolished).
+`store.stats()` exposes hits, misses and evictions so the cache is
+proven rather than assumed.
 `store.stats()` exposes hits, misses and evictions, so the cache is
 proven rather than assumed.
 
@@ -578,7 +677,7 @@ rollback truncates the journal buffer to its checkpoint. Overhead is
 measured and published: capture off ~6µs, journal ~17µs, session ~69µs
 per single-op commit, amortizing across a transaction.
 
-## Live queries (`src/live.js`, `src/window.js`)
+## Live queries (`src/live.js`, `src/live-time.js`, `src/window.js`)
 
 A live query classifies its document against the normative maintenance
 table by reading the COMPILED PLAN — translated filters, order terms
@@ -605,6 +704,23 @@ proven by a seeded oracle that holds the maintained result equal to a
 fresh re-query after every mutation. Incremental beats re-run 13× at
 1k rows, 51× at 10k.
 
+**Event time** (`src/live-time.js`) adds two more strategies for the
+one document shape whose answer is a function of instants rather than
+of rows: a `$resample` bucket state and a `$rolling` window state, both
+over a FIXED width. There is no clock under either — the watermark is a
+finite epoch the host supplies and `advance()` is the only way it moves
+— and both fold through `@jarenjs/core/series` itself rather than
+carrying a second aggregate, so a maintained answer cannot drift from
+what a fresh query gives. A write touches one bucket (two when it
+crosses a boundary) or the windows ending in `[t, t + width)`, and
+exactly those are recomputed. Everything a per-key state cannot place
+exactly re-runs with its member named: a calendar ladder, a named zone,
+a `locf`/`linear` fill, a `first`/`last` aggregate, a retention under
+`width + allowedLateness`. A reading behind the lateness boundary is
+never folded in silently — the view re-reads and the emission carries a
+`lateData` record — which is the one place this layer spends a full
+re-query to keep a promise rather than a number.
+
 ## Durable runs and the job queue (`src/jobs.js`, `src/dag-job.js`)
 
 The queue's whole correctness story is one guarded statement: the