@ultimat3/query 1.1.0 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLAUDE.md +393 -0
- package/README.md +179 -10
- package/package.json +7 -5
- package/src/cache.ts +180 -71
- package/src/client.ts +83 -2
- package/src/cursor-value.ts +65 -0
- package/src/deprecation.ts +81 -0
- package/src/errors.ts +97 -1
- package/src/facade.ts +2 -0
- package/src/http.ts +109 -0
- package/src/index.ts +40 -9
- package/src/input-shape.ts +74 -0
- package/src/live.ts +27 -3
- package/src/matcher.ts +36 -9
- package/src/mcp-tool.ts +11 -5
- package/src/naming.ts +5 -9
- package/src/pagination.ts +25 -3
- package/src/policy-gate.ts +13 -2
- package/src/query.ts +108 -8
- package/src/read.ts +116 -16
- package/src/shape.ts +103 -6
- package/src/source.ts +137 -49
- package/src/sql.ts +2 -2
- package/src/stable.ts +23 -12
- package/src/tags.ts +0 -17
package/CLAUDE.md
ADDED
|
@@ -0,0 +1,393 @@
|
|
|
1
|
+
# @ultimat3/query
|
|
2
|
+
|
|
3
|
+
Owns the `query` primitive: reads, live reads, cursors, the incremental matcher. Tier 3.
|
|
4
|
+
|
|
5
|
+
## Boundary
|
|
6
|
+
|
|
7
|
+
- May import: `core`, `schema` (t0), `cache`, `i18n`, `time` (t1), `entity`, `policy`, `http` (t2).
|
|
8
|
+
- Never import: `action`, `jobs`, `realtime` (sideways), or any tier 4-5 package.
|
|
9
|
+
- Reads only. A query that writes is an `action` in the wrong file.
|
|
10
|
+
|
|
11
|
+
## Files
|
|
12
|
+
|
|
13
|
+
| File | Job |
|
|
14
|
+
|---|---|
|
|
15
|
+
| `query.ts` | the primitive: `query()`, `describeQuery`, `queryHash`; the package's front door for the read path |
|
|
16
|
+
| `read.ts` | **the one read path** (`runQuery`, `sourceFor`) + the private declaration store `sql` lives in |
|
|
17
|
+
| `facade.ts` | the fluent surface — binds each projection to the query, re-implements none |
|
|
18
|
+
| `http.ts` | route projection (`GET /_x/query/<kebab>`, `enforcedBy: 'handler'`) |
|
|
19
|
+
| `mcp-tool.ts` | MCP read descriptor, same `sourceFor` |
|
|
20
|
+
| `client.ts` | typed read client (browser-safe: no server imports) |
|
|
21
|
+
| `naming.ts` | export name → `/_x/query/<kebab>`. Pure string math. **Paths only** — no tool name |
|
|
22
|
+
| `registry.ts` | export-name registration, `describeQueries()`, and the `registerPrimitiveRegistrar('query', …)` announcement |
|
|
23
|
+
| `live.ts` | `LiveQuery` descriptor + cursor arithmetic |
|
|
24
|
+
| `matcher.ts` | change event → minimal patch, or `X_MATCHER_UNSUPPORTED` |
|
|
25
|
+
| `pagination.ts` | `paginate()` over core's cursor codec — no offset, ever |
|
|
26
|
+
| `cursor-value.ts` | what a sort value becomes inside a cursor, and what it becomes again |
|
|
27
|
+
| `input-shape.ts` | what a read's `input:` may be, given that its route is a query STRING |
|
|
28
|
+
| `sql.ts` | `explain()` / `describeSql()` |
|
|
29
|
+
| `cache.ts` | the read path: the request memo, and the fill through `@ultimat3/cache`'s registered tiers |
|
|
30
|
+
| `source.ts` | `SqlSource` contract + `from()` in-memory reference |
|
|
31
|
+
| `shape.ts` | shared read vocabulary (filters, ordering, seek keys) |
|
|
32
|
+
| `policy-gate.ts` | **the only** file that touches `@ultimat3/policy` |
|
|
33
|
+
| `deprecation.ts` | `Deprecation` + the RFC 9745/8594 render + the `deprecated_calls_total` counter — TWINNED with `@ultimat3/action`'s |
|
|
34
|
+
|
|
35
|
+
## Invariants
|
|
36
|
+
|
|
37
|
+
- Every surface goes through `sourceFor`: parse input, evaluate policy, build the source.
|
|
38
|
+
Adding a second read path is the one unforgivable change here.
|
|
39
|
+
- **An explicit `ctx` is INSTALLED, never merely passed** (`As of 2026-08`). `asActor` used to hand
|
|
40
|
+
`options.ctx` to `run(ctx)` and enter no `runWithContext` unless an `actor` was also given, so
|
|
41
|
+
`guard()` decided about that actor while everything reading the AMBIENT context — above all
|
|
42
|
+
`@ultimat3/entity`'s tenant guard, which derives from `tryUseContext()` and not from the ctx it is
|
|
43
|
+
handed — saw a different identity, or none. A read was authorised as one caller and scoped to
|
|
44
|
+
nobody. Absent a `ctx` it reinstalls the ambient one, which is a no-op on every path that already
|
|
45
|
+
worked. The twin fix is `@ultimat3/action`'s `invoke`, and `read-context.test.ts` is written as an
|
|
46
|
+
equality between the three spellings of one caller — ambient, `options.actor`, `options.ctx` —
|
|
47
|
+
because three independent expectations are exactly what let this ship.
|
|
48
|
+
- **Skipping a read's policy costs a WRITTEN REASON, never a boolean** (`As of 2026-08`).
|
|
49
|
+
`SourceOptions.enforce?: boolean` is gone; it is `unenforced?: string`, and a blank one is
|
|
50
|
+
refused before the source is built. The bar is `@ultimat3/entity`'s `cross-tenant.ts`: a boolean
|
|
51
|
+
argument "reads exactly like forgetting the tenant", and a forgotten policy reads the same way —
|
|
52
|
+
so the reason IS the mechanism and every skipped policy is one `grep` away with its justification
|
|
53
|
+
attached. It is deliberately NOT capability-gated the way `crossTenant` is: `explain` runs from
|
|
54
|
+
the CLI with no actor to check a scope against, and gating it would close the one surface it
|
|
55
|
+
exists for. **`ToLiveOptions.enforce` stays a boolean**, because it is that one use with exactly
|
|
56
|
+
one reason — `toLiveQuery` translates it into the reason string spelled once as
|
|
57
|
+
`SHARED_WINDOW_REASON`, so a sync node and this file cannot disagree about why the shared window
|
|
58
|
+
has no subject. Two shipped callers, both reading no rows for a subscriber: `sql.ts` and that
|
|
59
|
+
window.
|
|
60
|
+
- The declaration never leaves `read.ts`. `defOf`/`stashDef`/`hasDef` are internal and must
|
|
61
|
+
never be re-exported from `src/index.ts` — that omission is the enforcement.
|
|
62
|
+
- A query has no `.def`. Inside the package read it with `defOf(target)`; outside, read the
|
|
63
|
+
lifted `.input`/`.policy`/`.cache`/`.mcp`/`.isLive` or `describe()`.
|
|
64
|
+
- App code reaches a projection through the query (`liveFeed.tool()`), never through `.def`
|
|
65
|
+
and never by importing the projection function. `facade.ts` is where a new method is bound;
|
|
66
|
+
the projection itself keeps living in its own file.
|
|
67
|
+
- `src/index.ts` re-exports `t` from `@ultimat3/schema` **verbatim**, so a query file imports
|
|
68
|
+
one package. Never wrap, spread or re-declare it: `t` delegates to `schemaProvider()` on every
|
|
69
|
+
access, and a copy would freeze the provider at import time. `index.test.ts` asserts identity.
|
|
70
|
+
- **`LiveQuery` describes the read *and* runs it.** `execute()` is the source the shape, the reads
|
|
71
|
+
and `sqlText` were taken from — one build of one `(query, input)`. A caller that wanted rows and
|
|
72
|
+
called `sourceFor` itself was a second build: twice the parse, twice the `sql()`, and a matcher
|
|
73
|
+
describing a source the rows never came from. `@ultimat3/realtime`'s shared window is the one
|
|
74
|
+
consumer, and it reads through `execute()`. It never memoises — a subscriber joining an existing
|
|
75
|
+
subscription must see the rows as they are now, not the window someone else opened.
|
|
76
|
+
- `isLive` is the declared boolean, `live()` is the subscription. Never name one after the other.
|
|
77
|
+
`QueryDescriptor.live` keeps its name — `@ultimat3/manifest` and `@ultimat3/admin` read it.
|
|
78
|
+
- `mcp` is opt-in (`expose: true`), exactly as it is for an action: rows reach an agent only when
|
|
79
|
+
the author said so. `isExposed` here delegates to `isMcpExposed` in `@ultimat3/core` — the one
|
|
80
|
+
predicate every reader in the framework asks — rather than spelling `=== true` a second time.
|
|
81
|
+
- **A read has ONE tool name, and it is the export name verbatim** (`As of 2026-08`). `toQueryTool`
|
|
82
|
+
snake_cased it (`liveFeed` → `live_feed`) while `@ultimat3/mcp` serves the read under
|
|
83
|
+
`queryName(target)` and answers `tools/call` for nothing else — so anything that read the name off
|
|
84
|
+
the descriptor rather than off `tools/list` called a tool the server had never heard of, and the
|
|
85
|
+
scope map in `defineAppMcp` is keyed on the verbatim name too. `toToolName` is **deleted**, not
|
|
86
|
+
merely unused: an exported derivation is a second way to spell one tool. Two pins, both in this
|
|
87
|
+
package because this is where the rule can be broken — `mcp-tool.test.ts` asserts
|
|
88
|
+
`toQueryTool(q).name === queryName(q)` (the presence), and `index.test.ts` asserts the barrel
|
|
89
|
+
exports no key matching `/tool_?name/i` (the absence, which nothing else catches: the only other
|
|
90
|
+
guard is `packages/mcp/src/cross-surface.test.ts`, tier 4 and unimportable from here).
|
|
91
|
+
`naming.ts` derives PATHS only.
|
|
92
|
+
- `client.ts` stays free of server imports — it is bundled into the browser. `@ultimat3/action`
|
|
93
|
+
is the same tier, so its naming is ported here, never imported.
|
|
94
|
+
- **`queryClient` is the map-wide read client and the mirror of `rpc`; both spellings run
|
|
95
|
+
`queryClientMethodFor`.** `queryClient<Api['queries']>({ baseUrl })` is how a surface that must
|
|
96
|
+
not import a feature reaches every registered read — `site/` in an app, whose one edge into
|
|
97
|
+
`app/` is a boundary violation, so `.client()` (which needs the query object) is unreachable
|
|
98
|
+
there. The map-wide client re-deriving a path from the property name would be the second URL
|
|
99
|
+
derivation this package spent a release removing; it proxies to the per-query method instead.
|
|
100
|
+
- **`toQueryRoute` is the other half of `client()`, and the two derive the same URL from the same
|
|
101
|
+
`naming.ts`.** The client shipped fetching `/_x/query/<kebab>` while nothing built a route for
|
|
102
|
+
it, so every typed read compiled and 404'd; a projection whose only consumer is a URL string is
|
|
103
|
+
the failure this pairing exists to prevent. Named for the primitive rather than `toRoute`,
|
|
104
|
+
because a host mounts it beside `@ultimat3/action`'s — the same reason the tool projection here
|
|
105
|
+
is `toQueryTool`.
|
|
106
|
+
- **The route coerces, `runQuery` validates, and only the first belongs to the wire.** A search
|
|
107
|
+
string is characters, so the boundary decodes it with `@ultimat3/schema`'s `coerceQuery` — the
|
|
108
|
+
one HTTP-boundary decoder, which never invents data and hands on what it cannot convert.
|
|
109
|
+
Validating there as well (`request.query(schema)`) would be the second parser: the same read
|
|
110
|
+
would answer `X_BODY_INVALID` where every other surface answers `X_INPUT_INVALID` and prints
|
|
111
|
+
its schema. For the same reason `meta.input` stays **absent** — the pipeline's body stage
|
|
112
|
+
validates it against a body, and a GET has none, so declaring it fails every read on nothing.
|
|
113
|
+
- **`rateLimit:` is declarable on a read, and `toQueryRoute` sets the NAME and the NUMBERS.**
|
|
114
|
+
`QueryDef` had neither until 2026-08, so every `GET /_x/query/*` fell through `bucketFor` to
|
|
115
|
+
`default` — 120 burst, 2/s per actor — and one authenticated caller could hold 120 cross-tenant
|
|
116
|
+
aggregates in flight and then 2/s indefinitely, from a single account, with no declaration able
|
|
117
|
+
to say otherwise. The conversion is `toBucket` from **`@ultimat3/http`**, never a copy here:
|
|
118
|
+
http owns `Bucket` and the maths, `@ultimat3/action` is the same tier as this package, and a
|
|
119
|
+
copy in either is a second answer for the other. The field is lifted onto the facade so
|
|
120
|
+
`toQueryRoute` reads it without `defOf`, exactly as `cache` and `mcp` are.
|
|
121
|
+
- **`deprecated:` is a compat WINDOW; versioning is two deployments behind one ingress.** Headers
|
|
122
|
+
are rendered ONCE at projection, so an unparseable date is `X_QUERY_DEPRECATION_INVALID` at
|
|
123
|
+
mount and not on the first read. `deprecation.ts` is a TWIN of `@ultimat3/action`'s — both are
|
|
124
|
+
tier 3, so neither may import the other, and the shared home is `@ultimat3/http` if it grows
|
|
125
|
+
one. The same compromise `naming.ts` is ported under; keep the two files identical in behaviour.
|
|
126
|
+
- **The span wraps the whole read, not `source.execute()`.** Parse, policy and `sql()`'s own
|
|
127
|
+
construction were outside every span, so a read whose cost was in building the source reported
|
|
128
|
+
milliseconds under a parent that reported seconds — a gap with no name. `readRows` holds the
|
|
129
|
+
span and `readRowsIn` is the body; attributes are bounded (surface, actor KIND, `live`,
|
|
130
|
+
`cached`, `fresh`) plus the row count, and never the input or an actor id — a read is keyed per
|
|
131
|
+
tenant and per cursor, so either would be unbounded. `telemetry.test.ts` asserts the extent
|
|
132
|
+
structurally through `currentSpan()`, because the test clock is frozen. `sourceFor` still has no
|
|
133
|
+
span of its own: adding one would double-span every read that goes through `readRows`.
|
|
134
|
+
- **`policyCapability` is a display label and `policyPermissions` is what a report matches on.**
|
|
135
|
+
A composite renders as `or(feed:read, org:administer)`, which equals no permission string, so
|
|
136
|
+
`x policy list` matching on `capability` reported every non-trivially-guarded read's permissions
|
|
137
|
+
as unenforced. `QueryDescriptor.permissions` is the flattened list, published beside
|
|
138
|
+
`capability` and never instead of it.
|
|
139
|
+
- **`client.ts` injects `traceparent`**, before the caller's own headers so an explicit one wins,
|
|
140
|
+
and sends nothing when the span context is incomplete — `00-<trace>--01` is a header every
|
|
141
|
+
collector drops. The twin of `@ultimat3/action`'s, ported for the same tier reason.
|
|
142
|
+
- **A read is `no-store`, and its policy is `enforcedBy: 'handler'`.** The URL names no actor
|
|
143
|
+
while the answer is scoped to one, so `public` would hand one reader's rows to the next caller
|
|
144
|
+
of that URL; and `runQuery` is the read's one evaluation, deciding from the parsed input, so an
|
|
145
|
+
authz stage deciding first would be a second authz system holding raw strings — and would
|
|
146
|
+
demand an `authorize` hook to decide at all. `http.test.ts` drives both over the real pipeline
|
|
147
|
+
with no hook wired and counts the evaluations: exactly one.
|
|
148
|
+
- `registry.ts` announces `registerQueries` in core's registrar table at import. That is how
|
|
149
|
+
`defineApi({ queries })` in `@ultimat3/action` registers a read without importing this package
|
|
150
|
+
sideways. Never remove the announcement: `defineApi` would then throw `X_REGISTRAR_MISSING`.
|
|
151
|
+
- Policy runs per subscriber for live queries. Never cache a decision across actors.
|
|
152
|
+
- The matcher patches from `QueryShape`, never from SQL text.
|
|
153
|
+
- `paginate` has no `offset` parameter and must never grow one, and it is reachable **only** as
|
|
154
|
+
`query.page(input, { first, after })` — a page is the read's own answer, not an imported helper.
|
|
155
|
+
`src/index.ts` exports `Page` and `PaginateArgs` and not the function: re-exporting it would be
|
|
156
|
+
a second way to ask for the thing `.page()` already does.
|
|
157
|
+
- **A cursor is a position, not a row.** `isAfterKey` in `source.ts` is the one definition of
|
|
158
|
+
"after this position": `Builder.seek()` compiles it to SQL and `paginate()` applies it when a
|
|
159
|
+
source cannot push the seek down. The fallback used to find the cursor's row by id and slice
|
|
160
|
+
after it — which restarts the listing from the top the moment that row is deleted, the exact
|
|
161
|
+
failure keyset pagination exists to prevent. Never reintroduce a row lookup here.
|
|
162
|
+
- The seek predicate is spelled out per key (`(a < $1) or (a = $2 and id > $3)`), the way
|
|
163
|
+
`@ultimat3/entity`'s `seekSql` spells it. A row-value comparison cannot express a mixed
|
|
164
|
+
`createdAt desc, id asc` ordering, and the id-tiebreak-only fallback it replaced returned rows
|
|
165
|
+
the ordering was already past — with `execute()` disagreeing with the SQL it printed.
|
|
166
|
+
- **NULL has one meaning, and `isNull` is it.** `null` and a column the row omits are the same
|
|
167
|
+
absence, in the SQL and in memory alike. `=`/`!=`/`in` read NULL as a **value** — `is null`,
|
|
168
|
+
`is not null`, `is distinct from`, `in (…) or is null`, the pair `@ultimat3/entity`'s
|
|
169
|
+
`predicateSql` emits; `>`/`>=`/`<`/`<=` read it as **unknown**, matching nothing on either side,
|
|
170
|
+
which is why they need no special emission; `order by` reads it as the **largest value**, spelled
|
|
171
|
+
`asc nulls last` / `desc nulls first` rather than inherited from the driver. `= $n` with a NULL
|
|
172
|
+
argument is never true in Postgres, so `where({ deletedAt: null })` matched every row in memory
|
|
173
|
+
and none in the database, and `"col" > $n` blanked page two at the first NULL. Never emit a bound
|
|
174
|
+
parameter where NULL is the value being tested, and never let `compareValues` sort a NULL as the
|
|
175
|
+
string `"null"` again — `compareRows`, `isAfterKey` and the matcher's insertion position all
|
|
176
|
+
read it, so one string compare moves rows on three surfaces. `in` takes a list or nothing: an
|
|
177
|
+
empty one is `1 = 0`, and so is an operand that is not an array at all — `matchesFilter` answers
|
|
178
|
+
no rows for it, and `"col" in $n` is a syntax error the driver reports instead of that answer.
|
|
179
|
+
- **The id is the tiebreak that makes the order total.** A row without one is
|
|
180
|
+
`X_QUERY_NOT_PAGEABLE` at `seekKeyOf` **and** at the matcher's `idOf`, never `String(undefined)`:
|
|
181
|
+
`"undefined"` is a position every row matches, signed and opaque, so page two would be page one
|
|
182
|
+
forever and one row's patch would land on another's index.
|
|
183
|
+
- **`totalOrder` is the order a read is served in, and all three readers use it.** The declared
|
|
184
|
+
keys then `id asc`, unless the ordering already names `id` — `Builder.servedOrder()` compiles it,
|
|
185
|
+
the in-memory sort applies it, and `positionFor` places a row by it. The matcher comparing
|
|
186
|
+
`shape.orderBy` alone appended a tied row after its whole tie group, which is a position no
|
|
187
|
+
re-read returns and a cursor that skips every tie it was pushed past. `SeekKey` is the same list
|
|
188
|
+
decomposed — `key` for the declared part, `id` for the tiebreak — so never add `id` to
|
|
189
|
+
`QueryShape.orderBy` to get it: `seekKeyOf` would then sign the id twice. An unordered query
|
|
190
|
+
appends, because SQL promises no position there to get wrong.
|
|
191
|
+
- **A live read asks for that order explicitly, and `sourceFor` is where it asks.** `total()` is
|
|
192
|
+
the `SqlSource` method for "the same read, served in `totalOrder`" — no cursor, no window — and
|
|
193
|
+
`buildSource` calls it when `surface === 'live'`, for nothing else, and only when the source
|
|
194
|
+
implements it. A live window served by the declared keys alone puts a tied row wherever the
|
|
195
|
+
database returned it, while `positionFor` places the patch by id and the resume re-read seeks by
|
|
196
|
+
id: the client then renders an order no re-read answers. Never reach for `seek(null, limit)`
|
|
197
|
+
instead — a live query need not carry a limit, and inventing one is a window nobody asked for.
|
|
198
|
+
- **A sort value carries its own TYPE through the cursor** (`cursor-value.ts`, `As of 2026-08`).
|
|
199
|
+
The codec is JSON, so `paginate()` putting raw column values in meant a `Date` went out and an
|
|
200
|
+
ISO STRING came back: `isAfterKey` compared `"1769904000000"` against `"2026-02-01T…"` through
|
|
201
|
+
`compareValues`' string branch and **page two came back empty** — and a `bigint` sort key was a
|
|
202
|
+
bare `TypeError` out of `JSON.stringify`, with no code and no fix. `@ultimat3/entity`'s
|
|
203
|
+
`cursor.ts` solves the same problem by reading the column's declared kind; a `query` has no
|
|
204
|
+
column kinds — `QueryShape.orderBy` is a name and a direction — so the value is TAGGED instead
|
|
205
|
+
(`{ $x: 'date' | 'bigint', v }`) and `reviveSortKey` is total without knowing which read minted
|
|
206
|
+
it. `undefined` encodes as `null`, because SQL has one absence and dropping the key would shift
|
|
207
|
+
every later one a position left. Anything JSON cannot carry and this cannot tag — an object, an
|
|
208
|
+
array, `NaN`, `±Infinity` — is `X_CURSOR_VALUE_UNSUPPORTED` where the cursor is MINTED, never
|
|
209
|
+
`X_CURSOR_INVALID`: the mistake is the read's own `orderBy` and no retry repairs it.
|
|
210
|
+
- **`compareValues` orders numbers and bigints in ONE order, because Postgres does.** The numeric
|
|
211
|
+
fast path was `typeof === 'number'` on both sides, so an `int8` fell to
|
|
212
|
+
`String(left) < String(right)`: `compareValues(9n, 10n)` answered `1` and a sort came out
|
|
213
|
+
`["10", "100", "9"]`. `bigint` is the physical type of every `<p>_minor` column and
|
|
214
|
+
`@ultimat3/entity`'s `count-by.ts` lists it as groupable, so the in-memory source, the live
|
|
215
|
+
matcher and the seek fallback ALL disagreed with the database on any bigint-ordered read.
|
|
216
|
+
`shape-order.test.ts` is the pin: one case per `ColumnKind`, each asserting the order Postgres
|
|
217
|
+
returns, plus the NULL rule and the cursor round trip. The kind list is spelled out rather than
|
|
218
|
+
imported — `tsconfig.json` excludes `*.test.ts`, so a `satisfies Record<ColumnKind, …>` written
|
|
219
|
+
in a test is a type assertion `tsc` never reads; a runtime count is the enforceable half.
|
|
220
|
+
- **A read's `input:` must survive a query STRING, and `query()` refuses one that cannot**
|
|
221
|
+
(`input-shape.ts`, `X_QUERY_INPUT_UNENCODABLE`, `As of 2026-08`). `client.ts` encoded a nested
|
|
222
|
+
member as `JSON.stringify(item)` and skipped a `null`, while `coerceQuery` has no inverse for
|
|
223
|
+
either — `case 'object'` hands the raw value back untouched — so the typed client type-checked
|
|
224
|
+
calls the server's own route then rejected, which is the exact failure `client.ts`'s header
|
|
225
|
+
claims to prevent. **The declaration is the fix, not the encoder**: teaching `coerceQuery` to
|
|
226
|
+
`JSON.parse` a string would make the ONE HTTP-boundary decoder invent structure for every
|
|
227
|
+
surface that shares it — forms and route params included — against that file's own rule that it
|
|
228
|
+
never invents data, and a `null` sentinel would be a reserved string colliding with the value
|
|
229
|
+
`"null"`. Refused: a structural member (`object`, `record`, `money`, or an array/union of one),
|
|
230
|
+
a REQUIRED nullable member, and a top-level input that is not an object. A schema
|
|
231
|
+
`tryIntrospect` cannot read is left alone, or `configureSchemaProvider` would be unusable.
|
|
232
|
+
- **A move OUT of a full window is a `refill`, never an `add`** (`matcher.ts`). `insert()` places a
|
|
233
|
+
moved row among the `limit - 1` rows the client still holds, so its position can never reach
|
|
234
|
+
`shape.limit` and the `position >= shape.limit` bail is unreachable on that path — the row was
|
|
235
|
+
re-inserted INSIDE the window. Proven with `limit: 3`, window `[a:1, b:2, c:3]` and a server also
|
|
236
|
+
holding `d:4, e:5`: moving `a` to `99` rendered `[b, c, a:99]` where the true window is
|
|
237
|
+
`[b, c, d]`. Only the server can answer the tail, and whether the moved row is still in the
|
|
238
|
+
window is its answer too.
|
|
239
|
+
- **A page is bounded whether or not the caller bounded it.** `paginate` asserts `first` is a whole
|
|
240
|
+
number of rows in `1…MAX_PAGE_SIZE` (10,000) before anything else — `args.first + 1` bound
|
|
241
|
+
whatever an action's input or a route parameter carried, so one request could ask for five
|
|
242
|
+
million rows. The constant is a TWIN of `@ultimat3/entity`'s, under the same tier compromise
|
|
243
|
+
`naming.ts` and `deprecation.ts` are ported under.
|
|
244
|
+
- **`tagKeys` is `@ultimat3/cache`'s, not this package's — moved 2026-08.** `src/tags.ts` here and
|
|
245
|
+
`@ultimat3/action`'s were byte-identical, and both packages are tier 3, so neither can import the
|
|
246
|
+
other and a copy in either is a second answer for the other — the same move `toBucket` made into
|
|
247
|
+
`@ultimat3/http`. `tagKey` went with it: `serializeTag` under a second name, zero call sites.
|
|
248
|
+
`@ultimat3/render` exports a *different* function under the same name (declaration order kept);
|
|
249
|
+
never import that one here.
|
|
250
|
+
- **A fingerprint is an identity, so two different inputs may not share one** (`stable.ts`).
|
|
251
|
+
`NaN`, `±Infinity` and JSON `null` all encoded as `'null'`, and `String(-0)` is `"0"` — so four
|
|
252
|
+
distinct inputs shared one read-cache entry and one cursor scope. They are bare tokens now
|
|
253
|
+
(`NaN`, `Infinity`, `-Infinity`, `-0`), which the `string` branch cannot spell because it always
|
|
254
|
+
quotes. Ordinary numbers are byte-identical, so no existing cursor scope moved.
|
|
255
|
+
- The cursor codec is `@ultimat3/core`'s (`encodeCursor` / `decodeCursor` / `configureCursorSigning`).
|
|
256
|
+
This package supplies only the scope a cursor is bound to — `queryHash(name, input)` — and never
|
|
257
|
+
signs, encodes or parses one itself. An unverified or foreign cursor is `X_CURSOR_INVALID`, thrown
|
|
258
|
+
by core's `CursorInvalidError`, which `errors.ts` re-exports so the name stays on this surface.
|
|
259
|
+
- **The request memo holds the read, not the rows.** `readOnce` publishes the in-flight promise
|
|
260
|
+
before its first await, so two reads of one key in one request are one execution and one tier
|
|
261
|
+
round trip whether the second follows the first or races it. A value-keyed map could not express
|
|
262
|
+
that, and could not tell a memoized `undefined` from a miss either. A rejection is evicted — a
|
|
263
|
+
failed read is not the request's answer, and the next read retries. `requestMemo(ctx)` is
|
|
264
|
+
therefore `Map<string, Promise<unknown>>`; never put a settled value in it.
|
|
265
|
+
- **Every read is memoized; only a `cache:` read goes through the tier.** `readThrough` is
|
|
266
|
+
`readOnce` plus the fill through the ladder, and `readRows` picks between them on `def.cache`
|
|
267
|
+
alone. The memo is not what `cache:` buys — a list that renders one uncached lookup per row pays for
|
|
268
|
+
every row otherwise, which is the N+1 this collapses. Never gate `readOnce` on `def.cache`
|
|
269
|
+
again, and never let a second key function grow beside `cacheKeyFor`.
|
|
270
|
+
- **A cache key carries the read's AUTHORITY, and `cache.scope` is what widens it** (`As of
|
|
271
|
+
2026-08`). `cacheKeyFor` held the name, the input and the tags — nothing about who asked — while
|
|
272
|
+
`sql(input, ctx)` is handed the context and `@ultimat3/entity` derives every tenant predicate
|
|
273
|
+
from `ctx.actor.orgId`. The tier is process-wide, so the first actor to ask filled the entry and
|
|
274
|
+
the next was served it: a query filtering on `ctx.actor.orgId` returned `org-a`'s row to an
|
|
275
|
+
`org-b` actor. `readAuthority(actor, scope)` is the ONE producer of the component and
|
|
276
|
+
`cacheKeyFor`'s fourth argument is **required and positional**, because an optional one is one a
|
|
277
|
+
call site forgets and a forgotten one is that read. `scope` defaults to `'actor'` and the default
|
|
278
|
+
is the mechanism: declaring nothing gets the narrowest key. `'tenant'` and `'global'` are written
|
|
279
|
+
statements about the rows — the `unenforced:` shape one field over — and `'tenant'` with no
|
|
280
|
+
`orgId` narrows to the actor rather than widening to everyone, because nothing here can prove two
|
|
281
|
+
org-less callers share a tenant. The authority is JSON, never a joined string, for the reason
|
|
282
|
+
`@ultimat3/entity`'s `scopeKey` gives: an actor id is app data and may carry the separator.
|
|
283
|
+
- **`cache.ttlMs` is judged at `query()`, not on the first read.** Every `CacheTier` refuses a
|
|
284
|
+
lease that is not positive and finite (`assertTtl`), and the read tier's one catch absorbs
|
|
285
|
+
`X_CACHE_TOO_LARGE` only — so `ttlMs: Infinity` turned a typo into a permanently failing business
|
|
286
|
+
read whose cause named a cache key. `X_QUERY_CACHE_TTL_INVALID`, on the line that wrote it. It
|
|
287
|
+
restates `assertTtl`'s bar as a refusal and never as a second resolution.
|
|
288
|
+
- **`fingerprint` is SHA-256/16, never a 32-bit hash** (`stable.ts`, `As of 2026-08`). It is a
|
|
289
|
+
SHARING key over client-chosen input — which read-cache entry two callers are served from, which
|
|
290
|
+
scope a cursor is bound to — so FNV-1a/32's 4×10⁹ values are a collision found offline in
|
|
291
|
+
seconds. Same primitive and width as `@ultimat3/realtime`'s `stableDigest`. `stableStringify` did
|
|
292
|
+
not move, so the only cost is one cold cache and every open cursor answering `X_CURSOR_INVALID`
|
|
293
|
+
with its own "request the first page again" fix.
|
|
294
|
+
- **A fill is FENCED, and the fence is `@ultimat3/cache`'s** (`As of 2026-08`). `run()` answers with
|
|
295
|
+
rows it read in the past: a mutator committing in between busts a key not yet in the tier, so the
|
|
296
|
+
drop is a no-op reporting `errors: []`, and the fill then publishes the pre-write rows for the
|
|
297
|
+
full TTL — invisible to every reader until it expires. The sample happens inside
|
|
298
|
+
`createCacheStack.read`, immediately before `load()`, and is re-asked per rung before each write.
|
|
299
|
+
This package no longer samples one of its own — that copy went with the private store. The caller
|
|
300
|
+
is answered either way: those rows ARE its answer, and only publishing is refused.
|
|
301
|
+
`cache-fence.test.ts` is the proof the property survived the move.
|
|
302
|
+
- **This package owns NO cache store, and that is the enforcement** (`As of 2026-08`). A `cache:`
|
|
303
|
+
read fills `createCacheStack(registeredTiers(), { clock })` — the tiers `@ultimat3/cache` has
|
|
304
|
+
registered — and there is nothing here to install, swap or wire. There used to be: a private
|
|
305
|
+
`ReadCache` seam (`setReadCache`/`getReadCache`/`MemoryReadCache`) that `invalidateTags` could not
|
|
306
|
+
reach, because that fan-out walks the registered `CacheTier`s and nothing else. The gap was closed
|
|
307
|
+
by `packages/cli/src/dev-cache.ts` installing the read cache **over** an object it also
|
|
308
|
+
registered — a correctness property held by a wiring line in a CLI file, one edit from being
|
|
309
|
+
wrong, and carrying a second `ReadCache` implementation (`tierReadCache`) that dated entries with
|
|
310
|
+
`Date.now()`. And `invalidateQueryTags` was a second fan-out path, which
|
|
311
|
+
`packages/cache/CLAUDE.md` forbids in as many words. One registry, one fan-out, no seam. **Never
|
|
312
|
+
reintroduce a store here**, and never call `tier.invalidateTags()` from this package.
|
|
313
|
+
- **A tier refusal degrades the cache, never the read** — and so does every other tier concern.
|
|
314
|
+
`bestEffort`, the fence, the single flight, the promotion and the TTL are all `createCacheStack`'s,
|
|
315
|
+
which is what "no store here" buys: a refused `get` reads as a miss, a refused `set` as "that tier
|
|
316
|
+
is unchanged", and the failure lands in `recentTierFailures()` under the name of the tier that
|
|
317
|
+
actually refused rather than under a `'query-read'` label for a rung in no registry. Never wrap a
|
|
318
|
+
tier call here in a private try/catch, and never sample a second fence.
|
|
319
|
+
- **The read path reads NO clock, and that is what makes it drivable.** It hands the stack a
|
|
320
|
+
RELATIVE `ttlMs`; the tier's own clock turns it into an absolute expiry. `fill` used to compute
|
|
321
|
+
`nowMs(clock) + ttlMs` and `tierReadCache` used to compute `expiresAt - Date.now()`, so the two
|
|
322
|
+
`ReadCache` implementations disagreed about "now" and no frozen clock could drive the
|
|
323
|
+
Redis-backed one. `read-tier.test.ts` pins it: a `createLruTier({ clock, jitterFraction: 0 })` and
|
|
324
|
+
a `ctx.clock` frozen at the same instant produce an expiry a test can assert exactly.
|
|
325
|
+
- **A `cache:` read always expires, and the bound is the tier's.**
|
|
326
|
+
`def.cache.ttlMs ?? DEFAULT_READ_CACHE_TTL_MS` (60s) is what `readRows` passes — a query keyed on
|
|
327
|
+
`{ orgId, cursor }` has as many distinct keys as the deployment has tenants, and unbounded that is
|
|
328
|
+
one permanent entry per page per org. `null` is "the caller named none", never "never": it reaches
|
|
329
|
+
the stack as an OMITTED `ttlMs`, which is how a tier is asked for its own default. Every tier
|
|
330
|
+
refuses a non-positive lease outright, so there is no immortal entry to spell.
|
|
331
|
+
- **A process that registered no tier reads uncached.** `createCacheStack([])` loads, answers and
|
|
332
|
+
writes nowhere — correct, and slower. That is the trade for deleting the module-default store: a
|
|
333
|
+
script, a worker boot or a test that wants caching calls `registerTier`, the same call every
|
|
334
|
+
other cached surface in the framework already uses.
|
|
335
|
+
- **The memo holds an execution, never a decision.** `readRows` runs `buildSource` — parse, guard,
|
|
336
|
+
`sql()` — *before* it reaches the memo, on every call, and `.as()` reads in a child context whose
|
|
337
|
+
identity is its own memo. So a memoized answer is still one this actor was allowed to ask for,
|
|
338
|
+
and no impersonated read can join a read made as someone else. Moving the memo above
|
|
339
|
+
`buildSource` would turn it into an authz bypass.
|
|
340
|
+
- **`fresh: true` skips the memo on the way in and publishes to it on the way out.** A memo is a
|
|
341
|
+
cache whose lifetime is the request, so `fresh` refuses to *join* an entry — `readFresh` is
|
|
342
|
+
`readOnce` minus the join, both sharing one `publish` — but it must still *become* one. Returning
|
|
343
|
+
the rows early instead left the pre-write entry standing, so "the one way to read past a write
|
|
344
|
+
made earlier in the same request" ended at the single call that asked for it and the next plain
|
|
345
|
+
read of that key got the stale answer back. Invalidation still drops tier entries only.
|
|
346
|
+
- **A read can declare a `rateLimit:`, and `toQueryRoute` enforces it — added 2026-08.** `QueryDef`
|
|
347
|
+
had no such field and the route set no bucket, so **every** `GET /_x/query/*` fell through
|
|
348
|
+
`bucketFor` to `default` — 120 burst, 2/s per actor. One authenticated caller could hold 120
|
|
349
|
+
cross-tenant aggregates in flight and then 2/s indefinitely, from a single account, and the
|
|
350
|
+
declaration that would have throttled it did not exist in the type. `meta.rateLimit` (the name)
|
|
351
|
+
**and** `meta.rateLimitBucket` (the numbers) are both set, because a name nothing registers is
|
|
352
|
+
the same silent fall-through. The conversion is `toBucket` from **`@ultimat3/http`** — http owns
|
|
353
|
+
`Bucket` and the maths, and `@ultimat3/action` is this tier, so a copy here would be a second
|
|
354
|
+
answer for the write half. Never derive a bucket locally.
|
|
355
|
+
- **`deprecated:` is a compat WINDOW; versioning is not here and will not be.** `Deprecation`
|
|
356
|
+
(RFC 9745, `@<unix seconds>`) and `Sunset` (RFC 8594, IMF-fixdate) on every answer including the
|
|
357
|
+
failures, a `rel="successor-version"` link built through `derivePath` — the same derivation
|
|
358
|
+
`client()` uses, never a second one — the dates on the descriptor, and
|
|
359
|
+
`deprecated_calls_total{primitive,name}`, which is the only way to answer "is anyone still
|
|
360
|
+
reading it?" before deleting the read. Rendered ONCE at projection, so a date that cannot become
|
|
361
|
+
a header is `X_QUERY_DEPRECATION_INVALID` at mount rather than on the first read. Running two
|
|
362
|
+
versions side by side is two deployments behind one ingress (axiom 7). `deprecation.ts` is a
|
|
363
|
+
twin of `@ultimat3/action`'s: both are tier 3, the shared home is `@ultimat3/http` if it ever
|
|
364
|
+
grows one, and this is the same compromise `naming.ts` is ported under.
|
|
365
|
+
- **The span wraps the whole read, not `source.execute()`.** Wrapping the execution alone left the
|
|
366
|
+
input parse, the policy evaluation and `sql()`'s own construction outside every span, so a read
|
|
367
|
+
whose cost was in building the source reported milliseconds under a parent reporting seconds —
|
|
368
|
+
a gap with no name, which reads as framework overhead. Attributes are bounded: surface, actor
|
|
369
|
+
KIND, `live`, `cached`, `fresh`, and the row count. Never the input and never an actor id — a
|
|
370
|
+
read is keyed per tenant and per cursor, so either would be unbounded. `telemetry.test.ts`
|
|
371
|
+
asserts the EXTENT structurally, reading `currentSpan()` from inside the policy predicate and
|
|
372
|
+
`sql:`, because the test clock is frozen and a timing assertion would hang on it.
|
|
373
|
+
- **`policyCapability` is a display label; `policyPermissions` is what a report matches on.** A
|
|
374
|
+
composite renders as `or(feed:read, org:administer)`, which equals no permission string, so
|
|
375
|
+
`x policy list` matching on `capability` reported every non-trivially-guarded read's permissions
|
|
376
|
+
as *unenforced*. `QueryDescriptor.permissions` is the flattened list from `@ultimat3/policy`,
|
|
377
|
+
published beside `capability` and never instead of it.
|
|
378
|
+
- **`queryClient`/`client()` inject `traceparent`.** Core's `traceparent()` had no caller in the
|
|
379
|
+
repo, so a service-to-service read began a fresh root trace on the far side. Set BEFORE the
|
|
380
|
+
caller's own headers so an explicit one wins; an incomplete span context (`spanId: ''`) sends
|
|
381
|
+
nothing rather than a header every collector drops. In a browser there is no ambient context, so
|
|
382
|
+
a cross-origin read gains no CORS preflight it did not already have. The helper is twinned in
|
|
383
|
+
`@ultimat3/action`'s client for the same tier reason `naming.ts` is.
|
|
384
|
+
- Authz goes through `enforce(surface, policy, { input, actor, ctx })` from
|
|
385
|
+
`@ultimat3/policy`; a live denial keeps its 4403 close code on `QueryDeniedError.denial`.
|
|
386
|
+
`policy-gate.ts` is the only file that imports the policy package.
|
|
387
|
+
|
|
388
|
+
## Commands
|
|
389
|
+
|
|
390
|
+
```
|
|
391
|
+
bun test packages/query
|
|
392
|
+
bun run typecheck
|
|
393
|
+
```
|
package/README.md
CHANGED
|
@@ -29,15 +29,51 @@ Every projection is a method on the query itself. A query has no `.def`.
|
|
|
29
29
|
| `liveFeed.as(actor, { orgId })` | the same read as another actor — the surrounding context is untouched, `null` is signed out |
|
|
30
30
|
| `liveFeed.page({ orgId }, { first: 20, after })` | one bounded page plus the signed cursor that continues it. There is no `offset` |
|
|
31
31
|
| `liveFeed.live({ orgId })` | the `LiveQuery` `@ultimat3/realtime` subscribes to, carrying the same policy object |
|
|
32
|
-
| `liveFeed.tool()` | the MCP read tool
|
|
32
|
+
| `liveFeed.tool()` | the MCP read tool, named `liveFeed`. `tool().policy === liveFeed.policy`, and it reads fresh |
|
|
33
33
|
| `liveFeed.client({ baseUrl })` | `GET /_x/query/live-feed?orgId=…`, typed both ways |
|
|
34
34
|
| `liveFeed.describe()` | the manifest row |
|
|
35
35
|
|
|
36
|
+
The route on the other end of that client is `toQueryRoute(liveFeed)`, and — `As of 2026-08` —
|
|
37
|
+
`x dev` and a container both mount it for every registered read, the framework's job, not the
|
|
38
|
+
app's. The search string is coerced at the wire and validated by the read's own schema, so a bad
|
|
39
|
+
`orgId` is the query's `X_INPUT_INVALID` and a 400; the answer is `no-store`, because the URL
|
|
40
|
+
names no actor while the rows are scoped to one.
|
|
41
|
+
|
|
42
|
+
`queryClient` is the same method for **every** registered read at once — the read half of
|
|
43
|
+
`@ultimat3/action`'s `rpc`, and the one spelling available to a surface that may not import a
|
|
44
|
+
feature:
|
|
45
|
+
|
|
46
|
+
```ts
|
|
47
|
+
import { queryClient } from '@ultimat3/query';
|
|
48
|
+
import type { Api } from '../api'; // a TYPE, so no module-graph edge
|
|
49
|
+
|
|
50
|
+
export const queries = queryClient<Api['queries']>({ baseUrl });
|
|
51
|
+
const [post] = await queries.publicPost({ slug }); // typed input, typed rows
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
Both spellings run `queryClientMethodFor`, so a read has one URL however it is addressed.
|
|
55
|
+
|
|
36
56
|
The declaration is lifted too: `.input`, `.policy`, `.cache`, `.mcp`, `.isLive`. `sql` is not
|
|
37
57
|
among them — it lives in a private store inside `read.ts`, so `sourceFor` is the only thing
|
|
38
58
|
that can build a source and there is nowhere for a second authz path to hide. Something that
|
|
39
59
|
merely looks like a query (`kind: 'query'`, no declaration) is `X_QUERY_FOREIGN`.
|
|
40
60
|
|
|
61
|
+
### Skipping the policy costs a written reason
|
|
62
|
+
|
|
63
|
+
Two reads have no subscriber to decide about: developer tooling that returns the statement and no
|
|
64
|
+
rows (`explain`, `describeSql`), and the shared, subject-less window a sync node builds once per
|
|
65
|
+
`(query, input)`. Both say so, in words:
|
|
66
|
+
|
|
67
|
+
```ts
|
|
68
|
+
const source = await sourceFor(target, input, {
|
|
69
|
+
unenforced: 'a scaffolded test asserts the SQL text; the policy is asserted separately',
|
|
70
|
+
});
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
A blank reason is refused before the source is built. It is a string and not a boolean for the
|
|
74
|
+
reason `@ultimat3/entity`'s `crossTenant(reason, fn)` is: `enforce: false` reads exactly like
|
|
75
|
+
forgetting the policy, and the reason is what tells the next reader which of the two it is.
|
|
76
|
+
|
|
41
77
|
## What each file owns
|
|
42
78
|
|
|
43
79
|
| File | Job |
|
|
@@ -47,7 +83,8 @@ merely looks like a query (`kind: 'query'`, no declaration) is `X_QUERY_FOREIGN`
|
|
|
47
83
|
| `facade.ts` | binds each projection to the query; re-implements none of them |
|
|
48
84
|
| `mcp-tool.ts` | the MCP read descriptor |
|
|
49
85
|
| `client.ts` | the typed read client (browser-safe) |
|
|
50
|
-
| `
|
|
86
|
+
| `http.ts` | the route projection — `GET /_x/query/<kebab>`, the URL the client derives |
|
|
87
|
+
| `naming.ts` | export name → wire path. The MCP tool name is the export name verbatim |
|
|
51
88
|
| `live.ts` | the `LiveQuery` descriptor `@ultimat3/realtime` subscribes to |
|
|
52
89
|
| `matcher.ts` | change event → minimal patch (`add` / `update` / `remove` / `refill`) |
|
|
53
90
|
| `pagination.ts` | `paginate()` — keyset pages over core's cursor codec |
|
|
@@ -106,6 +143,14 @@ the package, and rotating the secret is what invalidates every open cursor. This
|
|
|
106
143
|
the only thing that is its business — the scope, `queryHash(name, input)` — and re-exports
|
|
107
144
|
`CursorInvalidError` so the failure keeps its name on this surface.
|
|
108
145
|
|
|
146
|
+
The scope's hash is **SHA-256, first 16 hex** (`fingerprint` in `stable.ts`), the primitive and
|
|
147
|
+
width `@ultimat3/realtime`'s `stableDigest` and `@ultimat3/entity`'s `planScope` already use. It
|
|
148
|
+
was FNV-1a/32 until 2026-08 — 4×10⁹ values over input a client chooses, brute-forceable offline in
|
|
149
|
+
seconds, and a fingerprint here is a *sharing* key: which read-cache entry two callers are served
|
|
150
|
+
from, and which scope a cursor is bound to. The canonical serialization did not change, only the
|
|
151
|
+
hash, so a cursor issued before it fails its scope check as `X_CURSOR_INVALID` with "request the
|
|
152
|
+
first page again" as its fix, and a warm read cache is cold once.
|
|
153
|
+
|
|
109
154
|
A cursor names a **position in the ordering**, never a row and never a count. Both seek paths
|
|
110
155
|
answer "is this row after that position?" through the one predicate, `isAfterKey`: `Builder.seek()`
|
|
111
156
|
compiles it to SQL — spelled out per key, so a mixed `createdAt desc, id asc` listing is a real
|
|
@@ -116,23 +161,147 @@ gone. A row with no `id` cannot name a position at all: that is `X_QUERY_NOT_PAG
|
|
|
116
161
|
cursor signed over `"undefined"`.
|
|
117
162
|
|
|
118
163
|
Because the predicate always carries that id, the ordering carries it too: a paged read is served
|
|
119
|
-
`order by <declared keys>, "id" asc
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
164
|
+
`order by <declared keys>, "id" asc` — that list is `totalOrder(orderBy)`, exported for the reason
|
|
165
|
+
`isAfterKey` is — and the in-memory sort and the live matcher's insertion position read the same
|
|
166
|
+
one. Ordering by the declared keys alone leaves rows with equal sort values in whatever order the
|
|
167
|
+
database chose, while the cursor reads them as if id had decided — so one of a tied pair comes back
|
|
168
|
+
on both pages and the other on neither, and a row the matcher appends after a tie group is a
|
|
169
|
+
position no re-read returns.
|
|
170
|
+
|
|
171
|
+
A **live** read is served that way too, and `SqlSource.total()` is how it says so: the same read
|
|
172
|
+
with no cursor and no window, ordered `<declared keys>, "id" asc`. `sourceFor` calls it for
|
|
173
|
+
`surface: 'live'` and for nothing else, so the initial window, the patch positions the matcher
|
|
174
|
+
computes and the keyset re-read a reconnect resumes with are one ordering. A source that does not
|
|
175
|
+
implement `total()` is left alone — it already serves one order it can be resumed in.
|
|
176
|
+
|
|
177
|
+
## NULL
|
|
178
|
+
|
|
179
|
+
One rule, three readers: the SQL a source generates, the in-memory execution behind `from()`, and
|
|
180
|
+
the live matcher. `null` and a column the row omits are the same absence — `isNull` is the one
|
|
181
|
+
definition, exported for the same reason `isAfterKey` is.
|
|
182
|
+
|
|
183
|
+
| Operator | NULL is | Emitted as |
|
|
184
|
+
|---|---|---|
|
|
185
|
+
| `=` `!=` `in` | a value — it matches itself and nothing else | `is null` · `is not null` · `is distinct from` · `in (…) or is null` |
|
|
186
|
+
| `>` `>=` `<` `<=` | unknown — a NULL on either side matches nothing | `"col" > $n`, which already matches no NULL |
|
|
187
|
+
| `order by`, the cursor | the largest value: last ascending, first descending | `asc nulls last` · `desc nulls first` |
|
|
188
|
+
|
|
189
|
+
`where({ deletedAt: null })` compiles to `"deletedAt" is null` and binds no parameter: `= $1` with
|
|
190
|
+
a NULL argument is unknown in Postgres and unknown is never true, so that filter used to match
|
|
191
|
+
every row in memory and none in the database. The seek predicate had the same defect one page
|
|
192
|
+
later — `"publishedAt" > $1` is unknown for every draft, so page two stopped at the first NULL and
|
|
193
|
+
the rows after it were unreachable through a cursor. An ascending key now reaches them
|
|
194
|
+
(`("col" > $1 or "col" is null)`); a NULL cursor value drops its own term, nothing sorting after a
|
|
195
|
+
NULL under `nulls last`, and the page continues on the id tiebreak, which is never NULL.
|
|
196
|
+
|
|
197
|
+
`nulls last` / `nulls first` are Postgres' own defaults, written down rather than inherited: it is
|
|
198
|
+
the rule `compareValues` implements, so the in-memory sort and the seek predicate can only agree
|
|
199
|
+
with it, and a driver whose default differs cannot re-open the divergence.
|
|
123
200
|
|
|
124
201
|
## Caching
|
|
125
202
|
|
|
126
|
-
Request memo (same read twice in one render ⇒ one round trip), then the tier
|
|
127
|
-
`
|
|
128
|
-
|
|
129
|
-
`@ultimat3/cache
|
|
203
|
+
Request memo (same read twice in one render ⇒ one round trip), then the tier ladder
|
|
204
|
+
`@ultimat3/cache` has registered — `createCacheStack(registeredTiers())`, read down and promoted
|
|
205
|
+
up. Keys are `query:<name>:<authority>:<input fingerprint>:<tags>`. An action's `cache.invalidates`
|
|
206
|
+
and a query's `cache.tags` meet in the one graph owned by `@ultimat3/cache`, because there is one
|
|
207
|
+
registry and this package holds no store of its own.
|
|
208
|
+
|
|
209
|
+
**The authority is who the read was answered for, and it is not optional.** `sql(input, ctx)` is
|
|
210
|
+
handed the context and `@ultimat3/entity` derives every tenant predicate from `ctx.actor.orgId`,
|
|
211
|
+
never from the input — so a key made of the name, the input and the tags did not identify a read's
|
|
212
|
+
answer, and the process-wide tier served one org's rows to the next org that asked. `cache.scope`
|
|
213
|
+
declares who may be served one entry:
|
|
214
|
+
|
|
215
|
+
| `scope` | Key holds | Use it when |
|
|
216
|
+
|---|---|---|
|
|
217
|
+
| `actor` (default) | the actor's kind, id and org | anything. Declaring nothing gets this, and this is always correct |
|
|
218
|
+
| `tenant` | the actor's org — the actor itself when there is none | every member of one org sees the same rows |
|
|
219
|
+
| `global` | nothing | the rows are the same for everyone, signed-in or not |
|
|
220
|
+
|
|
221
|
+
The default is the mechanism: forgetting to declare a scope gives the narrowest key. Widening it
|
|
222
|
+
is a written statement about the rows, one `grep` away — the same shape `unenforced:` uses for a
|
|
223
|
+
skipped policy. `readAuthority(actor, scope)` is the only producer of the component, and it is a
|
|
224
|
+
required positional argument of `cacheKeyFor`, because an optional one is one a call site forgets.
|
|
225
|
+
|
|
226
|
+
**The fill is fenced, best-effort and single-flighted, and none of that is written here.**
|
|
227
|
+
`createCacheStack` samples `@ultimat3/cache`'s fence immediately before it runs the source and
|
|
228
|
+
re-asks it per rung before each write, so a bust that lands mid-read cannot be republished for the
|
|
229
|
+
whole TTL — the caller still gets the rows it read, because those are its answer; only publishing is
|
|
230
|
+
refused. Every tier call goes through `bestEffort`, so a Redis refusal is a miss rather than a
|
|
231
|
+
failed business read and shows up in `recentTierFailures()` under the name of the tier that
|
|
232
|
+
refused. Concurrent misses of one key share one `load()`. `@ultimat3/query` used to carry its own
|
|
233
|
+
copy of the first two and none of the third.
|
|
234
|
+
|
|
235
|
+
`cache.ttlMs` is refused at `query()` unless it is positive and finite
|
|
236
|
+
(`X_QUERY_CACHE_TTL_INVALID`): every tier refuses such a lease, so `ttlMs: Infinity` used to make
|
|
237
|
+
one read fail permanently at run time with a cause about a cache key.
|
|
238
|
+
|
|
239
|
+
An entry is written with the read's `cache.tags`, so a row bust (`post:1`) drops the lists that
|
|
240
|
+
held the row, exactly as `tagMatches` defines it — and `invalidateTags(tags)`, the call an action's
|
|
241
|
+
`cache.invalidates` makes, is the whole of what drops it. There is nothing to install and nothing
|
|
242
|
+
to wire: a `cache:` read fills the registered tiers, so a process that registered none reads
|
|
243
|
+
uncached rather than filling a store no fan-out can see.
|
|
244
|
+
|
|
245
|
+
A `cache:` block that omits `ttlMs` gets `DEFAULT_READ_CACHE_TTL_MS` (60s) rather than immortality.
|
|
246
|
+
Tags are the primary eviction; the TTL is the backstop for the read whose tags never fire. The
|
|
247
|
+
lease handed to the ladder is **relative** — the tier's own clock turns it into an expiry, so a
|
|
248
|
+
tier registered with a frozen clock is drivable end to end — and omitting it is how a tier is asked
|
|
249
|
+
for its own default. `@ultimat3/cache`'s tiers refuse a non-positive `ttlMs` and have no immortal
|
|
250
|
+
entry to offer, so there is no "never" to spell.
|
|
251
|
+
|
|
252
|
+
The memo entry is the read **in flight**, not its value, so "twice" covers reads that race as
|
|
253
|
+
well as reads that follow: five holes rendering concurrently share one execution and one tier
|
|
254
|
+
round trip. A rejection is evicted — a failed read is not the request's answer.
|
|
255
|
+
|
|
256
|
+
| Layer | Applies to | Lifetime |
|
|
257
|
+
|---|---|---|
|
|
258
|
+
| request memo (`readOnce`) | **every** query, `cache:` or not | the request — a `Ctx`'s identity is the key |
|
|
259
|
+
| tag-keyed tier (`readThrough`) | a query that declares `cache:` | `ttlMs`, or until an `invalidates` fan-out drops the tag |
|
|
260
|
+
|
|
261
|
+
`cache:` buys the tier, never the memo: an uncached lookup called once per row of a list would
|
|
262
|
+
otherwise cost one round trip per row. Parsing, the policy and `sql()` still run on every call —
|
|
263
|
+
the memo holds the execution, not the decision — and `.as()` reads in a child context, so an
|
|
264
|
+
impersonated read never joins one made as someone else.
|
|
265
|
+
|
|
266
|
+
`fresh: true` skips both on the way in, and **publishes into the memo on the way out**: it is how
|
|
267
|
+
a caller reads past a write made earlier in the same request, and the read it just made is the
|
|
268
|
+
request's answer from then on, so a later plain read of the same key joins it rather than the entry
|
|
269
|
+
the write moved past. Invalidation drops tier entries, not memo entries.
|
|
270
|
+
|
|
271
|
+
## `rateLimit:` — the read half, and it is enforced
|
|
272
|
+
|
|
273
|
+
```ts
|
|
274
|
+
rateLimit: { limit: 3, windowMs: 600_000 }, // 3 held, one back every three and a bit minutes
|
|
275
|
+
```
|
|
276
|
+
|
|
277
|
+
Symmetric with an action's, and for a reason: without it **every** `GET /_x/query/*` fell to the
|
|
278
|
+
`default` bucket — 120 burst, 2/s per actor — so one authenticated caller could hold 120
|
|
279
|
+
cross-tenant aggregates in flight and then 2/s indefinitely, from a single account, with no
|
|
280
|
+
declaration able to say otherwise. `toQueryRoute` sets the bucket NAME and the NUMBERS, and
|
|
281
|
+
`@ultimat3/http`'s `withRouteBuckets` registers them: a name alone falls through to `default`.
|
|
282
|
+
The conversion is `toBucket` from `@ultimat3/http` — the same one the action route uses, because
|
|
283
|
+
http owns `Bucket` and `action` is the same tier as this package. A pair the limiter cannot run
|
|
284
|
+
on is `X_RATE_LIMIT_INVALID`, at projection.
|
|
285
|
+
|
|
286
|
+
## `deprecated:` — a compat window, not a version
|
|
287
|
+
|
|
288
|
+
```ts
|
|
289
|
+
deprecated: { since: '2026-08-01T00:00:00Z', sunset: '2026-12-31T23:59:59Z', replacedBy: 'searchOrders' },
|
|
290
|
+
```
|
|
291
|
+
|
|
292
|
+
`Deprecation` (RFC 9745) and `Sunset` (RFC 8594) on every answer, a
|
|
293
|
+
`link: </_x/query/search-orders>; rel="successor-version"`, the dates on the descriptor, and a
|
|
294
|
+
`deprecated_calls_total{primitive,name}` counter so "is anyone still reading it?" has an answer
|
|
295
|
+
before the read is deleted. A date that cannot be rendered is `X_QUERY_DEPRECATION_INVALID` at
|
|
296
|
+
projection, not on the first read. Versioning is two deployments behind one ingress, never a
|
|
297
|
+
router feature here.
|
|
130
298
|
|
|
131
299
|
## Errors
|
|
132
300
|
|
|
133
301
|
| Code | When | Fix |
|
|
134
302
|
|---|---|---|
|
|
135
303
|
| `X_QUERY_DUPLICATE` | two queries under one name | rename one export |
|
|
304
|
+
| `X_QUERY_DEPRECATION_INVALID` | `deprecated:` with a `since`/`sunset` that is not a date | use an ISO-8601 instant |
|
|
136
305
|
| `X_QUERY_POLICY_MISSING` | registration without `policy:` | add `policy: can('…')` |
|
|
137
306
|
| `X_MATCHER_UNSUPPORTED` | live query the matcher cannot patch | reshape it, or `live: false` |
|
|
138
307
|
| `X_CURSOR_INVALID` | tampered / foreign / malformed cursor | request the first page again |
|