@niscorp/vex 0.2.2 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +172 -0
- package/DESIGN.md +66 -3
- package/DOCS.md +156 -12
- package/dist/adapters/express/index.cjs +102 -17
- package/dist/adapters/express/index.cjs.map +1 -1
- package/dist/adapters/express/index.d.cts +2 -2
- package/dist/adapters/express/index.d.ts +2 -2
- package/dist/adapters/express/index.js +2 -2
- package/dist/adapters/hono/index.cjs +121 -22
- package/dist/adapters/hono/index.cjs.map +1 -1
- package/dist/adapters/hono/index.d.cts +2 -2
- package/dist/adapters/hono/index.d.ts +2 -2
- package/dist/adapters/hono/index.js +21 -7
- package/dist/adapters/hono/index.js.map +1 -1
- package/dist/agent/index.cjs +2 -0
- package/dist/agent/index.cjs.map +1 -1
- package/dist/agent/index.d.cts +1 -1
- package/dist/agent/index.d.ts +1 -1
- package/dist/agent/index.js +1 -1
- package/dist/{chunk-HF25Q53I.js → chunk-H4BHW76Q.js} +6 -3
- package/dist/chunk-H4BHW76Q.js.map +1 -0
- package/dist/{chunk-HCEH55PP.js → chunk-IQM4SQE2.js} +123 -21
- package/dist/chunk-IQM4SQE2.js.map +1 -0
- package/dist/{handler-Cql8SJ3F.d.cts → handler-BGCL6_Si.d.ts} +3 -2
- package/dist/{handler-mICf9ins.d.ts → handler-CMrjTXNx.d.cts} +3 -2
- package/dist/index.cjs +163 -35
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +3 -3
- package/dist/index.d.ts +3 -3
- package/dist/index.js +44 -22
- package/dist/index.js.map +1 -1
- package/dist/{types-BRvMB6IM.d.cts → types-Bi-Lg86b.d.cts} +48 -2
- package/dist/{types-BRvMB6IM.d.ts → types-Bi-Lg86b.d.ts} +48 -2
- package/package.json +5 -5
- package/dist/chunk-HCEH55PP.js.map +0 -1
- package/dist/chunk-HF25Q53I.js.map +0 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,177 @@
|
|
|
1
1
|
# @niscorp/vex
|
|
2
2
|
|
|
3
|
+
## 0.3.0
|
|
4
|
+
|
|
5
|
+
### Minor Changes
|
|
6
|
+
|
|
7
|
+
- f834f3f: `evaluate` checks a config object once and only evaluates it after that; `prismTransform` is gone. **Breaking.**
|
|
8
|
+
|
|
9
|
+
BREAKING — approved by moccadroid, 2026-10-09: replace `prismTransform` with `evaluate` (`transform: evaluate`), and pass limits as `{ limits: … }`.
|
|
10
|
+
|
|
11
|
+
Prism had three ways to run a config, and the one a host was told to hand to its seam (`prismTransform`) was the one that reported worst, while the hosts on a hot path each wrapped `evaluate`, which checked the config on every call. There are two now, with one job each.
|
|
12
|
+
|
|
13
|
+
**`evaluate(config, source, options?)`** — for a config a program holds.
|
|
14
|
+
- The first time it is handed a config object it checks it and keeps its tree; the second time it prepares the tree as `compile` would; from then on a call only evaluates. Four fields picked and renamed, the same config object again: 3.5 µs before, 0.4 µs now. A nested choice on two fields: 18 µs before, 0.7 µs now. A config that is a new object on every call costs what it did.
|
|
15
|
+
- It is kept **by the object**: a config changed in place after its first call is not read again. Hand over a new object.
|
|
16
|
+
- Both arguments are `unknown`, so it is what a host's transform seam takes: `createTide({ transform: evaluate })`, `createUpgrader(grammars, { transform: evaluate })`.
|
|
17
|
+
- The third argument is `{ limits?, check? }`. Limits were the third argument themselves: `evaluate(config, source, { maxSteps: 100 })` is now `evaluate(config, source, { limits: { maxSteps: 100 } })`.
|
|
18
|
+
- `check: 'always'` is for writing a config, for tests and for tools: nothing is kept, the config is checked on every call, and the source must be plain JSON (a source holding `undefined`, a function, a `Date` or a non-finite number is refused with `E_TYPE`).
|
|
19
|
+
- An invalid config is a `PrismError` (`E_SCHEMA`) naming the part that is wrong, however deep the config. `prismTransform` threw zod's own error, and a `RangeError` for a config nested far past the limit.
|
|
20
|
+
|
|
21
|
+
`evaluateSafe` takes the same arguments.
|
|
22
|
+
|
|
23
|
+
**`compile` and `execute`** — for a config that is stored. Unchanged.
|
|
24
|
+
|
|
25
|
+
**`prismTransform` is removed**, from `@niscorp/prism` and from `@niscorp/prism/migrations`. What it added over `evaluate` was a check that the source is plain JSON; that check is now `check: 'always'`, and a seam's source is taken as it is.
|
|
26
|
+
|
|
27
|
+
**moss, nova, vex, loom, nisc, create-nisc and the cli move with it.** Each has Prism as a peer or a dependency, or has one of those that do, and a Prism at 0.3 is outside the range their last versions accept: an app takes them together. Nothing in vex, loom or the cli changed beyond that range.
|
|
28
|
+
|
|
29
|
+
moss's shell and nova's `$prism` binding called `evaluate` on every use and so paid the check each time: an endpoint's request and response under moss, and a `$prism` value on each draw and for each row of a list, are 8 to 45 times faster for the configs measured. tide's and strata's documentation name `evaluate` where they named `prismTransform`.
|
|
30
|
+
|
|
31
|
+
**What to change**
|
|
32
|
+
- `import { prismTransform } from '@niscorp/prism'` or `'@niscorp/prism/migrations'` → `import { evaluate } from '@niscorp/prism'`, and `transform: prismTransform` → `transform: evaluate`. Every app made by `create-nisc` has this in `src/dev/strata.ts`; the app's typecheck and `npm run strata` both stop on that line until it is changed.
|
|
33
|
+
- A third argument of limits becomes `{ limits: … }`.
|
|
34
|
+
- A wrapper that parsed the config before calling `evaluate` (`evaluate(ConfigSchema.parse(config), …)`) makes a new object on every call and so is checked on every call: hand `evaluate` the config itself.
|
|
35
|
+
- Code that changes a config object in place and evaluates it again must hand over a new object, or pass `{ check: 'always' }`.
|
|
36
|
+
- A host that relied on `prismTransform` refusing a source that is not plain JSON checks it itself, or passes `{ check: 'always' }`.
|
|
37
|
+
|
|
38
|
+
### Patch Changes
|
|
39
|
+
|
|
40
|
+
- Updated dependencies [023bcf9]
|
|
41
|
+
- Updated dependencies [1db6df4]
|
|
42
|
+
- Updated dependencies [6d098e4]
|
|
43
|
+
- Updated dependencies [8b4d97e]
|
|
44
|
+
- Updated dependencies [f834f3f]
|
|
45
|
+
- Updated dependencies [b084e30]
|
|
46
|
+
- Updated dependencies [663d724]
|
|
47
|
+
- Updated dependencies [6512d91]
|
|
48
|
+
- Updated dependencies [7d682b9]
|
|
49
|
+
- Updated dependencies [d2e9d22]
|
|
50
|
+
- Updated dependencies [e0e8abc]
|
|
51
|
+
- Updated dependencies [7fe1135]
|
|
52
|
+
- @niscorp/prism@0.3.0
|
|
53
|
+
|
|
54
|
+
## 0.2.3
|
|
55
|
+
|
|
56
|
+
### Patch Changes
|
|
57
|
+
|
|
58
|
+
- 1e5ad2d: An insert requires every key its `values` bind, whatever `onConflict.set` puts in the same column.
|
|
59
|
+
|
|
60
|
+
An `insert` or `insertEach` may set the same column in `values` and in `onConflict.set`. Vex read the two by column name, so where they put different things in a column it saw only what `onConflict.set` put there. A `$context` key that `values` used for that column, and that nothing else in the statement used, was not in the entry's derived signature and was not required. A call that left it out was not refused. The statement ran with that parameter unbound, and a new row was written with NULL in the column — or, where the column is NOT NULL, the database's own error came back in place of vex's. DOCS.md says a write never executes with holes; now this one does not.
|
|
61
|
+
|
|
62
|
+
```ts
|
|
63
|
+
{ op: 'insert', table: 'people',
|
|
64
|
+
values: { email: { $context: 'email' }, name: { $context: 'name' } },
|
|
65
|
+
onConflict: { target: ['email'], set: { name: { $context: 'newName' } } } }
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
- A call without `name` is refused before any SQL runs: 400 `missing_context`, `Mutation is missing context: name.` (was 200, and a row with `name` NULL).
|
|
69
|
+
- Discovery, `collectMutationContext` and the refusal's `details.expected` list both `name` and `newName` (listed `newName` only). `requiredContextKeys` returns both.
|
|
70
|
+
- The same where `onConflict.set` puts a literal or a `$lookup` in the column rather than another key, and for the keys a `$lookup` in `values` binds.
|
|
71
|
+
|
|
72
|
+
An entry that puts different things in one column in the two places has its signature read `values` first, then `onConflict.set`. Its keys can come in another order than before, and a key used in both is listed with the column and type of its place in `values`.
|
|
73
|
+
|
|
74
|
+
Nothing else moves. An entry whose `values` and `onConflict.set` share no column, or put the same thing in every column they share (the create-or-fetch touch, `set: { email: { $context: 'email' } }`), lists and requires what it did, in the same order. `update`, `delete` and `mutationEffect` are unchanged.
|
|
75
|
+
|
|
76
|
+
**What to change:** nothing, unless an entry sets a column in `values` from a `$context` key, sets the same column in `onConflict.set` from anything else, and some caller leaves that key out. That call wrote a NULL and is now answered 400. Send the key — `null` to write NULL on purpose; only an absent key is refused — or, if the column was never meant to be set on insert, take it out of `values`.
|
|
77
|
+
|
|
78
|
+
- 2011951: A mutation batch can use what an earlier statement wrote: `{ $returned: 'table.column' }`.
|
|
79
|
+
|
|
80
|
+
A batch ran in one transaction but no statement could see another's result. A row whose id the database generates could not be referenced by the rows written after it in the same batch.
|
|
81
|
+
|
|
82
|
+
A later statement of a batch can now read a column of the one row an earlier statement wrote:
|
|
83
|
+
|
|
84
|
+
```ts
|
|
85
|
+
[
|
|
86
|
+
{ op: 'insert', table: 'orders', values: { note: { $context: 'note' } } },
|
|
87
|
+
{
|
|
88
|
+
op: 'insertEach',
|
|
89
|
+
table: 'order_lines',
|
|
90
|
+
items: { $context: 'lines' },
|
|
91
|
+
values: { order_id: { $returned: 'orders.id' }, sku: { $item: 'sku' } },
|
|
92
|
+
},
|
|
93
|
+
];
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
The reference names the earlier statement — an insert, update, upsert or delete — by the table it writes. It goes in any value position (`values`, `set`, `onConflict.set`, an `upsert`'s `columns` and `insert`), and not in a `where`, a `$lookup`, or an `insertEach`'s `items`. It is for keys: the value passes through the driver, and a timestamp read back that way keeps milliseconds only.
|
|
97
|
+
|
|
98
|
+
The statement it names must write exactly one row. When it wrote none (its WHERE matched nothing, or the scope boundary kept the row back) or several, the batch fails with `execution_error` and nothing is written. `lintMutation` — so `seedCache` — refuses a reference that no earlier statement can answer, that two earlier statements could answer, that names an `insertEach`, or that names an `insert` whose `onConflict` has no `set`. Scope is unchanged: each statement is scoped on its own, as before.
|
|
99
|
+
|
|
100
|
+
Every existing entry, request and reply behaves as it did. `MutationValue` gains one union member, `ReturnedRef`.
|
|
101
|
+
|
|
102
|
+
**What to change:** nothing. To use it, upgrade every process that reads the cache first: a vex older than this one deletes a stored entry whose grammar it does not know, as it does for any form added since it was built.
|
|
103
|
+
|
|
104
|
+
- 2c57a38: Under `require`, an engine error on a read keeps its HTTP status.
|
|
105
|
+
|
|
106
|
+
The CommonJS build gives each entry point its own copy of `VexError`, and the HTTP handler recognised an engine's error by `instanceof`. With the engine from `require('@niscorp/vex')` and the endpoint from `require('@niscorp/vex/hono')` or `require('@niscorp/vex/express')`, every error the engine threw on a read was therefore answered as an unexpected one: 500, without its `details`, and with a `[vex] unhandled error` line in the log. The ESM build has one copy of the class and answered as DOCS.md says. Now both do:
|
|
107
|
+
- an unknown fingerprint is 404 `cache_miss` (was 500);
|
|
108
|
+
- a protected entry asked to change is 409 `fingerprint_protected` (was 500);
|
|
109
|
+
- `locked`, `agent_failed`, `unsatisfiable`, `scope_denied`, `invalid_dsl` and an `execution_error` thrown by the engine are 400, with `details` where the error has them (were 500, without);
|
|
110
|
+
- `missing_scope` is still 500, and its reply no longer names the scope keys the host left out — they go to the log, as they always did under ESM.
|
|
111
|
+
|
|
112
|
+
The same was true of an app that loads vex in both formats at once, and of moss loaded with `require` (its `/api/…/vex` endpoints). Writes were never affected: the handler runs them itself.
|
|
113
|
+
|
|
114
|
+
The engine has the same check for a generation hook's `unsatisfiable`. With the hook from `require('@niscorp/vex/agent')` it never matched, so nothing was negative-cached and the model was asked again on every repeat of a request it had already called impossible. The refusal is cached now, for `unsatisfiableTtlMs` (five minutes unless set), as it is under ESM.
|
|
115
|
+
|
|
116
|
+
Vex now recognises its own errors by a mark every copy of the class carries. Nothing is added to the exports, and nothing an error prints or serializes changes. An app's own `err instanceof VexError` is as it was: it holds for an error the engine throws, and under `require` it still does not hold for one made in `@niscorp/vex/agent` — read `err.code` there (DOCS.md, "Error handling").
|
|
117
|
+
|
|
118
|
+
**What to change:** nothing in an ESM app, and nothing in a CommonJS app that calls `handleQuery` from `@niscorp/vex` itself. A CommonJS app that had come to depend on those 500s now sees the documented status: a client that reads 500 as "unknown fingerprint", a retry or an alert on 5xx, a log search for `[vex] unhandled error`.
|
|
119
|
+
|
|
120
|
+
- 2d20daf: The Hono adapter answers a body that is not JSON with 400 `invalid_request`, not 500.
|
|
121
|
+
|
|
122
|
+
`POST`, `PATCH` and `DELETE` parsed the body unguarded, so a parse failure threw out of the route: text, a multipart or urlencoded form, truncated JSON, or no body at all was answered `500 Internal Server Error` in plain text, with the `SyntaxError` on the server's log — a server fault for what is the request's mistake, and the one failure on this surface without the `{ error, message }` shape. It is now `400 { "error": "invalid_request", "message": "Body must be JSON" }`, on a locked endpoint as well. A moss app's vex surfaces (`/api/vex`, `/api/<resource>/vex`) are this adapter, so they answer the same.
|
|
123
|
+
|
|
124
|
+
Nothing else moves. Every request that was answered before is answered the same, byte for byte: JSON of the wrong shape keeps its own 400; JSON sent as `text/plain` or with no content type is still read (the content type is never looked at); `getScope` still runs before the body is read; `onExecute` hears nothing for a body that was never a request, as before. The body is still read through hono's own `c.req.json()`, so a middleware that read it first shares hono's cache with the adapter exactly as before, on every hono 4. Only the parse failing is caught — a body that cannot be read at all, or a stream that broke, still throws. The Express adapter was not affected: it is handed `req.body` and already answered 400.
|
|
125
|
+
|
|
126
|
+
**What to change:** nothing, unless something of yours was built on the 500. A host's own `onError` (or a middleware reading `c.error`) no longer sees a `SyntaxError` for these requests — the response is the 400 above, not the one the handler made. A client that retried on 5xx stops retrying them. A nova endpoint aimed at a vex url with no `request` declared now fails with `@error.status` 400 and the message `Body must be JSON`, where it had 500 and `HTTP 500`.
|
|
127
|
+
|
|
128
|
+
- b711c38: The reply says when the engine's own limit may have cut a list.
|
|
129
|
+
|
|
130
|
+
A list query that states no `limit` is given `defaultLimit` (100), and a generated one asking for more than `maxLimit` is clamped. Either way the reply carried the rows and nothing else: a list of 100 looked like a list that had 100 things in it.
|
|
131
|
+
|
|
132
|
+
When the `LIMIT` a query ran with was the engine's number — the default, or the maximum — and exactly that many rows came back for a list, `meta.warnings` now carries one line saying the list may have been cut and which number did it. For a request that named a fingerprint the same line is written once per fingerprint with `console.warn`, because a host that hands callers `result` without `meta` would show it to no one.
|
|
133
|
+
|
|
134
|
+
It says "may": no extra row is fetched to find out, so a list of exactly that many rows gets the line too. A list that ended on a `limit` its author stated gets none, and neither does a single-row answer. No `result` changes, and no query reads a different number of rows.
|
|
135
|
+
|
|
136
|
+
**What to change:** nothing. If the line appears for one of your entries, state the `limit` that list should have.
|
|
137
|
+
|
|
138
|
+
- 3d8518c: `mutationMisfits(def, schema)`: why a stored write does not fit a schema, without running it.
|
|
139
|
+
|
|
140
|
+
Every write passes two column gates before it runs: its table, the columns it writes, its WHERE, its `$lookup`s and its conflict target must be the schema's, and so must a column it reads from an earlier statement (`$returned`). Those gates could only be met by running the write.
|
|
141
|
+
|
|
142
|
+
`mutationMisfits` asks them and runs nothing: one sentence for each refusal, none when the write fits. An upsert is asked as both statements it can become. Scope is not applied — what a policy stamps or bounds is checked where the principal is known.
|
|
143
|
+
|
|
144
|
+
**What to change:** nothing.
|
|
145
|
+
|
|
146
|
+
- b711c38: A seeded entry runs with the `limit` its author wrote, past `maxLimit`.
|
|
147
|
+
|
|
148
|
+
`maxLimit` (1000 by default) clamped every query, including one a developer wrote and seeded: a seeded entry that stated a larger `limit` ran with `maxLimit` in its place, and the reply did not say so.
|
|
149
|
+
|
|
150
|
+
`maxLimit` now clamps only a query nobody reviewed — one generated for a request. An entry seeded with `seedCache` runs with the `limit` it states. Which kind an entry is comes from the stored row and not from the request: protected, with no request hash. That is what `seedCache` writes, and what a host writes when it stores an entry through the cache backend itself; no request can produce it. A generated entry stays clamped when it is replayed, when it is stored under a name the caller chose, and when it is protected afterwards. `engine.compile` takes a bare query and clamps as before; `engine.test` still runs with a limit of 5.
|
|
151
|
+
|
|
152
|
+
One edge: a generated entry that was protected and is then seeded with the identical definition keeps its request hash, because `seedCache` leaves a row that already matches alone. It stays clamped, and its reply says so, until it is unprotected or deleted and seeded again.
|
|
153
|
+
|
|
154
|
+
**This changes what an existing app receives in one setup:** a read entry the host stored itself — seeded, or written protected through the cache backend — whose `limit` is above the engine's `maxLimit` returned `maxLimit` rows and now returns up to its stated `limit`. Nothing changes for an entry that states no limit, one that states `maxLimit` or less, a generated query, or a mutation.
|
|
155
|
+
|
|
156
|
+
**What to change:** nothing, if the larger answer is what the entry was written for. To keep the clamp on seeded entries, set `config.capAuthored: true` on the engine (under moss: `runtime.vexConfig: { capAuthored: true }`).
|
|
157
|
+
|
|
158
|
+
- 1903342: An upsert's signature lists the key its `columns` bind, also for a column its `insert` sets too.
|
|
159
|
+
|
|
160
|
+
An `upsert` may set the same column in `columns` and in `insert`: the first is written when it updates, the second when it creates. Its derived signature read the two merged by column name, so where they used different `$context` keys only the one in `insert` was listed. The key the update binds was missing from discovery and from `collectMutationContext` — and from `details.expected` of the very `missing_context` refusal that named it. What an upsert requires was right all along, on both branches; only the listing was short.
|
|
161
|
+
|
|
162
|
+
```ts
|
|
163
|
+
{ op: 'upsert', table: 'people', key: 'id',
|
|
164
|
+
columns: { name: { $context: 'name' } },
|
|
165
|
+
insert: { name: { $context: 'firstName' } } }
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
- The signature lists `name` beside `firstName` (listed `firstName` only). `firstName` is still marked `insert only`.
|
|
169
|
+
- Where both halves use the same key for the column, that key is no longer marked `insert only`: the update binds it too.
|
|
170
|
+
|
|
171
|
+
Nothing else moves. An upsert whose `columns` and `insert` share no column is listed as it was, and no write runs, is refused or lands differently. Inside, the signature, the required keys and the `$returned` check now read what a statement sets through one function.
|
|
172
|
+
|
|
173
|
+
**What to change:** nothing.
|
|
174
|
+
|
|
3
175
|
## 0.2.2
|
|
4
176
|
|
|
5
177
|
### Patch Changes
|
package/DESIGN.md
CHANGED
|
@@ -188,6 +188,31 @@ extra filter clauses.
|
|
|
188
188
|
omitted so the runtime can return a valid-but-empty response instead of
|
|
189
189
|
failing.
|
|
190
190
|
|
|
191
|
+
**The limit, and whose number it is.** Every query compiles with a `LIMIT`. A
|
|
192
|
+
query that states none is given `defaultLimit`; one that states more than
|
|
193
|
+
`maxLimit` is clamped to it — unless its author seeded it. `maxLimit` exists
|
|
194
|
+
for the query nobody reviewed: a generated one must not be able to ask for the
|
|
195
|
+
table. Applied to a number a developer wrote into a seeded entry it did
|
|
196
|
+
something else: it replaced that number with a smaller one, and the reply did
|
|
197
|
+
not say so. So the clamp reads provenance off the stored entry: `protected`
|
|
198
|
+
with no `requestHash` is what `seedCache` writes, and what a host writes when
|
|
199
|
+
it stores an entry through the cache backend itself. No request can produce it
|
|
200
|
+
(a generated entry always carries the hash of the request it was born from,
|
|
201
|
+
and the one field a wire call can change on a stored entry is `protected`),
|
|
202
|
+
and an entry that has it runs with its stated limit. It records who put the
|
|
203
|
+
entry there, not whether a person read it: a host that stores a query it did
|
|
204
|
+
not write has vouched for it. `config.capAuthored` puts the
|
|
205
|
+
clamp back for a host that wants one ceiling over everything.
|
|
206
|
+
|
|
207
|
+
What is left is the case nothing can decide for the author: a list that states
|
|
208
|
+
no limit. Returning every row is not offered: nothing would bound the read,
|
|
209
|
+
and an unbounded read of a large table is the failure `defaultLimit` is there
|
|
210
|
+
to stop. So the default
|
|
211
|
+
stays, and the engine stops being silent about it: when the `LIMIT` was its own
|
|
212
|
+
number and exactly that many rows came back for a list, the reply's
|
|
213
|
+
`meta.warnings` says the list may have been cut (and the console, once per
|
|
214
|
+
named entry). No extra row is fetched to make it certain; the line says "may".
|
|
215
|
+
|
|
191
216
|
### Adapters
|
|
192
217
|
|
|
193
218
|
`DatabaseAdapter` is the whole database boundary: `introspect`, `compile`
|
|
@@ -376,6 +401,37 @@ values bind as SQL parameters exactly like reads) → execute. Batches run in
|
|
|
376
401
|
one transaction; the client is structural (`MutationClient` — PGlite, a pg
|
|
377
402
|
wrapper, a test double), so the core imports no driver.
|
|
378
403
|
|
|
404
|
+
**A batch that uses what it wrote (`$returned`).** `{ $returned: 'table.column' }`
|
|
405
|
+
in a later statement reads the one row an earlier statement of the same batch
|
|
406
|
+
wrote — the create-a-parent-then-its-children case, where the parent's id is
|
|
407
|
+
the database's to generate and no natural key exists to `$lookup` it by. Three
|
|
408
|
+
decisions shape it:
|
|
409
|
+
|
|
410
|
+
- *Named by table, nothing else.* A statement has no name of its own, and
|
|
411
|
+
giving it one (`as`) would be a second vocabulary inside the batch. A
|
|
412
|
+
reference names the table an earlier statement writes; if two earlier
|
|
413
|
+
statements write that table the lint refuses the reference, because nothing
|
|
414
|
+
could say which was meant.
|
|
415
|
+
- *Bound between statements, not compiled into one.* The statements stay the
|
|
416
|
+
separate, sequential statements they were: each compiles to an ordinary
|
|
417
|
+
parameter where the reference sits, and the engine fills that parameter from
|
|
418
|
+
the earlier statement's `RETURNING` row before running it. Folding the batch
|
|
419
|
+
into one statement of CTEs was tried and dropped — every CTE in a statement
|
|
420
|
+
sees the same snapshot, so a `$lookup` could no longer read a row an earlier
|
|
421
|
+
statement of the batch had inserted, which a batch can do today.
|
|
422
|
+
- *Exactly one row, or the batch fails.* A `$lookup` that finds nothing writes
|
|
423
|
+
NULL, because the row it reads may honestly not exist. A reference reads a
|
|
424
|
+
row the batch claims to have written; when the scope boundary or a WHERE
|
|
425
|
+
left that statement with no row, writing NULL (or skipping the child) would
|
|
426
|
+
commit half of what the author described. So it throws inside the
|
|
427
|
+
transaction and everything rolls back. What can be known without running —
|
|
428
|
+
no earlier writer, two of them, an `insertEach`, a do-nothing `onConflict` —
|
|
429
|
+
is refused at seed by `lintMutation`.
|
|
430
|
+
|
|
431
|
+
Scope is untouched by it: each statement is scoped on its own exactly as
|
|
432
|
+
before, and the value a reference carries comes from a row that statement was
|
|
433
|
+
already allowed to write.
|
|
434
|
+
|
|
379
435
|
**Derived context signatures** close the discoverability loop for both
|
|
380
436
|
kinds: a `$context` ref sits at a position whose column the schema types, so
|
|
381
437
|
"what do I pass this fingerprint" is computed from the stored def — never
|
|
@@ -831,9 +887,16 @@ byte for byte, so nothing a cache already holds under one has moved.
|
|
|
831
887
|
- Dynamic mutation generation — icebox, on purpose. If it ever returns, it
|
|
832
888
|
feeds the existing pipeline (author → validate → seed) behind an effect
|
|
833
889
|
contract and an approval step; nothing shipped today gets replaced.
|
|
834
|
-
- Mutation transactions with cross-statement refs
|
|
835
|
-
|
|
836
|
-
|
|
890
|
+
- ~~Mutation transactions with cross-statement refs~~ — built: see
|
|
891
|
+
**A batch that uses what it wrote (`$returned`)** under Mutations.
|
|
892
|
+
- A reply shape for writes. A mutation still answers with the rows its
|
|
893
|
+
statements returned (one row as that row, several as an array); choosing
|
|
894
|
+
what of a batch the caller is handed — a mapping over the writes — is not
|
|
895
|
+
built.
|
|
896
|
+
- Engine-derived paging (a cursor the engine mints from a query's sort).
|
|
897
|
+
Considered with the limit work and left out: an author can already page
|
|
898
|
+
with a `$context` bound in the filter, and a second way to do it was not
|
|
899
|
+
earned.
|
|
837
900
|
- MySQL / SQLite adapters — the interface is ready; implementation on demand.
|
|
838
901
|
- Multi-level field paths — addable in the resolver without DSL changes.
|
|
839
902
|
- ~~Intent-aware positive caching~~ — superseded: collisions did bite, and the
|
package/DOCS.md
CHANGED
|
@@ -165,6 +165,7 @@ type QueryEngineConfig = {
|
|
|
165
165
|
maxNestingDepth?: number; // default: 2
|
|
166
166
|
defaultLimit?: number; // default: 100
|
|
167
167
|
maxLimit?: number; // default: 1000
|
|
168
|
+
capAuthored?: boolean; // default: false — true holds seeded entries to maxLimit too
|
|
168
169
|
rejectCartesianProducts?: boolean; // default: true
|
|
169
170
|
warnUnindexedFilters?: boolean; // default: true
|
|
170
171
|
rejectUnindexedFilters?: boolean; // default: false
|
|
@@ -190,8 +191,10 @@ type QueryEngineConfig = {
|
|
|
190
191
|
throws `execution_error` if a rule names a column the database does not have.
|
|
191
192
|
- **`rows`** sizes the reactive rows cache — see
|
|
192
193
|
[Reactive reads](#reactive-reads-refresh).
|
|
193
|
-
- `defaultLimit` is applied when a DSL omits `limit
|
|
194
|
-
|
|
194
|
+
- `defaultLimit` is applied when a DSL omits `limit`. `maxLimit` clamps a larger
|
|
195
|
+
`limit` on a query nobody reviewed — one generated for a request. An entry
|
|
196
|
+
the host stored itself — **seeded** — runs with the `limit` it states, past
|
|
197
|
+
`maxLimit`; `capAuthored: true` clamps those as well. See [Limit](#limit).
|
|
195
198
|
- **`embed`** turns text into a vector for `semantic` filters, at parameter-
|
|
196
199
|
binding time. It's an injected provider (wire it to Signal) — *not* a database
|
|
197
200
|
concern, so it lives on the engine, not the adapter (the adapter only emits the
|
|
@@ -299,7 +302,8 @@ type QueryResponse = {
|
|
|
299
302
|
absent?: true; // optional, and not supplied this run (so: no type)
|
|
300
303
|
}>;
|
|
301
304
|
timing?: { agentMs?: number; executionMs: number; mappingMs?: number };
|
|
302
|
-
warnings?: string[]; // analyzer warnings,
|
|
305
|
+
warnings?: string[]; // analyzer warnings, and the line that says the
|
|
306
|
+
// engine's own limit may have cut a list (see Limit)
|
|
303
307
|
missingContext?: string[]; // present when REQUIRED keys were not supplied;
|
|
304
308
|
// optional keys never appear here
|
|
305
309
|
};
|
|
@@ -380,6 +384,53 @@ type Source = string | { as: string; query: Query }; // entity name or subque
|
|
|
380
384
|
type FieldRef = string | { field: string; as: string }; // bare column, or aliased output key
|
|
381
385
|
```
|
|
382
386
|
|
|
387
|
+
### Limit
|
|
388
|
+
|
|
389
|
+
Every query runs with a `LIMIT`. Whose number it is depends on who wrote the
|
|
390
|
+
query:
|
|
391
|
+
|
|
392
|
+
| The query | States no `limit` | States one |
|
|
393
|
+
|---|---|---|
|
|
394
|
+
| **seeded** — stored by the host itself | `defaultLimit` (100) | that number, whatever `maxLimit` is |
|
|
395
|
+
| **generated** for a request | `defaultLimit` (100) | that number, at most `maxLimit` (1000) |
|
|
396
|
+
|
|
397
|
+
`maxLimit` guards against a query nobody reviewed asking for the table. A
|
|
398
|
+
number a developer wrote into a seeded entry is a reviewed decision, and it is
|
|
399
|
+
honoured: a seeded entry that states `limit: 5000` reads up to 5000 rows. Set
|
|
400
|
+
`config.capAuthored: true` to hold seeded entries to `maxLimit` as well.
|
|
401
|
+
|
|
402
|
+
"Seeded" is not something a request can claim. It is read off the stored
|
|
403
|
+
entry: protected, and carrying no request hash. That is what `seedCache`
|
|
404
|
+
writes, and what a host writes when it stores an entry through the cache
|
|
405
|
+
backend itself (`cache.set(key, { …, protected: true })`). No request can
|
|
406
|
+
produce it: a generated entry always carries the hash of the request it was
|
|
407
|
+
born from, and protecting one afterwards does not remove it. One consequence:
|
|
408
|
+
a generated entry that was protected and is then seeded with the *identical*
|
|
409
|
+
definition keeps its hash — `seedCache` leaves a row that already matches
|
|
410
|
+
alone — and stays clamped, with the warning below, until it is unprotected or
|
|
411
|
+
deleted and seeded again.
|
|
412
|
+
|
|
413
|
+
`compile(dsl)` takes a bare query with no stored entry behind it, and clamps.
|
|
414
|
+
`test(dsl)` runs with a limit of 5, as it always has.
|
|
415
|
+
|
|
416
|
+
**When the engine's number may have ended a list, the reply says so.** If the
|
|
417
|
+
`LIMIT` was the engine's — the default a query was given, or the maximum it was
|
|
418
|
+
clamped to — and exactly that many rows came back for a list, `meta.warnings`
|
|
419
|
+
carries a line:
|
|
420
|
+
|
|
421
|
+
```
|
|
422
|
+
This list states no limit, so it ran with the engine's defaultLimit of 100, and exactly 100 rows came back — it may have been cut. State a `limit`.
|
|
423
|
+
This query asks for 5000 rows, the engine's maxLimit is 1000, and exactly 1000 came back — the list may have been cut. Raise `maxLimit` to read more.
|
|
424
|
+
```
|
|
425
|
+
|
|
426
|
+
It is "may": no extra row is fetched to find out, so a list of exactly that
|
|
427
|
+
many rows is told the same. A list that ended on a `limit` its author stated
|
|
428
|
+
is never told anything, and neither is a single-row answer (a shape that is
|
|
429
|
+
not an array). For a request that named a fingerprint the same line is also
|
|
430
|
+
written once per fingerprint to `console.warn` — a host that hands callers
|
|
431
|
+
`result` without `meta` would otherwise show it to no one. The fix is in the
|
|
432
|
+
entry: state the `limit` the list should have.
|
|
433
|
+
|
|
383
434
|
### Fields
|
|
384
435
|
|
|
385
436
|
Each entry is `entity.field`, or `{ field, as }` to set a distinct output key
|
|
@@ -789,8 +840,9 @@ as a `kind: 'mutation'` entry under a named fingerprint (normally
|
|
|
789
840
|
|
|
790
841
|
### The grammar
|
|
791
842
|
|
|
792
|
-
Five ops, closed and strict. Values are literals, `{ $context: "key" }`,
|
|
793
|
-
`{ $lookup }` scalar subquery
|
|
843
|
+
Five ops, closed and strict. Values are literals, `{ $context: "key" }`, a
|
|
844
|
+
`{ $lookup }` scalar subquery, or — in a batch — a `{ $returned }` read of the
|
|
845
|
+
row an earlier statement wrote (both below) — **never `$scope`** (identity is
|
|
794
846
|
engine-injected, unforgeable) and never a field path. `update`/`delete`
|
|
795
847
|
REQUIRE a `where` (a real vex `Filter`).
|
|
796
848
|
|
|
@@ -848,6 +900,11 @@ Two guarantees, both enforced before any SQL runs:
|
|
|
848
900
|
entry as the insert-plus-update it really is, so discovery visibility
|
|
849
901
|
follows the same rule.
|
|
850
902
|
|
|
903
|
+
`values` and `set` are separate positions, each binding its own parameter —
|
|
904
|
+
also where they set the same column. Every `$context` key in either is in the
|
|
905
|
+
entry's signature and must be sent on every call: the caller cannot know
|
|
906
|
+
which half will run.
|
|
907
|
+
|
|
851
908
|
### `$lookup` — a value read from another table
|
|
852
909
|
|
|
853
910
|
Any value position (insert `values`, update `set`, `onConflict.set`) may be a
|
|
@@ -869,12 +926,72 @@ a unique key: more than one matching row is a runtime error (deliberately —
|
|
|
869
926
|
silently picking an arbitrary row would be worse), and zero rows write NULL
|
|
870
927
|
(the column's constraints decide whether that lands).
|
|
871
928
|
|
|
929
|
+
### `$returned` — a value from the row an earlier statement wrote
|
|
930
|
+
|
|
931
|
+
In a batch, a later statement may read a column of the one row an earlier
|
|
932
|
+
statement wrote — how a child row gets the id the database generated for its
|
|
933
|
+
parent, in the same transaction:
|
|
934
|
+
|
|
935
|
+
```ts
|
|
936
|
+
[
|
|
937
|
+
{ op: 'insert', table: 'orders', values: { note: { $context: 'note' } } },
|
|
938
|
+
{ op: 'insertEach', table: 'order_lines', items: { $context: 'lines' },
|
|
939
|
+
values: { order_id: { $returned: 'orders.id' }, sku: { $item: 'sku' }, qty: { $item: 'qty' } } },
|
|
940
|
+
]
|
|
941
|
+
```
|
|
942
|
+
|
|
943
|
+
`{ $returned: 'table.column' }` names the earlier statement by the **table** it
|
|
944
|
+
writes — an `insert`, `update`, `upsert` or `delete` (a delete returns the row
|
|
945
|
+
it removed); `column` is a column of that table, including one the database
|
|
946
|
+
generated. It goes in any value position, the same ones `$lookup` does (the
|
|
947
|
+
`values`, `set` and `onConflict.set` of every op, an `upsert`'s
|
|
948
|
+
`columns`/`insert` among them), and nowhere else: not in a `where`, not inside
|
|
949
|
+
a `$lookup`, not as an `insertEach`'s `items`.
|
|
950
|
+
|
|
951
|
+
Where `$lookup` reads a row that may honestly not exist (and writes NULL when
|
|
952
|
+
it does not), `$returned` reads a row this batch claims to have written. So:
|
|
953
|
+
|
|
954
|
+
- **Exactly one row, or nothing is written.** If the statement it names wrote
|
|
955
|
+
no row — its WHERE matched nothing, the scope boundary kept the row back, an
|
|
956
|
+
`upsert` updated a row that is not there — or wrote several, the batch fails
|
|
957
|
+
with `execution_error` (`details: { returned, rows }`) and the transaction
|
|
958
|
+
rolls back, earlier statements included.
|
|
959
|
+
- **Refused before any request** by `lintMutation` (so by `seedCache`), and
|
|
960
|
+
again by the engine for a definition that was never seeded:
|
|
961
|
+
- no **earlier** statement of the batch writes that table (this covers a
|
|
962
|
+
reference pointing forwards, and one in a statement that is not in a batch);
|
|
963
|
+
- **two** earlier statements write it — a reference could not tell them apart;
|
|
964
|
+
- the statement it names is an `insertEach` (it writes a row per item);
|
|
965
|
+
- the statement it names is an `insert` whose `onConflict` has no `set` — on a
|
|
966
|
+
conflict that returns no row. Give it a `set` (the create-or-fetch idiom
|
|
967
|
+
above), and the reference holds on both paths.
|
|
968
|
+
- **An unknown column** is `invalid_dsl`, checked against the introspected
|
|
969
|
+
schema before any SQL runs.
|
|
970
|
+
|
|
971
|
+
The value never passes through the request: the engine binds it from the
|
|
972
|
+
returned row between statements, and a context key spelled `orders.id` is not
|
|
973
|
+
it. Scope is unchanged — every statement is scoped on its own, and a `set` rule
|
|
974
|
+
on a column still wins over anything the author put there, a reference
|
|
975
|
+
included.
|
|
976
|
+
|
|
977
|
+
Two things to know when using it:
|
|
978
|
+
|
|
979
|
+
- **It is for keys.** The value passes through the database driver on its way
|
|
980
|
+
from one statement to the next. An id, a number, text and json arrive as
|
|
981
|
+
they were; a timestamp read back as a JS `Date` keeps its milliseconds and
|
|
982
|
+
no more.
|
|
983
|
+
- **The reply is the rows of every statement, in order, as for any batch** —
|
|
984
|
+
and a reply of exactly one row is that row, not an array of one. A parent
|
|
985
|
+
with two children answers `[parent, child, child]`; the same batch with an
|
|
986
|
+
empty `items` answers `parent`. A caller that needs the new id reads the
|
|
987
|
+
first row in both forms.
|
|
988
|
+
|
|
872
989
|
### `insertEach` — one statement for a caller-sized list
|
|
873
990
|
|
|
874
991
|
`items` names a context key holding an array of objects; `{ $item: "key" }`
|
|
875
992
|
values read from the current element (cast to the column's type from the
|
|
876
|
-
schema); every other value — literal, `$context`, `$lookup`,
|
|
877
|
-
`$scope` — is constant across rows. Compiles to a single
|
|
993
|
+
schema); every other value — literal, `$context`, `$lookup`, `$returned`,
|
|
994
|
+
engine-injected `$scope` — is constant across rows. Compiles to a single
|
|
878
995
|
`INSERT … SELECT … FROM jsonb_array_elements($items)`, so "one row per ticked
|
|
879
996
|
weekday" is one authored statement, not a code loop:
|
|
880
997
|
|
|
@@ -935,14 +1052,26 @@ for reads / `effect` = `{ op, table, columns }` for writes), and the
|
|
|
935
1052
|
collectMutationContext(def, schema); // { id: { type: 'string', column: 'tasks.id', note: 'upsert key — …' }, … }
|
|
936
1053
|
collectQueryContext(dsl, schema); // same idea over a query's filter/compute/subqueries
|
|
937
1054
|
mutationEffect(def); // [{ op: 'update', table: 'tasks', columns: ['done'] }]
|
|
1055
|
+
mutationMisfits(def, schema); // [] when it fits — else why not: ['"tasks.id" in the WHERE is not a column of "tasks".']
|
|
938
1056
|
```
|
|
939
1057
|
|
|
1058
|
+
`mutationMisfits` asks the column gates every write passes before it runs —
|
|
1059
|
+
table, written columns, WHERE, `$lookup`, conflict target, `$returned` —
|
|
1060
|
+
without running anything, an upsert as both statements it can become. Scope is
|
|
1061
|
+
not applied.
|
|
1062
|
+
moss's `migrateTables` asks it of every write entry before a migration run may
|
|
1063
|
+
commit.
|
|
1064
|
+
|
|
940
1065
|
### Authoring lint
|
|
941
1066
|
|
|
942
1067
|
`lintMutation(def)` returns a list of issues (empty when clean). It flags an
|
|
943
1068
|
`update`/`delete` whose WHERE binds no `$context` — such a write is not
|
|
944
|
-
caller-bounded (its only row limit is the scope policy) —
|
|
945
|
-
containing an `optional` condition
|
|
1069
|
+
caller-bounded (its only row limit is the scope policy) — any statement
|
|
1070
|
+
containing an `optional` condition, and a `$returned` whose table is not
|
|
1071
|
+
written by exactly one earlier statement, or is written by an `insertEach` or
|
|
1072
|
+
a do-nothing `insert` (the four cases under
|
|
1073
|
+
[`$returned`](#returned--a-value-from-the-row-an-earlier-statement-wrote)).
|
|
1074
|
+
How many rows that statement returns is known only when it runs. `seedCache` runs it on every mutation entry
|
|
946
1075
|
and throws on an issue, so an unkeyed write never ships: loud at boot, never at
|
|
947
1076
|
runtime.
|
|
948
1077
|
|
|
@@ -1119,6 +1248,9 @@ await seedCache(engine.cache, [ordersRecent, taskSetDone]);
|
|
|
1119
1248
|
(`{ $ref: '$.result' }`) is stored, so a replay never falls through to
|
|
1120
1249
|
`mapToShape`.
|
|
1121
1250
|
- A mutation is linted (`lintMutation`); a failing statement throws.
|
|
1251
|
+
- A read's `limit` is the author's: a seeded entry runs with the number it
|
|
1252
|
+
states even past `maxLimit` (see [Limit](#limit)). One that states none gets
|
|
1253
|
+
`defaultLimit` like any other query — state a `limit` on every list.
|
|
1122
1254
|
- It is idempotent against a durable cache: a stored row that already matches
|
|
1123
1255
|
its authored definition is left alone; anything else at that fingerprint
|
|
1124
1256
|
converges to the authored definition, protected.
|
|
@@ -1337,7 +1469,11 @@ other than `GET`/`POST`/`PATCH`/`DELETE` answers 405.
|
|
|
1337
1469
|
`fingerprint_protected`, a write with missing context → 400
|
|
1338
1470
|
`missing_context` with the full derived signature in `details.expected`,
|
|
1339
1471
|
other `VexError`s (including `locked` and `scope_denied`) → 400 with
|
|
1340
|
-
`{ error, message, details }`. A
|
|
1472
|
+
`{ error, message, details }`. A body that is not JSON → 400
|
|
1473
|
+
`invalid_request` (`Body must be JSON`): the Hono adapter parses the body
|
|
1474
|
+
itself and never reads the content type, so JSON sent as `text/plain` is
|
|
1475
|
+
read; the Express adapter is handed `req.body` as the host's parser left
|
|
1476
|
+
it. A reach the host cannot serve for this
|
|
1341
1477
|
principal → 403 `scope_denied`. 500s are the host's faults: `missing_scope`
|
|
1342
1478
|
(key names stay in the log), an entry demanding a reach with no
|
|
1343
1479
|
`policyForReach`, a mutation fingerprint on a handler with no `mutations`
|
|
@@ -1346,7 +1482,9 @@ other than `GET`/`POST`/`PATCH`/`DELETE` answers 405.
|
|
|
1346
1482
|
**`DELETE`** with `{ fingerprint }` → evict (a protected entry, read or mutation,
|
|
1347
1483
|
is refused with 409 `fingerprint_protected`). Both are refused with 403 `locked` on a
|
|
1348
1484
|
locked endpoint and answer 404 `cache_miss` for an unknown fingerprint.
|
|
1349
|
-
Fingerprints ride the body — names contain
|
|
1485
|
+
Fingerprints ride the body — names contain `/` — and a body that is not
|
|
1486
|
+
JSON, or carries no fingerprint, is a 400 `invalid_request` before anything
|
|
1487
|
+
else is checked.
|
|
1350
1488
|
|
|
1351
1489
|
Under the hood all of these call the framework-agnostic `handleDiscovery` /
|
|
1352
1490
|
`handleQuery` / `handleFingerprintPatch` / `handleFingerprintDelete` from
|
|
@@ -1428,6 +1566,12 @@ try {
|
|
|
1428
1566
|
}
|
|
1429
1567
|
```
|
|
1430
1568
|
|
|
1569
|
+
Under `require`, each entry point carries its own copy of the class. An error the
|
|
1570
|
+
engine itself throws is an `instanceof` the `VexError` exported beside it; one made
|
|
1571
|
+
by a hook from `@niscorp/vex/agent` (`agent_failed`, `unsatisfiable`) is not — it
|
|
1572
|
+
has the same `name`, `code` and `message`, so read `err.code` there. The framework
|
|
1573
|
+
handlers recognise both.
|
|
1574
|
+
|
|
1431
1575
|
`VexError.code` is one of:
|
|
1432
1576
|
|
|
1433
1577
|
| Code | Meaning |
|
|
@@ -1437,7 +1581,7 @@ try {
|
|
|
1437
1581
|
| `missing_scope` | A `$scope` slot the host did not fill. The host's fault, not the request's: the HTTP handler answers 500 and keeps the key names in its log |
|
|
1438
1582
|
| `scope_denied` | An entity is denied by the scope policy (`VexScopeError`) |
|
|
1439
1583
|
| `missing_context` | A required context value was absent. Reads surface it softly via `meta.missingContext`; a WRITE hard-400s with the full derived signature in `details.expected` |
|
|
1440
|
-
| `execution_error` | Schema not loaded, or a database/runtime failure |
|
|
1584
|
+
| `execution_error` | Schema not loaded, or a database/runtime failure. Also a batch whose `$returned` found no row, or several, where it needs exactly one — `details: { returned, rows }`, nothing written |
|
|
1441
1585
|
| `agent_failed` | Generation was needed and there is no `generateDsl` hook, or the reference agent's run failed |
|
|
1442
1586
|
| `cache_miss` | A fingerprint was sent alone and nothing is stored under it |
|
|
1443
1587
|
| `fingerprint_protected` | A protected entry was asked to change: a named-slot request that no longer matches its stored one |
|