@niscorp/vex 0.4.4 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +90 -0
- package/DESIGN.md +72 -45
- package/DOCS.md +219 -21
- package/README.md +18 -0
- package/dist/adapters/express/index.cjs +606 -173
- package/dist/adapters/express/index.cjs.map +1 -1
- package/dist/adapters/express/index.d.cts +6 -2
- package/dist/adapters/express/index.d.ts +6 -2
- package/dist/adapters/express/index.js +16 -3
- package/dist/adapters/express/index.js.map +1 -1
- package/dist/adapters/hono/index.cjs +597 -173
- package/dist/adapters/hono/index.cjs.map +1 -1
- package/dist/adapters/hono/index.d.cts +5 -2
- package/dist/adapters/hono/index.d.ts +5 -2
- package/dist/adapters/hono/index.js +7 -3
- package/dist/adapters/hono/index.js.map +1 -1
- package/dist/agent/index.cjs +12 -12
- package/dist/agent/index.cjs.map +1 -1
- package/dist/agent/index.d.cts +1 -1
- package/dist/agent/index.d.ts +1 -1
- package/dist/agent/index.js +12 -12
- package/dist/agent/index.js.map +1 -1
- package/dist/{chunk-W5MYFJKT.js → chunk-F7OXXJIH.js} +562 -192
- package/dist/chunk-F7OXXJIH.js.map +1 -0
- package/dist/{chunk-F7RDSWYF.js → chunk-VHNCSQPO.js} +2 -2
- package/dist/{chunk-F7RDSWYF.js.map → chunk-VHNCSQPO.js.map} +1 -1
- package/dist/{handler-CoJxInw-.d.ts → handler-DdfaWN_J.d.ts} +7 -1
- package/dist/{handler-CsFbanZA.d.cts → handler-kvZkxwaN.d.cts} +7 -1
- package/dist/index.cjs +667 -259
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +5 -3
- package/dist/index.d.ts +5 -3
- package/dist/index.js +290 -248
- package/dist/index.js.map +1 -1
- package/dist/migrations/index.cjs.map +1 -1
- package/dist/migrations/index.js +1 -1
- package/dist/{types-B2HEmyNq.d.cts → types-BxkRxTTx.d.cts} +38 -2
- package/dist/{types-DC68ZUuq.d.ts → types-DgUZNKii.d.ts} +38 -2
- package/package.json +8 -8
- package/dist/chunk-W5MYFJKT.js.map +0 -1
package/DOCS.md
CHANGED
|
@@ -15,6 +15,7 @@ In-depth usage guide for `@niscorp/vex`. For the high-level pitch see
|
|
|
15
15
|
- [The DSL](#the-dsl)
|
|
16
16
|
- [Scope policies](#scope-policies)
|
|
17
17
|
- [Grants and behaviors](#grants-and-behaviors)
|
|
18
|
+
- [Upgraded columns](#upgraded-columns)
|
|
18
19
|
- [Mutations](#mutations)
|
|
19
20
|
- [Caching](#caching)
|
|
20
21
|
- [Wiring the LLM agents](#wiring-the-llm-agents)
|
|
@@ -161,6 +162,8 @@ type QueryEngineConfig = {
|
|
|
161
162
|
mapToShape?: (rows, shape, intent?) => Promise<{ ir: CompiledIr; config: unknown; transformed: JsonValue }>;
|
|
162
163
|
embed?: (text, dimensions?) => Promise<number[]>; // text→vector for semantic filters
|
|
163
164
|
behaviors?: ScopeBehaviors; // which columns a scope key is bound to (see below)
|
|
165
|
+
upgradedColumns?: UpgradedColumns; // JSON columns a read returns upgraded (see below)
|
|
166
|
+
upgrader?: ColumnUpgrader; // strata's Upgrader; required with upgradedColumns
|
|
164
167
|
rows?: LiveRowsConfig; // the rows cache behind `refresh: 'reactive'`
|
|
165
168
|
config?: {
|
|
166
169
|
maxNestingDepth?: number; // default: 2
|
|
@@ -182,6 +185,21 @@ type QueryEngineConfig = {
|
|
|
182
185
|
- `createPostgresAdapter({ pool, exclude })` names tables the adapter never
|
|
183
186
|
reports: `introspect()` leaves them out. It is for tables the host owns that
|
|
184
187
|
are not application data, such as a migration journal.
|
|
188
|
+
- `createPostgresAdapter({ pool, schema })` makes `introspect()` list the tables
|
|
189
|
+
of that Postgres schema (`public` without it). Queries and mutations name
|
|
190
|
+
their tables without a schema, so they resolve through the connection's
|
|
191
|
+
`search_path`: with `schema`, set the `search_path` to that schema too, or
|
|
192
|
+
reads and writes reach the tables of another schema.
|
|
193
|
+
- `createPostgresAdapter({ pool, schema, qualifyTables: true })` writes the
|
|
194
|
+
schema's name on every table instead (`FROM "app".tasks`), in reads and in
|
|
195
|
+
mutations, so the `search_path` does not matter. `introspect()` then returns
|
|
196
|
+
the name as `namespace` on the `DatabaseSchema`, and the mutation compiler
|
|
197
|
+
reads it there. It is not part of the schema fingerprint, so cached entries
|
|
198
|
+
stay fresh when the option is turned on. Without the option, tables are named
|
|
199
|
+
bare, as above: a host that points the `search_path` at a schema per tenant
|
|
200
|
+
keeps doing so. With the option, `adapter.introspect({ schema: 'other' })`
|
|
201
|
+
lists `other`'s tables but still names them, and reports `namespace`, with the
|
|
202
|
+
adapter's own `schema`.
|
|
185
203
|
- `config.statementTimeoutMs` is enforced by the database (`statement_timeout`),
|
|
186
204
|
set up at `introspect()`. A database that cannot enforce it (PGlite) runs
|
|
187
205
|
reads unbounded and the engine warns each time `introspect()` runs. A `PgPool`
|
|
@@ -201,6 +219,7 @@ type QueryEngineConfig = {
|
|
|
201
219
|
`scopeBindings(behaviors, tables, scope)` returns that list for the given
|
|
202
220
|
tables: one entry per table, column and key, sorted, and only for keys
|
|
203
221
|
`scope` holds a value for.
|
|
222
|
+
- **`upgradedColumns`** and **`upgrader`**: see [Upgraded columns](#upgraded-columns).
|
|
204
223
|
- **`rows`** sizes the reactive rows cache — see
|
|
205
224
|
[Reactive reads](#reactive-reads-refresh).
|
|
206
225
|
- `defaultLimit` is applied when a DSL omits `limit`. `maxLimit` clamps a larger
|
|
@@ -221,6 +240,7 @@ type ExecuteOptions = {
|
|
|
221
240
|
entities?: string[]; // restrict generation to this entity subset for this call
|
|
222
241
|
locked?: boolean; // replay-only: unknown/changed fingerprints throw `locked` instead of generating
|
|
223
242
|
scopePolicy?: ScopePolicy; // per-request policy; overrides the engine's configured one for this call
|
|
243
|
+
policyForReach?: (reach: string) => ScopePolicy | undefined | Promise<ScopePolicy | undefined>; // the same principal's policy at a reach a replayed entry declares
|
|
224
244
|
signal?: AbortSignal; // with onChange: follow a `refresh: 'reactive'` entry until it aborts
|
|
225
245
|
onChange?: (response: QueryResponse) => void; // every later answer that differs from the last
|
|
226
246
|
};
|
|
@@ -253,6 +273,8 @@ type QueryRequest = {
|
|
|
253
273
|
```
|
|
254
274
|
|
|
255
275
|
The request is Zod-validated; `execute` throws `invalid_request` on failure.
|
|
276
|
+
A `fingerprint` or `intent` sent as `null` is read as left out. A `null` in
|
|
277
|
+
`context` is a value and stays.
|
|
256
278
|
It must carry a `fingerprint`, a `shape`, or both — the three postures:
|
|
257
279
|
|
|
258
280
|
1. **Explore** — `{ intent, shape, context }`: generate, cache, mint a
|
|
@@ -357,6 +379,7 @@ type QueryEngine = {
|
|
|
357
379
|
getDslSchema: () => object;
|
|
358
380
|
getSchema: () => DatabaseSchema | undefined;
|
|
359
381
|
cache: CacheBackend;
|
|
382
|
+
readonly scopePolicy?: ScopePolicy;
|
|
360
383
|
invalidate: (tables: readonly string[]) => void;
|
|
361
384
|
rows: { stats: () => LiveRowsStats; stop: () => void };
|
|
362
385
|
};
|
|
@@ -380,6 +403,9 @@ type QueryEngine = {
|
|
|
380
403
|
feed an LLM so it learns the DSL.
|
|
381
404
|
- **`getSchema()`** — the last introspected `DatabaseSchema`, or `undefined`.
|
|
382
405
|
- **`cache`** — the live `CacheBackend` for inspection or manual eviction.
|
|
406
|
+
- **`scopePolicy`** — the policy given to `createQueryEngine` (`scope`), absent
|
|
407
|
+
when none was. A call that passes no policy of its own runs under it, and
|
|
408
|
+
discovery lists under it when the handler has none.
|
|
383
409
|
- **`invalidate(tables)`** — these tables were written: every reactive read
|
|
384
410
|
over any of them is refetched (once per burst). The vex handler calls it after
|
|
385
411
|
every committed mutation; a host calls it for writes vex did not make.
|
|
@@ -465,7 +491,13 @@ Each entry is `entity.field`, or `{ field, as }` to set a distinct output key
|
|
|
465
491
|
without a compute). There is no `SELECT *`. `fields` is **optional** — omit it
|
|
466
492
|
for an aggregate-only query (e.g. a bare `COUNT(*)`); a query must have at least
|
|
467
493
|
one of `fields` / `aggregate`. Every entity referenced anywhere must also appear
|
|
468
|
-
in `from`.
|
|
494
|
+
in `from`. A table with a foreign key to itself can be listed twice and is
|
|
495
|
+
joined to itself along it; every path names the last copy, and scope rules
|
|
496
|
+
apply to both.
|
|
497
|
+
|
|
498
|
+
A query is refused when an optional key is `null`: `"limit": null` is an
|
|
499
|
+
error, not a missing limit. The query agent's tools read a top-level `null`
|
|
500
|
+
argument as left out.
|
|
469
501
|
|
|
470
502
|
```json
|
|
471
503
|
{ "from": ["order", "customer"],
|
|
@@ -653,8 +685,10 @@ const engine = createQueryEngine({
|
|
|
653
685
|
});
|
|
654
686
|
```
|
|
655
687
|
|
|
656
|
-
At execution, the `query` ref's text is read from context
|
|
657
|
-
|
|
688
|
+
At execution, the `query` ref's text is read from context (`$context`) or from
|
|
689
|
+
the scope values the host supplies (`$scope`), embedded, and bound as the vector
|
|
690
|
+
parameter. A missing `$scope` value is refused with `missing_scope`, as any
|
|
691
|
+
other scope slot is. (Never select a vector column directly in `fields` — it
|
|
658
692
|
is for `semantic` filters only.)
|
|
659
693
|
|
|
660
694
|
---
|
|
@@ -838,6 +872,16 @@ holding two additive rather than a conflict to arbitrate.
|
|
|
838
872
|
practice. A phase only one role grants keeps that role's rules intact, which is
|
|
839
873
|
what carries a member's identity stamp onto a write no other role grants.
|
|
840
874
|
|
|
875
|
+
Two rule sets with the same number of matches are not combined: the role listed
|
|
876
|
+
first wins, and the principal reads only the rows its rules allow. The merged
|
|
877
|
+
policy is `default: 'deny'`, so a role's `default: 'allow'` is not carried over.
|
|
878
|
+
Each entity is read as the phases it grants before the rule sets are compared: a
|
|
879
|
+
`public` entity grants all four phases with no rule; a `write` list is added to
|
|
880
|
+
insert, update and delete (its `match` rules only, for delete), as
|
|
881
|
+
`createScopePolicy` does; a `deny` entity grants nothing, so another role's grant
|
|
882
|
+
stands. `{ write: [tenant], update: [] }` therefore merges to
|
|
883
|
+
`update: [tenant]`.
|
|
884
|
+
|
|
841
885
|
### A read that must stay narrow
|
|
842
886
|
|
|
843
887
|
The merge above widens. For a read whose answer legitimately widens with the
|
|
@@ -857,11 +901,103 @@ only narrow rows, never widen phases. A caller with no verb for the table is
|
|
|
857
901
|
still refused. A mutation entry takes `reach` the same way (`SeedMutation.reach`),
|
|
858
902
|
and the write is compiled at that profile whoever asks.
|
|
859
903
|
|
|
860
|
-
The host supplies `policyForReach(reach)` (hono: `getPolicyForReach(c, reach)`),
|
|
861
|
-
because vex
|
|
862
|
-
entry declaring a reach no host can resolve is a 500, and a host
|
|
863
|
-
`undefined` is a 403. Falling back to the caller's own reach would be
|
|
864
|
-
and silence is the whole risk — the read would answer, with too much.
|
|
904
|
+
The host supplies `policyForReach(reach)` (hono: `getPolicyForReach(c, reach)`; express: `getPolicyForReach(req, reach)`),
|
|
905
|
+
because vex knows neither grants nor behaviors. **The handler fails closed both
|
|
906
|
+
ways:** an entry declaring a reach no host can resolve is a 500, and a host
|
|
907
|
+
returning `undefined` is a 403. Falling back to the caller's own reach would be
|
|
908
|
+
silent, and silence is the whole risk — the read would answer, with too much.
|
|
909
|
+
|
|
910
|
+
That is `handleQuery` and the hono and express adapters. `engine.execute`
|
|
911
|
+
serves a replayed entry's `reach` through `ExecuteOptions.policyForReach`: the
|
|
912
|
+
run uses the policy it returns, and `undefined` throws `scope_denied`. Without
|
|
913
|
+
`policyForReach` it runs under the policy it is handed (`scopePolicy`, else the
|
|
914
|
+
engine's own), whatever reach the entry declares, and logs one warning per
|
|
915
|
+
fingerprint saying so.
|
|
916
|
+
|
|
917
|
+
## Upgraded columns
|
|
918
|
+
|
|
919
|
+
A JSON column may hold values written at an older grammar, with the stamp each
|
|
920
|
+
was written at in another JSON column of the same row: a strata document (rule
|
|
921
|
+
19 of the nisc rulebook). `upgradedColumns` names those columns, keyed by
|
|
922
|
+
table and column, and every read that returns one hands it to the mapping, and
|
|
923
|
+
to the caller, upgraded to the grammars of the engine's `upgrader`:
|
|
924
|
+
|
|
925
|
+
```typescript
|
|
926
|
+
import { createUpgrader } from '@niscorp/strata';
|
|
927
|
+
import { evaluate } from '@niscorp/prism';
|
|
928
|
+
|
|
929
|
+
const engine = createQueryEngine({
|
|
930
|
+
adapter,
|
|
931
|
+
upgrader: await createUpgrader(grammars, { transform: evaluate }),
|
|
932
|
+
upgradedColumns: {
|
|
933
|
+
// The kind is fixed for the column.
|
|
934
|
+
answers: { plan: { stamp: 'grammar', kind: 'acme.answers/plan', unreadable: 'omit' } },
|
|
935
|
+
// The kind is `<grammar>/<the row's value of column>`: `acme.notes/list` for a row whose `kind` is `list`.
|
|
936
|
+
notes: { body: { stamp: 'grammar', kind: { grammar: 'acme.notes', column: 'kind' } } },
|
|
937
|
+
},
|
|
938
|
+
});
|
|
939
|
+
```
|
|
940
|
+
|
|
941
|
+
`stamp` and `kind.column` name columns of the same table. `UpgradedColumnsSchema`
|
|
942
|
+
is the setting's schema, and `ColumnUpgrader` the part of strata's `Upgrader`
|
|
943
|
+
the engine uses (`upgrade`, `locate`, `stamp`).
|
|
944
|
+
|
|
945
|
+
**Checked at start.** `createQueryEngine` throws `invalid_request` when
|
|
946
|
+
`upgradedColumns` do not parse, when there is no `upgrader`, or when a fixed
|
|
947
|
+
`kind` (or the `grammar` of a kind read from the row) is not one the
|
|
948
|
+
upgrader's grammars declare. `introspect()` throws `execution_error` when a
|
|
949
|
+
column, its stamp or its kind column is not in the database, or when the
|
|
950
|
+
column or its stamp is not `json`/`jsonb`.
|
|
951
|
+
|
|
952
|
+
**What a read must say.** Every query is checked as it is compiled: a replay,
|
|
953
|
+
`engine.compile`, `engine.test` and the generator's own test runs. A seeded
|
|
954
|
+
entry is therefore checked when it is first replayed, unless its app compiles
|
|
955
|
+
it before (moss does, at boot).
|
|
956
|
+
|
|
957
|
+
- A read that returns an upgraded column returns its stamp column too, and for
|
|
958
|
+
a kind read from the row, that column. Otherwise: `invalid_dsl`, naming the
|
|
959
|
+
field to add.
|
|
960
|
+
- An upgraded column is returned only as one of the read's own fields
|
|
961
|
+
(`table.column` or `{ field, as }`). Named anywhere else (a subquery's fields
|
|
962
|
+
or computes, a `compute`, an `aggregate`, a `groupBy`), the read is refused,
|
|
963
|
+
because the value would arrive un-upgraded. A filter or a sort on it, a
|
|
964
|
+
subquery's included, is allowed and compares the stored value.
|
|
965
|
+
- Two values returned under one key (two fields, or a field and a compute or
|
|
966
|
+
aggregate of that name) are refused when the key is an upgraded column, its
|
|
967
|
+
stamp or its kind: give each its own key with `as`.
|
|
968
|
+
|
|
969
|
+
**What a read does.** After the rows come back and before the mapping, each
|
|
970
|
+
row's value is upgraded from the stamp in that row, and every key the stamp was
|
|
971
|
+
returned under is set to the stamp the upgrade gives back, so a caller that
|
|
972
|
+
upgrades the value again changes nothing. A null value is left as it is. This
|
|
973
|
+
holds for a replay, a generated read, every answer a reactive read delivers,
|
|
974
|
+
`engine.test` and the generator's test runs. The rows a reactive read shares
|
|
975
|
+
between callers are not changed. A write's returned rows are as stored.
|
|
976
|
+
|
|
977
|
+
**A row that does not upgrade.** A value does not upgrade when its stamp is
|
|
978
|
+
ahead of the upgrader's grammars (strata's `TOO_NEW`), a migration step fails
|
|
979
|
+
on it (`STEP_FAILED`), its kind is one no grammar declares (`UNKNOWN_KIND`),
|
|
980
|
+
its stamp is neither null nor an object of grammar names to whole numbers, or
|
|
981
|
+
its kind column is not text. The column's `unreadable` says what the read does
|
|
982
|
+
then (`UNREADABLE_MODES`):
|
|
983
|
+
|
|
984
|
+
- `'refuse'` (the default): the whole read fails. Strata's errors are answered
|
|
985
|
+
by `handleQuery` as a 500 with strata's code; a stamp or kind column that is
|
|
986
|
+
not one is vex's `execution_error`, a 400. A reactive read's refetch that
|
|
987
|
+
fails reports `rows.error`, and its followers keep their last answer.
|
|
988
|
+
- `'omit'`: the row is left out, and `meta.warnings` says which of the
|
|
989
|
+
database's rows (counting from 0) and why. A list can come back shorter than
|
|
990
|
+
its `limit`; the "may have been cut" warning counts the rows the database
|
|
991
|
+
returned. A read whose shape is an object answers the first row left, so a
|
|
992
|
+
row left out makes `$.result` the next one: use `'null'` or `'refuse'` where
|
|
993
|
+
that would mislead.
|
|
994
|
+
- `'null'`: the row stays with that value null, and `meta.warnings` says so.
|
|
995
|
+
This column does not change the stamp; another upgraded column of the row
|
|
996
|
+
that shares it may.
|
|
997
|
+
|
|
998
|
+
A reactive read delivers again when only its warnings change.
|
|
999
|
+
|
|
1000
|
+
An engine given no `upgradedColumns` returns every column as stored.
|
|
865
1001
|
|
|
866
1002
|
## Mutations
|
|
867
1003
|
|
|
@@ -894,7 +1030,16 @@ REQUIRE a `where` (a real vex `Filter`).
|
|
|
894
1030
|
```
|
|
895
1031
|
|
|
896
1032
|
An `upsert` updates the row when the call's context carries its `key`, and
|
|
897
|
-
inserts when the key is absent or an empty string.
|
|
1033
|
+
inserts when the key is absent or an empty string. A `null` key inserts where
|
|
1034
|
+
the database makes the key: the introspected key column has a default (such as
|
|
1035
|
+
`gen_random_uuid()` or a sequence) or is an identity column, and the entry does
|
|
1036
|
+
not write the key itself in `columns` or `insert`. Like an absent key, it is
|
|
1037
|
+
then refused with `scope_denied` for a caller who may update but not insert.
|
|
1038
|
+
Anywhere else a `null` key is an update of no row and answers `[]`, since an
|
|
1039
|
+
insert there would write NULL into the key. A write that matched no row answers
|
|
1040
|
+
`[]`: check the length. Introspection gives an identity column the
|
|
1041
|
+
`defaultValue` `GENERATED ALWAYS AS IDENTITY` or `GENERATED BY DEFAULT AS
|
|
1042
|
+
IDENTITY`.
|
|
898
1043
|
|
|
899
1044
|
An array of ops is a batch and runs in ONE transaction (the client must
|
|
900
1045
|
expose `transaction`; a batch on a non-transactional client is refused).
|
|
@@ -1076,6 +1221,34 @@ handleQuery({ engine, locked: true, mutations: { client: db, policy } }, body, s
|
|
|
1076
1221
|
double). `locked` does not affect writes: they are replay-only under every
|
|
1077
1222
|
posture. Direct library use: `executeMutation(client, def, { context, scope, policy, schema })`.
|
|
1078
1223
|
|
|
1224
|
+
**A write keeps its rows inside the caller's update rules.** When an
|
|
1225
|
+
`update`, the update half of an `upsert`, or an insert's `onConflict … set`
|
|
1226
|
+
writes a column one of the caller's update `match` rules names, the new value
|
|
1227
|
+
must match that rule too. An update that would move a row out is refused before
|
|
1228
|
+
anything is written: 400 `scope_denied`, naming the rule and the column (`the
|
|
1229
|
+
new "tasks.project_id" is not one of the scope's "projectIds"`); in a batch,
|
|
1230
|
+
the whole batch is refused. An `onConflict` update leaves such a row as it is
|
|
1231
|
+
and answers no row for it, as for a conflicting row outside the rules; a host
|
|
1232
|
+
with `default: 'allow'` and only update rules on a table gets this check on
|
|
1233
|
+
its `onConflict … set` too. A column
|
|
1234
|
+
a `set` rule stamps is the policy's own value and is not checked. To let a write
|
|
1235
|
+
move a row out of its own rules — handing a task over by setting `owner_id`
|
|
1236
|
+
under an `owner_id = me` rule — set `mutations: { …, checkMovedRows: false }`
|
|
1237
|
+
(`executeWrites(client, def, { …, checkMovedRows: false })` at the library
|
|
1238
|
+
level).
|
|
1239
|
+
|
|
1240
|
+
**What a write answers.** By default a write answers every row it wrote,
|
|
1241
|
+
whole (`RETURNING *`), whatever the caller's read rules say: renaming a task
|
|
1242
|
+
the update rules reach answers the whole row, columns and all, even when the
|
|
1243
|
+
read rules would not show it. `mutations: { …, answerReadableRows: true }`
|
|
1244
|
+
answers only the rows the caller's read rules show; the write itself is
|
|
1245
|
+
unchanged. An `update`, a `delete`, and an insert with `onConflict … set` are
|
|
1246
|
+
filtered; a plain insert answers the row it created. A read rule whose scope
|
|
1247
|
+
value the host left out hides the row; it does not refuse the write. Off by
|
|
1248
|
+
default, because some setups depend on the whole answer: a sign-in entry that
|
|
1249
|
+
deletes a one-time link and reads who signs in from the deleted row, run by a
|
|
1250
|
+
role that has no read grant on the link table, would answer `[]` with it on.
|
|
1251
|
+
|
|
1079
1252
|
A host that resolves a policy **per principal** (e.g. compiled from an ACL
|
|
1080
1253
|
layer) passes it per request: `VexHandlerConfig.scopePolicy` overrides the
|
|
1081
1254
|
engine's configured read policy for that request (`ExecuteOptions.scopePolicy`
|
|
@@ -1128,9 +1301,10 @@ successful mutation replay, AFTER the commit:
|
|
|
1128
1301
|
mutations: {
|
|
1129
1302
|
client, policy,
|
|
1130
1303
|
onWrite: ({ fingerprint, writes, scope }) => { … },
|
|
1131
|
-
// writes: [{ table, op, rows }] — per statement, resolved post-desugar
|
|
1304
|
+
// writes: [{ table, op, rows, readable? }] — per statement, resolved post-desugar
|
|
1132
1305
|
// (an upsert reports the branch it took; insertEach reports 'insert'),
|
|
1133
|
-
// rows as the database returned them
|
|
1306
|
+
// rows as the database returned them; readable, under answerReadableRows,
|
|
1307
|
+
// the ones the caller's read rules show. scope: what the request could
|
|
1134
1308
|
// not forge.
|
|
1135
1309
|
}
|
|
1136
1310
|
```
|
|
@@ -1178,7 +1352,14 @@ are not asking the same thing).
|
|
|
1178
1352
|
|
|
1179
1353
|
### Lifetime and protection
|
|
1180
1354
|
|
|
1181
|
-
- A replay stamps `lastUsedAt` (at most once a minute per entry)
|
|
1355
|
+
- A replay stamps `lastUsedAt` (at most once a minute per entry). It calls the
|
|
1356
|
+
backend's optional `touch(key, lastUsedAt)`, which writes that field alone, so
|
|
1357
|
+
a `PATCH` or a reseed landing between the replay's read and its stamp is kept.
|
|
1358
|
+
The memory, Postgres and tiered caches have it (the tiered one when both its
|
|
1359
|
+
levels do). A backend without it is stamped through `set` with the whole entry
|
|
1360
|
+
the replay read, and keeps the race. A wrapper that spreads a built-in cache
|
|
1361
|
+
and overrides `set` inherits that cache's `touch`, so its stamps go through
|
|
1362
|
+
`touch` and never reach the wrapper's `set`.
|
|
1182
1363
|
`sweepCache(cache, { maxIdleMs })` evicts entries idle past the limit,
|
|
1183
1364
|
skipping protected ones, and returns how many it evicted. Alive = used.
|
|
1184
1365
|
Nothing in vex calls it: a host runs it at startup or on a timer.
|
|
@@ -1524,9 +1705,10 @@ The rest of `VexHonoConfig`, all optional:
|
|
|
1524
1705
|
| Option | Purpose |
|
|
1525
1706
|
|--------|---------|
|
|
1526
1707
|
| `locked` | Replay-only endpoint: no generation, no fingerprint management |
|
|
1708
|
+
| `manageFingerprints` | `false` refuses `PATCH` and `DELETE` with 403 `locked` on an unlocked endpoint, which still generates. Default on: an unlocked endpoint with it on logs one warning as it is mounted |
|
|
1527
1709
|
| `getPolicy(c)` | Per-request `ScopePolicy`; governs reads, mutation replay and discovery on this endpoint. Returning `undefined` leaves the engine's policy and `mutations.policy` in force |
|
|
1528
1710
|
| `getPolicyForReach(c, reach)` | The same principal's policy at a reach an entry demands |
|
|
1529
|
-
| `mutations` | `{ client, policy?, onWrite? }` — enables mutation replay; `policy` is the fallback when `getPolicy` is absent. A request that gets a policy from neither replays no write: a mutation fingerprint answers 500 |
|
|
1711
|
+
| `mutations` | `{ client, policy?, onWrite?, checkMovedRows?, answerReadableRows? }` — enables mutation replay; `policy` is the fallback when `getPolicy` is absent. A request that gets a policy from neither replays no write: a mutation fingerprint answers 500 |
|
|
1530
1712
|
| `onExecute` | The execution observer (see [Events](#events)) |
|
|
1531
1713
|
|
|
1532
1714
|
`vex` and `VexHonoConfig` are generic over the hono `Env`: `vex<AppEnv>({ … })`
|
|
@@ -1549,9 +1731,12 @@ app.all('/api/orders/vex', vex({
|
|
|
1549
1731
|
}));
|
|
1550
1732
|
```
|
|
1551
1733
|
|
|
1552
|
-
`VexExpressConfig` also takes `locked`
|
|
1553
|
-
(`
|
|
1554
|
-
|
|
1734
|
+
`VexExpressConfig` also takes `locked`, `manageFingerprints`, `mutations` (`{ client, policy, onWrite?, checkMovedRows?, answerReadableRows? }`),
|
|
1735
|
+
`getPolicy(req)`, `getPolicyForReach(req, reach)` and `onExecute`. These mean
|
|
1736
|
+
what they mean for Hono: `getPolicy` governs reads, mutation replay and
|
|
1737
|
+
discovery, and when it returns `undefined` the engine's policy and
|
|
1738
|
+
`mutations.policy` stay in force. `PATCH` and `DELETE` carry no per-request
|
|
1739
|
+
policy. Any method other than `GET`/`POST`/`PATCH`/`DELETE` answers 405.
|
|
1555
1740
|
|
|
1556
1741
|
### The endpoints
|
|
1557
1742
|
|
|
@@ -1563,8 +1748,12 @@ other than `GET`/`POST`/`PATCH`/`DELETE` answers 405.
|
|
|
1563
1748
|
shape|effect }]` — `context` is the derived, typed input signature; `effect`
|
|
1564
1749
|
summarizes a write), and the full DSL JSON Schema. A client or agent can read
|
|
1565
1750
|
this to learn how to call the resource — and exactly what each fingerprint
|
|
1566
|
-
takes.
|
|
1567
|
-
|
|
1751
|
+
takes. Each is listed only where the policy it runs under lets it run.
|
|
1752
|
+
Entities and read entries go by the policy a query there runs under: the
|
|
1753
|
+
per-request policy when there is one, otherwise the engine's
|
|
1754
|
+
(`createQueryEngine({ scope })`). A write entry goes by the policy its write
|
|
1755
|
+
runs under, `mutations.policy` (the per-request policy on an adapter with
|
|
1756
|
+
`getPolicy`).
|
|
1568
1757
|
- **`POST`** → runs a query OR replays a write, one body shape. A
|
|
1569
1758
|
`fingerprint` naming a `kind: 'mutation'` entry dispatches to the write
|
|
1570
1759
|
pipeline (when `mutations` is configured); anything else is the read path
|
|
@@ -1582,11 +1771,20 @@ other than `GET`/`POST`/`PATCH`/`DELETE` answers 405.
|
|
|
1582
1771
|
principal → 403 `scope_denied`. 500s are the host's faults: `missing_scope`
|
|
1583
1772
|
(key names stay in the log), an entry demanding a reach with no
|
|
1584
1773
|
`policyForReach`, a mutation fingerprint on a handler with no `mutations`
|
|
1585
|
-
configured, and unexpected errors
|
|
1774
|
+
configured, and unexpected errors, a context value that does not fit its
|
|
1775
|
+
column among them. The log line of an unexpected error carries its code,
|
|
1776
|
+
message and stack, and the SQL text where the driver's error carries it
|
|
1777
|
+
(PGlite's does, node-postgres's does not); never the error object, which can
|
|
1778
|
+
hold the statement's bound values.
|
|
1586
1779
|
- **`PATCH`** with `{ fingerprint, protected }` → flip the protection bit.
|
|
1587
1780
|
**`DELETE`** with `{ fingerprint }` → evict (a protected entry, read or mutation,
|
|
1588
1781
|
is refused with 409 `fingerprint_protected`). Both are refused with 403 `locked` on a
|
|
1589
|
-
locked endpoint
|
|
1782
|
+
locked endpoint, or one with `manageFingerprints: false`, and answer 404
|
|
1783
|
+
`cache_miss` for an unknown fingerprint. Otherwise they check nobody: no
|
|
1784
|
+
policy, no scope. Whoever reaches the endpoint can unprotect any entry, a
|
|
1785
|
+
seeded one included, and delete it; the adapter logs one warning saying so
|
|
1786
|
+
when it mounts such an endpoint. An agent's endpoint that generates but
|
|
1787
|
+
should manage nothing sets `manageFingerprints: false`.
|
|
1590
1788
|
Fingerprints ride the body — names contain `/` — and a body that is not
|
|
1591
1789
|
JSON, or carries no fingerprint, is a 400 `invalid_request` before anything
|
|
1592
1790
|
else is checked.
|
|
@@ -1631,7 +1829,7 @@ log); the engine itself emits the `query.*` and `rows.*` events.
|
|
|
1631
1829
|
|
|
1632
1830
|
### The execution observer
|
|
1633
1831
|
|
|
1634
|
-
`onExecute` (handler config; the hono
|
|
1832
|
+
`onExecute` (handler config; the hono and express adapters pass it through) fires once per
|
|
1635
1833
|
`handleQuery`, read or write, after the outcome is known:
|
|
1636
1834
|
|
|
1637
1835
|
```typescript
|
package/README.md
CHANGED
|
@@ -64,6 +64,23 @@ Then keep **one** unlocked endpoint (wired with `generateDsl`) for genuine
|
|
|
64
64
|
ad-hoc queries — an agent, an admin console — under its own scope policy. Built
|
|
65
65
|
surfaces are locked; exploration is open and scoped.
|
|
66
66
|
|
|
67
|
+
An unlocked endpoint also answers `PATCH` and `DELETE`, and they check nobody:
|
|
68
|
+
whoever reaches it can unprotect any entry, a seeded one included, and then
|
|
69
|
+
delete it or generate over it. Fingerprints are global, so that changes what
|
|
70
|
+
the locked endpoints replay. An endpoint for an agent turns that off:
|
|
71
|
+
|
|
72
|
+
```typescript
|
|
73
|
+
app.route('/api/agent/vex', vex({
|
|
74
|
+
engine, // wired with generateDsl
|
|
75
|
+
getScope: async (c) => ({ accountId: await resolveAccount(c) }),
|
|
76
|
+
getPolicy: (c) => agentPolicy(c),
|
|
77
|
+
manageFingerprints: false, // PATCH and DELETE answer 403 `locked`
|
|
78
|
+
}));
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
Left on, the adapter logs one warning as it mounts the endpoint; keep it on
|
|
82
|
+
only where everybody who reaches the endpoint may manage every entry.
|
|
83
|
+
|
|
67
84
|
## Documentation
|
|
68
85
|
|
|
69
86
|
- **[DOCS.md](./DOCS.md)** — Full usage guide: configuration, DSL reference, scope, caching, framework adapters, error handling, local dev.
|
|
@@ -96,6 +113,7 @@ engine.rows; // the reactive rows cache: stats(), stop()
|
|
|
96
113
|
| Writes | Mutation entries in the same cache — a closed grammar, replay-only by fingerprint, never generated; `lintMutation` checks a statement at seed |
|
|
97
114
|
| Seeding and freshness | `seedCache` turns authored `SeedEntry` / `SeedMutation` rows into protected cache entries; an entry's `refresh` is `'snapshot'` or `'reactive'` |
|
|
98
115
|
| Safety | Server-side scope policies, query analyzer (cartesian/nesting/index checks), parameterized SQL only |
|
|
116
|
+
| Upgraded columns | `upgradedColumns` + `upgrader` on the engine: a JSON column stamped in another column is returned upgraded by every read, before the mapping |
|
|
99
117
|
| Stored entries | `@niscorp/vex/migrations`: `VEX_SEQUENCE`, the strata sequence an entry is stamped with as a stored document |
|
|
100
118
|
| Its own table | The Postgres cache's table is a [strata](../strata/README.md) sequence (`cache.sequence`, `nisc.vex.cache`): `cache.init()` applies it through the ledger once, instead of `IF NOT EXISTS` on every boot. Requires a pool with `transaction` (the PGlite pool and `createPgPool`'s have one; a bare `pg.Pool` does not). |
|
|
101
119
|
|