@niscorp/vex 0.1.0 → 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/DOCS.md CHANGED
@@ -86,15 +86,15 @@ fingerprint accepts without failing a request first.
86
86
  ## Installation
87
87
 
88
88
  ```bash
89
- pnpm add @niscorp/vex zod
89
+ pnpm add @niscorp/vex @niscorp/prism @niscorp/strata zod
90
90
  ```
91
91
 
92
- `zod` is the only required peer. Add the others for the paths you use:
92
+ `@niscorp/prism` (mappings), `@niscorp/strata` (the cache table's ledger) and `zod` are required peers. Add the others for the paths you use:
93
93
 
94
94
  ```bash
95
95
  pnpm add pg # Postgres adapter + Postgres cache
96
96
  pnpm add hono # or: express # framework adapter
97
- pnpm add @niscorp/cortex @niscorp/signal @niscorp/prism # reference LLM agents
97
+ pnpm add @niscorp/cortex @niscorp/signal # reference LLM agents
98
98
  ```
99
99
 
100
100
  For semantic (vector) search you need a Postgres with `pgvector` (the dev setup
@@ -124,9 +124,11 @@ indexes, and caches the result plus a schema fingerprint. The engine throws
124
124
  `execution_error` ("schema not loaded") if you call `compile`/`execute`/`test`
125
125
  before introspecting.
126
126
 
127
- With no `generateDsl` hook, `execute` serves only requests whose shape is
128
- already cached (and otherwise throws `agent_failed`). `compile` and `test` work
129
- fully because you pass them a DSL directly.
127
+ With no `generateDsl` hook, `execute` serves only requests that replay a
128
+ cached fingerprint — a seeded entry or a named slot. An unknown fingerprint sent
129
+ alone throws `cache_miss`; a request that would need generation throws
130
+ `agent_failed`. `compile` and `test` work fully because you pass them a DSL
131
+ directly.
130
132
 
131
133
  ### With LLM-backed generation and mapping
132
134
 
@@ -154,9 +156,11 @@ type QueryEngineConfig = {
154
156
  scope?: ScopePolicy; // server-side access control
155
157
  cache?: CacheBackend; // default: createMemoryCache()
156
158
  onEvent?: VexEventHandler; // pipeline event stream
157
- generateDsl?: (request, schema) => Promise<Query>;
159
+ generateDsl?: (request, schema, caller) => Promise<Query>;
158
160
  mapToShape?: (rows, shape) => Promise<{ ir: CompiledIr; transformed: JsonValue }>;
159
161
  embed?: (text, dimensions?) => Promise<number[]>; // text→vector for semantic filters
162
+ behaviors?: ScopeBehaviors; // which columns a scope key is bound to (see below)
163
+ rows?: LiveRowsConfig; // the rows cache behind `refresh: 'reactive'`
160
164
  config?: {
161
165
  maxNestingDepth?: number; // default: 2
162
166
  defaultLimit?: number; // default: 100
@@ -166,11 +170,26 @@ type QueryEngineConfig = {
166
170
  rejectUnindexedFilters?: boolean; // default: false
167
171
  entities?: string[]; // whitelist for introspection
168
172
  unsatisfiableTtlMs?: number; // negative-cache TTL, default 300_000 (5 min)
173
+ statementTimeoutMs?: number; // how long one read may run, default 10_000 (0: unbounded)
174
+ maxPresenceVariants?: number; // optional-key combinations per entry before a warning, default 32
169
175
  };
170
176
  };
171
177
  ```
172
178
 
173
179
  - `config.entities`, when set, restricts `introspect()` to those tables.
180
+ - `config.statementTimeoutMs` is enforced by the database (`statement_timeout`),
181
+ set up at `introspect()`. A database that cannot enforce it (PGlite) runs
182
+ reads unbounded and the engine warns each time `introspect()` runs.
183
+ - `config.maxPresenceVariants` is not a limit on what runs: past it, a response
184
+ carries a warning that the entry has compiled that many distinct
185
+ optional-key combinations.
186
+ - **`behaviors`** is the same `ScopeBehaviors` document the host's policies are
187
+ compiled from (see [Grants and behaviors](#grants-and-behaviors)). The engine
188
+ reads it for one thing: which columns a scope key is bound to, so a generation
189
+ can be told who its caller is (`GenerationCaller.bindings`). `introspect()`
190
+ throws `execution_error` if a rule names a column the database does not have.
191
+ - **`rows`** sizes the reactive rows cache — see
192
+ [Reactive reads](#reactive-reads-refresh).
174
193
  - `defaultLimit` is applied when a DSL omits `limit`; `maxLimit` clamps any
175
194
  larger value.
176
195
  - **`embed`** turns text into a vector for `semantic` filters, at parameter-
@@ -186,6 +205,9 @@ type ExecuteOptions = {
186
205
  scope?: ScopeValues; // { [key]: value } for $scope references
187
206
  entities?: string[]; // restrict generation to this entity subset for this call
188
207
  locked?: boolean; // replay-only: unknown/changed fingerprints throw `locked` instead of generating
208
+ scopePolicy?: ScopePolicy; // per-request policy; overrides the engine's configured one for this call
209
+ signal?: AbortSignal; // with onChange: follow a `refresh: 'reactive'` entry until it aborts
210
+ onChange?: (response: QueryResponse) => void; // every later answer that differs from the last
189
211
  };
190
212
  ```
191
213
 
@@ -271,7 +293,7 @@ type QueryResponse = {
271
293
  intent?: string; // the stored intent (descriptive only)
272
294
  };
273
295
  context: Record<string, { // the resolved parameter contract
274
- type?: 'string' | 'number' | 'boolean' | 'string[]' | 'number[]';
296
+ type?: 'string' | 'number' | 'boolean' | 'string[]' | 'number[]' | 'json';
275
297
  kind: 'context' | 'scope' | 'semantic';
276
298
  optional?: true; // supplying it switches a condition on
277
299
  absent?: true; // optional, and not supplied this run (so: no type)
@@ -285,9 +307,10 @@ type QueryResponse = {
285
307
  ```
286
308
 
287
309
  If the compiled query needs context/scope values the caller did not provide,
288
- `execute` returns a **valid** response with an empty `result` and
310
+ `execute` returns a **valid** response with an empty `result` (`[]`) and
289
311
  `meta.missingContext` listing the missing keys — it does not throw. Retry with
290
- the values filled in. `meta.context` always describes the full parameter
312
+ the values filled in. (A missing **scope** value is different: it throws
313
+ `missing_scope`.) `meta.context` always describes the full parameter
291
314
  contract so a caller knows what to provide.
292
315
 
293
316
  ---
@@ -298,11 +321,13 @@ contract so a caller knows what to provide.
298
321
  type QueryEngine = {
299
322
  introspect: () => Promise<DatabaseSchema>;
300
323
  execute: (request: QueryRequest, options?: ExecuteOptions) => Promise<QueryResponse>;
301
- compile: (dsl: Query, scope?: ScopeValues) => CompiledQuery;
324
+ compile: (dsl: Query) => CompiledQuery;
302
325
  test: (dsl: Query, scope?: ScopeValues) => Promise<TestResult>;
303
326
  getDslSchema: () => object;
304
327
  getSchema: () => DatabaseSchema | undefined;
305
328
  cache: CacheBackend;
329
+ invalidate: (tables: readonly string[]) => void;
330
+ rows: { stats: () => LiveRowsStats; stop: () => void };
306
331
  };
307
332
  ```
308
333
 
@@ -312,15 +337,24 @@ type QueryEngine = {
312
337
  - **`compile(dsl)`** — run the deterministic pipeline (clamp limit →
313
338
  discover → scope → resolve → analyze → adapter compile) and return the
314
339
  `CompiledQuery` (`{ sql, paramSlots, contextContract }`). No execution, no
315
- LLM. Throws `invalid_dsl` if the analyzer finds errors.
340
+ LLM. Throws `invalid_dsl` if the analyzer finds errors. Every `optional`
341
+ condition is kept, so this is the widest form of the query.
316
342
  - **`test(dsl, scope?)`** — compile with `limit` forced to 5, build synthetic
317
343
  context values, execute, and return `{ rows, warnings, errors }`. Errors are
318
344
  caught and returned in `errors` rather than thrown. Useful for validating a
319
- hand-written DSL against real data.
345
+ hand-written DSL against real data. Scope is never synthetic: a query whose
346
+ policy binds `$scope` values needs them passed, or the missing-scope refusal
347
+ comes back in `errors`.
320
348
  - **`getDslSchema()`** — the DSL as a JSON Schema (draft-7). This is what you
321
349
  feed an LLM so it learns the DSL.
322
350
  - **`getSchema()`** — the last introspected `DatabaseSchema`, or `undefined`.
323
351
  - **`cache`** — the live `CacheBackend` for inspection or manual eviction.
352
+ - **`invalidate(tables)`** — these tables were written: every reactive read
353
+ over any of them is refetched (once per burst). The vex handler calls it after
354
+ every committed mutation; a host calls it for writes vex did not make.
355
+ - **`rows`** — the reactive rows cache: `stats()` (entries, bytes, follows,
356
+ followers, hits, misses, evictions, oversized, refreshes) and `stop()` (clears
357
+ its timers, for a host that outlives the engine).
324
358
 
325
359
  ---
326
360
 
@@ -486,8 +520,9 @@ Grouped, with a renamed field:
486
520
  }
487
521
  ```
488
522
 
489
- Functions: `count` (field path or `"*"`); `sum` / `avg` / `min` / `max` (a field
490
- path or a compute expression).
523
+ Functions: `count` (field path or `"*"`); `countDistinct` (a field path —
524
+ `COUNT(DISTINCT field)`); `sum` / `avg` / `min` / `max` (a field path or a
525
+ compute expression).
491
526
 
492
527
  ### Subqueries
493
528
 
@@ -569,7 +604,9 @@ type ScopeEntityRule =
569
604
  delete?: ScopeMatch[]; // specific: just DELETE (match only — nothing to set)
570
605
  };
571
606
 
572
- type ScopeMatch = { match: string; to: string }; // row.<match> = scope[to]
607
+ type ScopeMatch =
608
+ | { match: string; to: string } // row.<match> = scope[to]
609
+ | { match: string; in: string }; // row.<match> = ANY(scope[in]) — filters only, never pins an INSERT
573
610
  type ScopeSet = { set: string; to: string }; // INSERT/UPDATE row.<set> := scope[to]
574
611
  ```
575
612
 
@@ -609,8 +646,12 @@ Behavior:
609
646
  where binding it would stamp NULL. A missing `$context` value is different: it
610
647
  is the caller's, and a read surfaces it via `meta.missingContext`
611
648
  (empty-but-valid).
612
- - `match` is equality today; an `op` for `in` / multi-valued boundaries is additive
613
- when a real one appears.
649
+ - `match` is equality (`to`) or set membership (`in`, compiled to
650
+ `col = ANY($n)`). The set form only filters — a read, or the rows an
651
+ UPDATE/DELETE may touch: an empty set matches no row, an absent one is refused
652
+ with `missing_scope` like any other scope value, and a set-valued rule that
653
+ would have to pin an INSERT is refused (`scope_denied`) — a write's subject
654
+ must be a single value.
614
655
 
615
656
  The policy applies whenever one is configured; the values fill it. Omitting
616
657
  the values means nobody, never everybody.
@@ -725,7 +766,8 @@ same principal under that profile:
725
766
 
726
767
  Grants are unchanged — the union of every role the caller holds — so this can
727
768
  only narrow rows, never widen phases. A caller with no verb for the table is
728
- still refused.
769
+ still refused. A mutation entry takes `reach` the same way (`SeedMutation.reach`),
770
+ and the write is compiled at that profile whoever asks.
729
771
 
730
772
  The host supplies `policyForReach(reach)` (hono: `getPolicyForReach(c, reach)`),
731
773
  because vex holds neither grants nor behaviors. **It fails closed both ways:** an
@@ -893,10 +935,12 @@ mutationEffect(def); // [{ op: 'update', table: 'tasks', column
893
935
 
894
936
  ### Authoring lint
895
937
 
896
- `lintMutation(def)` flags an `update`/`delete` whose WHERE binds no
897
- `$context` — such a write is not caller-bounded (its only row limit is the
898
- scope policy). Run it in your seed path so an unkeyed write never ships:
899
- loud at boot, never at runtime.
938
+ `lintMutation(def)` returns a list of issues (empty when clean). It flags an
939
+ `update`/`delete` whose WHERE binds no `$context` — such a write is not
940
+ caller-bounded (its only row limit is the scope policy) — and any statement
941
+ containing an `optional` condition. `seedCache` runs it on every mutation entry
942
+ and throws on an issue, so an unkeyed write never ships: loud at boot, never at
943
+ runtime.
900
944
 
901
945
  ### The write observer
902
946
 
@@ -953,14 +997,16 @@ everything:
953
997
  `fingerprint_protected` instead of being replaced.
954
998
 
955
999
  Concurrent identical generations collapse to one (single-flight, keyed by the
956
- request hash).
1000
+ request hash and the active policy — two callers who can see different tables
1001
+ are not asking the same thing).
957
1002
 
958
1003
  ### Lifetime and protection
959
1004
 
960
- - Every replay stamps `lastUsedAt`; `sweepCache(cache, { maxIdleMs })` evicts
961
- entries idle past the limit, skipping protected ones. Alive = used.
962
- - `protected: true` is written only by a seed path or an explicit
963
- `handleFingerprintPatch` — never by a runtime query.
1005
+ - A replay stamps `lastUsedAt` (at most once a minute per entry);
1006
+ `sweepCache(cache, { maxIdleMs })` evicts entries idle past the limit,
1007
+ skipping protected ones, and returns how many it evicted. Alive = used.
1008
+ - `protected: true` is written only by the seed path (`seedCache`) or an
1009
+ explicit `handleFingerprintPatch` — never by a runtime query.
964
1010
  - `ExecuteOptions.locked` (and `VexHandlerConfig.locked`) makes an engine call
965
1011
  or endpoint replay-only: asking it to generate throws `locked`.
966
1012
 
@@ -968,7 +1014,7 @@ request hash).
968
1014
 
969
1015
  - **Request hash** — intent + normalized shape + the *names* of context keys
970
1016
  (values excluded). Stored with named slots (the match/replace test) and the
971
- key of the **negative cache**: a `cannotSatisfy` result is cached with a TTL
1017
+ key (with the active policy) of the **negative cache**: a `cannotSatisfy` result is cached with a TTL
972
1018
  (`config.unsatisfiableTtlMs`, default 5 min) so an impossible request isn't
973
1019
  re-run until it expires.
974
1020
  - Every entry stores a **schema fingerprint**. After a schema change it no
@@ -1005,6 +1051,11 @@ const l2 = createPostgresCache({
1005
1051
  await l2.init(); // applies l2.sequence through strata's ledger — once, recorded
1006
1052
  ```
1007
1053
 
1054
+ `init()` runs a strata migration, so the pool must expose `transaction` — the
1055
+ PGlite pool has one; a bare `pg.Pool` needs a wrapper that checks a client out
1056
+ per transaction (see `PgPool`). A host that migrates several owners in one run
1057
+ collects `l2.sequence` (`nisc.vex.cache`) instead of calling `init()`.
1058
+
1008
1059
  **Tiered** — L1 (memory) + L2 (durable), read-through/write-through. Reads hit
1009
1060
  L1; on an L1 miss it promotes a validated L2 entry into L1. Writes go to L1
1010
1061
  synchronously and L2 fire-and-forget.
@@ -1026,6 +1077,90 @@ demand; `partial` preloads only the listed fingerprints and lazy-fills the
1026
1077
  rest. The tiered cache is single-instance: separate processes keep separate
1027
1078
  L1s.
1028
1079
 
1080
+ ### Seeding
1081
+
1082
+ `seedCache(cache, entries)` turns authored entries into **protected** cache
1083
+ rows — the API surface of a locked endpoint. Call it at boot, after
1084
+ `cache.init()`.
1085
+
1086
+ ```typescript
1087
+ import { seedCache } from '@niscorp/vex';
1088
+ import type { SeedEntry, SeedMutation } from '@niscorp/vex';
1089
+
1090
+ const ordersRecent: SeedEntry = {
1091
+ fingerprint: 'orders/recent',
1092
+ intent: 'A customer\'s orders, newest first',
1093
+ shape: [{ order_id: '', total: 0, created_at: '' }],
1094
+ dsl: {
1095
+ from: ['orders'],
1096
+ fields: [{ field: 'orders.id', as: 'order_id' }, 'orders.total', 'orders.created_at'],
1097
+ filter: { eq: ['orders.customer_id', { $context: 'customerId' }] },
1098
+ sort: [{ field: 'orders.created_at', dir: 'desc' }],
1099
+ },
1100
+ // mapping?: uncompiled Prism over { result, context, scope }; omitted = identity
1101
+ // reach?: the scope profile this read must be served at
1102
+ // refresh?: 'snapshot' (default) | 'reactive'
1103
+ };
1104
+
1105
+ const taskSetDone: SeedMutation = {
1106
+ fingerprint: 'tasks/setDone',
1107
+ mutation: { op: 'update', table: 'tasks', set: { done: { $context: 'done' } },
1108
+ where: { eq: ['tasks.id', { $context: 'id' }] } },
1109
+ };
1110
+
1111
+ await seedCache(engine.cache, [ordersRecent, taskSetDone]);
1112
+ ```
1113
+
1114
+ - A read's `mapping` is compiled to Prism IR here; with none, the identity
1115
+ (`{ $ref: '$.result' }`) is stored, so a replay never falls through to
1116
+ `mapToShape`.
1117
+ - A mutation is linted (`lintMutation`); a failing statement throws.
1118
+ - It is idempotent against a durable cache: a stored row that already matches
1119
+ its authored definition is left alone; anything else at that fingerprint
1120
+ converges to the authored definition, protected.
1121
+
1122
+ `@niscorp/vex/pglite` exports `createPglitePool(db, parsers?)`, which shapes a
1123
+ PGlite database as the `PgPool` the Postgres adapter and cache take (with
1124
+ `transaction`), and `RAW_DATE_PARSERS`, which returns `DATE`/`TIMESTAMPTZ`
1125
+ columns as raw strings. Pass the parsers to the adapter's pool; give the
1126
+ Postgres cache a pool without them — it reads its own timestamp columns as
1127
+ `Date`s.
1128
+
1129
+ ### Reactive reads (`refresh`)
1130
+
1131
+ An entry's `refresh` says when its answer is refreshed: `'snapshot'` (the
1132
+ default) answers when asked; `'reactive'` answers again whenever a write lands
1133
+ on a table the query reads, for as long as somebody follows it. The mode is
1134
+ authored on a seeded entry — never generated (`REFRESH_MODES`, `isRefresh`,
1135
+ type `Refresh`).
1136
+
1137
+ - **Following.** Pass `{ signal, onChange }` in `ExecuteOptions`: the returned
1138
+ promise is the first answer; every later answer that differs goes to
1139
+ `onChange` until `signal` aborts. Over HTTP there is no callback, so
1140
+ `handleQuery(config, body, scope, live?)` takes an in-process `VexLive`
1141
+ (`{ signal, onChange }`), and the hono adapter reads one from the request's
1142
+ env under `VEX_LIVE_ENV` (the third argument of `app.request`).
1143
+ - **Invalidation.** The handler calls
1144
+ `engine.invalidate(tablesChangedBy(writes, schema))` after every committed
1145
+ mutation; for a delete, `tablesChangedBy` adds every table that references a
1146
+ deleted one. Writes vex never sees are the host's to announce with
1147
+ `engine.invalidate`.
1148
+ - **The rows cache** is per engine, in process memory, sized by
1149
+ `QueryEngineConfig.rows`:
1150
+
1151
+ ```typescript
1152
+ type LiveRowsConfig = {
1153
+ ttlMs?: number; // how long stored rows are trusted; default 60_000
1154
+ maxEntries?: number; // default 5_000
1155
+ maxBytes?: number; // JSON length of stored results; default 64 MiB
1156
+ maxEntryBytes?: number; // largest single result stored; default maxBytes / 10
1157
+ debounceMs?: number; // burst window before followed reads refetch; default 10
1158
+ };
1159
+ ```
1160
+
1161
+ A reactive entry should `sort`, and reads time from `$scope`, never context.
1162
+ Design: [DESIGN.md](./DESIGN.md) **Reactive reads**.
1163
+
1029
1164
  ---
1030
1165
 
1031
1166
  ## Wiring the LLM agents
@@ -1034,38 +1169,46 @@ The engine consumes two hooks. Both are optional; supply them to answer
1034
1169
  free-form requests.
1035
1170
 
1036
1171
  ```typescript
1037
- type GenerateDsl = (request: QueryRequest, schema: DatabaseSchema) => Promise<Query>;
1172
+ type GenerateDsl = (request: QueryRequest, schema: DatabaseSchema, caller: GenerationCaller) => Promise<Query>;
1038
1173
  type MapToShape = (rows: Row[], shape: unknown) =>
1039
- Promise<{ ir: CompiledIr; transformed: unknown[] }>;
1174
+ Promise<{ ir: CompiledIr; transformed: JsonValue }>;
1175
+
1176
+ type GenerationCaller = {
1177
+ read: (dsl: Query) => Promise<{ rows: Row[]; sql: string; warnings: string[] }>;
1178
+ bindings: ScopeBinding[]; // { entity, field, key } — columns bound to scope keys the caller carries
1179
+ };
1040
1180
  ```
1041
1181
 
1042
1182
  - `generateDsl` runs on a cache miss; it must return a DSL that passes
1043
1183
  `QuerySchema`. To signal an impossible request, throw
1044
1184
  `new VexError('unsatisfiable', reason)` — the runtime negative-caches it.
1045
- - `mapToShape` runs when the requested shape differs from the raw rows; it
1046
- returns a compiled Prism `ir` (cached for reuse) plus the transformed rows.
1185
+ - `mapToShape` runs when a shape is in play (the request's, or the stored one on
1186
+ a fingerprint-only replay) and the entry has no stored mapping; it returns a
1187
+ compiled Prism `ir` plus the mapped result. The `ir` is stored with the entry
1188
+ only on a miss — a replayed entry that has none calls the hook every time.
1189
+ Without the hook the raw rows are returned.
1047
1190
 
1048
1191
  You can implement these however you like. Vex also ships a **reference
1049
1192
  implementation** built on `@niscorp/cortex` and `@niscorp/prism`, exported from
1050
- the `@niscorp/vex/agent` subpath (`@niscorp/cortex`, `@niscorp/signal`, and
1051
- `@niscorp/prism` become required peers only if you import it):
1193
+ the `@niscorp/vex/agent` subpath (`@niscorp/cortex` is an optional peer, needed
1194
+ only if you import it; the model client comes from `@niscorp/signal`):
1052
1195
 
1053
1196
  ```typescript
1054
1197
  import { createQueryDsl, createShapeMapper } from '@niscorp/vex/agent';
1055
1198
  import { createSignal } from '@niscorp/signal';
1056
1199
 
1057
- const schema = await engine.introspect();
1058
- const queryJsonSchema = engine.getDslSchema();
1059
-
1060
1200
  const llm = createSignal('openrouter', { apiKey, model: 'qwen/qwen3.8-27b' });
1061
1201
 
1062
- const generateDsl = createQueryDsl({ llm, queryJsonSchema });
1202
+ const generateDsl = createQueryDsl({ llm, queryJsonSchema: bare.getDslSchema() });
1063
1203
  const mapToShape = createShapeMapper(llm);
1064
1204
 
1065
1205
  const engine = createQueryEngine({ adapter, generateDsl, mapToShape, cache });
1066
1206
  await engine.introspect();
1067
1207
  ```
1068
1208
 
1209
+ (`bare` is any engine — `getDslSchema()` needs no introspection, so
1210
+ `createQueryEngine({ adapter })` is enough to read it from.)
1211
+
1069
1212
  - `createQueryDsl(config)` builds the `generateDsl` hook by running
1070
1213
  `vexQueryDslAgent`. It is built from a model and the DSL spec only — who it
1071
1214
  runs for arrives with every call. The engine calls the hook as
@@ -1074,8 +1217,12 @@ await engine.introspect();
1074
1217
  engine's own pipeline under the caller's policy and scope values (`$context`
1075
1218
  binds NULL). That capability is the tools' only way to the database.
1076
1219
  - `createShapeMapper(llm)` builds the `mapToShape` hook by running Prism's
1077
- `mappingAgent`. The `@niscorp/vex/agent` subpath also exports `vexQueryDslAgent`
1078
- itself plus `createQueryTools` for customizing the tool set.
1220
+ `mappingAgent`. When the rows already are the shape (`rowsFitShape` — a flat
1221
+ shape whose keys and value kinds the first row matches exactly) it stores the
1222
+ identity mapping and makes no model call. The `@niscorp/vex/agent` subpath
1223
+ also exports `vexQueryDslAgent` itself, `createQueryTools` for customizing the
1224
+ tool set, and `describeCaller`, which turns `caller.bindings` into the
1225
+ context line the agent reads.
1079
1226
 
1080
1227
  The reference query agent (`vexQueryDslAgent`, id `vex.query`) is a Cortex
1081
1228
  agent whose envelope payload is a `QuerySchema` query, with six tools:
@@ -1086,10 +1233,10 @@ agent whose envelope payload is a `QuerySchema` query, with six tools:
1086
1233
  | `getSampleRows` | `{ entity, limit? }` | See real rows (default 5) |
1087
1234
  | `getDistinctValues` | `{ entity, field, limit? }` | Field cardinality (default 20) |
1088
1235
  | `describeField` | `{ entity, field }` | type, nullable, cardinality, null count, min/max |
1089
- | `testQuery` | `{ dsl }` | Validate + compile + execute with synthetic params (LIMIT 5) |
1236
+ | `testQuery` | the draft query itself (not wrapped) | Validate + compile + execute with synthetic params (LIMIT 5) |
1090
1237
  | `cannotSatisfy` | `{ reason }` | Declare the request impossible (`createQueryDsl` aborts the run and throws `unsatisfiable`) |
1091
1238
 
1092
- Bounds: `stepCount(20)` + `outputRetries(3)` stop conditions on the agent. Every
1239
+ Bounds: `stepCount(20)`, `outputRetries(3)` and `repeatedCalls(2)` stop conditions on the agent. Every
1093
1240
  data-touching tool is a DSL run through `caller.read` — there is no SQL in the
1094
1241
  tool set — so a sample row, a distinct value and a statistic are all things the
1095
1242
  caller could have read. (If you build your own generator,
@@ -1130,6 +1277,16 @@ app.route('/api/orders/vex', vex({
1130
1277
  }));
1131
1278
  ```
1132
1279
 
1280
+ The rest of `VexHonoConfig`, all optional:
1281
+
1282
+ | Option | Purpose |
1283
+ |--------|---------|
1284
+ | `locked` | Replay-only endpoint: no generation, no fingerprint management |
1285
+ | `getPolicy(c)` | Per-request `ScopePolicy`; governs reads, mutation replay and discovery on this endpoint |
1286
+ | `getPolicyForReach(c, reach)` | The same principal's policy at a reach an entry demands |
1287
+ | `mutations` | `{ client, policy?, onWrite? }` — enables mutation replay; `policy` is the fallback when `getPolicy` is absent |
1288
+ | `onExecute` | The execution observer (see [Events](#events)) |
1289
+
1133
1290
  ### Express
1134
1291
 
1135
1292
  ```typescript
@@ -1146,29 +1303,41 @@ app.all('/api/orders/vex', vex({
1146
1303
  }));
1147
1304
  ```
1148
1305
 
1306
+ `VexExpressConfig` also takes `locked` and `mutations`
1307
+ (`{ client, policy, onWrite? }`). It has no per-request policy hook. Any method
1308
+ other than `GET`/`POST`/`PATCH`/`DELETE` answers 405.
1309
+
1149
1310
  ### The endpoints
1150
1311
 
1151
1312
  - **`GET`** → discovery JSON: `vex` protocol version, description, the filtered
1152
1313
  entities (fields, relations, row counts), the request body contract (the
1153
1314
  three fingerprint postures), the protection summary (`protection:
1154
1315
  "all"|"some"|"none"`, `locked`), the fingerprint overview (`[{ fingerprint,
1155
- kind, protected, schemaFresh, intent, context, shape|effect, lastUsedAt }]`
1156
- — `context` is the derived, typed input signature; `effect` summarizes a
1157
- write), and the full DSL JSON Schema. A client or agent can read this to
1158
- learn how to call the resource — and exactly what each fingerprint takes.
1316
+ kind, protected, schemaFresh, intent, createdAt, lastUsedAt, context,
1317
+ shape|effect }]` — `context` is the derived, typed input signature; `effect`
1318
+ summarizes a write), and the full DSL JSON Schema. A client or agent can read
1319
+ this to learn how to call the resource — and exactly what each fingerprint
1320
+ takes. Under a per-request policy, entities and entries the policy cannot
1321
+ touch are not listed.
1159
1322
  - **`POST`** → runs a query OR replays a write, one body shape. A
1160
1323
  `fingerprint` naming a `kind: 'mutation'` entry dispatches to the write
1161
1324
  pipeline (when `mutations` is configured); anything else is the read path
1162
1325
  (fingerprint replay, generation, or named slot); scope comes from
1163
1326
  `getScope`. Returns `{ status, body }` mapped to the HTTP response —
1164
1327
  unknown fingerprint → 404 `cache_miss`, protected mismatch → 409
1165
- `fingerprint_protected`, locked → 403, a write with missing context → 400
1328
+ `fingerprint_protected`, a write with missing context → 400
1166
1329
  `missing_context` with the full derived signature in `details.expected`,
1167
- other `VexError`s → 400 with `{ error, message, details }`, unexpected
1168
- errors → 500.
1330
+ other `VexError`s (including `locked` and `scope_denied`) → 400 with
1331
+ `{ error, message, details }`. A reach the host cannot serve for this
1332
+ principal → 403 `scope_denied`. 500s are the host's faults: `missing_scope`
1333
+ (key names stay in the log), an entry demanding a reach with no
1334
+ `policyForReach`, a mutation fingerprint on a handler with no `mutations`
1335
+ configured, and unexpected errors.
1169
1336
  - **`PATCH`** with `{ fingerprint, protected }` → flip the protection bit.
1170
- **`DELETE`** with `{ fingerprint }` → evict (unprotected only). Fingerprints
1171
- ride the body — names contain `/`.
1337
+ **`DELETE`** with `{ fingerprint }` → evict (a protected entry, read or mutation,
1338
+ is refused with 409 `fingerprint_protected`). Both are refused with 403 `locked` on a
1339
+ locked endpoint and answer 404 `cache_miss` for an unknown fingerprint.
1340
+ Fingerprints ride the body — names contain `/`.
1172
1341
 
1173
1342
  Under the hood all of these call the framework-agnostic `handleDiscovery` /
1174
1343
  `handleQuery` / `handleFingerprintPatch` / `handleFingerprintDelete` from
@@ -1195,6 +1364,8 @@ const onEvent = (e: VexEvent) => {
1195
1364
  case 'query.mapped': /* mappingMs */ break;
1196
1365
  case 'query.done': /* totalMs */ break;
1197
1366
  case 'query.error': /* code, message */ break;
1367
+ case 'rows.evict': /* reason: 'capacity' | 'oversized', bytes */ break;
1368
+ case 'rows.error': /* message — a reactive refetch failed; the last good answer stands */ break;
1198
1369
  // llm.request / llm.response — only if you emit them from your LLM wrapper
1199
1370
  }
1200
1371
  };
@@ -1204,7 +1375,29 @@ const engine = createQueryEngine({ adapter, onEvent });
1204
1375
 
1205
1376
  The `llm.request`/`llm.response` variants exist for callers that wrap their LLM
1206
1377
  client to record a transcript (the dev server does this to write a full debug
1207
- log); the engine itself emits the `query.*` events.
1378
+ log); the engine itself emits the `query.*` and `rows.*` events.
1379
+
1380
+ ### The execution observer
1381
+
1382
+ `onExecute` (handler config; the hono adapter passes it through) fires once per
1383
+ `handleQuery`, read or write, after the outcome is known:
1384
+
1385
+ ```typescript
1386
+ type ExecuteRecord = {
1387
+ kind: 'query' | 'mutation';
1388
+ status: 'ok' | 'error' | 'refused'; // 200 | anything else | 403
1389
+ fingerprint?: string;
1390
+ reach?: string;
1391
+ cacheHit?: boolean; // reads
1392
+ rows?: number; // returned (reads) or written (mutations)
1393
+ startUnixNano: number;
1394
+ endUnixNano: number;
1395
+ scope: ScopeValues;
1396
+ };
1397
+ ```
1398
+
1399
+ With no observer the record is not built. A listener that throws is logged and
1400
+ contained.
1208
1401
 
1209
1402
  ---
1210
1403
 
@@ -1236,8 +1429,10 @@ try {
1236
1429
  | `scope_denied` | An entity is denied by the scope policy (`VexScopeError`) |
1237
1430
  | `missing_context` | A required context value was absent. Reads surface it softly via `meta.missingContext`; a WRITE hard-400s with the full derived signature in `details.expected` |
1238
1431
  | `execution_error` | Schema not loaded, or a database/runtime failure |
1239
- | `agent_failed` | No `generateDsl` hook and no cached DSL for the shape |
1240
- | `cache_miss` | Cache mode `only` and nothing cached |
1432
+ | `agent_failed` | Generation was needed and there is no `generateDsl` hook, or the reference agent's run failed |
1433
+ | `cache_miss` | A fingerprint was sent alone and nothing is stored under it |
1434
+ | `fingerprint_protected` | A protected entry was asked to change: a named-slot request that no longer matches its stored one |
1435
+ | `locked` | A replay-only call (`locked: true`) would have needed generation |
1241
1436
  | `unsatisfiable` | The agent declared the request impossible (negative-cached) |
1242
1437
 
1243
1438
  ---
package/README.md CHANGED
@@ -7,11 +7,12 @@ The LLM never writes SQL. It fills in a validated JSON DSL; a deterministic pipe
7
7
  ## Install
8
8
 
9
9
  ```bash
10
- pnpm add @niscorp/vex zod
11
- # Postgres adapter (optional peer):
12
- pnpm add pg
13
- # Only if you wire the bundled LLM agents:
14
- pnpm add @niscorp/cortex @niscorp/signal @niscorp/prism
10
+ pnpm add @niscorp/vex @niscorp/prism @niscorp/strata zod
11
+ # prism (mappings), strata (the cache table's ledger) and zod are required
12
+ # peers. Add the others for the paths you use:
13
+ pnpm add pg # a Postgres pool for the adapter and the cache
14
+ pnpm add hono # the Hono adapter (or: express)
15
+ pnpm add @niscorp/cortex @niscorp/signal # only if you wire the bundled LLM agents
15
16
  ```
16
17
 
17
18
  ## Quick Example
@@ -85,10 +86,12 @@ engine.cache; // CacheBackend — the live cache
85
86
 
86
87
  | Area | Provided |
87
88
  |------|----------|
88
- | Database adapters | PostgreSQL (`createPostgresAdapter`) — pluggable `DatabaseAdapter` interface for others |
89
+ | Database adapters | PostgreSQL (`createPostgresAdapter`) — pluggable `DatabaseAdapter` interface for others. `createPglitePool` (`@niscorp/vex/pglite`) shapes a PGlite database as the pool it takes |
89
90
  | Cache backends | In-memory, Postgres (durable), tiered L1/L2 with warm-up |
90
91
  | Framework adapters | Hono (`@niscorp/vex/hono`), Express (`@niscorp/vex/express`) — discovery + query endpoints |
91
- | LLM integration | `generateDsl` / `mapToShape` hooks; reference agents built on `@niscorp/cortex` |
92
+ | LLM integration | `generateDsl` / `mapToShape` hooks; reference agents built on `@niscorp/cortex` (`@niscorp/vex/agent`: `createQueryDsl`, `createShapeMapper`) |
93
+ | Writes | Mutation entries in the same cache — a closed grammar, replay-only by fingerprint, never generated; `lintMutation` checks a statement at seed |
94
+ | Seeding and freshness | `seedCache` turns authored `SeedEntry` / `SeedMutation` rows into protected cache entries; an entry's `refresh` is `'snapshot'` or `'reactive'` |
92
95
  | Safety | Server-side scope policies, query analyzer (cartesian/nesting/index checks), parameterized SQL only |
93
96
  | Its own table | The Postgres cache's table is a [strata](../strata/README.md) sequence (`cache.sequence`, `nisc.vex.cache`): `cache.init()` applies it through the ledger once, instead of `IF NOT EXISTS` on every boot. Requires a pool with `transaction` (the PGlite pool has one). |
94
97
 
@@ -1340,7 +1340,7 @@ var handleFingerprintDelete = async (config, fingerprint) => {
1340
1340
  if (entry === void 0) {
1341
1341
  return { status: 404, body: { error: "cache_miss", message: `Unknown fingerprint "${fingerprint}"` } };
1342
1342
  }
1343
- if (entry.kind === "ok" && entry.protected === true) {
1343
+ if ((entry.kind === "ok" || entry.kind === "mutation") && entry.protected === true) {
1344
1344
  return { status: 409, body: { error: "fingerprint_protected", message: `"${fingerprint}" is protected \u2014 unprotect it first.` } };
1345
1345
  }
1346
1346
  await config.engine.cache.delete(fingerprint);