@niscorp/vex 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (45) hide show
  1. package/DESIGN.md +881 -0
  2. package/DOCS.md +1295 -0
  3. package/LICENSE +202 -0
  4. package/README.md +97 -0
  5. package/dist/adapters/express/index.cjs +1406 -0
  6. package/dist/adapters/express/index.cjs.map +1 -0
  7. package/dist/adapters/express/index.d.cts +25 -0
  8. package/dist/adapters/express/index.d.ts +25 -0
  9. package/dist/adapters/express/index.js +60 -0
  10. package/dist/adapters/express/index.js.map +1 -0
  11. package/dist/adapters/hono/index.cjs +1426 -0
  12. package/dist/adapters/hono/index.cjs.map +1 -0
  13. package/dist/adapters/hono/index.d.cts +25 -0
  14. package/dist/adapters/hono/index.d.ts +25 -0
  15. package/dist/adapters/hono/index.js +78 -0
  16. package/dist/adapters/hono/index.js.map +1 -0
  17. package/dist/adapters/pglite/index.cjs +17 -0
  18. package/dist/adapters/pglite/index.cjs.map +1 -0
  19. package/dist/adapters/pglite/index.d.cts +21 -0
  20. package/dist/adapters/pglite/index.d.ts +21 -0
  21. package/dist/adapters/pglite/index.js +14 -0
  22. package/dist/adapters/pglite/index.js.map +1 -0
  23. package/dist/agent/index.cjs +478 -0
  24. package/dist/agent/index.cjs.map +1 -0
  25. package/dist/agent/index.d.cts +29 -0
  26. package/dist/agent/index.d.ts +29 -0
  27. package/dist/agent/index.js +313 -0
  28. package/dist/agent/index.js.map +1 -0
  29. package/dist/chunk-FPKW4TRX.js +1528 -0
  30. package/dist/chunk-FPKW4TRX.js.map +1 -0
  31. package/dist/chunk-HF25Q53I.js +165 -0
  32. package/dist/chunk-HF25Q53I.js.map +1 -0
  33. package/dist/handler-Cql8SJ3F.d.cts +133 -0
  34. package/dist/handler-mICf9ins.d.ts +133 -0
  35. package/dist/index.cjs +4078 -0
  36. package/dist/index.cjs.map +1 -0
  37. package/dist/index.d.cts +199 -0
  38. package/dist/index.d.ts +199 -0
  39. package/dist/index.js +2340 -0
  40. package/dist/index.js.map +1 -0
  41. package/dist/introspect-D3OaXHBc.d.cts +16 -0
  42. package/dist/introspect-D3OaXHBc.d.ts +16 -0
  43. package/dist/types-BRvMB6IM.d.cts +1491 -0
  44. package/dist/types-BRvMB6IM.d.ts +1491 -0
  45. package/package.json +125 -0
package/DESIGN.md ADDED
@@ -0,0 +1,881 @@
1
+ # Vex — Design Document
2
+
3
+ ## Purpose
4
+
5
+ Intent-based data access. A caller describes *what* data they need and the
6
+ *shape* they want it in. Vex synthesizes a query, executes it, and reshapes the
7
+ result to match. Because it caches each generation under a fingerprint, replaying that
8
+ fingerprint costs zero LLM calls.
9
+
10
+ **One sentence:** ask for data in English, get it in any shape, from any database.
11
+
12
+ The thesis: most applications have a small, stable set of data-access patterns.
13
+ Each pattern has a consistent output shape. Once the DSL (and any result
14
+ mapping) is generated for a shape, it is reused for every later request with
15
+ that shape — the LLM cost amortizes to zero, and the hot path is pure compile +
16
+ execute.
17
+
18
+ ---
19
+
20
+ ## Core principles
21
+
22
+ 1. **The DSL is the source of truth.** The LLM never emits SQL. It fills a
23
+ constrained JSON DSL that is Zod-validated before anything else touches it.
24
+ Scope, discovery, analysis, and compilation all operate on the DSL or on a
25
+ value derived from it.
26
+ 2. **Deterministic core, injected intelligence.** The engine itself contains no
27
+ LLM code. Natural-language→DSL and rows→shape are two optional function
28
+ hooks (`generateDsl`, `mapToShape`). With them, Vex answers free-form
29
+ requests. Without them, it is a deterministic DSL compiler + executor + cache
30
+ that still serves any already-cached shape. Cortex/Signal/Prism are optional
31
+ peer dependencies, not hard ones.
32
+ 3. **Shape-only caching.** The positive cache key is a hash of the request's
33
+ *shape* with all values replaced by type tokens. Two requests with the same
34
+ structure but different values share a cache entry. Intent is deliberately
35
+ excluded from this key.
36
+ 4. **Scope is server-side and LLM-invisible.** Access control is injected into
37
+ the DSL *after* generation, as ordinary filter clauses bound to server
38
+ `$scope` values. The model cannot see or forge them, so prompt injection
39
+ cannot bypass them.
40
+ 5. **One-level field paths.** Always `entity.field`, never
41
+ `entity.relation.field`. Every entity appears explicitly in `from`. This
42
+ keeps the resolver simple and entity discovery a pure string walk.
43
+ 6. **Postgres first, adapter interface from day one.** SQL generation is the
44
+ only database-specific step. It lives entirely behind the `DatabaseAdapter`
45
+ interface so other backends can follow without changing the core.
46
+ 7. **Schemas teach the model.** Every DSL schema field carries a Zod
47
+ `.describe()`. The JSON Schema generated from it — not prose — is what the
48
+ query agent reads to learn the DSL.
49
+
50
+ ---
51
+
52
+ ## Architecture
53
+
54
+ ```
55
+ ┌──────────────────────────────────────┐
56
+ QueryRequest │ { intent?, shape, context } │
57
+ + ExecuteOptions │ options: { scope?, cache?, entities?} │
58
+ └────────────────┬─────────────────────┘
59
+ ↓
60
+ ┌──────────────────────────────────────┐
61
+ │ Validate (Zod) + hash │
62
+ │ shapeHash (positive key) │
63
+ │ requestHash (negative key / single- │
64
+ │ flight) │
65
+ └────────────────┬─────────────────────┘
66
+ ↓
67
+ ┌──────────────────────────────────────┐
68
+ │ Cache read │
69
+ │ positive HIT → cached DSL (+ Prism IR)│
70
+ │ negative HIT → throw unsatisfiable │
71
+ │ miss + mode 'only' → throw cache_miss │
72
+ └────────────────┬─────────────────────┘
73
+ miss ↓ ↑ hit (skip generation)
74
+ ┌──────────────────────────────────────┐
75
+ │ generateDsl hook (LLM, optional) │
76
+ │ single-flighted; negative-caches a │
77
+ │ "cannot satisfy" result │
78
+ └────────────────┬─────────────────────┘
79
+ ↓
80
+ ── deterministic pipeline (runPipeline) ───────────────────────────
81
+ ┌──────────────────────────────────────┐
82
+ │ clamp limit → discover entities → │
83
+ │ apply scope → resolve → analyze → │
84
+ │ adapter.compile → CompiledQuery │
85
+ └────────────────┬─────────────────────┘
86
+ ↓
87
+ ┌──────────────────────────────────────┐
88
+ │ Missing-context check │
89
+ │ → return empty result + meta. │
90
+ │ missingContext (valid response) │
91
+ └────────────────┬─────────────────────┘
92
+ ↓
93
+ ┌──────────────────────────────────────┐
94
+ │ Bind params (context/scope/embed) → │
95
+ │ adapter.execute → raw rows │
96
+ └────────────────┬─────────────────────┘
97
+ ↓
98
+ ┌──────────────────────────────────────┐
99
+ │ Map to shape (optional) │
100
+ │ cached Prism IR → apply; else │
101
+ │ mapToShape hook (LLM) → IR + rows │
102
+ └────────────────┬─────────────────────┘
103
+ ↓
104
+ ┌──────────────────────────────────────┐
105
+ │ Cache write (by fingerprint) │
106
+ │ DSL + Prism IR + schema fingerprint │
107
+ └────────────────┬─────────────────────┘
108
+ ↓
109
+ ┌──────────────────────────────────────┐
110
+ │ { result, meta: { cache, context, │
111
+ │ timing, warnings? } } │
112
+ └──────────────────────────────────────┘
113
+ ```
114
+
115
+ On a cache hit, generation and mapping are skipped: the pipeline re-runs
116
+ deterministically against the cached DSL and applies the cached Prism IR. No
117
+ LLM calls. That re-run (resolve → analyze → compile → bind → execute) is what
118
+ makes scope and context values current on every request, even cached ones.
119
+
120
+ ---
121
+
122
+ ## Subsystems
123
+
124
+ ### The DSL
125
+
126
+ A constrained JSON structure (`schemas/`) that can only express safe, valid
127
+ queries. Top level (`QuerySchema`):
128
+
129
+ - `from` — array of entity names or `{ as, query }` subqueries (min 1).
130
+ - `fields` — array of `entity.field` paths (min 1; **required** — no `SELECT *`).
131
+ - `filter` — a recursive `FilterSchema` union.
132
+ - `compute` — record of `alias → ComputeExpression` (arithmetic, `concat`,
133
+ `coalesce`, `case`).
134
+ - `aggregate` — record of `alias → AggregateExpression` (`count`/`sum`/`avg`/
135
+ `min`/`max`).
136
+ - `groupBy`, `sort`, `limit`, `distinct`.
137
+
138
+ Filter operators: `eq`/`neq`/`gt`/`gte`/`lt`/`lte`, `in`/`notIn`, `like`/
139
+ `ilike`, `isNull`/`isNotNull`, `and`/`or`/`not`, plus `semantic` (vector
140
+ similarity) and `fuzzy` (string distance). Values are literals, `{ $context }`
141
+ (caller-provided) or `{ $scope }` (server-injected).
142
+
143
+ Two refinements over a naive design, both for the model's benefit: there is no
144
+ `offset` (cursor pagination via a `gt` filter is safer and cacheable) and no
145
+ `between` (it is sugar for `and(gte, lte)`). Fewer operators means a smaller
146
+ JSON Schema and better model adherence.
147
+
148
+ ### Scope
149
+
150
+ Two pure functions over the DSL:
151
+
152
+ - `discoverEntities(dsl)` walks every place an entity name can appear — `from`,
153
+ field paths, filter paths (recursively through `and`/`or`/`not`), compute and
154
+ aggregate references, `groupBy`, `sort`, and into subqueries — and returns the
155
+ set of referenced entities. No schema needed; pure string extraction.
156
+ - `applyScope(dsl, entities, policy)` looks up each discovered entity in the
157
+ policy and produces a new DSL. `public` rules pass through; `deny` (or no rule
158
+ under a `default: 'deny'` policy) throws `VexScopeError`; filter rules become
159
+ `{ eq|neq|in: ["entity.field", { $scope: "source" }] }` clauses AND-merged
160
+ into the existing filter. It recurses into subqueries.
161
+
162
+ Scope runs after generation and before resolution. The resolver, analyzer, and
163
+ compiler never know scope exists — they just process a DSL that happens to carry
164
+ extra filter clauses.
165
+
166
+ ### Engine (deterministic pipeline)
167
+
168
+ - **resolver** (`resolve`) turns the scoped DSL into a `ResolvedQuery`: field
169
+ paths → table aliases + columns, FK-based join discovery between entity
170
+ sources (a nullable FK resolves to a LEFT join so a null FK never drops the
171
+ referencing row; subquery sources are not FK-joined — the compiler cross-joins them,
172
+ and the outer query can reference their field/compute/aggregate outputs),
173
+ subquery recursion.
174
+ - **analyzer** (`analyze`) validates the resolved query and returns
175
+ `{ warnings, errors }`. Errors halt the pipeline; warnings ride along in the
176
+ response. Checks: nesting depth vs `maxNestingDepth`; cartesian products
177
+ (multiple entity sources with no connecting join); unindexed filter columns
178
+ (warn or reject); aggregates without `groupBy` (warn); vector field without an
179
+ HNSW/IVFFlat index for semantic search (warn).
180
+ - **executor** (`executeQuery`, `buildContextContract`, `findMissingContext`)
181
+ binds parameters and calls `adapter.execute`. Binding resolves each
182
+ `ParamSlot` from context (`context`), scope (`scope`), or by embedding text
183
+ (`semantic`). `findMissingContext` reports which required keys the caller
184
+ omitted so the runtime can return a valid-but-empty response instead of
185
+ failing.
186
+
187
+ ### Adapters
188
+
189
+ `DatabaseAdapter` is the whole database boundary: `introspect`, `compile`
190
+ (`ResolvedQuery → CompiledQuery`), `execute`, and a `capabilities` flag set.
191
+ Compilation is adapter-internal — there is no shared compiler in `engine/`; each
192
+ adapter owns its full SQL dialect (the Postgres adapter has its own
193
+ filter/compute/aggregate SQL generation under `adapters/postgres/`).
194
+
195
+ A `CompiledQuery` is `{ sql, paramSlots, contextContract }`. The ordered
196
+ `paramSlots` array drives positional binding (`$1`, `$2`, …); each slot knows
197
+ its `kind` (context/scope/semantic), `type`, and (for semantic) `dimensions`.
198
+ The text→vector `embed` function is an engine-level injected provider
199
+ (`createQueryEngine({ embed })`), used by the executor during binding for
200
+ `semantic` slots — the adapter only emits the vector-distance SQL, never embeds.
201
+
202
+ The Postgres adapter advertises vector search, JSON fields, full-text, CTEs,
203
+ window functions, RETURNING, and statement timeout; it coerces numeric column
204
+ strings (int8/numeric/float) back to JS numbers on the way out.
205
+
206
+ ### Cache — mechanics
207
+
208
+ Identity is the **fingerprint** (see Cache v2 below). The supporting hashes:
209
+
210
+ - **Request hash** (`computeRequestHash`) hashes intent + normalized shape +
211
+ *context key names* (not values). It identifies the *request* stored with a
212
+ named slot (match → hit, differ → regenerate-and-replace), and keys the
213
+ negative cache and single-flight de-duplication.
214
+ - **Schema fingerprint** (`computeSchemaFingerprint`) is a stable hash of the
215
+ structural schema (names, types, nullability, PKs, relations, indexes —
216
+ excluding volatile rowCount and cosmetic fields). It is stored on each entry;
217
+ a mismatch means the cached DSL may reference changed columns, so the entry is
218
+ treated as a miss and evicted. Stable across restarts so warm-up entries
219
+ aren't wrongly discarded.
220
+ - **Entry kinds.** A discriminated union: `ok` (DSL + optional Prism IR) and
221
+ `unsatisfiable` (a TTL'd negative result — a request the agent declared
222
+ impossible, so it isn't re-run until the TTL lapses or the schema changes).
223
+ - **Backends.** In-memory (`createMemoryCache`), durable Postgres
224
+ (`createPostgresCache`, jsonb table, server-side TTL, `protected`/
225
+ `last_used_at`/`request_hash` columns with idempotent ALTER migrations,
226
+ validate-on-write and evict-on-read), and tiered (`createTieredCache`, L1 +
227
+ L2 read-through/write-through with `full`/`lazy`/`partial` warm-up). Every
228
+ backend implements the same `CacheBackend` contract; `init`/`entries` are
229
+ optional capabilities used for warm-up and the discovery overview.
230
+ - **Validation** (`validateEntry`) guards every write and every promotion into
231
+ memory, so a corrupt or schema-drifted row can never poison the cache.
232
+ - **GC** (`sweepCache`) evicts entries idle past `maxIdleMs`
233
+ (`lastUsedAt ?? createdAt`), skipping protected ones. Hygiene, not necessity.
234
+
235
+ ### Cache v2 — fingerprints
236
+
237
+ Agreed and implemented 2026-07-10; replaced shape-keyed caching everywhere
238
+ (engine, HTTP layer, and all consumers — relay, fable, mythos, showroom).
239
+
240
+ **Why v1 has to go.** Shape-hash identity collides: two different intents with
241
+ the same normalized shape share one slot, so the second silently returns the
242
+ first's data stamped `hit: true`. Entries are anonymous — no way to pin,
243
+ promote, iterate, or roll back a generation. And running a query writes the
244
+ cache as a side effect, so *testing* and *publishing* are the same physical
245
+ act: callers either bypass the cache and lose the artifact, or hit it and can
246
+ never change it.
247
+
248
+ **The model: one identity, `fingerprint`.**
249
+
250
+ - A request WITHOUT a fingerprint generates and caches; the server mints a
251
+ fingerprint (`fp_…`) and returns it in `meta.cache.fingerprint`.
252
+ - A minted fingerprint is an **immutable pin** — embed it and replay the exact
253
+ generation forever. Changing the query means proving again and embedding the
254
+ new fingerprint. (git: a commit hash.)
255
+ - A caller-chosen fingerprint (any string) is a **mutable named slot** — the
256
+ name is yours, its content follows your request. (git: a branch.)
257
+ - Intent NEVER keys anything: natural language is fake identity in both
258
+ directions. It remains generator input + human metadata.
259
+
260
+ **Three request postures** — the body replaces every cache mode:
261
+
262
+ ```jsonc
263
+ // 1. Explore: generate, cache, mint
264
+ { "intent": "top 10 open deals by value…", "shape": [{ "id": "", "value_display": "" }] }
265
+ // → meta.cache: { "fingerprint": "fp_9k27ax", "hit": false }
266
+
267
+ // 2. Replay: fingerprint alone (+ context VALUES welcome) — never generates
268
+ { "fingerprint": "fp_9k27ax", "context": { "sortBy": "deals.value" } }
269
+ // → meta.cache: { "fingerprint": "fp_9k27ax", "hit": true }; unknown fp → 404
270
+
271
+ // 3. Own a slot: fingerprint + request
272
+ { "fingerprint": "deals/table", "intent": "…", "shape": [ … ] }
273
+ // stored request matches → hit; differs → regenerate + REPLACE the slot
274
+ // (meta.cache.replaced: true) — unless protected → 409
275
+ ```
276
+
277
+ Request equality = intent string + normalized shape + context KEY NAMES
278
+ (`normalizeShape` survives for this comparison; context values are runtime
279
+ data and never identity — which is what makes replays parameterizable:
280
+ `sortBy`/search terms may vary per call against one cached DSL).
281
+
282
+ **Lifetime = usage.** Every replay stamps `lastUsedAt`; a GC sweep evicts
283
+ entries unused for N days. An embedded pin is touched on every mount and lives
284
+ forever by being used; an abandoned exploration ages out. There are NO
285
+ provisional/permanent tiers and NO promote API — the distinction is replaced
286
+ by the convention *alive = used*. (Entries are single-digit KB; the sweep is
287
+ hygiene, not necessity.)
288
+
289
+ **Protection: one stored bit, two writers, one knob.**
290
+
291
+ - `protected: true` on an entry means a differing request 409s instead of
292
+ replacing. Written ONLY by (i) the seed path and (ii) an explicit `PATCH`
293
+ (which is what "save/lock this view" calls). Runtime writes are never
294
+ protected by default.
295
+ - One endpoint knob: `locked: true` = replay-only posture (no generation at
296
+ all — the production stance; v1's `cache: 'only'` promoted from caller flag
297
+ to endpoint config).
298
+ - No governance fields ever ride in the query body.
299
+
300
+ **The REST surface.** Vex is an API endpoint over REST and is designed as one.
301
+ Scoped mounts (`/api/deals/vex`) ride the existing `options.entities` filter —
302
+ scope constrains *generation* (smaller schema, no out-of-scope joins);
303
+ identity stays global.
304
+
305
+ - `POST /api/<scope>/vex` — execute (the three postures above). Errors:
306
+ unknown fingerprint replay → 404 `cache_miss`; protected slot + differing
307
+ request → 409 `fingerprint_protected`; locked endpoint asked to generate →
308
+ 403 `locked`.
309
+ - `GET /api/<scope>/vex` — self-description: `{ protection:
310
+ "all"|"some"|"none", locked, fingerprints: [{ fingerprint, protected,
311
+ schemaFresh, intent, lastUsedAt }] }` plus the schema/DSL/body docs. Doubles
312
+ as agent-facing discovery: "does a named query already cover this?" is one
313
+ call.
314
+ - `PATCH /…/vex` with `{ fingerprint, protected }` — flip the bit. `DELETE`
315
+ with `{ fingerprint }` — evict (unprotected only). Fingerprints ride the
316
+ body, not the path — names contain `/`.
317
+
318
+ **What dies with v1:**
319
+
320
+ - `CacheMode` (`use`/`refresh`/`bypass`/`only`), `ExecuteOptions.cache`, the
321
+ `?cache=` query param — the request body says everything now.
322
+ - The shape hash as a cache key (and with it the entire collision class).
323
+ - The embed-VERBATIM requirement for generated screens: a pin has no wording
324
+ to drift, and context may vary per replay — sorting and parameterized
325
+ reloads become legal on cached queries again.
326
+
327
+ **Unchanged:** the negative cache (request-hash keyed, TTL'd), schema-
328
+ fingerprint freshness/eviction, backends and tiering, `validateEntry`.
329
+
330
+ **Where it landed:**
331
+
332
+ - Engine (`engine/runtime.ts`): `resolveFingerprint` — no fp → mint; fp alone
333
+ → replay or `cache_miss`; fp + request → hit on matching request hash,
334
+ regenerate-and-replace on mismatch (409 when protected). Fingerprint-only
335
+ replays use the entry's STORED shape for the array-vs-single envelope.
336
+ Replays touch `lastUsedAt`.
337
+ - Request schema: `fingerprint` in; `intent`/`shape` optional. Meta:
338
+ `cache: { hit, fingerprint, replaced?, intent? }`.
339
+ - Backends: keys are fingerprint strings; entries carry `protected`,
340
+ `lastUsedAt`, `requestHash`; `sweepCache` GC; postgres migrates in place.
341
+ - HTTP: `handleQuery` (no cache param), async `handleDiscovery` overview,
342
+ `handleFingerprintPatch`/`handleFingerprintDelete`; express + hono adapters.
343
+ - Consumers: seeds re-keyed to protected names (`deals/list`, `todos/open`,
344
+ …); every hand-authored read replays `{ fingerprint, context }`; relay's
345
+ architect embeds `{ fingerprint: meta.cache.fingerprint, context }`; mythos'
346
+ `/api/query` is `locked` (replay-only); showroom's demos own
347
+ `vex-demo/<id>` slots (canned = seed + replay, live = delete + regenerate).
348
+
349
+ ### Mutations (write pipeline)
350
+
351
+ Writes are first-class but deliberately NARROWER than reads. The asymmetry is
352
+ the design: a bad generated query shows wrong data, a bad generated write
353
+ destroys it — so mutations have **no generation path at all**. A mutation is
354
+ a dev-authored artifact that enters through the seed path, lives in the cache
355
+ as a `kind: 'mutation'` entry (the def in the same jsonb slot a query's DSL
356
+ uses), and is invoked by fingerprint. The def never travels: an inline
357
+ `{ mutation }` body is not a request shape, and the wire is ONE shape for
358
+ both kinds — `POST { fingerprint, context }`, dispatched by the entry's kind.
359
+ Replay-only holds under every posture (`locked` only ever governed query
360
+ generation); the query engine refuses a mutation fingerprint explicitly.
361
+
362
+ The pipeline: validate (closed grammar — `$scope` unauthorable, update/delete
363
+ require a WHERE) → desugar (upsert → insert/update by key presence) →
364
+ require context (a write never executes with holes; the `missing_context`
365
+ error carries the FULL derived signature) → scope (the same `ScopePolicy`
366
+ reads use: `set` writes identity on insert and update, `match` pins rows) → column
367
+ check against the introspected schema → compile (through the read
368
+ pipeline's own `compileFilter`/`compileFieldOrValue`/`resolveParams`, so
369
+ values bind as SQL parameters exactly like reads) → execute. Batches run in
370
+ one transaction; the client is structural (`MutationClient` — PGlite, a pg
371
+ wrapper, a test double), so the core imports no driver.
372
+
373
+ **Derived context signatures** close the discoverability loop for both
374
+ kinds: a `$context` ref sits at a position whose column the schema types, so
375
+ "what do I pass this fingerprint" is computed from the stored def — never
376
+ authored, never stale. Discovery lists every entry with kind, intent, the
377
+ typed context contract, and shape (reads) or effect (`{op, table, columns}`,
378
+ writes). An authoring lint (`lintMutation`) refuses un-keyed update/delete
379
+ defs at seed time.
380
+
381
+ ### Reactive reads (`refresh`)
382
+
383
+ Agreed and built 2026-09-26. A read can keep answering.
384
+
385
+ **The declaration is on the entry**, because freshness is a property of the
386
+ question — "the current standings" is live, "the answer as asked" is not:
387
+
388
+ ```ts
389
+ { fingerprint: 'members/roster', refresh: 'reactive', dsl: { … } }
390
+ ```
391
+
392
+ `refresh` is a mode, not a flag: `'snapshot'` (the default, and what every read
393
+ was before this existed — answered when asked, never again) or `'reactive'`. It
394
+ will grow; `'clock'` — time as a source of change, for a read that depends on
395
+ `today` — is the next state, and is not accepted until it is built, because a
396
+ value the schema takes and the engine ignores would read as live and behave as
397
+ a snapshot. Only a stored entry can be reactive (the mode is authored, never
398
+ generated), and only one whose mapping is compiled, because a later answer must
399
+ never wait on a model. It persists like `reach`: a column in the Postgres
400
+ cache, validated on write, compared by the seed path.
401
+
402
+ **The caller follows by passing `{ signal, onChange }`** to `execute` — the
403
+ first answer is the returned promise as always; every later answer that
404
+ differs goes to `onChange` until `signal` aborts. Over HTTP there is no
405
+ callback, so `handleQuery` takes an optional in-process `VexLive`, and the
406
+ hono adapter reads one from the request's env — the third argument of hono's
407
+ `app.request`, which only an in-process caller can set. A read nobody follows
408
+ is simply a read answered from the rows cache.
409
+
410
+ **The rows cache** (`engine/live.ts`) holds two things apart:
411
+
412
+ - **rows** — key → the rows a reactive query returned. The only large thing,
413
+ bounded by age (`ttlMs`, default 60 s), count (`maxEntries`) and text
414
+ (`maxBytes`, measured as each result's JSON length), least-recently-used
415
+ first out; a result over `maxEntryBytes` is never stored. Evicting rows never
416
+ breaks anything; it costs one query. It lives in process memory, per engine.
417
+ - **follows** — key → the tables the query reads, how to run it again, and who
418
+ to tell. Bounded by what is followed, not by this module: a follow ends when
419
+ its last follower's signal aborts.
420
+
421
+ **The key is the compiled SQL and its bound values.** Scope compiles into both
422
+ — row rules into the SQL, scope values into the parameters — so two callers
423
+ share rows exactly when they would provably get the same rows. The cache never
424
+ reasons about policy. The MAPPING, which reads `$.scope` (a reader's
425
+ language), runs per caller over the shared rows: one query for twenty
426
+ viewers, twenty cheap mappings.
427
+
428
+ **A write bumps its tables' versions.** Stored rows remember the versions they
429
+ were read at, and are stale when those moved — invalidation is a counter bump,
430
+ not a walk over the cache. The same stamp closes the race that would otherwise
431
+ poison it: rows fetched by a read that started before a write and finished
432
+ after it carry the versions from before, so they are stale the moment they
433
+ land, and a new read does not join that fetch.
434
+
435
+ **Who invalidates: the handler, after every committed mutation** —
436
+ `engine.invalidate(tablesChangedBy(writes, schema))`. It sits there, and not in
437
+ any host's observer, because this is the one door every vex write passes: an
438
+ HTTP request, a host's in-process machinery write, a reflex's effect. A read
439
+ that heard about only some writes would be live only sometimes, which is worse
440
+ than a snapshot that says what it is. `tablesChangedBy` adds, for a delete,
441
+ every table that references a deleted one, transitively — a cascade changes
442
+ rows vex never wrote, and the introspected schema carries the reference but
443
+ not its rule, so a delete is assumed to reach them all (over-telling costs a
444
+ refetch; under-telling leaves a screen stale). Writes vex never sees — raw SQL,
445
+ a database trigger — are not invalidations; a host that makes them calls
446
+ `engine.invalidate` itself, and the TTL heals the rest.
447
+
448
+ **Followed reads refetch once per burst** (`debounceMs`, trailing) — thirty
449
+ writes are one query — and are numbered, so a slow refetch landing after a
450
+ newer one is dropped. Change is decided twice: once per key on the rows (the
451
+ same rows → nobody is mapped), then per follower on its own mapped answer
452
+ (canonical JSON, hashed), so a follower hears only when *its* answer changed.
453
+ Every followed read is also re-checked once per TTL, bypassing the cache — a
454
+ lost invalidation heals within one TTL. A refetch that fails keeps the last
455
+ good answer and says so (`rows.error`).
456
+
457
+ **Visibility.** `engine.rows.stats()` — entries, bytes, follows, followers,
458
+ hits, misses, evictions, oversized, refreshes — and `rows.evict` /
459
+ `rows.error` events on the engine's `onEvent`. An engine evicting faster than
460
+ it answers warns once a minute: its `rows` limits are too small for what it
461
+ holds.
462
+
463
+ **Rules for authors.** A reactive entry should `sort` — an unordered read can
464
+ come back in another order after a write and count as changed (harmless: one
465
+ extra answer). It reads time from `$scope`, never context: the values bound
466
+ when the read was made are the ones it refetches with.
467
+
468
+ ### Who "me" is (`behaviors`, `GenerationCaller.bindings`)
469
+
470
+ Built 2026-09-27. A generated query can mean the caller — "what's my name?",
471
+ "my orders", "after me" — without the model ever seeing who that is.
472
+
473
+ **The meaning comes from what the host declared, never from a name.** The
474
+ host's behaviors (the `ScopeBehaviors` its policies are compiled from) bind
475
+ columns to scope keys: `{ set: 'member_id', to: 'userId' }`,
476
+ `{ match: 'studio_id', to: 'tenantId' }`. Scope values are always the caller's
477
+ own, so a column bound to a key the caller carries holds that value on the
478
+ caller's own rows — whether it is called `user_id`, `guest_id` or `id`.
479
+
480
+ **Why the behaviors and not the policy.** A compiled policy keeps only the
481
+ phases a caller is granted. A member who may only read `members` has
482
+ `members: { read: [] }`; the rule that says `member_id` is them lives on an
483
+ insert they do not hold. So the engine takes the behaviors themselves
484
+ (`QueryEngineConfig.behaviors`) and reads every reach and every phase.
485
+
486
+ **What a generation is handed**: `GenerationCaller.bindings` — every column
487
+ bound to a scope key this caller actually carries, on a table their policy lets
488
+ them read. Keys and columns, never values. A caller carrying none of the keys —
489
+ machinery, an `executeAs` job — has no self, and is told so; nothing else about
490
+ it is special. The reference agent turns the bindings into one context line
491
+ (`describeCaller`): filter such a column with `{ $scope: key }` to mean the
492
+ caller, and read the caller's row as an aliased subquery source to compare other
493
+ rows against it.
494
+
495
+ **Safety is unchanged.** The bindings are guidance for the model; access is
496
+ still the policy, and the value is still bound by the engine. A query written
497
+ this way is stored like any other and answers for whoever replays it — one
498
+ "what's my name?" serves everybody.
499
+
500
+ **Checked at `introspect`**: every column a behavior names must exist, or the
501
+ engine refuses to start.
502
+
503
+ ### LLM integration (decoupled)
504
+
505
+ The engine takes two optional hooks:
506
+
507
+ - `generateDsl(request, schema, caller) → Query` — natural language + shape →
508
+ DSL. `schema` is only the tables the caller's policy can read; `caller.read`
509
+ runs a DSL through the engine's pipeline as that caller. A generation sees
510
+ what the person asking could see — and it is keyed (single-flight, negative
511
+ cache) by request AND policy, because two callers who see different tables
512
+ are asking different agents.
513
+ - `mapToShape(rows, shape) → { ir, transformed }` — raw rows → requested shape,
514
+ returning a compiled Prism IR that is cached for reuse.
515
+
516
+ This is the seam that keeps the core deterministic. Vex ships a reference
517
+ implementation of both, built on `@niscorp/cortex` and `@niscorp/prism` and
518
+ exported from the `@niscorp/vex/agent` subpath — `createQueryDsl` (fills
519
+ `generateDsl`) and `createShapeMapper` (fills `mapToShape`):
520
+
521
+ - The **query agent** (`vexQueryDslAgent`, id `vex.query`) is a structured-output
522
+ Cortex agent over `QuerySchema` with six tools — `getSchema`, `getSampleRows`,
523
+ `getDistinctValues`, `describeField`, `testQuery`, `cannotSatisfy` — and two
524
+ rules: a tool-call limiter (nudge at 8, abort at 10) and an unsatisfiable
525
+ abort. Context producers inject the live schema and the DSL JSON Schema so the
526
+ static instructions stay stable. Every data-touching tool is a DSL run through
527
+ `caller.read`; the tool set holds no adapter and no SQL.
528
+ `createQueryDsl` wraps it into the `generateDsl` hook.
529
+ - The **mapping agent** is Prism's exported `mappingAgent`; `createShapeMapper`
530
+ builds the same envelope the runtime replays against — `{ result: rows }` for an
531
+ array shape, `{ result: rows[0] ?? null }` for an object shape — runs the agent
532
+ once to get a transform config, compiles it to IR, and applies the IR once over
533
+ that envelope (the identity, with no model call, when the rows already fit).
534
+
535
+ > The reference agents are a separate concern from the deterministic core: they
536
+ > live behind the `@niscorp/vex/agent` subpath, so `@niscorp/cortex`,
537
+ > `@niscorp/signal`, and `@niscorp/prism` are pulled in only when you import it.
538
+ > The engine itself depends on none of them — provide your own
539
+ > `generateDsl`/`mapToShape`, or use the reference factories.
540
+
541
+ ### HTTP layer (optional)
542
+
543
+ Vex is a library; it does not own a server. But it ships two thin framework
544
+ adapters and a framework-agnostic handler so a resource can be exposed with a
545
+ few lines:
546
+
547
+ - `handler.ts` — `handleDiscovery` returns a self-describing JSON document
548
+ (entities, fields, relations, the request contract, and the DSL JSON Schema);
549
+ `handleQuery` parses a body, runs `engine.execute`, and maps `VexError`s to
550
+ HTTP status codes.
551
+ - `@niscorp/vex/hono` and `@niscorp/vex/express` wrap that handler: `GET`
552
+ returns discovery, `POST` runs a query, scope is resolved per-request via a
553
+ `getScope` callback, and the `cache` query param selects the cache mode.
554
+
555
+ ### Events
556
+
557
+ An optional `onEvent` handler receives a typed `VexEvent` stream across the
558
+ pipeline — `query.start`, `query.cache`, `query.dsl`, `query.sql`,
559
+ `query.rows`, `query.mapped`, `query.done`, `query.error`, plus `llm.request`/
560
+ `llm.response` for callers that wrap their LLM client. This is how the dev
561
+ server renders per-stage timing and a full LLM transcript log.
562
+
563
+ ---
564
+
565
+ ## File structure
566
+
567
+ ```
568
+ src/
569
+ index.ts Public API barrel
570
+ types.ts QueryEngine, QueryEngineConfig, ExecuteOptions, hook types
571
+ errors.ts VexError (code + details)
572
+ events.ts VexEvent union, VexEventHandler
573
+ handler.ts Framework-agnostic discovery + query handlers
574
+
575
+ schemas/
576
+ query.schema.ts QuerySchema (top-level DSL) + Source, SortEntry
577
+ filter.schema.ts FilterSchema (recursive union)
578
+ compute.schema.ts ComputeExpressionSchema
579
+ aggregate.schema.ts AggregateExpressionSchema
580
+ value.schema.ts FieldOrValue, ContextRef ($context), ScopeRef ($scope)
581
+ database.schema.ts DatabaseSchema, EntitySchema, FieldSchema, ...
582
+ request.schema.ts QueryRequestSchema, QueryResponse, error codes
583
+ index.ts Barrel
584
+
585
+ scope/
586
+ discover.ts discoverEntities(dsl)
587
+ apply.ts applyScope(dsl, entities, policy), VexScopeError
588
+ scope.types.ts ScopePolicy, ScopeEntityRule, ScopeFilterRule
589
+
590
+ engine/
591
+ runtime.ts createQueryEngine — orchestration + cache + single-flight
592
+ resolver.ts resolve(dsl, schema) → ResolvedQuery
593
+ analyzer.ts analyze(resolved, config) → { warnings, errors }
594
+ executor.ts executeQuery, buildContextContract, findMissingContext
595
+ live.ts createLiveRows — the rows cache and follows behind refresh: 'reactive'
596
+ engine.types.ts ResolvedQuery and friends, AnalysisConfig, TestResult
597
+
598
+ adapters/
599
+ adapter.types.ts DatabaseAdapter, CompiledQuery, ParamSlot, capabilities
600
+ postgres/
601
+ postgres.adapter.ts createPostgresAdapter
602
+ introspect.ts schema discovery from pg_catalog
603
+ compile.ts ResolvedQuery → parameterized SQL
604
+ operators.ts per-operator SQL generation
605
+ hono/index.ts vex() Hono app (@niscorp/vex/hono)
606
+ express/index.ts vex() Express handler (@niscorp/vex/express)
607
+ index.ts adapter barrel
608
+
609
+ mutations/
610
+ schema.ts MutationSchema (closed grammar), MutationDefinitionSchema
611
+ engine.ts executeMutation — desugar/scope/validate/compile/execute
612
+ signature.ts collectMutationContext, collectQueryContext, mutationEffect, lintMutation
613
+ index.ts Barrel
614
+
615
+ cache/
616
+ cache.types.ts CacheBackend, CacheEntry (ok | mutation | unsatisfiable)
617
+ hash.ts computeShapeHash, computeRequestHash, computeSchemaFingerprint
618
+ memory.ts createMemoryCache (L1)
619
+ postgres.ts createPostgresCache (L2, durable)
620
+ tiered.ts createTieredCache (L1 + L2, warm-up)
621
+ validate.ts validateEntry
622
+ util.ts isEntryFresh, fireAndForget
623
+ index.ts Barrel
624
+
625
+ agent/ Reference Cortex agents (exported via ./agent)
626
+ query.agent.ts defineAgent: vexQueryDslAgent (vex.query → Query)
627
+ tools.ts the six query tools
628
+ rules.ts tool-limit + unsatisfiable rules
629
+ producers.ts schema + DSL-spec context producers
630
+ index.ts createQueryDsl, createShapeMapper, vexQueryDslAgent
631
+
632
+ utils/
633
+ context.ts buildValidationContext, resolveParams
634
+
635
+ scripts/ Dev-only: docker, seed, dev server, fixtures
636
+ ```
637
+
638
+ ---
639
+
640
+ ## Dependencies
641
+
642
+ | Package | Relationship | Purpose |
643
+ |---------|--------------|---------|
644
+ | `zod` | peer (required) | Schema validation + JSON Schema generation |
645
+ | `pg` | peer (optional) | Postgres adapter and Postgres cache backend |
646
+ | `hono` | peer (optional) | Hono framework adapter |
647
+ | `express` | peer (optional) | Express framework adapter |
648
+ | `@niscorp/cortex` | peer (optional) | Reference query agent (`defineAgent`/`defineTool`/`defineRule`, standalone runtime) |
649
+ | `@niscorp/signal` | peer (optional) | LLM calls + embeddings, via Cortex |
650
+ | `@niscorp/prism` | peer (optional) | Result mapping (`mappingAgent`, IR compile/execute) |
651
+
652
+ Only `zod` is mandatory. Everything else is pulled in only by the path you use.
653
+
654
+ ---
655
+
656
+ ## Key design decisions
657
+
658
+ 1. **Constrained DSL, not text-to-SQL.** Security (the model can't emit
659
+ injection through a validated JSON union), cacheability (a structure hashes
660
+ more reliably than free text), and safety (the analyzer rejects cartesian
661
+ products and over-deep nesting *before* SQL exists).
662
+
663
+ A validated JSON union is only injection-proof if every string in it reaches
664
+ SQL as something the union can vouch for. Authored text reaches a statement
665
+ exactly three ways: a **column** that resolved against the introspected
666
+ schema, a **literal** quoted the way Postgres's `quote_literal` does (safe
667
+ under either `standard_conforming_strings`), or an **output name** held to
668
+ `ident` and quoted. Values from a request bind as parameters and never reach
669
+ text at all. The name grammar (`identifier.schema.ts`) is checked three
670
+ times — at parse, in the resolver for a DSL that skipped parsing, and by a
671
+ compiler that refuses any field position that did not resolve.
672
+
673
+ ⟲ It was not, until 2026-09-24. Field positions (`isNull`, `in`, `like`,
674
+ aggregate arguments) were typed `z.string()`; the resolver skipped any
675
+ string without a dot and the compiler pasted it into the statement, so
676
+ `{ isNotNull: "(SELECT secret FROM …)" }` was a boolean oracle over a table
677
+ the policy denied. Output names went into `AS "…"` unescaped, and a literal
678
+ containing a backslash could close its own string on a server with
679
+ `standard_conforming_strings` off. `test/security/injection.test.ts` is the
680
+ attack list, against a real Postgres; it failed 35 of 41 before the fix.
681
+
682
+ 2. **The LLM is injected, not embedded.** Making `generateDsl`/`mapToShape`
683
+ plain hooks keeps the engine deterministic and testable without a model,
684
+ lets the heavy Cortex/Signal/Prism stack stay optional, and lets a consumer
685
+ swap in their own generator. The bundled Cortex agents are a reference, not a
686
+ requirement.
687
+
688
+ 3. **Fingerprint identity, minted or named.** The output shape still
689
+ determines the DSL and the mapping — but the cache *key* is a fingerprint,
690
+ not the shape. A first request mints one (`fp_…`); a caller may instead own
691
+ a named slot. Shape-hashing collided (two intents, one shape, one slot) and
692
+ left every entry anonymous; a fingerprint makes a generation an addressable,
693
+ replayable artifact — pin it, replay it, roll it back. (See *Cache v2*.)
694
+
695
+ 4. **Scope injected server-side, after RESOLUTION.** If the model could see
696
+ scope filters it could be talked out of them. Binding them to `$scope`
697
+ values after the DSL is fixed makes them unforgeable.
698
+
699
+ The policy decides whether scope applies; the values only fill it. An
700
+ engine with a policy enforces it on every run, and a `$scope` slot nobody
701
+ filled refuses (`missing_scope`) rather than binding NULL — which matches no
702
+ row on a read and stamps NULL on a write.
703
+
704
+ ⟲ Until 2026-09-24 scoping switched on only when a policy AND values were
705
+ both present. Omitting the values turned the policy off entirely, table
706
+ access included — a call that forgot to say who it was for read as
707
+ everybody, and one did (relay's record lookup). Writes had always required
708
+ both; reads now agree. The generating agent's tools had the same hole by a
709
+ different route: they ran their own SQL, unscoped, beside the pipeline.
710
+ They now reach the database only through `caller.read`.
711
+
712
+ Scope answers two questions and they run at different stages, which is the
713
+ correction to an earlier version that ran both at once:
714
+
715
+ - **May this caller touch this table?** — `checkScope`, over entity names,
716
+ before resolution. Needs no schema and no aliases.
717
+ - **Where does the row rule go?** — `scopeResolved`, after resolution,
718
+ because the answer depends on HOW the table is reached.
719
+
720
+ That second stage exists because merging row rules into the DSL's `filter`
721
+ compiles them to WHERE, and **a WHERE predicate on a left-joined table is a
722
+ LEFT JOIN demoted to an INNER one.** A driving row whose optional FK is null
723
+ gets null-padded columns, `null = $tenant` is null rather than true, and the
724
+ row disappears. No error, no warning, a shorter list — and invisible until
725
+ the data actually contains a null FK, which in one consumer took months.
726
+ The resolver had already gone to the trouble of inferring `left` for exactly
727
+ this reason; the scope layer was undoing it two steps later.
728
+
729
+ Row rules for left-joined tables now go in that join's `ON` clause. The
730
+ boundary is unchanged — another tenant's row still cannot contribute
731
+ columns — it simply no longer annihilates the row that had none.
732
+
733
+ A consequence worth having on its own: scope writes nothing into the
734
+ authored `Query`. The document integrations ship over the wire carries no
735
+ engine-trusted field, so there is no key to strip at intake and no forgery
736
+ surface to remember.
737
+
738
+ 10. **`exists` shadows SQL rather than inventing a vocabulary.** Correlated
739
+ subqueries were the one thing the filter grammar could not express, so
740
+ "issues nobody was sent to" and "rooms nobody is in" became two round trips
741
+ each — a read returning a flat array of ids, then a `notIn` against it.
742
+ That is fine at three dozen rows, wrong at three thousand, and racy in
743
+ between.
744
+
745
+ `{ exists: { from, filter } }` is the same `{ from, filter }` a query
746
+ already is, and nothing more: no fields, no sort, no limit. EXISTS asks
747
+ whether a row is there, so anything shaping output would be noise — and
748
+ refusing it is what keeps this an operator rather than an invitation to
749
+ nest arbitrary SQL. The correlation needs no new syntax at all, because a
750
+ dotted string on either side of a comparison is already a field path:
751
+
752
+ ```json
753
+ { "exists": { "from": ["tasks"],
754
+ "filter": { "eq": ["tasks.issue_id", "issues.id"] } } }
755
+ ```
756
+
757
+ `not` composes for NOT EXISTS. Whoever knows SQL writes this on the first
758
+ try, which was the entire design constraint.
759
+
760
+ Two implementation notes that are load-bearing. It compiles **inline**,
761
+ into the parent's own parameter counter — the other subquery path compiles
762
+ independently and renumbers afterwards, which is how two parameterised
763
+ subqueries came to collide on `$1`; there is nothing to renumber if nothing
764
+ was numbered apart. And its inner aliases come from the **shared** alias
765
+ counter, so an `exists` over the same table as the outer query cannot take
766
+ the same alias and silently correlate a table with itself.
767
+
768
+ Scope recurses into it. Access-checking the inner table is not enough: an
769
+ EXISTS returns a boolean, but a boolean about somebody else's rows is still
770
+ an answer about somebody else's rows, and an uncorrelated `exists` over a
771
+ scoped table would otherwise report whether ANY tenant had one.
772
+
773
+ 5. **Negative caching with a TTL.** An "impossible" request is expensive to
774
+ re-discover every time, so a `cannotSatisfy` result is cached — but TTL'd and
775
+ fingerprinted, because a schema change can make yesterday's impossible
776
+ request possible.
777
+
778
+ 6. **Single-flight on the default path.** A burst of identical cache misses
779
+ collapses to one generation; the rest await it. `refresh`/`bypass` opt out,
780
+ since they explicitly mean "don't share cache state."
781
+
782
+ 7. **Adapters own compilation.** The DSL and agent loop are database-agnostic;
783
+ only SQL generation is not. Keeping the entire compile step inside the
784
+ adapter means a new backend is a self-contained unit and the core never grows
785
+ dialect knowledge.
786
+
787
+ 8. **A discovery endpoint, not just a query endpoint.** A self-describing
788
+ resource (`GET` → entities + contract + DSL JSON Schema) lets a client — or
789
+ another agent — learn how to query it without out-of-band documentation.
790
+
791
+ 9. **Writes are replay-only, one wire shape with reads.** Mutations are
792
+ dev-authored cache entries (`kind: 'mutation'`) invoked by fingerprint;
793
+ the def never travels and no generation path exists. The client-facing
794
+ API is uniformly `{ fingerprint, context }` — the entry's kind picks the
795
+ pipeline. Context signatures are DERIVED from the stored def (typed from
796
+ the schema by position), so the input contract can't drift.
797
+
798
+ ---
799
+
800
+ ## Deferred
801
+
802
+ - Dynamic mutation generation — icebox, on purpose. If it ever returns, it
803
+ feeds the existing pipeline (author → validate → seed) behind an effect
804
+ contract and an approval step; nothing shipped today gets replaced.
805
+ - Mutation transactions with cross-statement refs (`$returned.id` — the
806
+ create-parent-then-children case). Batches are atomic today but cannot
807
+ reference earlier statements' results.
808
+ - MySQL / SQLite adapters — the interface is ready; implementation on demand.
809
+ - Multi-level field paths — addable in the resolver without DSL changes.
810
+ - ~~Intent-aware positive caching~~ — superseded: collisions did bite, and the
811
+ answer is identity, not intent-matching. See **Cache v2 — fingerprints**.
812
+ - Cross-process cache coherence for the tiered backend (an L1 TTL refresh).
813
+ - `refresh: 'clock'` — time as a source of change: a reactive read whose bound
814
+ scope includes a clock value (`today`) refetches when that value next moves.
815
+ - Cross-process invalidation for reactive reads — a host carries
816
+ `engine.invalidate` between processes (moss's fabric); until then each
817
+ process's TTL bounds the staleness.
818
+ - Row-level pruning — for a single-table reactive read, test a written row
819
+ against the scoped filter and skip the refetch. An optimisation, not a design.
820
+ - Adapter-owned vector serialization — the pgvector `[...]` literal is currently
821
+ formed in the generic binding step; a second adapter would move it behind the
822
+ adapter.
823
+
824
+ ## Reach is the caller's, not the table's
825
+
826
+ A `ScopePolicy` fuses two questions: which phases an entity has, and what a
827
+ granted phase does. The second used to be answered per TABLE — one `match` rule
828
+ per entity, applied to every caller that held any grant on it.
829
+
830
+ That is right until two callers need the same grant at different distances. "The
831
+ desk reads every booking; a member reads their own" cannot be said with one rule
832
+ per table, and the workaround is always the same: a SECOND TABLE carrying the
833
+ tighter rule, kept level with the first by a trigger. One fact in two places is
834
+ a drift bug waiting for the first ordering nobody thought about.
835
+
836
+ So a table may declare NAMED rule sets, and `createScopePolicy` takes the
837
+ caller's profile:
838
+
839
+ bookings: { default: { read: [tenant] },
840
+ personal: { read: [tenant, own] } }
841
+
842
+ createScopePolicy(grants, behaviors) // studio-wide
843
+ createScopePolicy(grants, behaviors, 'personal') // + their own rows
844
+
845
+ The profile is chosen ONCE per caller rather than per grant, because "acts for
846
+ themselves" is a property of the caller and not of each thing they read — and
847
+ saying it once is one place to get it wrong instead of eight. A table declaring
848
+ no variant under a profile falls back to its `default`, which is what keeps
849
+ shared reads (a timetable everyone sees the same way) free of per-profile
850
+ entries.
851
+
852
+ It fails closed in the direction that matters. An unknown profile denies
853
+ everything rather than falling back: a mistyped name that quietly meant "the
854
+ default" would widen a caller to every row of every table they hold a grant on,
855
+ which is the one failure mode a policy layer may not have. A table that declares
856
+ named variants and no `default` likewise refuses an unprofiled caller instead of
857
+ guessing that "no rule" is safe.
858
+
859
+ AND A CALLER IS NOT ONE ROLE. Reach belongs to a role, and people hold several —
860
+ the instructor who trains at the studio they teach at is the ordinary case, not
861
+ the edge. So the unit of compilation is the ROLE: one policy each, merged
862
+ (`mergeScopePolicies`), broadest rule set winning per entity and phase. A
863
+ principal may do anything any of their roles permits.
864
+
865
+ That merge has a consequence worth stating plainly: for somebody holding both, a
866
+ staff role's studio-wide reach WINS over a member role's personal one on the
867
+ tables they share. Correct for the roster they are paid to read, wrong for their
868
+ own membership card.
869
+
870
+ SO REACH IS SAID TWICE, and by two different parties. The CALLER carries a floor
871
+ — how far this principal may see, from their role. The ENTRY may carry a ceiling
872
+ — how far this READ may see, whoever asks (`reach`). "Their own card" is a
873
+ property of the question, not of whoever happens to be asking it, and an app that
874
+ could only say it about people was an app that got it wrong for anybody holding
875
+ two roles.
876
+
877
+ The two compose in one direction only: an entry's reach recompiles the caller's
878
+ OWN grants under a different profile, so it narrows rows and never widens verbs.
879
+ A principal without the verb is refused either way. And a declared reach that
880
+ cannot be compiled refuses the read rather than falling back — the fallback would
881
+ be an answer with too much in it, and nothing would say so.