@niscorp/vex 0.4.0 → 0.4.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (50) hide show
  1. package/CHANGELOG.md +25 -0
  2. package/DESIGN.md +12 -7
  3. package/DOCS.md +64 -19
  4. package/README.md +2 -2
  5. package/dist/adapters/express/index.cjs +1 -1
  6. package/dist/adapters/express/index.cjs.map +1 -1
  7. package/dist/adapters/express/index.d.cts +2 -2
  8. package/dist/adapters/express/index.d.ts +2 -2
  9. package/dist/adapters/express/index.js +1 -1
  10. package/dist/adapters/hono/index.cjs +1 -1
  11. package/dist/adapters/hono/index.cjs.map +1 -1
  12. package/dist/adapters/hono/index.d.cts +2 -2
  13. package/dist/adapters/hono/index.d.ts +2 -2
  14. package/dist/adapters/hono/index.js +1 -1
  15. package/dist/adapters/pg/index.cjs +44 -0
  16. package/dist/adapters/pg/index.cjs.map +1 -0
  17. package/dist/adapters/pg/index.d.cts +28 -0
  18. package/dist/adapters/pg/index.d.ts +28 -0
  19. package/dist/adapters/pg/index.js +31 -0
  20. package/dist/adapters/pg/index.js.map +1 -0
  21. package/dist/adapters/pglite/index.cjs +3 -1
  22. package/dist/adapters/pglite/index.cjs.map +1 -1
  23. package/dist/adapters/pglite/index.d.cts +5 -4
  24. package/dist/adapters/pglite/index.d.ts +5 -4
  25. package/dist/adapters/pglite/index.js +3 -3
  26. package/dist/adapters/pglite/index.js.map +1 -1
  27. package/dist/agent/index.cjs +144 -22
  28. package/dist/agent/index.cjs.map +1 -1
  29. package/dist/agent/index.d.cts +1 -1
  30. package/dist/agent/index.d.ts +1 -1
  31. package/dist/agent/index.js +144 -22
  32. package/dist/agent/index.js.map +1 -1
  33. package/dist/chunk-DFWYYBJU.js +7 -0
  34. package/dist/chunk-DFWYYBJU.js.map +1 -0
  35. package/dist/{chunk-7F2OH4IP.js → chunk-RGJRUOGQ.js} +7 -4
  36. package/dist/chunk-RGJRUOGQ.js.map +1 -0
  37. package/dist/date-parsers-BaUuE_ax.d.cts +4 -0
  38. package/dist/date-parsers-BaUuE_ax.d.ts +4 -0
  39. package/dist/{handler-Bw7xVmBM.d.ts → handler-CoJxInw-.d.ts} +1 -1
  40. package/dist/{handler-Cy0gNM6K.d.cts → handler-CsFbanZA.d.cts} +1 -1
  41. package/dist/index.cjs +6 -3
  42. package/dist/index.cjs.map +1 -1
  43. package/dist/index.d.cts +3 -3
  44. package/dist/index.d.ts +3 -3
  45. package/dist/index.js +3 -3
  46. package/dist/index.js.map +1 -1
  47. package/dist/{types-DO-rYZci.d.cts → types-B2HEmyNq.d.cts} +1 -1
  48. package/dist/{types-Bs_9VM9p.d.ts → types-DC68ZUuq.d.ts} +1 -1
  49. package/package.json +13 -1
  50. package/dist/chunk-7F2OH4IP.js.map +0 -1
package/CHANGELOG.md CHANGED
@@ -1,5 +1,30 @@
1
1
  # @niscorp/vex
2
2
 
3
+ ## 0.4.2
4
+
5
+ ### Patch Changes
6
+
7
+ - 2a6f8d6: An insert's `onConflict.set` is held to every `match` rule of the update phase, one on a column of the conflict target included: each becomes a WHERE on the DO UPDATE half, so a conflict with a row the caller may not update writes nothing and returns no row.
8
+
9
+ Before, a `match` on a conflict-target column was left out, on the reasoning that the insert pins that column from scope. Where the policy's insert phase does not pin it — no insert rule for that column, or two merged roles of which one may insert anywhere — an entry that takes that column's value from the caller let them overwrite, and read back, a row their update rule excludes.
10
+
11
+ - 915efe6: `@niscorp/vex/pg` exports `createPgPool(pool, parsers?)`, which shapes a node-postgres `pg.Pool` as the `PgPool` the Postgres adapter and cache take, with a `transaction` that checks one client out of the pool and runs `BEGIN`/`COMMIT` (`ROLLBACK` when the callback throws) on it — what the cache's `init()`, batch mutations and per-read statement timeouts need. It also exports `RAW_DATE_PARSERS`, the same constant as `@niscorp/vex/pglite`'s. `pg` is now listed as an optional peer.
12
+
13
+ ## 0.4.1
14
+
15
+ ### Patch Changes
16
+
17
+ - 6680ad6: `createQueryDsl` hands the query agent the schema and the DSL spec as compact text instead of JSON: an entity per line with its relations and indexes, and the DSL as types that keep every description, property and allowed value. On a 15-entity CRM schema the fixed part of each call drops from about 36,000 characters to about 13,000. On 19 queries a screen builder asked for, the agent answered all 19, with the same row counts as before, at 44% of the cost and a median of 3.5 s per query instead of 5.1 s. The query it writes is still validated against the full DSL schema.
18
+ - 6680ad6: A shape can declare a field as a Prism expression over the row's stored fields, and `createShapeMapper` computes it with no model call.
19
+
20
+ `[{ close_date: '', close_date_display: { $date: { value: { $ref: '$.close_date' }, format: 'MMM D' } } }]`: `createQueryDsl` asks the query for the stored fields a declared field reads, and `createShapeMapper` computes the declared field with Prism, so the stored value beside it is never overwritten; a mapping a model wrote overwrote it about a quarter of the time. `normalizeShape` keeps a declared field whole, so two formats are two cache entries.
21
+
22
+ A flat shape whose fields the first row carries, each plain one of the kind its example shows, is mapped with no model call, as rows that fit the shape exactly were; rows with more columns than the shape names are now mapped the same way rather than by the agent. Any other shape — nested, a field the rows lack, a value of another kind — goes to the mapping agent, whose mapping is run on the rows before it is used: one that fails there is written again, the agent told why, up to three attempts in all. The agent is told the rows already answer the request (filtered, sorted, limited) and keeps every row, in order.
23
+
24
+ - 1c78d6d: The engine hands `mapToShape` the request's intent, and `createShapeMapper` gives it to the mapping agent.
25
+
26
+ `MapToShape` is `(rows, shape, intent?)`: the engine passes the intent a query was asked in, so the mapping can give a value the form the request named (money as `$1.2M`, a date as `Apr 6`). Before, the mapper saw the rows and the shape alone, whose placeholders say nothing of a form. `createShapeMapper` passes it to prism's `mappingAgent` as its `notes`. A mapper that takes two arguments keeps working.
27
+
3
28
  ## 0.4.0
4
29
 
5
30
  ### Minor Changes
package/DESIGN.md CHANGED
@@ -587,9 +587,11 @@ The engine takes two optional hooks:
587
587
  what the person asking could see — and it is keyed (single-flight, negative
588
588
  cache) by request AND policy, because two callers who see different tables
589
589
  are asking different agents.
590
- - `mapToShape(rows, shape) → { ir, config, transformed }` — raw rows → requested
590
+ - `mapToShape(rows, shape, intent?) → { ir, config, transformed }` — raw rows → requested
591
591
  shape, returning a compiled Prism IR that is cached for reuse, and the Prism
592
- config it was compiled from, which is kept beside it (`mapping`).
592
+ config it was compiled from, which is kept beside it (`mapping`). `intent` is
593
+ the request's own words, when it has them, so the mapping can give a value the
594
+ form they ask for (money as `$1.2M`, a date as `Apr 6`).
593
595
 
594
596
  This is the seam that keeps the core deterministic. Vex ships a reference
595
597
  implementation of both, built on `@niscorp/cortex` and `@niscorp/prism` and
@@ -601,15 +603,16 @@ exported from the `@niscorp/vex/agent` subpath — `createQueryDsl` (fills
601
603
  `getDistinctValues`, `describeField`, `testQuery`, `cannotSatisfy` — and three
602
604
  stop conditions (`stepCount(20)`, `outputRetries(3)`, `repeatedCalls(2)`);
603
605
  `createQueryDsl` aborts the run when `cannotSatisfy` is called. Context
604
- producers inject the live schema, the caller line and the DSL JSON Schema so the
605
- static instructions stay stable. Every data-touching tool is a DSL run through
606
+ producers inject the live schema and the DSL specification as compact text, and
607
+ the caller line, so the static instructions stay stable. Every data-touching tool is a DSL run through
606
608
  `caller.read`; the tool set holds no adapter and no SQL.
607
609
  `createQueryDsl` wraps it into the `generateDsl` hook.
608
610
  - The **mapping agent** is Prism's exported `mappingAgent`; `createShapeMapper`
609
611
  builds the same envelope the runtime replays against — `{ result: rows }` for an
610
612
  array shape, `{ result: rows[0] ?? null }` for an object shape — runs the agent
611
- once to get a transform config, compiles it to IR, and applies the IR once over
612
- that envelope (the identity, with no model call, when the rows already fit).
613
+ once, with the intent as its `notes`, to get a transform config, compiles it to IR, and applies the IR once over
614
+ that envelope. A flat shape whose fields the rows carry is mapped by Prism with no model call, its declared
615
+ fields computed from the stored ones; the agent writes the mapping for any other.
613
616
 
614
617
  > The reference agents are a separate concern from the deterministic core: they
615
618
  > live behind the `@niscorp/vex/agent` subpath, so `@niscorp/cortex` (and
@@ -688,7 +691,9 @@ src/
688
691
  compile.ts ResolvedQuery → parameterized SQL
689
692
  operators.ts per-operator SQL generation
690
693
  statement-timeout.ts how a read's time limit is enforced
694
+ date-parsers.ts RAW_DATE_PARSERS, shared by the two pool shims
691
695
  pglite/index.ts createPglitePool (@niscorp/vex/pglite)
696
+ pg/index.ts createPgPool (@niscorp/vex/pg)
692
697
  hono/index.ts vex() Hono app (@niscorp/vex/hono)
693
698
  express/index.ts vex() Express handler (@niscorp/vex/express)
694
699
  index.ts adapter barrel
@@ -730,7 +735,7 @@ scripts/ Dev-only: docker, seed, dev server, fixtures
730
735
  | Package | Relationship | Purpose |
731
736
  |---------|--------------|---------|
732
737
  | `zod` | peer (required) | Schema validation + JSON Schema generation |
733
- | `pg` | peer (optional) | Postgres adapter and Postgres cache backend |
738
+ | `pg` | peer (optional) | A Postgres pool for the adapter and the cache; `@niscorp/vex/pg` imports it |
734
739
  | `hono` | peer (optional) | Hono framework adapter |
735
740
  | `express` | peer (optional) | Express framework adapter |
736
741
  | `@niscorp/cortex` | peer (optional) | Reference query agent (`defineAgent`/`defineTool`, standalone runtime) |
package/DOCS.md CHANGED
@@ -185,7 +185,10 @@ type QueryEngineConfig = {
185
185
  set up at `introspect()`. A database that cannot enforce it (PGlite) runs
186
186
  reads unbounded and the engine warns each time `introspect()` runs. A `PgPool`
187
187
  declares such a database with `statementTimeouts: false`, as the pool from
188
- `createPglitePool` does.
188
+ `createPglitePool` does. When the server's own `statement_timeout` is not
189
+ already within the limit, a pool with `transaction` (`createPgPool`'s) runs
190
+ each read in a transaction that sets it first; a bare `pg.Pool` runs it
191
+ unbounded.
189
192
  - `config.maxPresenceVariants` is not a limit on what runs: past it, a response
190
193
  carries a warning that the entry has compiled that many distinct
191
194
  optional-key combinations.
@@ -266,6 +269,21 @@ The **shape** is an example, not a schema: use empty strings, zeros, and
266
269
  booleans as type markers. `[{ id: '', total: 0 }]` means "an array of objects
267
270
  with a string `id` and a number `total`." A bare object means a single record.
268
271
 
272
+ A field can be **declared** rather than shown by example: a Prism expression over
273
+ the row's own stored fields, where `$.close_date` is the row's `close_date`.
274
+
275
+ ```typescript
276
+ shape: [{ id: '', close_date: '', close_date_display: { $date: { value: { $ref: '$.close_date' }, format: 'MMM D' } } }]
277
+ // → [{ id: 'deal_085', close_date: '2026-06-08', close_date_display: 'Jun 8' }]
278
+ ```
279
+
280
+ The query is written for the stored fields a declared field reads, and
281
+ `createShapeMapper` computes the declared field with Prism, never a model, so
282
+ the stored value beside it is never overwritten. A declared field is part of the
283
+ request's identity: two formats are two cache entries. A field the rows lack,
284
+ such as a form shown only by example (`close_date_display: 'Apr 6'`), is left
285
+ to the mapping agent; a field the rows already carry is copied as it is.
286
+
269
287
  The shape also picks the result envelope: an **array** shape maps over the whole
270
288
  row set (the mapping sees `$.result` as the rows array — a `$map`/identity); a
271
289
  **non-array** shape maps the single (first) row (the mapping sees `$.result` as
@@ -909,11 +927,9 @@ Two guarantees, both enforced before any SQL runs:
909
927
  unique column sets in the message — never a runtime surprise.
910
928
  - **DO UPDATE is an update and is governed like one.** Declaring `set`
911
929
  requires the update (or `write` umbrella) phase — an insert-only grant
912
- refuses it. Update-phase `set` rules stamp identity into the DO UPDATE SET;
913
- `match` rules whose column the conflict target already pins are inherently
914
- satisfied (the inserted, scope-pinned value must equal the existing row's
915
- for a conflict to arise), and any other `match` compiles to a WHERE on the
916
- DO UPDATE half — the RLS boundary, again. `mutationEffect` declares the
930
+ refuses it. Update-phase `set` rules stamp identity into the DO UPDATE SET,
931
+ and every update-phase `match` compiles to a WHERE on the DO UPDATE half —
932
+ the RLS boundary, again. `mutationEffect` declares the
917
933
  entry as the insert-plus-update it really is, so discovery visibility
918
934
  follows the same rule.
919
935
 
@@ -1204,10 +1220,21 @@ const l2 = createPostgresCache({
1204
1220
  await l2.init(); // applies l2.sequence through strata's ledger — once, recorded
1205
1221
  ```
1206
1222
 
1207
- `init()` runs a strata migration, so the pool must expose `transaction` — the
1208
- PGlite pool has one; a bare `pg.Pool` needs a wrapper that checks a client out
1209
- per transaction (see `PgPool`). A host that migrates several owners in one run
1210
- collects `l2.sequence` (`nisc.vex.cache`) instead of calling `init()`.
1223
+ `init()` runs a strata migration, so the pool must expose `transaction`. A bare
1224
+ `pg.Pool` does not; `createPgPool` from `@niscorp/vex/pg` gives it one that
1225
+ checks one client out per transaction (the PGlite pool has one too):
1226
+
1227
+ ```typescript
1228
+ import pg from 'pg';
1229
+ import { createPgPool } from '@niscorp/vex/pg';
1230
+
1231
+ const pool = createPgPool(new pg.Pool({ connectionString: process.env.DATABASE_URL }));
1232
+ const l2 = createPostgresCache({ pool });
1233
+ await l2.init();
1234
+ ```
1235
+
1236
+ A host that migrates several owners in one run collects `l2.sequence`
1237
+ (`nisc.vex.cache`) instead of calling `init()`.
1211
1238
 
1212
1239
  **Tiered** — L1 (memory) + L2 (durable), read-through/write-through. Reads hit
1213
1240
  L1; on an L1 miss it promotes a validated L2 entry into L1. Writes go to L1
@@ -1300,10 +1327,22 @@ await seedCache(engine.cache, [ordersRecent, taskSetDone]);
1300
1327
 
1301
1328
  `@niscorp/vex/pglite` exports `createPglitePool(db, parsers?)`, which shapes a
1302
1329
  PGlite database as the `PgPool` the Postgres adapter and cache take (with
1303
- `transaction`), and `RAW_DATE_PARSERS`, which returns `DATE`/`TIMESTAMPTZ`
1304
- columns as raw strings. Pass the parsers to the adapter's pool; give the
1305
- Postgres cache a pool without them — it reads its own timestamp columns as
1306
- `Date`s.
1330
+ `transaction`). `@niscorp/vex/pg` exports `createPgPool(pool, parsers?)`, the
1331
+ same for a node-postgres `pg.Pool`: its `transaction` checks one client out of
1332
+ the pool, runs `BEGIN`, the callback and `COMMIT` on it (`ROLLBACK` and rethrow
1333
+ when the callback throws), and releases it. Both export `RAW_DATE_PARSERS`,
1334
+ which returns `DATE`/`TIMESTAMPTZ` columns as raw strings. Pass the parsers to
1335
+ the adapter's pool; give the Postgres cache a pool without them — it reads its
1336
+ own timestamp columns as `Date`s:
1337
+
1338
+ ```typescript
1339
+ import pg from 'pg';
1340
+ import { createPgPool, RAW_DATE_PARSERS } from '@niscorp/vex/pg';
1341
+
1342
+ const db = new pg.Pool({ connectionString: process.env.DATABASE_URL });
1343
+ const adapter = createPostgresAdapter({ pool: createPgPool(db, RAW_DATE_PARSERS) });
1344
+ const cache = createPostgresCache({ pool: createPgPool(db) });
1345
+ ```
1307
1346
 
1308
1347
  `@niscorp/vex` and `@niscorp/vex/pglite` import no Node built-in, so a bundler
1309
1348
  builds them for a page with no alias and no polyfill. Vex's hashes are the same
@@ -1397,15 +1436,21 @@ await engine.introspect();
1397
1436
 
1398
1437
  - `createQueryDsl(config)` builds the `generateDsl` hook by running
1399
1438
  `vexQueryDslAgent`. It is built from a model and the DSL spec only — who it
1400
- runs for arrives with every call. The engine calls the hook as
1439
+ runs for arrives with every call. The agent reads the schema and the DSL spec
1440
+ as compact text — an entity per line with its relations and indexes, the DSL
1441
+ as types with every description — and, for a shape with declared fields, the
1442
+ stored fields those read. The engine calls the hook as
1401
1443
  `generateDsl(request, schema, caller)`: `schema` holds only the tables the
1402
1444
  caller's policy can read, and `caller.read(dsl)` runs a DSL through the
1403
1445
  engine's own pipeline under the caller's policy and scope values (`$context`
1404
1446
  binds NULL). That capability is the tools' only way to the database.
1405
- - `createShapeMapper(llm)` builds the `mapToShape` hook by running Prism's
1406
- `mappingAgent`. When the rows already are the shape (`rowsFitShape` — a flat
1407
- shape whose keys and value kinds the first row matches exactly) it stores the
1408
- identity mapping and makes no model call. The `@niscorp/vex/agent` subpath
1447
+ - `createShapeMapper(llm)` builds the `mapToShape` hook. A flat shape whose
1448
+ fields the first row carries, each plain one of the kind its example shows —
1449
+ plain fields by key, declared fields computed from the stored fields they
1450
+ read — is mapped by Prism with no model call. Any other shape (nested, a field
1451
+ the rows lack, a value of another kind) goes to Prism's `mappingAgent`, handed the request's intent and run on the rows
1452
+ before it is used: a mapping that fails there is written again, up to three
1453
+ attempts. `rowsFitShape` is still exported. The `@niscorp/vex/agent` subpath
1409
1454
  also exports `vexQueryDslAgent` itself, `createQueryTools` for customizing the
1410
1455
  tool set, and `describeCaller`, which turns `caller.bindings` into the
1411
1456
  context line the agent reads. A run of `vexQueryDslAgent` given no `caller`
package/README.md CHANGED
@@ -87,14 +87,14 @@ engine.cache; // CacheBackend — the live cache
87
87
 
88
88
  | Area | Provided |
89
89
  |------|----------|
90
- | Database adapters | PostgreSQL (`createPostgresAdapter`) — pluggable `DatabaseAdapter` interface for others. `createPglitePool` (`@niscorp/vex/pglite`) shapes a PGlite database as the pool it takes |
90
+ | Database adapters | PostgreSQL (`createPostgresAdapter`) — pluggable `DatabaseAdapter` interface for others. `createPglitePool` (`@niscorp/vex/pglite`) shapes a PGlite database as the pool it takes, `createPgPool` (`@niscorp/vex/pg`) a node-postgres `pg.Pool` |
91
91
  | Cache backends | In-memory, Postgres (durable), tiered L1/L2 with warm-up |
92
92
  | Framework adapters | Hono (`@niscorp/vex/hono`), Express (`@niscorp/vex/express`) — discovery + query endpoints |
93
93
  | LLM integration | `generateDsl` / `mapToShape` hooks; reference agents built on `@niscorp/cortex` (`@niscorp/vex/agent`: `createQueryDsl`, `createShapeMapper`) |
94
94
  | Writes | Mutation entries in the same cache — a closed grammar, replay-only by fingerprint, never generated; `lintMutation` checks a statement at seed |
95
95
  | Seeding and freshness | `seedCache` turns authored `SeedEntry` / `SeedMutation` rows into protected cache entries; an entry's `refresh` is `'snapshot'` or `'reactive'` |
96
96
  | Safety | Server-side scope policies, query analyzer (cartesian/nesting/index checks), parameterized SQL only |
97
- | Its own table | The Postgres cache's table is a [strata](../strata/README.md) sequence (`cache.sequence`, `nisc.vex.cache`): `cache.init()` applies it through the ledger once, instead of `IF NOT EXISTS` on every boot. Requires a pool with `transaction` (the PGlite pool has one). |
97
+ | Its own table | The Postgres cache's table is a [strata](../strata/README.md) sequence (`cache.sequence`, `nisc.vex.cache`): `cache.init()` applies it through the ledger once, instead of `IF NOT EXISTS` on every boot. Requires a pool with `transaction` (the PGlite pool and `createPgPool`'s have one; a bare `pg.Pool` does not). |
98
98
 
99
99
  ## License
100
100
 
@@ -1017,7 +1017,7 @@ var scopeInsertParts = (table, rawValues, rawConflict, policy) => {
1017
1017
  for (const r of [...gate.rule.write ?? [], ...gate.rule.update ?? []]) {
1018
1018
  if ("set" in r) {
1019
1019
  conflict = { ...conflict, set: { ...conflict.set ?? {}, [r.set]: { $scope: r.to } } };
1020
- } else if (!conflict.target.includes(r.match)) {
1020
+ } else {
1021
1021
  const boundary = boundaryOf(table, r);
1022
1022
  conflict = { ...conflict, where: conflict.where === void 0 ? boundary : { and: [conflict.where, boundary] } };
1023
1023
  }