@niscorp/vex 0.4.2 → 0.4.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,24 @@
1
1
  # @niscorp/vex
2
2
 
3
+ ## 0.4.4
4
+
5
+ ### Patch Changes
6
+
7
+ - d70f22c: The docs say entries are keyed by fingerprint (a minted one is replaced like a named one when the request differs), `MapToShape` takes `intent`, the mapping agent retries up to three times, and list `exists`, `optional`, `countDistinct`, `invalidate`, `rows` and the `./migrations` entry.
8
+
9
+ **What to change:** nothing. Documentation only.
10
+
11
+ ## 0.4.3
12
+
13
+ ### Patch Changes
14
+
15
+ - c5f8e64: `insertEach` converts each `{ $item }` value by its column's own type, as Postgres converts an `insert`'s value. It cast through a fixed list of five types before, so Postgres refused every other column type (`column is of type … but expression is of type text`).
16
+ - **Columns it now writes:** `time`, enums, `inet`, `interval`, arrays (an item is a JSON array) and pgvector `vector`, also when the extension lives in another schema. A vector item is a JSON array of numbers (`[0.1, 0.2]`) or the pgvector text form (`"[0.1,0.2]"`), so a list of embeddings goes in one statement.
17
+ - **A `timestamp` (without time zone) is stored as `insert` stores it.** It was read as `timestamptz`, so with the session's time zone at Europe/Vienna, `2026-10-26T09:30:00Z` landed as 10:30; it now lands as 09:30.
18
+ - **A fraction for an integer column is refused, as `insert` refuses it.** It was rounded: 7.5 stored 8.
19
+
20
+ **What to change:** an app that sends fractions to an integer column through `insertEach` rounds them itself. An app whose database sessions are not on UTC and that wrote a zoned value into a `timestamp` column through `insertEach` has shifted rows from before; new rows are not shifted.
21
+
3
22
  ## 0.4.2
4
23
 
5
24
  ### Patch Changes
package/DESIGN.md CHANGED
@@ -30,10 +30,11 @@ execute.
30
30
  that still serves any already-cached shape. Cortex (and Signal behind it) is
31
31
  an optional peer dependency, not a hard one; Prism is required, because a
32
32
  stored mapping is Prism IR and the engine executes it.
33
- 3. **Shape-only caching.** The positive cache key is a hash of the request's
34
- *shape* with all values replaced by type tokens. Two requests with the same
35
- structure but different values share a cache entry. Intent is deliberately
36
- excluded from this key.
33
+ 3. **Fingerprint caching.** The positive cache is keyed by a fingerprint,
34
+ minted by the engine (`fp_…`) or named by the caller. A request reuses an
35
+ entry while it matches the stored one: the same intent, the same *shape* with
36
+ all values replaced by type tokens, the same context key names. Context
37
+ values never count, so one entry serves every value. (See *Cache v2*.)
37
38
  4. **Scope is server-side and LLM-invisible.** Access control is injected into
38
39
  the DSL *after* generation, as ordinary filter clauses bound to server
39
40
  `$scope` values. The model cannot see or forge them, so prompt injection
@@ -135,12 +136,13 @@ queries. Top level (`QuerySchema`):
135
136
  - `filter` — a recursive `FilterSchema` union.
136
137
  - `compute` — record of `alias → ComputeExpression` (arithmetic, `concat`,
137
138
  `coalesce`, `case`).
138
- - `aggregate` — record of `alias → AggregateExpression` (`count`/`sum`/`avg`/
139
- `min`/`max`).
139
+ - `aggregate` — record of `alias → AggregateExpression` (`count`/`countDistinct`/
140
+ `sum`/`avg`/`min`/`max`).
140
141
  - `groupBy`, `sort`, `limit`, `distinct`.
141
142
 
142
143
  Filter operators: `eq`/`neq`/`gt`/`gte`/`lt`/`lte`, `in`/`notIn`, `like`/
143
- `ilike`, `isNull`/`isNotNull`, `and`/`or`/`not`, plus `semantic` (vector
144
+ `ilike`, `isNull`/`isNotNull`, `and`/`or`/`not`, `exists` (a correlated subquery),
145
+ `optional` (a condition the caller turns on by supplying context keys), plus `semantic` (vector
144
146
  similarity) and `fuzzy` (string distance). Values are literals, `{ $context }`
145
147
  (caller-provided) or `{ $scope }` (server-injected).
146
148
 
@@ -295,9 +297,9 @@ never change it.
295
297
 
296
298
  - A request WITHOUT a fingerprint generates and caches; the server mints a
297
299
  fingerprint (`fp_…`) and returns it in `meta.cache.fingerprint`.
298
- - A minted fingerprint is an **immutable pin** — embed it and replay the exact
299
- generation forever. Changing the query means proving again and embedding the
300
- new fingerprint. (git: a commit hash.)
300
+ - A minted fingerprint replays its generation for as long as it is sent
301
+ alone. Sent with a request that differs, it is regenerated and replaced like
302
+ a named slot, unless protected.
301
303
  - A caller-chosen fingerprint (any string) is a **mutable named slot** — the
302
304
  name is yours, its content follows your request. (git: a branch.)
303
305
  - Intent NEVER keys anything: natural language is fake identity in both
@@ -609,9 +611,10 @@ exported from the `@niscorp/vex/agent` subpath — `createQueryDsl` (fills
609
611
  `createQueryDsl` wraps it into the `generateDsl` hook.
610
612
  - The **mapping agent** is Prism's exported `mappingAgent`; `createShapeMapper`
611
613
  builds the same envelope the runtime replays against — `{ result: rows }` for an
612
- array shape, `{ result: rows[0] ?? null }` for an object shape — runs the agent
613
- once, with the intent as its `notes`, to get a transform config, compiles it to IR, and applies the IR once over
614
- that envelope. A flat shape whose fields the rows carry is mapped by Prism with no model call, its declared
614
+ array shape, `{ result: rows[0] ?? null }` for an object shape — runs the agent,
615
+ with the intent as its `notes`, to get a transform config, compiles it to IR, and applies the IR once over
616
+ that envelope. A config that fails to compile or run on the rows sends the agent round again with the error in
617
+ its notes, up to three attempts (`MAPPING_ATTEMPTS`). A flat shape whose fields the rows carry is mapped by Prism with no model call, its declared
615
618
  fields computed from the stored ones; the agent writes the mapping for any other.
616
619
 
617
620
  > The reference agents are a separate concern from the deterministic core: they
@@ -718,8 +721,13 @@ src/
718
721
  agent/ Reference Cortex agents (exported via ./agent)
719
722
  query.agent.ts defineAgent: vexQueryDslAgent (vex.query → Query)
720
723
  tools.ts the six query tools
724
+ compact.ts compactSchema, compactSpec — the schema and DSL spec as compact text
725
+ declared.ts declaredMapping, storedShape — fields a shape declares as prism expressions
721
726
  index.ts createQueryDsl, createShapeMapper, vexQueryDslAgent
722
727
 
728
+ migrations/
729
+ index.ts VEX_SEQUENCE, VEX_SCHEMAS — an entry as a stored document (./migrations)
730
+
723
731
  utils/
724
732
  context.ts buildValidationContext, resolveParams
725
733
  canonical.ts canonical JSON text + hash
@@ -737,7 +745,7 @@ scripts/ Dev-only: docker, seed, dev server, fixtures
737
745
  | `zod` | peer (required) | Schema validation + JSON Schema generation |
738
746
  | `pg` | peer (optional) | A Postgres pool for the adapter and the cache; `@niscorp/vex/pg` imports it |
739
747
  | `hono` | peer (optional) | Hono framework adapter |
740
- | `express` | peer (optional) | Express framework adapter |
748
+ | `express` | none | `@niscorp/vex/express` imports nothing from Express; it types the request and response it uses |
741
749
  | `@niscorp/cortex` | peer (optional) | Reference query agent (`defineAgent`/`defineTool`, standalone runtime) |
742
750
  | `@niscorp/signal` | via Cortex (its peer) | LLM calls + embeddings |
743
751
  | `@niscorp/prism` | peer (required) | Result mapping (IR compile/execute; `mappingAgent` for the reference mapper) |
package/DOCS.md CHANGED
@@ -44,8 +44,9 @@ path with no model and no network beyond the database.
44
44
  server-minted (`fp_…`, returned in `meta.cache.fingerprint`) or caller-chosen
45
45
  (a name like `deals/list`). Replaying a fingerprint skips generation entirely
46
46
  and re-runs only the deterministic path — so scope and context stay current
47
- while LLM cost drops to zero. A minted fingerprint is an immutable pin; a
48
- named one is a mutable slot whose content follows your request. Design and
47
+ while LLM cost drops to zero. A fingerprint sent alone replays its generation;
48
+ sent with a request that differs, minted or named, it is regenerated and its
49
+ content replaced. Design and
49
50
  rationale: [DESIGN.md](./DESIGN.md) **Cache v2 — fingerprints**.
50
51
 
51
52
  **Scope is separate from context.** `context` is caller data, visible to the
@@ -157,7 +158,7 @@ type QueryEngineConfig = {
157
158
  cache?: CacheBackend; // default: createMemoryCache()
158
159
  onEvent?: VexEventHandler; // pipeline event stream
159
160
  generateDsl?: (request, schema, caller) => Promise<Query>;
160
- mapToShape?: (rows, shape) => Promise<{ ir: CompiledIr; config: unknown; transformed: JsonValue }>;
161
+ mapToShape?: (rows, shape, intent?) => Promise<{ ir: CompiledIr; config: unknown; transformed: JsonValue }>;
161
162
  embed?: (text, dimensions?) => Promise<number[]>; // text→vector for semantic filters
162
163
  behaviors?: ScopeBehaviors; // which columns a scope key is bound to (see below)
163
164
  rows?: LiveRowsConfig; // the rows cache behind `refresh: 'reactive'`
@@ -1022,11 +1023,15 @@ Two things to know when using it:
1022
1023
  ### `insertEach` — one statement for a caller-sized list
1023
1024
 
1024
1025
  `items` names a context key holding an array of objects; `{ $item: "key" }`
1025
- values read from the current element (cast to the column's type from the
1026
- schema); every other value — literal, `$context`, `$lookup`, `$returned`,
1027
- engine-injected `$scope` — is constant across rows. Compiles to a single
1028
- `INSERT … SELECT … FROM jsonb_array_elements($items)`, so "one row per ticked
1029
- weekday" is one authored statement, not a code loop:
1026
+ values read from the current element; every other value — literal,
1027
+ `$context`, `$lookup`, `$returned`, engine-injected `$scope` — is constant
1028
+ across rows. vex binds each element under the column names its `$item`s fill
1029
+ (a key no `$item` names is dropped) and compiles to a single
1030
+ `INSERT … SELECT … FROM jsonb_to_recordset($items) AS item(<column> <type>, …)`,
1031
+ each type the column's own from the introspected schema. Postgres converts
1032
+ every item as it converts an `insert`'s value, a column the statement does not
1033
+ write takes its default, and "one row per ticked weekday" is one authored
1034
+ statement, not a code loop:
1030
1035
 
1031
1036
  ```ts
1032
1037
  { op: 'insertEach', table: 'class_templates', items: { $context: 'slots' },
@@ -1040,6 +1045,10 @@ and `set`/`match` rules pin their columns identically on every row. A
1040
1045
  non-array `items` value is refused loudly before any SQL runs; an empty array
1041
1046
  inserts nothing and returns no rows.
1042
1047
 
1048
+ An array column's item is a JSON array, a `jsonb` column's any JSON value, and
1049
+ a pgvector `vector` column's a JSON array of numbers (`[0.1, 0.2]`) or the
1050
+ pgvector text form (`"[0.1,0.2]"`).
1051
+
1043
1052
  ### Replay
1044
1053
 
1045
1054
  The handler dispatches by the entry's kind — one wire shape:
@@ -1153,8 +1162,9 @@ The positive cache is keyed by the request's `fingerprint` (see the request
1153
1162
  contract above). There are no cache modes — the body's posture says
1154
1163
  everything:
1155
1164
 
1156
- - **No fingerprint** → generate, cache, mint (`fp_…`). The mint is an
1157
- immutable pin: embed it and replay that exact generation forever.
1165
+ - **No fingerprint** → generate, cache, mint (`fp_…`). Sent alone, the mint
1166
+ replays that generation; sent with a differing request, it is replaced like
1167
+ any named fingerprint.
1158
1168
  - **Fingerprint alone** → replay-or-`cache_miss`. Never generates. Context
1159
1169
  values may vary per call — sorting and parameterized reloads are legal
1160
1170
  against one cached DSL.
@@ -1394,7 +1404,7 @@ free-form requests.
1394
1404
 
1395
1405
  ```typescript
1396
1406
  type GenerateDsl = (request: QueryRequest, schema: DatabaseSchema, caller: GenerationCaller) => Promise<Query>;
1397
- type MapToShape = (rows: Row[], shape: unknown) =>
1407
+ type MapToShape = (rows: Row[], shape: unknown, intent?: string) =>
1398
1408
  Promise<{ ir: CompiledIr; config: unknown; transformed: JsonValue }>;
1399
1409
 
1400
1410
  type GenerationCaller = {
package/README.md CHANGED
@@ -81,6 +81,8 @@ engine.test(dsl, scope?); // Promise<TestResult> — compile + run with
81
81
  engine.getDslSchema(); // object — the DSL as JSON Schema (draft-7)
82
82
  engine.getSchema(); // DatabaseSchema | undefined — last introspected schema
83
83
  engine.cache; // CacheBackend — the live cache
84
+ engine.invalidate(tables); // these tables were written: reactive reads over them refetch
85
+ engine.rows; // the reactive rows cache: stats(), stop()
84
86
  ```
85
87
 
86
88
  ## What's in the box
@@ -94,6 +96,7 @@ engine.cache; // CacheBackend — the live cache
94
96
  | Writes | Mutation entries in the same cache — a closed grammar, replay-only by fingerprint, never generated; `lintMutation` checks a statement at seed |
95
97
  | Seeding and freshness | `seedCache` turns authored `SeedEntry` / `SeedMutation` rows into protected cache entries; an entry's `refresh` is `'snapshot'` or `'reactive'` |
96
98
  | Safety | Server-side scope policies, query analyzer (cartesian/nesting/index checks), parameterized SQL only |
99
+ | Stored entries | `@niscorp/vex/migrations`: `VEX_SEQUENCE`, the strata sequence an entry is stamped with as a stored document |
97
100
  | Its own table | The Postgres cache's table is a [strata](../strata/README.md) sequence (`cache.sequence`, `nisc.vex.cache`): `cache.init()` applies it through the ledger once, instead of `IF NOT EXISTS` on every boot. Requires a pool with `transaction` (the PGlite pool and `createPgPool`'s have one; a bare `pg.Pool` does not). |
98
101
 
99
102
  ## License
@@ -1055,6 +1055,10 @@ var scopeMutation = (m, policy) => {
1055
1055
  }
1056
1056
  return { op: "delete", table: m.table, where };
1057
1057
  };
1058
+ var itemRow = (element, itemKeys) => {
1059
+ const fields = element !== null && typeof element === "object" && !Array.isArray(element) ? element : {};
1060
+ return Object.fromEntries(Object.entries(itemKeys).map(([column, key]) => [column, fields[key]]));
1061
+ };
1058
1062
  var newCtx = () => ({
1059
1063
  resolvedPaths: /* @__PURE__ */ new Map(),
1060
1064
  aliasMap: /* @__PURE__ */ new Map(),
@@ -1101,13 +1105,6 @@ var compileConflict = (m, ctx, schema) => {
1101
1105
  }
1102
1106
  return ` ON CONFLICT (${c.target.join(", ")}) DO UPDATE SET ${sets.join(", ")}${where}`;
1103
1107
  };
1104
- var ITEM_CAST = {
1105
- number: "numeric",
1106
- boolean: "boolean",
1107
- date: "date",
1108
- timestamp: "timestamptz",
1109
- uuid: "uuid"
1110
- };
1111
1108
  var compileMutation = (m, schema) => {
1112
1109
  const ctx = newCtx();
1113
1110
  if (m.op === "insert") {
@@ -1117,26 +1114,28 @@ var compileMutation = (m, schema) => {
1117
1114
  return { sql: `INSERT INTO ${m.table} (${cols.join(", ")}) VALUES (${vals.join(", ")})${compileConflict(m, ctx, schema)} RETURNING *`, slots: ctx.paramSlots, returned: ctx.returned };
1118
1115
  }
1119
1116
  if (m.op === "insertEach") {
1120
- const entity = findEntity(m.table, schema);
1117
+ const types = new Map(findEntity(m.table, schema)?.fields.map((f) => [f.name, f.type]));
1121
1118
  const entries = Object.entries(m.values);
1122
1119
  const cols = entries.map(([c]) => c);
1120
+ const itemKeys = {};
1121
+ const itemColumns = [];
1123
1122
  const exprs = entries.map(([col, v]) => {
1124
1123
  if (isItemRef(v)) {
1125
- const nt = entity?.fields.find((f) => f.name === col)?.normalizedType ?? "string";
1126
- if (nt === "json") return `(item.value->${quoteLiteral(v.$item)})`;
1127
- const cast = ITEM_CAST[nt];
1128
- const raw = `(item.value->>${quoteLiteral(v.$item)})`;
1129
- return cast === void 0 ? raw : `${raw}::${cast}`;
1124
+ itemKeys[col] = v.$item;
1125
+ itemColumns.push(`${col} ${String(types.get(col))}`);
1126
+ return `item.${col}`;
1130
1127
  }
1131
1128
  return isReturned(v) ? compileColumnValue(m.table, col, v, ctx, schema) : compileValue(v, ctx);
1132
1129
  });
1133
1130
  ctx.paramCounter.value += 1;
1134
1131
  ctx.paramSlots.push({ key: m.items.$context, kind: "context", type: "json" });
1135
- const source = `jsonb_array_elements($${ctx.paramCounter.value}::jsonb) AS item`;
1132
+ const items = `$${ctx.paramCounter.value}::jsonb`;
1133
+ const source = itemColumns.length === 0 ? `jsonb_array_elements(${items}) AS item` : `jsonb_to_recordset(${items}) AS item(${itemColumns.join(", ")})`;
1136
1134
  return {
1137
1135
  sql: `INSERT INTO ${m.table} (${cols.join(", ")}) SELECT ${exprs.join(", ")} FROM ${source}${compileConflict(m, ctx, schema)} RETURNING *`,
1138
1136
  slots: ctx.paramSlots,
1139
- returned: ctx.returned
1137
+ returned: ctx.returned,
1138
+ itemKeys
1140
1139
  };
1141
1140
  }
1142
1141
  if (m.op === "update") {
@@ -1209,7 +1208,8 @@ var executeWrites = async (client, def, mctx) => {
1209
1208
  if (!Array.isArray(value)) {
1210
1209
  throw new VexError("invalid_request", `"${c.slots[i]?.key ?? ""}" must be an array of objects (one inserted row per element).`);
1211
1210
  }
1212
- return JSON.stringify(value);
1211
+ const { itemKeys } = c;
1212
+ return JSON.stringify(itemKeys === void 0 ? value : value.map((element) => itemRow(element, itemKeys)));
1213
1213
  });
1214
1214
  const res = await q.query(c.sql, bound);
1215
1215
  results.push({ table: c.table, op: c.op, rows: res.rows });