@niscorp/vex 0.4.2 → 0.4.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +19 -0
- package/DESIGN.md +22 -14
- package/DOCS.md +21 -11
- package/README.md +3 -0
- package/dist/adapters/express/index.cjs +16 -16
- package/dist/adapters/express/index.cjs.map +1 -1
- package/dist/adapters/express/index.js +1 -1
- package/dist/adapters/hono/index.cjs +16 -16
- package/dist/adapters/hono/index.cjs.map +1 -1
- package/dist/adapters/hono/index.js +1 -1
- package/dist/{chunk-RGJRUOGQ.js → chunk-W5MYFJKT.js} +18 -18
- package/dist/chunk-W5MYFJKT.js.map +1 -0
- package/dist/index.cjs +16 -16
- package/dist/index.cjs.map +1 -1
- package/dist/index.js +2 -2
- package/package.json +9 -8
- package/dist/chunk-RGJRUOGQ.js.map +0 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,24 @@
|
|
|
1
1
|
# @niscorp/vex
|
|
2
2
|
|
|
3
|
+
## 0.4.4
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- d70f22c: The docs say entries are keyed by fingerprint (a minted one is replaced like a named one when the request differs), `MapToShape` takes `intent`, the mapping agent retries up to three times, and list `exists`, `optional`, `countDistinct`, `invalidate`, `rows` and the `./migrations` entry.
|
|
8
|
+
|
|
9
|
+
**What to change:** nothing. Documentation only.
|
|
10
|
+
|
|
11
|
+
## 0.4.3
|
|
12
|
+
|
|
13
|
+
### Patch Changes
|
|
14
|
+
|
|
15
|
+
- c5f8e64: `insertEach` converts each `{ $item }` value by its column's own type, as Postgres converts an `insert`'s value. It cast through a fixed list of five types before, so Postgres refused every other column type (`column is of type … but expression is of type text`).
|
|
16
|
+
- **Columns it now writes:** `time`, enums, `inet`, `interval`, arrays (an item is a JSON array) and pgvector `vector`, also when the extension lives in another schema. A vector item is a JSON array of numbers (`[0.1, 0.2]`) or the pgvector text form (`"[0.1,0.2]"`), so a list of embeddings goes in one statement.
|
|
17
|
+
- **A `timestamp` (without time zone) is stored as `insert` stores it.** It was read as `timestamptz`, so with the session's time zone at Europe/Vienna, `2026-10-26T09:30:00Z` landed as 10:30; it now lands as 09:30.
|
|
18
|
+
- **A fraction for an integer column is refused, as `insert` refuses it.** It was rounded: 7.5 stored 8.
|
|
19
|
+
|
|
20
|
+
**What to change:** an app that sends fractions to an integer column through `insertEach` rounds them itself. An app whose database sessions are not on UTC and that wrote a zoned value into a `timestamp` column through `insertEach` has shifted rows from before; new rows are not shifted.
|
|
21
|
+
|
|
3
22
|
## 0.4.2
|
|
4
23
|
|
|
5
24
|
### Patch Changes
|
package/DESIGN.md
CHANGED
|
@@ -30,10 +30,11 @@ execute.
|
|
|
30
30
|
that still serves any already-cached shape. Cortex (and Signal behind it) is
|
|
31
31
|
an optional peer dependency, not a hard one; Prism is required, because a
|
|
32
32
|
stored mapping is Prism IR and the engine executes it.
|
|
33
|
-
3. **
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
33
|
+
3. **Fingerprint caching.** The positive cache is keyed by a fingerprint,
|
|
34
|
+
minted by the engine (`fp_…`) or named by the caller. A request reuses an
|
|
35
|
+
entry while it matches the stored one: the same intent, the same *shape* with
|
|
36
|
+
all values replaced by type tokens, the same context key names. Context
|
|
37
|
+
values never count, so one entry serves every value. (See *Cache v2*.)
|
|
37
38
|
4. **Scope is server-side and LLM-invisible.** Access control is injected into
|
|
38
39
|
the DSL *after* generation, as ordinary filter clauses bound to server
|
|
39
40
|
`$scope` values. The model cannot see or forge them, so prompt injection
|
|
@@ -135,12 +136,13 @@ queries. Top level (`QuerySchema`):
|
|
|
135
136
|
- `filter` — a recursive `FilterSchema` union.
|
|
136
137
|
- `compute` — record of `alias → ComputeExpression` (arithmetic, `concat`,
|
|
137
138
|
`coalesce`, `case`).
|
|
138
|
-
- `aggregate` — record of `alias → AggregateExpression` (`count`/`
|
|
139
|
-
`min`/`max`).
|
|
139
|
+
- `aggregate` — record of `alias → AggregateExpression` (`count`/`countDistinct`/
|
|
140
|
+
`sum`/`avg`/`min`/`max`).
|
|
140
141
|
- `groupBy`, `sort`, `limit`, `distinct`.
|
|
141
142
|
|
|
142
143
|
Filter operators: `eq`/`neq`/`gt`/`gte`/`lt`/`lte`, `in`/`notIn`, `like`/
|
|
143
|
-
`ilike`, `isNull`/`isNotNull`, `and`/`or`/`not`,
|
|
144
|
+
`ilike`, `isNull`/`isNotNull`, `and`/`or`/`not`, `exists` (a correlated subquery),
|
|
145
|
+
`optional` (a condition the caller turns on by supplying context keys), plus `semantic` (vector
|
|
144
146
|
similarity) and `fuzzy` (string distance). Values are literals, `{ $context }`
|
|
145
147
|
(caller-provided) or `{ $scope }` (server-injected).
|
|
146
148
|
|
|
@@ -295,9 +297,9 @@ never change it.
|
|
|
295
297
|
|
|
296
298
|
- A request WITHOUT a fingerprint generates and caches; the server mints a
|
|
297
299
|
fingerprint (`fp_…`) and returns it in `meta.cache.fingerprint`.
|
|
298
|
-
- A minted fingerprint
|
|
299
|
-
|
|
300
|
-
|
|
300
|
+
- A minted fingerprint replays its generation for as long as it is sent
|
|
301
|
+
alone. Sent with a request that differs, it is regenerated and replaced like
|
|
302
|
+
a named slot, unless protected.
|
|
301
303
|
- A caller-chosen fingerprint (any string) is a **mutable named slot** — the
|
|
302
304
|
name is yours, its content follows your request. (git: a branch.)
|
|
303
305
|
- Intent NEVER keys anything: natural language is fake identity in both
|
|
@@ -609,9 +611,10 @@ exported from the `@niscorp/vex/agent` subpath — `createQueryDsl` (fills
|
|
|
609
611
|
`createQueryDsl` wraps it into the `generateDsl` hook.
|
|
610
612
|
- The **mapping agent** is Prism's exported `mappingAgent`; `createShapeMapper`
|
|
611
613
|
builds the same envelope the runtime replays against — `{ result: rows }` for an
|
|
612
|
-
array shape, `{ result: rows[0] ?? null }` for an object shape — runs the agent
|
|
613
|
-
|
|
614
|
-
that envelope. A
|
|
614
|
+
array shape, `{ result: rows[0] ?? null }` for an object shape — runs the agent,
|
|
615
|
+
with the intent as its `notes`, to get a transform config, compiles it to IR, and applies the IR once over
|
|
616
|
+
that envelope. A config that fails to compile or run on the rows sends the agent round again with the error in
|
|
617
|
+
its notes, up to three attempts (`MAPPING_ATTEMPTS`). A flat shape whose fields the rows carry is mapped by Prism with no model call, its declared
|
|
615
618
|
fields computed from the stored ones; the agent writes the mapping for any other.
|
|
616
619
|
|
|
617
620
|
> The reference agents are a separate concern from the deterministic core: they
|
|
@@ -718,8 +721,13 @@ src/
|
|
|
718
721
|
agent/ Reference Cortex agents (exported via ./agent)
|
|
719
722
|
query.agent.ts defineAgent: vexQueryDslAgent (vex.query → Query)
|
|
720
723
|
tools.ts the six query tools
|
|
724
|
+
compact.ts compactSchema, compactSpec — the schema and DSL spec as compact text
|
|
725
|
+
declared.ts declaredMapping, storedShape — fields a shape declares as prism expressions
|
|
721
726
|
index.ts createQueryDsl, createShapeMapper, vexQueryDslAgent
|
|
722
727
|
|
|
728
|
+
migrations/
|
|
729
|
+
index.ts VEX_SEQUENCE, VEX_SCHEMAS — an entry as a stored document (./migrations)
|
|
730
|
+
|
|
723
731
|
utils/
|
|
724
732
|
context.ts buildValidationContext, resolveParams
|
|
725
733
|
canonical.ts canonical JSON text + hash
|
|
@@ -737,7 +745,7 @@ scripts/ Dev-only: docker, seed, dev server, fixtures
|
|
|
737
745
|
| `zod` | peer (required) | Schema validation + JSON Schema generation |
|
|
738
746
|
| `pg` | peer (optional) | A Postgres pool for the adapter and the cache; `@niscorp/vex/pg` imports it |
|
|
739
747
|
| `hono` | peer (optional) | Hono framework adapter |
|
|
740
|
-
| `express` |
|
|
748
|
+
| `express` | none | `@niscorp/vex/express` imports nothing from Express; it types the request and response it uses |
|
|
741
749
|
| `@niscorp/cortex` | peer (optional) | Reference query agent (`defineAgent`/`defineTool`, standalone runtime) |
|
|
742
750
|
| `@niscorp/signal` | via Cortex (its peer) | LLM calls + embeddings |
|
|
743
751
|
| `@niscorp/prism` | peer (required) | Result mapping (IR compile/execute; `mappingAgent` for the reference mapper) |
|
package/DOCS.md
CHANGED
|
@@ -44,8 +44,9 @@ path with no model and no network beyond the database.
|
|
|
44
44
|
server-minted (`fp_…`, returned in `meta.cache.fingerprint`) or caller-chosen
|
|
45
45
|
(a name like `deals/list`). Replaying a fingerprint skips generation entirely
|
|
46
46
|
and re-runs only the deterministic path — so scope and context stay current
|
|
47
|
-
while LLM cost drops to zero. A
|
|
48
|
-
|
|
47
|
+
while LLM cost drops to zero. A fingerprint sent alone replays its generation;
|
|
48
|
+
sent with a request that differs, minted or named, it is regenerated and its
|
|
49
|
+
content replaced. Design and
|
|
49
50
|
rationale: [DESIGN.md](./DESIGN.md) **Cache v2 — fingerprints**.
|
|
50
51
|
|
|
51
52
|
**Scope is separate from context.** `context` is caller data, visible to the
|
|
@@ -157,7 +158,7 @@ type QueryEngineConfig = {
|
|
|
157
158
|
cache?: CacheBackend; // default: createMemoryCache()
|
|
158
159
|
onEvent?: VexEventHandler; // pipeline event stream
|
|
159
160
|
generateDsl?: (request, schema, caller) => Promise<Query>;
|
|
160
|
-
mapToShape?: (rows, shape) => Promise<{ ir: CompiledIr; config: unknown; transformed: JsonValue }>;
|
|
161
|
+
mapToShape?: (rows, shape, intent?) => Promise<{ ir: CompiledIr; config: unknown; transformed: JsonValue }>;
|
|
161
162
|
embed?: (text, dimensions?) => Promise<number[]>; // text→vector for semantic filters
|
|
162
163
|
behaviors?: ScopeBehaviors; // which columns a scope key is bound to (see below)
|
|
163
164
|
rows?: LiveRowsConfig; // the rows cache behind `refresh: 'reactive'`
|
|
@@ -1022,11 +1023,15 @@ Two things to know when using it:
|
|
|
1022
1023
|
### `insertEach` — one statement for a caller-sized list
|
|
1023
1024
|
|
|
1024
1025
|
`items` names a context key holding an array of objects; `{ $item: "key" }`
|
|
1025
|
-
values read from the current element
|
|
1026
|
-
|
|
1027
|
-
|
|
1028
|
-
|
|
1029
|
-
|
|
1026
|
+
values read from the current element; every other value — literal,
|
|
1027
|
+
`$context`, `$lookup`, `$returned`, engine-injected `$scope` — is constant
|
|
1028
|
+
across rows. vex binds each element under the column names its `$item`s fill
|
|
1029
|
+
(a key no `$item` names is dropped) and compiles to a single
|
|
1030
|
+
`INSERT … SELECT … FROM jsonb_to_recordset($items) AS item(<column> <type>, …)`,
|
|
1031
|
+
each type the column's own from the introspected schema. Postgres converts
|
|
1032
|
+
every item as it converts an `insert`'s value, a column the statement does not
|
|
1033
|
+
write takes its default, and "one row per ticked weekday" is one authored
|
|
1034
|
+
statement, not a code loop:
|
|
1030
1035
|
|
|
1031
1036
|
```ts
|
|
1032
1037
|
{ op: 'insertEach', table: 'class_templates', items: { $context: 'slots' },
|
|
@@ -1040,6 +1045,10 @@ and `set`/`match` rules pin their columns identically on every row. A
|
|
|
1040
1045
|
non-array `items` value is refused loudly before any SQL runs; an empty array
|
|
1041
1046
|
inserts nothing and returns no rows.
|
|
1042
1047
|
|
|
1048
|
+
An array column's item is a JSON array, a `jsonb` column's any JSON value, and
|
|
1049
|
+
a pgvector `vector` column's a JSON array of numbers (`[0.1, 0.2]`) or the
|
|
1050
|
+
pgvector text form (`"[0.1,0.2]"`).
|
|
1051
|
+
|
|
1043
1052
|
### Replay
|
|
1044
1053
|
|
|
1045
1054
|
The handler dispatches by the entry's kind — one wire shape:
|
|
@@ -1153,8 +1162,9 @@ The positive cache is keyed by the request's `fingerprint` (see the request
|
|
|
1153
1162
|
contract above). There are no cache modes — the body's posture says
|
|
1154
1163
|
everything:
|
|
1155
1164
|
|
|
1156
|
-
- **No fingerprint** → generate, cache, mint (`fp_…`).
|
|
1157
|
-
|
|
1165
|
+
- **No fingerprint** → generate, cache, mint (`fp_…`). Sent alone, the mint
|
|
1166
|
+
replays that generation; sent with a differing request, it is replaced like
|
|
1167
|
+
any named fingerprint.
|
|
1158
1168
|
- **Fingerprint alone** → replay-or-`cache_miss`. Never generates. Context
|
|
1159
1169
|
values may vary per call — sorting and parameterized reloads are legal
|
|
1160
1170
|
against one cached DSL.
|
|
@@ -1394,7 +1404,7 @@ free-form requests.
|
|
|
1394
1404
|
|
|
1395
1405
|
```typescript
|
|
1396
1406
|
type GenerateDsl = (request: QueryRequest, schema: DatabaseSchema, caller: GenerationCaller) => Promise<Query>;
|
|
1397
|
-
type MapToShape = (rows: Row[], shape: unknown) =>
|
|
1407
|
+
type MapToShape = (rows: Row[], shape: unknown, intent?: string) =>
|
|
1398
1408
|
Promise<{ ir: CompiledIr; config: unknown; transformed: JsonValue }>;
|
|
1399
1409
|
|
|
1400
1410
|
type GenerationCaller = {
|
package/README.md
CHANGED
|
@@ -81,6 +81,8 @@ engine.test(dsl, scope?); // Promise<TestResult> — compile + run with
|
|
|
81
81
|
engine.getDslSchema(); // object — the DSL as JSON Schema (draft-7)
|
|
82
82
|
engine.getSchema(); // DatabaseSchema | undefined — last introspected schema
|
|
83
83
|
engine.cache; // CacheBackend — the live cache
|
|
84
|
+
engine.invalidate(tables); // these tables were written: reactive reads over them refetch
|
|
85
|
+
engine.rows; // the reactive rows cache: stats(), stop()
|
|
84
86
|
```
|
|
85
87
|
|
|
86
88
|
## What's in the box
|
|
@@ -94,6 +96,7 @@ engine.cache; // CacheBackend — the live cache
|
|
|
94
96
|
| Writes | Mutation entries in the same cache — a closed grammar, replay-only by fingerprint, never generated; `lintMutation` checks a statement at seed |
|
|
95
97
|
| Seeding and freshness | `seedCache` turns authored `SeedEntry` / `SeedMutation` rows into protected cache entries; an entry's `refresh` is `'snapshot'` or `'reactive'` |
|
|
96
98
|
| Safety | Server-side scope policies, query analyzer (cartesian/nesting/index checks), parameterized SQL only |
|
|
99
|
+
| Stored entries | `@niscorp/vex/migrations`: `VEX_SEQUENCE`, the strata sequence an entry is stamped with as a stored document |
|
|
97
100
|
| Its own table | The Postgres cache's table is a [strata](../strata/README.md) sequence (`cache.sequence`, `nisc.vex.cache`): `cache.init()` applies it through the ledger once, instead of `IF NOT EXISTS` on every boot. Requires a pool with `transaction` (the PGlite pool and `createPgPool`'s have one; a bare `pg.Pool` does not). |
|
|
98
101
|
|
|
99
102
|
## License
|
|
@@ -1055,6 +1055,10 @@ var scopeMutation = (m, policy) => {
|
|
|
1055
1055
|
}
|
|
1056
1056
|
return { op: "delete", table: m.table, where };
|
|
1057
1057
|
};
|
|
1058
|
+
var itemRow = (element, itemKeys) => {
|
|
1059
|
+
const fields = element !== null && typeof element === "object" && !Array.isArray(element) ? element : {};
|
|
1060
|
+
return Object.fromEntries(Object.entries(itemKeys).map(([column, key]) => [column, fields[key]]));
|
|
1061
|
+
};
|
|
1058
1062
|
var newCtx = () => ({
|
|
1059
1063
|
resolvedPaths: /* @__PURE__ */ new Map(),
|
|
1060
1064
|
aliasMap: /* @__PURE__ */ new Map(),
|
|
@@ -1101,13 +1105,6 @@ var compileConflict = (m, ctx, schema) => {
|
|
|
1101
1105
|
}
|
|
1102
1106
|
return ` ON CONFLICT (${c.target.join(", ")}) DO UPDATE SET ${sets.join(", ")}${where}`;
|
|
1103
1107
|
};
|
|
1104
|
-
var ITEM_CAST = {
|
|
1105
|
-
number: "numeric",
|
|
1106
|
-
boolean: "boolean",
|
|
1107
|
-
date: "date",
|
|
1108
|
-
timestamp: "timestamptz",
|
|
1109
|
-
uuid: "uuid"
|
|
1110
|
-
};
|
|
1111
1108
|
var compileMutation = (m, schema) => {
|
|
1112
1109
|
const ctx = newCtx();
|
|
1113
1110
|
if (m.op === "insert") {
|
|
@@ -1117,26 +1114,28 @@ var compileMutation = (m, schema) => {
|
|
|
1117
1114
|
return { sql: `INSERT INTO ${m.table} (${cols.join(", ")}) VALUES (${vals.join(", ")})${compileConflict(m, ctx, schema)} RETURNING *`, slots: ctx.paramSlots, returned: ctx.returned };
|
|
1118
1115
|
}
|
|
1119
1116
|
if (m.op === "insertEach") {
|
|
1120
|
-
const
|
|
1117
|
+
const types = new Map(findEntity(m.table, schema)?.fields.map((f) => [f.name, f.type]));
|
|
1121
1118
|
const entries = Object.entries(m.values);
|
|
1122
1119
|
const cols = entries.map(([c]) => c);
|
|
1120
|
+
const itemKeys = {};
|
|
1121
|
+
const itemColumns = [];
|
|
1123
1122
|
const exprs = entries.map(([col, v]) => {
|
|
1124
1123
|
if (isItemRef(v)) {
|
|
1125
|
-
|
|
1126
|
-
|
|
1127
|
-
|
|
1128
|
-
const raw = `(item.value->>${quoteLiteral(v.$item)})`;
|
|
1129
|
-
return cast === void 0 ? raw : `${raw}::${cast}`;
|
|
1124
|
+
itemKeys[col] = v.$item;
|
|
1125
|
+
itemColumns.push(`${col} ${String(types.get(col))}`);
|
|
1126
|
+
return `item.${col}`;
|
|
1130
1127
|
}
|
|
1131
1128
|
return isReturned(v) ? compileColumnValue(m.table, col, v, ctx, schema) : compileValue(v, ctx);
|
|
1132
1129
|
});
|
|
1133
1130
|
ctx.paramCounter.value += 1;
|
|
1134
1131
|
ctx.paramSlots.push({ key: m.items.$context, kind: "context", type: "json" });
|
|
1135
|
-
const
|
|
1132
|
+
const items = `$${ctx.paramCounter.value}::jsonb`;
|
|
1133
|
+
const source = itemColumns.length === 0 ? `jsonb_array_elements(${items}) AS item` : `jsonb_to_recordset(${items}) AS item(${itemColumns.join(", ")})`;
|
|
1136
1134
|
return {
|
|
1137
1135
|
sql: `INSERT INTO ${m.table} (${cols.join(", ")}) SELECT ${exprs.join(", ")} FROM ${source}${compileConflict(m, ctx, schema)} RETURNING *`,
|
|
1138
1136
|
slots: ctx.paramSlots,
|
|
1139
|
-
returned: ctx.returned
|
|
1137
|
+
returned: ctx.returned,
|
|
1138
|
+
itemKeys
|
|
1140
1139
|
};
|
|
1141
1140
|
}
|
|
1142
1141
|
if (m.op === "update") {
|
|
@@ -1209,7 +1208,8 @@ var executeWrites = async (client, def, mctx) => {
|
|
|
1209
1208
|
if (!Array.isArray(value)) {
|
|
1210
1209
|
throw new VexError("invalid_request", `"${c.slots[i]?.key ?? ""}" must be an array of objects (one inserted row per element).`);
|
|
1211
1210
|
}
|
|
1212
|
-
|
|
1211
|
+
const { itemKeys } = c;
|
|
1212
|
+
return JSON.stringify(itemKeys === void 0 ? value : value.map((element) => itemRow(element, itemKeys)));
|
|
1213
1213
|
});
|
|
1214
1214
|
const res = await q.query(c.sql, bound);
|
|
1215
1215
|
results.push({ table: c.table, op: c.op, rows: res.rows });
|