@niscorp/vex 0.3.1 → 0.4.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +27 -0
- package/DESIGN.md +11 -7
- package/DOCS.md +52 -9
- package/dist/adapters/express/index.cjs +1 -1
- package/dist/adapters/express/index.cjs.map +1 -1
- package/dist/adapters/express/index.d.cts +3 -2
- package/dist/adapters/express/index.d.ts +3 -2
- package/dist/adapters/express/index.js +3 -2
- package/dist/adapters/express/index.js.map +1 -1
- package/dist/adapters/hono/index.cjs +1 -1
- package/dist/adapters/hono/index.cjs.map +1 -1
- package/dist/adapters/hono/index.d.cts +3 -2
- package/dist/adapters/hono/index.d.ts +3 -2
- package/dist/adapters/hono/index.js +3 -2
- package/dist/adapters/hono/index.js.map +1 -1
- package/dist/agent/index.cjs +144 -21
- package/dist/agent/index.cjs.map +1 -1
- package/dist/agent/index.d.cts +3 -2
- package/dist/agent/index.d.ts +3 -2
- package/dist/agent/index.js +146 -22
- package/dist/agent/index.js.map +1 -1
- package/dist/{chunk-3HORTMZ7.js → chunk-AKR6HU34.js} +9 -5
- package/dist/chunk-AKR6HU34.js.map +1 -0
- package/dist/chunk-F7RDSWYF.js +7 -0
- package/dist/chunk-F7RDSWYF.js.map +1 -0
- package/dist/{chunk-J7JYKCUA.js → chunk-UPQMKAAX.js} +3 -18
- package/dist/chunk-UPQMKAAX.js.map +1 -0
- package/dist/chunk-XTOK7YOR.js +18 -0
- package/dist/chunk-XTOK7YOR.js.map +1 -0
- package/dist/{handler-B4NAi4k_.d.cts → handler-CoJxInw-.d.ts} +2 -1
- package/dist/{handler-BRVhWuNk.d.ts → handler-CsFbanZA.d.cts} +2 -1
- package/dist/index.cjs +26 -13
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +6 -4
- package/dist/index.d.ts +6 -4
- package/dist/index.js +30 -20
- package/dist/index.js.map +1 -1
- package/dist/migrations/index.cjs +173 -0
- package/dist/migrations/index.cjs.map +1 -0
- package/dist/migrations/index.d.cts +20 -0
- package/dist/migrations/index.d.ts +20 -0
- package/dist/migrations/index.js +23 -0
- package/dist/migrations/index.js.map +1 -0
- package/dist/query.schema-B-LVNfyX.d.cts +461 -0
- package/dist/query.schema-B-LVNfyX.d.ts +461 -0
- package/dist/{types-DUJp5B3o.d.cts → types-B2HEmyNq.d.cts} +11 -463
- package/dist/{types-DUJp5B3o.d.ts → types-DC68ZUuq.d.ts} +11 -463
- package/package.json +12 -1
- package/dist/chunk-3HORTMZ7.js.map +0 -1
- package/dist/chunk-J7JYKCUA.js.map +0 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,32 @@
|
|
|
1
1
|
# @niscorp/vex
|
|
2
2
|
|
|
3
|
+
## 0.4.1
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- 6680ad6: `createQueryDsl` hands the query agent the schema and the DSL spec as compact text instead of JSON: an entity per line with its relations and indexes, and the DSL as types that keep every description, property and allowed value. On a 15-entity CRM schema the fixed part of each call drops from about 36,000 characters to about 13,000. On 19 queries a screen builder asked for, the agent answered all 19, with the same row counts as before, at 44% of the cost and a median of 3.5 s per query instead of 5.1 s. The query it writes is still validated against the full DSL schema.
|
|
8
|
+
- 6680ad6: A shape can declare a field as a Prism expression over the row's stored fields, and `createShapeMapper` computes it with no model call.
|
|
9
|
+
|
|
10
|
+
`[{ close_date: '', close_date_display: { $date: { value: { $ref: '$.close_date' }, format: 'MMM D' } } }]`: `createQueryDsl` asks the query for the stored fields a declared field reads, and `createShapeMapper` computes the declared field with Prism, so the stored value beside it is never overwritten; a mapping a model wrote overwrote it about a quarter of the time. `normalizeShape` keeps a declared field whole, so two formats are two cache entries.
|
|
11
|
+
|
|
12
|
+
A flat shape whose fields the first row carries, each plain one of the kind its example shows, is mapped with no model call, as rows that fit the shape exactly were; rows with more columns than the shape names are now mapped the same way rather than by the agent. Any other shape — nested, a field the rows lack, a value of another kind — goes to the mapping agent, whose mapping is run on the rows before it is used: one that fails there is written again, the agent told why, up to three attempts in all. The agent is told the rows already answer the request (filtered, sorted, limited) and keeps every row, in order.
|
|
13
|
+
|
|
14
|
+
- 1c78d6d: The engine hands `mapToShape` the request's intent, and `createShapeMapper` gives it to the mapping agent.
|
|
15
|
+
|
|
16
|
+
`MapToShape` is `(rows, shape, intent?)`: the engine passes the intent a query was asked in, so the mapping can give a value the form the request named (money as `$1.2M`, a date as `Apr 6`). Before, the mapper saw the rows and the shape alone, whose placeholders say nothing of a form. `createShapeMapper` passes it to prism's `mappingAgent` as its `notes`. A mapper that takes two arguments keeps working.
|
|
17
|
+
|
|
18
|
+
## 0.4.0
|
|
19
|
+
|
|
20
|
+
### Minor Changes
|
|
21
|
+
|
|
22
|
+
- 4068f98: Vex keeps what an entry it writes was written from, and says who asks when it looks one up. **Breaking.**
|
|
23
|
+
|
|
24
|
+
BREAKING — approved by moccadroid, 2026-10-14: a `mapToShape` an app writes answers `config`, the Prism config its `ir` was compiled from, beside `ir` and `transformed`. `createShapeMapper` does.
|
|
25
|
+
- **A query vex writes keeps its mapping's Prism config** (`mapping`, on the entry), beside the compiled form. A compiled form can neither be read back as a config nor upgraded when Prism changes.
|
|
26
|
+
- **The Postgres cache's table gets a `mapping` column**, in a new migration, `nisc.vex.cache/2`. A deployment that runs its migrations before it boots runs that one first.
|
|
27
|
+
- **`CacheBackend.get(key, scope?)`**: the two lookups that replay an entry hand the cache the scope of whoever asks, so a cache can answer an entry to some callers and not to others. A backend that takes no scope works as before.
|
|
28
|
+
- **`@niscorp/vex/migrations`** exports `VEX_SEQUENCE`, one stored kind, `nisc.vex/entry` (an entry as written, its `mapping` a `nisc.prism/config`), and `VEX_SCHEMAS`.
|
|
29
|
+
|
|
3
30
|
## 0.3.1
|
|
4
31
|
|
|
5
32
|
### Patch Changes
|
package/DESIGN.md
CHANGED
|
@@ -266,7 +266,7 @@ Identity is the **fingerprint** (see Cache v2 below). The supporting hashes:
|
|
|
266
266
|
impossible, so it isn't re-run until the TTL lapses or the schema changes).
|
|
267
267
|
- **Backends.** In-memory (`createMemoryCache`), durable Postgres
|
|
268
268
|
(`createPostgresCache`, jsonb table, server-side TTL, `protected`/
|
|
269
|
-
`last_used_at`/`request_hash` columns; the table is a strata sequence,
|
|
269
|
+
`last_used_at`/`request_hash`/`mapping` columns; the table is a strata sequence,
|
|
270
270
|
`nisc.vex.cache`, applied once through the ledger by `init()`;
|
|
271
271
|
validate-on-write and evict-on-read), and tiered (`createTieredCache`, L1 +
|
|
272
272
|
L2 read-through/write-through with `full`/`lazy`/`partial` warm-up). Every
|
|
@@ -587,8 +587,11 @@ The engine takes two optional hooks:
|
|
|
587
587
|
what the person asking could see — and it is keyed (single-flight, negative
|
|
588
588
|
cache) by request AND policy, because two callers who see different tables
|
|
589
589
|
are asking different agents.
|
|
590
|
-
- `mapToShape(rows, shape) → { ir, transformed }` — raw rows → requested
|
|
591
|
-
returning a compiled Prism IR that is cached for reuse
|
|
590
|
+
- `mapToShape(rows, shape, intent?) → { ir, config, transformed }` — raw rows → requested
|
|
591
|
+
shape, returning a compiled Prism IR that is cached for reuse, and the Prism
|
|
592
|
+
config it was compiled from, which is kept beside it (`mapping`). `intent` is
|
|
593
|
+
the request's own words, when it has them, so the mapping can give a value the
|
|
594
|
+
form they ask for (money as `$1.2M`, a date as `Apr 6`).
|
|
592
595
|
|
|
593
596
|
This is the seam that keeps the core deterministic. Vex ships a reference
|
|
594
597
|
implementation of both, built on `@niscorp/cortex` and `@niscorp/prism` and
|
|
@@ -600,15 +603,16 @@ exported from the `@niscorp/vex/agent` subpath — `createQueryDsl` (fills
|
|
|
600
603
|
`getDistinctValues`, `describeField`, `testQuery`, `cannotSatisfy` — and three
|
|
601
604
|
stop conditions (`stepCount(20)`, `outputRetries(3)`, `repeatedCalls(2)`);
|
|
602
605
|
`createQueryDsl` aborts the run when `cannotSatisfy` is called. Context
|
|
603
|
-
producers inject the live schema
|
|
604
|
-
static instructions stay stable. Every data-touching tool is a DSL run through
|
|
606
|
+
producers inject the live schema and the DSL specification as compact text, and
|
|
607
|
+
the caller line, so the static instructions stay stable. Every data-touching tool is a DSL run through
|
|
605
608
|
`caller.read`; the tool set holds no adapter and no SQL.
|
|
606
609
|
`createQueryDsl` wraps it into the `generateDsl` hook.
|
|
607
610
|
- The **mapping agent** is Prism's exported `mappingAgent`; `createShapeMapper`
|
|
608
611
|
builds the same envelope the runtime replays against — `{ result: rows }` for an
|
|
609
612
|
array shape, `{ result: rows[0] ?? null }` for an object shape — runs the agent
|
|
610
|
-
once to get a transform config, compiles it to IR, and applies the IR once over
|
|
611
|
-
that envelope
|
|
613
|
+
once, with the intent as its `notes`, to get a transform config, compiles it to IR, and applies the IR once over
|
|
614
|
+
that envelope. A flat shape whose fields the rows carry is mapped by Prism with no model call, its declared
|
|
615
|
+
fields computed from the stored ones; the agent writes the mapping for any other.
|
|
612
616
|
|
|
613
617
|
> The reference agents are a separate concern from the deterministic core: they
|
|
614
618
|
> live behind the `@niscorp/vex/agent` subpath, so `@niscorp/cortex` (and
|
package/DOCS.md
CHANGED
|
@@ -157,7 +157,7 @@ type QueryEngineConfig = {
|
|
|
157
157
|
cache?: CacheBackend; // default: createMemoryCache()
|
|
158
158
|
onEvent?: VexEventHandler; // pipeline event stream
|
|
159
159
|
generateDsl?: (request, schema, caller) => Promise<Query>;
|
|
160
|
-
mapToShape?: (rows, shape) => Promise<{ ir: CompiledIr; transformed: JsonValue }>;
|
|
160
|
+
mapToShape?: (rows, shape) => Promise<{ ir: CompiledIr; config: unknown; transformed: JsonValue }>;
|
|
161
161
|
embed?: (text, dimensions?) => Promise<number[]>; // text→vector for semantic filters
|
|
162
162
|
behaviors?: ScopeBehaviors; // which columns a scope key is bound to (see below)
|
|
163
163
|
rows?: LiveRowsConfig; // the rows cache behind `refresh: 'reactive'`
|
|
@@ -266,6 +266,21 @@ The **shape** is an example, not a schema: use empty strings, zeros, and
|
|
|
266
266
|
booleans as type markers. `[{ id: '', total: 0 }]` means "an array of objects
|
|
267
267
|
with a string `id` and a number `total`." A bare object means a single record.
|
|
268
268
|
|
|
269
|
+
A field can be **declared** rather than shown by example: a Prism expression over
|
|
270
|
+
the row's own stored fields, where `$.close_date` is the row's `close_date`.
|
|
271
|
+
|
|
272
|
+
```typescript
|
|
273
|
+
shape: [{ id: '', close_date: '', close_date_display: { $date: { value: { $ref: '$.close_date' }, format: 'MMM D' } } }]
|
|
274
|
+
// → [{ id: 'deal_085', close_date: '2026-06-08', close_date_display: 'Jun 8' }]
|
|
275
|
+
```
|
|
276
|
+
|
|
277
|
+
The query is written for the stored fields a declared field reads, and
|
|
278
|
+
`createShapeMapper` computes the declared field with Prism, never a model, so
|
|
279
|
+
the stored value beside it is never overwritten. A declared field is part of the
|
|
280
|
+
request's identity: two formats are two cache entries. A field the rows lack,
|
|
281
|
+
such as a form shown only by example (`close_date_display: 'Apr 6'`), is left
|
|
282
|
+
to the mapping agent; a field the rows already carry is copied as it is.
|
|
283
|
+
|
|
269
284
|
The shape also picks the result envelope: an **array** shape maps over the whole
|
|
270
285
|
row set (the mapping sees `$.result` as the rows array — a `$map`/identity); a
|
|
271
286
|
**non-array** shape maps the single (first) row (the mapping sees `$.result` as
|
|
@@ -1232,6 +1247,27 @@ demand; `partial` preloads only the listed fingerprints and lazy-fills the
|
|
|
1232
1247
|
rest. The tiered cache is single-instance: separate processes keep separate
|
|
1233
1248
|
L1s.
|
|
1234
1249
|
|
|
1250
|
+
**Who asks.** The engine hands the two lookups that replay an entry the scope
|
|
1251
|
+
of whoever asks (`get(key, scope)`), so a cache a host writes can answer an
|
|
1252
|
+
entry to some callers and not to others. The three backends here answer every
|
|
1253
|
+
caller alike.
|
|
1254
|
+
|
|
1255
|
+
**What a query vex wrote keeps.** The query, the compiled mapping (`prismIr`)
|
|
1256
|
+
and the Prism config it was compiled from (`mapping`), which `mapToShape`
|
|
1257
|
+
answers as `config`. A compiled form can neither be read back as a config nor
|
|
1258
|
+
upgraded when Prism changes, so a host that keeps such an entry for longer
|
|
1259
|
+
keeps the config. The Postgres table has a column for it from
|
|
1260
|
+
`nisc.vex.cache/2` on: a deployment that runs its migrations before it boots
|
|
1261
|
+
runs that one first.
|
|
1262
|
+
|
|
1263
|
+
**An entry as a stored document.** `@niscorp/vex/migrations` exports
|
|
1264
|
+
`VEX_SEQUENCE`, a strata sequence with one kind, `nisc.vex/entry`: an entry as
|
|
1265
|
+
written, `{ dsl, mapping?, shape?, intent?, reach?, refresh? }`, without the
|
|
1266
|
+
fingerprint it is kept under, its `mapping` a `nisc.prism/config`. A host that
|
|
1267
|
+
stores entries stamps them with it and upgrades them where it reads them, as it
|
|
1268
|
+
does nova's and Prism's documents (`VEX_SCHEMAS` is what the grammar check
|
|
1269
|
+
snapshots).
|
|
1270
|
+
|
|
1235
1271
|
### Seeding
|
|
1236
1272
|
|
|
1237
1273
|
`seedCache(cache, entries)` turns authored entries into **protected** cache
|
|
@@ -1335,7 +1371,7 @@ free-form requests.
|
|
|
1335
1371
|
```typescript
|
|
1336
1372
|
type GenerateDsl = (request: QueryRequest, schema: DatabaseSchema, caller: GenerationCaller) => Promise<Query>;
|
|
1337
1373
|
type MapToShape = (rows: Row[], shape: unknown) =>
|
|
1338
|
-
Promise<{ ir: CompiledIr; transformed: JsonValue }>;
|
|
1374
|
+
Promise<{ ir: CompiledIr; config: unknown; transformed: JsonValue }>;
|
|
1339
1375
|
|
|
1340
1376
|
type GenerationCaller = {
|
|
1341
1377
|
read: (dsl: Query) => Promise<{ rows: Row[]; sql: string; warnings: string[] }>;
|
|
@@ -1348,8 +1384,9 @@ type GenerationCaller = {
|
|
|
1348
1384
|
`new VexError('unsatisfiable', reason)` — the runtime negative-caches it.
|
|
1349
1385
|
- `mapToShape` runs when a shape is in play (the request's, or the stored one on
|
|
1350
1386
|
a fingerprint-only replay) and the entry has no stored mapping; it returns a
|
|
1351
|
-
compiled Prism `ir
|
|
1352
|
-
only on a miss — a replayed entry that
|
|
1387
|
+
compiled Prism `ir`, the Prism `config` it was compiled from, and the mapped
|
|
1388
|
+
result. Both are stored with the entry only on a miss — a replayed entry that
|
|
1389
|
+
has none calls the hook every time.
|
|
1353
1390
|
Without the hook the raw rows are returned.
|
|
1354
1391
|
|
|
1355
1392
|
You can implement these however you like. Vex also ships a **reference
|
|
@@ -1375,15 +1412,21 @@ await engine.introspect();
|
|
|
1375
1412
|
|
|
1376
1413
|
- `createQueryDsl(config)` builds the `generateDsl` hook by running
|
|
1377
1414
|
`vexQueryDslAgent`. It is built from a model and the DSL spec only — who it
|
|
1378
|
-
runs for arrives with every call. The
|
|
1415
|
+
runs for arrives with every call. The agent reads the schema and the DSL spec
|
|
1416
|
+
as compact text — an entity per line with its relations and indexes, the DSL
|
|
1417
|
+
as types with every description — and, for a shape with declared fields, the
|
|
1418
|
+
stored fields those read. The engine calls the hook as
|
|
1379
1419
|
`generateDsl(request, schema, caller)`: `schema` holds only the tables the
|
|
1380
1420
|
caller's policy can read, and `caller.read(dsl)` runs a DSL through the
|
|
1381
1421
|
engine's own pipeline under the caller's policy and scope values (`$context`
|
|
1382
1422
|
binds NULL). That capability is the tools' only way to the database.
|
|
1383
|
-
- `createShapeMapper(llm)` builds the `mapToShape` hook
|
|
1384
|
-
|
|
1385
|
-
|
|
1386
|
-
|
|
1423
|
+
- `createShapeMapper(llm)` builds the `mapToShape` hook. A flat shape whose
|
|
1424
|
+
fields the first row carries, each plain one of the kind its example shows —
|
|
1425
|
+
plain fields by key, declared fields computed from the stored fields they
|
|
1426
|
+
read — is mapped by Prism with no model call. Any other shape (nested, a field
|
|
1427
|
+
the rows lack, a value of another kind) goes to Prism's `mappingAgent`, handed the request's intent and run on the rows
|
|
1428
|
+
before it is used: a mapping that fails there is written again, up to three
|
|
1429
|
+
attempts. `rowsFitShape` is still exported. The `@niscorp/vex/agent` subpath
|
|
1387
1430
|
also exports `vexQueryDslAgent` itself, `createQueryTools` for customizing the
|
|
1388
1431
|
tool set, and `describeCaller`, which turns `caller.bindings` into the
|
|
1389
1432
|
context line the agent reads. A run of `vexQueryDslAgent` given no `caller`
|
|
@@ -1405,7 +1405,7 @@ var runQuery = async (config, body, scope, probe, live) => {
|
|
|
1405
1405
|
try {
|
|
1406
1406
|
let entryReach;
|
|
1407
1407
|
if (parsed.data.fingerprint !== void 0) {
|
|
1408
|
-
const entry = await engine.cache.get(parsed.data.fingerprint);
|
|
1408
|
+
const entry = await engine.cache.get(parsed.data.fingerprint, scope);
|
|
1409
1409
|
if (entry !== void 0 && entry.kind === "ok") entryReach = entry.reach;
|
|
1410
1410
|
if (probe !== void 0 && entryReach !== void 0) probe.reach = entryReach;
|
|
1411
1411
|
if (entry !== void 0 && entry.kind === "mutation") {
|