@niscorp/vex 0.4.0 → 0.4.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,20 @@
1
1
  # @niscorp/vex
2
2
 
3
+ ## 0.4.1
4
+
5
+ ### Patch Changes
6
+
7
+ - 6680ad6: `createQueryDsl` hands the query agent the schema and the DSL spec as compact text instead of JSON: an entity per line with its relations and indexes, and the DSL as types that keep every description, property and allowed value. On a 15-entity CRM schema the fixed part of each call drops from about 36,000 characters to about 13,000. On 19 queries a screen builder asked for, the agent answered all 19, with the same row counts as before, at 44% of the cost and a median of 3.5 s per query instead of 5.1 s. The query it writes is still validated against the full DSL schema.
8
+ - 6680ad6: A shape can declare a field as a Prism expression over the row's stored fields, and `createShapeMapper` computes it with no model call.
9
+
10
+ `[{ close_date: '', close_date_display: { $date: { value: { $ref: '$.close_date' }, format: 'MMM D' } } }]`: `createQueryDsl` asks the query for the stored fields a declared field reads, and `createShapeMapper` computes the declared field with Prism, so the stored value beside it is never overwritten; a mapping a model wrote overwrote it about a quarter of the time. `normalizeShape` keeps a declared field whole, so two formats are two cache entries.
11
+
12
+ A flat shape whose fields the first row carries, each plain one of the kind its example shows, is mapped with no model call, as rows that fit the shape exactly were; rows with more columns than the shape names are now mapped the same way rather than by the agent. Any other shape — nested, a field the rows lack, a value of another kind — goes to the mapping agent, whose mapping is run on the rows before it is used: one that fails there is written again, the agent told why, up to three attempts in all. The agent is told the rows already answer the request (filtered, sorted, limited) and keeps every row, in order.
13
+
14
+ - 1c78d6d: The engine hands `mapToShape` the request's intent, and `createShapeMapper` gives it to the mapping agent.
15
+
16
+ `MapToShape` is `(rows, shape, intent?)`: the engine passes the intent a query was asked in, so the mapping can give a value the form the request named (money as `$1.2M`, a date as `Apr 6`). Before, the mapper saw the rows and the shape alone, whose placeholders say nothing of a form. `createShapeMapper` passes it to prism's `mappingAgent` as its `notes`. A mapper that takes two arguments keeps working.
17
+
3
18
  ## 0.4.0
4
19
 
5
20
  ### Minor Changes
package/DESIGN.md CHANGED
@@ -587,9 +587,11 @@ The engine takes two optional hooks:
587
587
  what the person asking could see — and it is keyed (single-flight, negative
588
588
  cache) by request AND policy, because two callers who see different tables
589
589
  are asking different agents.
590
- - `mapToShape(rows, shape) → { ir, config, transformed }` — raw rows → requested
590
+ - `mapToShape(rows, shape, intent?) → { ir, config, transformed }` — raw rows → requested
591
591
  shape, returning a compiled Prism IR that is cached for reuse, and the Prism
592
- config it was compiled from, which is kept beside it (`mapping`).
592
+ config it was compiled from, which is kept beside it (`mapping`). `intent` is
593
+ the request's own words, when it has them, so the mapping can give a value the
594
+ form they ask for (money as `$1.2M`, a date as `Apr 6`).
593
595
 
594
596
  This is the seam that keeps the core deterministic. Vex ships a reference
595
597
  implementation of both, built on `@niscorp/cortex` and `@niscorp/prism` and
@@ -601,15 +603,16 @@ exported from the `@niscorp/vex/agent` subpath — `createQueryDsl` (fills
601
603
  `getDistinctValues`, `describeField`, `testQuery`, `cannotSatisfy` — and three
602
604
  stop conditions (`stepCount(20)`, `outputRetries(3)`, `repeatedCalls(2)`);
603
605
  `createQueryDsl` aborts the run when `cannotSatisfy` is called. Context
604
- producers inject the live schema, the caller line and the DSL JSON Schema so the
605
- static instructions stay stable. Every data-touching tool is a DSL run through
606
+ producers inject the live schema and the DSL specification as compact text, and
607
+ the caller line, so the static instructions stay stable. Every data-touching tool is a DSL run through
606
608
  `caller.read`; the tool set holds no adapter and no SQL.
607
609
  `createQueryDsl` wraps it into the `generateDsl` hook.
608
610
  - The **mapping agent** is Prism's exported `mappingAgent`; `createShapeMapper`
609
611
  builds the same envelope the runtime replays against — `{ result: rows }` for an
610
612
  array shape, `{ result: rows[0] ?? null }` for an object shape — runs the agent
611
- once to get a transform config, compiles it to IR, and applies the IR once over
612
- that envelope (the identity, with no model call, when the rows already fit).
613
+ once, with the intent as its `notes`, to get a transform config, compiles it to IR, and applies the IR once over
614
+ that envelope. A flat shape whose fields the rows carry is mapped by Prism with no model call, its declared
615
+ fields computed from the stored ones; the agent writes the mapping for any other.
613
616
 
614
617
  > The reference agents are a separate concern from the deterministic core: they
615
618
  > live behind the `@niscorp/vex/agent` subpath, so `@niscorp/cortex` (and
package/DOCS.md CHANGED
@@ -266,6 +266,21 @@ The **shape** is an example, not a schema: use empty strings, zeros, and
266
266
  booleans as type markers. `[{ id: '', total: 0 }]` means "an array of objects
267
267
  with a string `id` and a number `total`." A bare object means a single record.
268
268
 
269
+ A field can be **declared** rather than shown by example: a Prism expression over
270
+ the row's own stored fields, where `$.close_date` is the row's `close_date`.
271
+
272
+ ```typescript
273
+ shape: [{ id: '', close_date: '', close_date_display: { $date: { value: { $ref: '$.close_date' }, format: 'MMM D' } } }]
274
+ // → [{ id: 'deal_085', close_date: '2026-06-08', close_date_display: 'Jun 8' }]
275
+ ```
276
+
277
+ The query is written for the stored fields a declared field reads, and
278
+ `createShapeMapper` computes the declared field with Prism, never a model, so
279
+ the stored value beside it is never overwritten. A declared field is part of the
280
+ request's identity: two formats are two cache entries. A field the rows lack,
281
+ such as a form shown only by example (`close_date_display: 'Apr 6'`), is left
282
+ to the mapping agent; a field the rows already carry is copied as it is.
283
+
269
284
  The shape also picks the result envelope: an **array** shape maps over the whole
270
285
  row set (the mapping sees `$.result` as the rows array — a `$map`/identity); a
271
286
  **non-array** shape maps the single (first) row (the mapping sees `$.result` as
@@ -1397,15 +1412,21 @@ await engine.introspect();
1397
1412
 
1398
1413
  - `createQueryDsl(config)` builds the `generateDsl` hook by running
1399
1414
  `vexQueryDslAgent`. It is built from a model and the DSL spec only — who it
1400
- runs for arrives with every call. The engine calls the hook as
1415
+ runs for arrives with every call. The agent reads the schema and the DSL spec
1416
+ as compact text — an entity per line with its relations and indexes, the DSL
1417
+ as types with every description — and, for a shape with declared fields, the
1418
+ stored fields those read. The engine calls the hook as
1401
1419
  `generateDsl(request, schema, caller)`: `schema` holds only the tables the
1402
1420
  caller's policy can read, and `caller.read(dsl)` runs a DSL through the
1403
1421
  engine's own pipeline under the caller's policy and scope values (`$context`
1404
1422
  binds NULL). That capability is the tools' only way to the database.
1405
- - `createShapeMapper(llm)` builds the `mapToShape` hook by running Prism's
1406
- `mappingAgent`. When the rows already are the shape (`rowsFitShape` — a flat
1407
- shape whose keys and value kinds the first row matches exactly) it stores the
1408
- identity mapping and makes no model call. The `@niscorp/vex/agent` subpath
1423
+ - `createShapeMapper(llm)` builds the `mapToShape` hook. A flat shape whose
1424
+ fields the first row carries, each plain one of the kind its example shows —
1425
+ plain fields by key, declared fields computed from the stored fields they
1426
+ read — is mapped by Prism with no model call. Any other shape (nested, a field
1427
+ the rows lack, a value of another kind) goes to Prism's `mappingAgent`, handed the request's intent and run on the rows
1428
+ before it is used: a mapping that fails there is written again, up to three
1429
+ attempts. `rowsFitShape` is still exported. The `@niscorp/vex/agent` subpath
1409
1430
  also exports `vexQueryDslAgent` itself, `createQueryTools` for customizing the
1410
1431
  tool set, and `describeCaller`, which turns `caller.bindings` into the
1411
1432
  context line the agent reads. A run of `vexQueryDslAgent` given no `caller`