@niscorp/vex 0.4.3 → 0.4.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,13 @@
1
1
  # @niscorp/vex
2
2
 
3
+ ## 0.4.4
4
+
5
+ ### Patch Changes
6
+
7
+ - d70f22c: The docs say entries are keyed by fingerprint (a minted one is replaced like a named one when the request differs), `MapToShape` takes `intent`, the mapping agent retries up to three times, and list `exists`, `optional`, `countDistinct`, `invalidate`, `rows` and the `./migrations` entry.
8
+
9
+ **What to change:** nothing. Documentation only.
10
+
3
11
  ## 0.4.3
4
12
 
5
13
  ### Patch Changes
package/DESIGN.md CHANGED
@@ -30,10 +30,11 @@ execute.
30
30
  that still serves any already-cached shape. Cortex (and Signal behind it) is
31
31
  an optional peer dependency, not a hard one; Prism is required, because a
32
32
  stored mapping is Prism IR and the engine executes it.
33
- 3. **Shape-only caching.** The positive cache key is a hash of the request's
34
- *shape* with all values replaced by type tokens. Two requests with the same
35
- structure but different values share a cache entry. Intent is deliberately
36
- excluded from this key.
33
+ 3. **Fingerprint caching.** The positive cache is keyed by a fingerprint,
34
+ minted by the engine (`fp_…`) or named by the caller. A request reuses an
35
+ entry while it matches the stored one: the same intent, the same *shape* with
36
+ all values replaced by type tokens, the same context key names. Context
37
+ values never count, so one entry serves every value. (See *Cache v2*.)
37
38
  4. **Scope is server-side and LLM-invisible.** Access control is injected into
38
39
  the DSL *after* generation, as ordinary filter clauses bound to server
39
40
  `$scope` values. The model cannot see or forge them, so prompt injection
@@ -135,12 +136,13 @@ queries. Top level (`QuerySchema`):
135
136
  - `filter` — a recursive `FilterSchema` union.
136
137
  - `compute` — record of `alias → ComputeExpression` (arithmetic, `concat`,
137
138
  `coalesce`, `case`).
138
- - `aggregate` — record of `alias → AggregateExpression` (`count`/`sum`/`avg`/
139
- `min`/`max`).
139
+ - `aggregate` — record of `alias → AggregateExpression` (`count`/`countDistinct`/
140
+ `sum`/`avg`/`min`/`max`).
140
141
  - `groupBy`, `sort`, `limit`, `distinct`.
141
142
 
142
143
  Filter operators: `eq`/`neq`/`gt`/`gte`/`lt`/`lte`, `in`/`notIn`, `like`/
143
- `ilike`, `isNull`/`isNotNull`, `and`/`or`/`not`, plus `semantic` (vector
144
+ `ilike`, `isNull`/`isNotNull`, `and`/`or`/`not`, `exists` (a correlated subquery),
145
+ `optional` (a condition the caller turns on by supplying context keys), plus `semantic` (vector
144
146
  similarity) and `fuzzy` (string distance). Values are literals, `{ $context }`
145
147
  (caller-provided) or `{ $scope }` (server-injected).
146
148
 
@@ -295,9 +297,9 @@ never change it.
295
297
 
296
298
  - A request WITHOUT a fingerprint generates and caches; the server mints a
297
299
  fingerprint (`fp_…`) and returns it in `meta.cache.fingerprint`.
298
- - A minted fingerprint is an **immutable pin** — embed it and replay the exact
299
- generation forever. Changing the query means proving again and embedding the
300
- new fingerprint. (git: a commit hash.)
300
+ - A minted fingerprint replays its generation for as long as it is sent
301
+ alone. Sent with a request that differs, it is regenerated and replaced like
302
+ a named slot, unless protected.
301
303
  - A caller-chosen fingerprint (any string) is a **mutable named slot** — the
302
304
  name is yours, its content follows your request. (git: a branch.)
303
305
  - Intent NEVER keys anything: natural language is fake identity in both
@@ -609,9 +611,10 @@ exported from the `@niscorp/vex/agent` subpath — `createQueryDsl` (fills
609
611
  `createQueryDsl` wraps it into the `generateDsl` hook.
610
612
  - The **mapping agent** is Prism's exported `mappingAgent`; `createShapeMapper`
611
613
  builds the same envelope the runtime replays against — `{ result: rows }` for an
612
- array shape, `{ result: rows[0] ?? null }` for an object shape — runs the agent
613
- once, with the intent as its `notes`, to get a transform config, compiles it to IR, and applies the IR once over
614
- that envelope. A flat shape whose fields the rows carry is mapped by Prism with no model call, its declared
614
+ array shape, `{ result: rows[0] ?? null }` for an object shape — runs the agent,
615
+ with the intent as its `notes`, to get a transform config, compiles it to IR, and applies the IR once over
616
+ that envelope. A config that fails to compile or run on the rows sends the agent round again with the error in
617
+ its notes, up to three attempts (`MAPPING_ATTEMPTS`). A flat shape whose fields the rows carry is mapped by Prism with no model call, its declared
615
618
  fields computed from the stored ones; the agent writes the mapping for any other.
616
619
 
617
620
  > The reference agents are a separate concern from the deterministic core: they
@@ -718,8 +721,13 @@ src/
718
721
  agent/ Reference Cortex agents (exported via ./agent)
719
722
  query.agent.ts defineAgent: vexQueryDslAgent (vex.query → Query)
720
723
  tools.ts the six query tools
724
+ compact.ts compactSchema, compactSpec — the schema and DSL spec as compact text
725
+ declared.ts declaredMapping, storedShape — fields a shape declares as prism expressions
721
726
  index.ts createQueryDsl, createShapeMapper, vexQueryDslAgent
722
727
 
728
+ migrations/
729
+ index.ts VEX_SEQUENCE, VEX_SCHEMAS — an entry as a stored document (./migrations)
730
+
723
731
  utils/
724
732
  context.ts buildValidationContext, resolveParams
725
733
  canonical.ts canonical JSON text + hash
@@ -737,7 +745,7 @@ scripts/ Dev-only: docker, seed, dev server, fixtures
737
745
  | `zod` | peer (required) | Schema validation + JSON Schema generation |
738
746
  | `pg` | peer (optional) | A Postgres pool for the adapter and the cache; `@niscorp/vex/pg` imports it |
739
747
  | `hono` | peer (optional) | Hono framework adapter |
740
- | `express` | peer (optional) | Express framework adapter |
748
+ | `express` | none | `@niscorp/vex/express` imports nothing from Express; it types the request and response it uses |
741
749
  | `@niscorp/cortex` | peer (optional) | Reference query agent (`defineAgent`/`defineTool`, standalone runtime) |
742
750
  | `@niscorp/signal` | via Cortex (its peer) | LLM calls + embeddings |
743
751
  | `@niscorp/prism` | peer (required) | Result mapping (IR compile/execute; `mappingAgent` for the reference mapper) |
package/DOCS.md CHANGED
@@ -44,8 +44,9 @@ path with no model and no network beyond the database.
44
44
  server-minted (`fp_…`, returned in `meta.cache.fingerprint`) or caller-chosen
45
45
  (a name like `deals/list`). Replaying a fingerprint skips generation entirely
46
46
  and re-runs only the deterministic path — so scope and context stay current
47
- while LLM cost drops to zero. A minted fingerprint is an immutable pin; a
48
- named one is a mutable slot whose content follows your request. Design and
47
+ while LLM cost drops to zero. A fingerprint sent alone replays its generation;
48
+ sent with a request that differs, minted or named, it is regenerated and its
49
+ content replaced. Design and
49
50
  rationale: [DESIGN.md](./DESIGN.md) **Cache v2 — fingerprints**.
50
51
 
51
52
  **Scope is separate from context.** `context` is caller data, visible to the
@@ -157,7 +158,7 @@ type QueryEngineConfig = {
157
158
  cache?: CacheBackend; // default: createMemoryCache()
158
159
  onEvent?: VexEventHandler; // pipeline event stream
159
160
  generateDsl?: (request, schema, caller) => Promise<Query>;
160
- mapToShape?: (rows, shape) => Promise<{ ir: CompiledIr; config: unknown; transformed: JsonValue }>;
161
+ mapToShape?: (rows, shape, intent?) => Promise<{ ir: CompiledIr; config: unknown; transformed: JsonValue }>;
161
162
  embed?: (text, dimensions?) => Promise<number[]>; // text→vector for semantic filters
162
163
  behaviors?: ScopeBehaviors; // which columns a scope key is bound to (see below)
163
164
  rows?: LiveRowsConfig; // the rows cache behind `refresh: 'reactive'`
@@ -1161,8 +1162,9 @@ The positive cache is keyed by the request's `fingerprint` (see the request
1161
1162
  contract above). There are no cache modes — the body's posture says
1162
1163
  everything:
1163
1164
 
1164
- - **No fingerprint** → generate, cache, mint (`fp_…`). The mint is an
1165
- immutable pin: embed it and replay that exact generation forever.
1165
+ - **No fingerprint** → generate, cache, mint (`fp_…`). Sent alone, the mint
1166
+ replays that generation; sent with a differing request, it is replaced like
1167
+ any named fingerprint.
1166
1168
  - **Fingerprint alone** → replay-or-`cache_miss`. Never generates. Context
1167
1169
  values may vary per call — sorting and parameterized reloads are legal
1168
1170
  against one cached DSL.
@@ -1402,7 +1404,7 @@ free-form requests.
1402
1404
 
1403
1405
  ```typescript
1404
1406
  type GenerateDsl = (request: QueryRequest, schema: DatabaseSchema, caller: GenerationCaller) => Promise<Query>;
1405
- type MapToShape = (rows: Row[], shape: unknown) =>
1407
+ type MapToShape = (rows: Row[], shape: unknown, intent?: string) =>
1406
1408
  Promise<{ ir: CompiledIr; config: unknown; transformed: JsonValue }>;
1407
1409
 
1408
1410
  type GenerationCaller = {
package/README.md CHANGED
@@ -81,6 +81,8 @@ engine.test(dsl, scope?); // Promise<TestResult> — compile + run with
81
81
  engine.getDslSchema(); // object — the DSL as JSON Schema (draft-7)
82
82
  engine.getSchema(); // DatabaseSchema | undefined — last introspected schema
83
83
  engine.cache; // CacheBackend — the live cache
84
+ engine.invalidate(tables); // these tables were written: reactive reads over them refetch
85
+ engine.rows; // the reactive rows cache: stats(), stop()
84
86
  ```
85
87
 
86
88
  ## What's in the box
@@ -94,6 +96,7 @@ engine.cache; // CacheBackend — the live cache
94
96
  | Writes | Mutation entries in the same cache — a closed grammar, replay-only by fingerprint, never generated; `lintMutation` checks a statement at seed |
95
97
  | Seeding and freshness | `seedCache` turns authored `SeedEntry` / `SeedMutation` rows into protected cache entries; an entry's `refresh` is `'snapshot'` or `'reactive'` |
96
98
  | Safety | Server-side scope policies, query analyzer (cartesian/nesting/index checks), parameterized SQL only |
99
+ | Stored entries | `@niscorp/vex/migrations`: `VEX_SEQUENCE`, the strata sequence an entry is stamped with as a stored document |
97
100
  | Its own table | The Postgres cache's table is a [strata](../strata/README.md) sequence (`cache.sequence`, `nisc.vex.cache`): `cache.init()` applies it through the ledger once, instead of `IF NOT EXISTS` on every boot. Requires a pool with `transaction` (the PGlite pool and `createPgPool`'s have one; a bare `pg.Pool` does not). |
98
101
 
99
102
  ## License
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@niscorp/vex",
3
- "version": "0.4.3",
3
+ "version": "0.4.4",
4
4
  "description": "Declarative query synthesis — intent to SQL with caching and scope policies",
5
5
  "license": "Apache-2.0",
6
6
  "repository": {
@@ -98,9 +98,9 @@
98
98
  "hono": "^4",
99
99
  "pg": "^8",
100
100
  "zod": "^4.2.0",
101
- "@niscorp/cortex": "^0.1.4",
102
- "@niscorp/prism": "^0.3.1",
103
- "@niscorp/strata": "^0.1.7"
101
+ "@niscorp/cortex": "^0.1.6",
102
+ "@niscorp/strata": "^0.1.9",
103
+ "@niscorp/prism": "^0.3.3"
104
104
  },
105
105
  "devDependencies": {
106
106
  "@electric-sql/pglite": "^0.5.1",
@@ -114,10 +114,10 @@
114
114
  "tsx": "^4.20.3",
115
115
  "vitest": "^4.1.2",
116
116
  "zod": "^4.6.5",
117
- "@niscorp/cortex": "^0.1.4",
118
- "@niscorp/prism": "^0.3.1",
119
- "@niscorp/signal": "^0.1.5",
120
- "@niscorp/strata": "^0.1.7"
117
+ "@niscorp/cortex": "^0.1.6",
118
+ "@niscorp/signal": "^0.1.9",
119
+ "@niscorp/prism": "^0.3.3",
120
+ "@niscorp/strata": "^0.1.9"
121
121
  },
122
122
  "peerDependenciesMeta": {
123
123
  "@niscorp/cortex": {