@niscorp/vex 0.4.3 → 0.4.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +8 -0
- package/DESIGN.md +22 -14
- package/DOCS.md +8 -6
- package/README.md +3 -0
- package/package.json +8 -8
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,13 @@
|
|
|
1
1
|
# @niscorp/vex
|
|
2
2
|
|
|
3
|
+
## 0.4.4
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- d70f22c: The docs say entries are keyed by fingerprint (a minted one is replaced like a named one when the request differs), `MapToShape` takes `intent`, the mapping agent retries up to three times, and list `exists`, `optional`, `countDistinct`, `invalidate`, `rows` and the `./migrations` entry.
|
|
8
|
+
|
|
9
|
+
**What to change:** nothing. Documentation only.
|
|
10
|
+
|
|
3
11
|
## 0.4.3
|
|
4
12
|
|
|
5
13
|
### Patch Changes
|
package/DESIGN.md
CHANGED
|
@@ -30,10 +30,11 @@ execute.
|
|
|
30
30
|
that still serves any already-cached shape. Cortex (and Signal behind it) is
|
|
31
31
|
an optional peer dependency, not a hard one; Prism is required, because a
|
|
32
32
|
stored mapping is Prism IR and the engine executes it.
|
|
33
|
-
3. **
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
33
|
+
3. **Fingerprint caching.** The positive cache is keyed by a fingerprint,
|
|
34
|
+
minted by the engine (`fp_…`) or named by the caller. A request reuses an
|
|
35
|
+
entry while it matches the stored one: the same intent, the same *shape* with
|
|
36
|
+
all values replaced by type tokens, the same context key names. Context
|
|
37
|
+
values never count, so one entry serves every value. (See *Cache v2*.)
|
|
37
38
|
4. **Scope is server-side and LLM-invisible.** Access control is injected into
|
|
38
39
|
the DSL *after* generation, as ordinary filter clauses bound to server
|
|
39
40
|
`$scope` values. The model cannot see or forge them, so prompt injection
|
|
@@ -135,12 +136,13 @@ queries. Top level (`QuerySchema`):
|
|
|
135
136
|
- `filter` — a recursive `FilterSchema` union.
|
|
136
137
|
- `compute` — record of `alias → ComputeExpression` (arithmetic, `concat`,
|
|
137
138
|
`coalesce`, `case`).
|
|
138
|
-
- `aggregate` — record of `alias → AggregateExpression` (`count`/`
|
|
139
|
-
`min`/`max`).
|
|
139
|
+
- `aggregate` — record of `alias → AggregateExpression` (`count`/`countDistinct`/
|
|
140
|
+
`sum`/`avg`/`min`/`max`).
|
|
140
141
|
- `groupBy`, `sort`, `limit`, `distinct`.
|
|
141
142
|
|
|
142
143
|
Filter operators: `eq`/`neq`/`gt`/`gte`/`lt`/`lte`, `in`/`notIn`, `like`/
|
|
143
|
-
`ilike`, `isNull`/`isNotNull`, `and`/`or`/`not`,
|
|
144
|
+
`ilike`, `isNull`/`isNotNull`, `and`/`or`/`not`, `exists` (a correlated subquery),
|
|
145
|
+
`optional` (a condition the caller turns on by supplying context keys), plus `semantic` (vector
|
|
144
146
|
similarity) and `fuzzy` (string distance). Values are literals, `{ $context }`
|
|
145
147
|
(caller-provided) or `{ $scope }` (server-injected).
|
|
146
148
|
|
|
@@ -295,9 +297,9 @@ never change it.
|
|
|
295
297
|
|
|
296
298
|
- A request WITHOUT a fingerprint generates and caches; the server mints a
|
|
297
299
|
fingerprint (`fp_…`) and returns it in `meta.cache.fingerprint`.
|
|
298
|
-
- A minted fingerprint
|
|
299
|
-
|
|
300
|
-
|
|
300
|
+
- A minted fingerprint replays its generation for as long as it is sent
|
|
301
|
+
alone. Sent with a request that differs, it is regenerated and replaced like
|
|
302
|
+
a named slot, unless protected.
|
|
301
303
|
- A caller-chosen fingerprint (any string) is a **mutable named slot** — the
|
|
302
304
|
name is yours, its content follows your request. (git: a branch.)
|
|
303
305
|
- Intent NEVER keys anything: natural language is fake identity in both
|
|
@@ -609,9 +611,10 @@ exported from the `@niscorp/vex/agent` subpath — `createQueryDsl` (fills
|
|
|
609
611
|
`createQueryDsl` wraps it into the `generateDsl` hook.
|
|
610
612
|
- The **mapping agent** is Prism's exported `mappingAgent`; `createShapeMapper`
|
|
611
613
|
builds the same envelope the runtime replays against — `{ result: rows }` for an
|
|
612
|
-
array shape, `{ result: rows[0] ?? null }` for an object shape — runs the agent
|
|
613
|
-
|
|
614
|
-
that envelope. A
|
|
614
|
+
array shape, `{ result: rows[0] ?? null }` for an object shape — runs the agent,
|
|
615
|
+
with the intent as its `notes`, to get a transform config, compiles it to IR, and applies the IR once over
|
|
616
|
+
that envelope. A config that fails to compile or run on the rows sends the agent round again with the error in
|
|
617
|
+
its notes, up to three attempts (`MAPPING_ATTEMPTS`). A flat shape whose fields the rows carry is mapped by Prism with no model call, its declared
|
|
615
618
|
fields computed from the stored ones; the agent writes the mapping for any other.
|
|
616
619
|
|
|
617
620
|
> The reference agents are a separate concern from the deterministic core: they
|
|
@@ -718,8 +721,13 @@ src/
|
|
|
718
721
|
agent/ Reference Cortex agents (exported via ./agent)
|
|
719
722
|
query.agent.ts defineAgent: vexQueryDslAgent (vex.query → Query)
|
|
720
723
|
tools.ts the six query tools
|
|
724
|
+
compact.ts compactSchema, compactSpec — the schema and DSL spec as compact text
|
|
725
|
+
declared.ts declaredMapping, storedShape — fields a shape declares as prism expressions
|
|
721
726
|
index.ts createQueryDsl, createShapeMapper, vexQueryDslAgent
|
|
722
727
|
|
|
728
|
+
migrations/
|
|
729
|
+
index.ts VEX_SEQUENCE, VEX_SCHEMAS — an entry as a stored document (./migrations)
|
|
730
|
+
|
|
723
731
|
utils/
|
|
724
732
|
context.ts buildValidationContext, resolveParams
|
|
725
733
|
canonical.ts canonical JSON text + hash
|
|
@@ -737,7 +745,7 @@ scripts/ Dev-only: docker, seed, dev server, fixtures
|
|
|
737
745
|
| `zod` | peer (required) | Schema validation + JSON Schema generation |
|
|
738
746
|
| `pg` | peer (optional) | A Postgres pool for the adapter and the cache; `@niscorp/vex/pg` imports it |
|
|
739
747
|
| `hono` | peer (optional) | Hono framework adapter |
|
|
740
|
-
| `express` |
|
|
748
|
+
| `express` | none | `@niscorp/vex/express` imports nothing from Express; it types the request and response it uses |
|
|
741
749
|
| `@niscorp/cortex` | peer (optional) | Reference query agent (`defineAgent`/`defineTool`, standalone runtime) |
|
|
742
750
|
| `@niscorp/signal` | via Cortex (its peer) | LLM calls + embeddings |
|
|
743
751
|
| `@niscorp/prism` | peer (required) | Result mapping (IR compile/execute; `mappingAgent` for the reference mapper) |
|
package/DOCS.md
CHANGED
|
@@ -44,8 +44,9 @@ path with no model and no network beyond the database.
|
|
|
44
44
|
server-minted (`fp_…`, returned in `meta.cache.fingerprint`) or caller-chosen
|
|
45
45
|
(a name like `deals/list`). Replaying a fingerprint skips generation entirely
|
|
46
46
|
and re-runs only the deterministic path — so scope and context stay current
|
|
47
|
-
while LLM cost drops to zero. A
|
|
48
|
-
|
|
47
|
+
while LLM cost drops to zero. A fingerprint sent alone replays its generation;
|
|
48
|
+
sent with a request that differs, minted or named, it is regenerated and its
|
|
49
|
+
content replaced. Design and
|
|
49
50
|
rationale: [DESIGN.md](./DESIGN.md) **Cache v2 — fingerprints**.
|
|
50
51
|
|
|
51
52
|
**Scope is separate from context.** `context` is caller data, visible to the
|
|
@@ -157,7 +158,7 @@ type QueryEngineConfig = {
|
|
|
157
158
|
cache?: CacheBackend; // default: createMemoryCache()
|
|
158
159
|
onEvent?: VexEventHandler; // pipeline event stream
|
|
159
160
|
generateDsl?: (request, schema, caller) => Promise<Query>;
|
|
160
|
-
mapToShape?: (rows, shape) => Promise<{ ir: CompiledIr; config: unknown; transformed: JsonValue }>;
|
|
161
|
+
mapToShape?: (rows, shape, intent?) => Promise<{ ir: CompiledIr; config: unknown; transformed: JsonValue }>;
|
|
161
162
|
embed?: (text, dimensions?) => Promise<number[]>; // text→vector for semantic filters
|
|
162
163
|
behaviors?: ScopeBehaviors; // which columns a scope key is bound to (see below)
|
|
163
164
|
rows?: LiveRowsConfig; // the rows cache behind `refresh: 'reactive'`
|
|
@@ -1161,8 +1162,9 @@ The positive cache is keyed by the request's `fingerprint` (see the request
|
|
|
1161
1162
|
contract above). There are no cache modes — the body's posture says
|
|
1162
1163
|
everything:
|
|
1163
1164
|
|
|
1164
|
-
- **No fingerprint** → generate, cache, mint (`fp_…`).
|
|
1165
|
-
|
|
1165
|
+
- **No fingerprint** → generate, cache, mint (`fp_…`). Sent alone, the mint
|
|
1166
|
+
replays that generation; sent with a differing request, it is replaced like
|
|
1167
|
+
any named fingerprint.
|
|
1166
1168
|
- **Fingerprint alone** → replay-or-`cache_miss`. Never generates. Context
|
|
1167
1169
|
values may vary per call — sorting and parameterized reloads are legal
|
|
1168
1170
|
against one cached DSL.
|
|
@@ -1402,7 +1404,7 @@ free-form requests.
|
|
|
1402
1404
|
|
|
1403
1405
|
```typescript
|
|
1404
1406
|
type GenerateDsl = (request: QueryRequest, schema: DatabaseSchema, caller: GenerationCaller) => Promise<Query>;
|
|
1405
|
-
type MapToShape = (rows: Row[], shape: unknown) =>
|
|
1407
|
+
type MapToShape = (rows: Row[], shape: unknown, intent?: string) =>
|
|
1406
1408
|
Promise<{ ir: CompiledIr; config: unknown; transformed: JsonValue }>;
|
|
1407
1409
|
|
|
1408
1410
|
type GenerationCaller = {
|
package/README.md
CHANGED
|
@@ -81,6 +81,8 @@ engine.test(dsl, scope?); // Promise<TestResult> — compile + run with
|
|
|
81
81
|
engine.getDslSchema(); // object — the DSL as JSON Schema (draft-7)
|
|
82
82
|
engine.getSchema(); // DatabaseSchema | undefined — last introspected schema
|
|
83
83
|
engine.cache; // CacheBackend — the live cache
|
|
84
|
+
engine.invalidate(tables); // these tables were written: reactive reads over them refetch
|
|
85
|
+
engine.rows; // the reactive rows cache: stats(), stop()
|
|
84
86
|
```
|
|
85
87
|
|
|
86
88
|
## What's in the box
|
|
@@ -94,6 +96,7 @@ engine.cache; // CacheBackend — the live cache
|
|
|
94
96
|
| Writes | Mutation entries in the same cache — a closed grammar, replay-only by fingerprint, never generated; `lintMutation` checks a statement at seed |
|
|
95
97
|
| Seeding and freshness | `seedCache` turns authored `SeedEntry` / `SeedMutation` rows into protected cache entries; an entry's `refresh` is `'snapshot'` or `'reactive'` |
|
|
96
98
|
| Safety | Server-side scope policies, query analyzer (cartesian/nesting/index checks), parameterized SQL only |
|
|
99
|
+
| Stored entries | `@niscorp/vex/migrations`: `VEX_SEQUENCE`, the strata sequence an entry is stamped with as a stored document |
|
|
97
100
|
| Its own table | The Postgres cache's table is a [strata](../strata/README.md) sequence (`cache.sequence`, `nisc.vex.cache`): `cache.init()` applies it through the ledger once, instead of `IF NOT EXISTS` on every boot. Requires a pool with `transaction` (the PGlite pool and `createPgPool`'s have one; a bare `pg.Pool` does not). |
|
|
98
101
|
|
|
99
102
|
## License
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@niscorp/vex",
|
|
3
|
-
"version": "0.4.
|
|
3
|
+
"version": "0.4.4",
|
|
4
4
|
"description": "Declarative query synthesis — intent to SQL with caching and scope policies",
|
|
5
5
|
"license": "Apache-2.0",
|
|
6
6
|
"repository": {
|
|
@@ -98,9 +98,9 @@
|
|
|
98
98
|
"hono": "^4",
|
|
99
99
|
"pg": "^8",
|
|
100
100
|
"zod": "^4.2.0",
|
|
101
|
-
"@niscorp/cortex": "^0.1.
|
|
102
|
-
"@niscorp/
|
|
103
|
-
"@niscorp/
|
|
101
|
+
"@niscorp/cortex": "^0.1.6",
|
|
102
|
+
"@niscorp/strata": "^0.1.9",
|
|
103
|
+
"@niscorp/prism": "^0.3.3"
|
|
104
104
|
},
|
|
105
105
|
"devDependencies": {
|
|
106
106
|
"@electric-sql/pglite": "^0.5.1",
|
|
@@ -114,10 +114,10 @@
|
|
|
114
114
|
"tsx": "^4.20.3",
|
|
115
115
|
"vitest": "^4.1.2",
|
|
116
116
|
"zod": "^4.6.5",
|
|
117
|
-
"@niscorp/cortex": "^0.1.
|
|
118
|
-
"@niscorp/
|
|
119
|
-
"@niscorp/
|
|
120
|
-
"@niscorp/strata": "^0.1.
|
|
117
|
+
"@niscorp/cortex": "^0.1.6",
|
|
118
|
+
"@niscorp/signal": "^0.1.9",
|
|
119
|
+
"@niscorp/prism": "^0.3.3",
|
|
120
|
+
"@niscorp/strata": "^0.1.9"
|
|
121
121
|
},
|
|
122
122
|
"peerDependenciesMeta": {
|
|
123
123
|
"@niscorp/cortex": {
|