@niscorp/vex 0.4.4 → 0.4.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +12 -0
- package/DOCS.md +89 -0
- package/README.md +1 -0
- package/dist/adapters/express/index.d.cts +2 -2
- package/dist/adapters/express/index.d.ts +2 -2
- package/dist/adapters/hono/index.d.cts +2 -2
- package/dist/adapters/hono/index.d.ts +2 -2
- package/dist/agent/index.d.cts +1 -1
- package/dist/agent/index.d.ts +1 -1
- package/dist/{handler-CsFbanZA.d.cts → handler-CB0BqKsY.d.cts} +1 -1
- package/dist/{handler-CoJxInw-.d.ts → handler-O6XslV-S.d.ts} +1 -1
- package/dist/index.cjs +174 -27
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +3 -3
- package/dist/index.d.ts +3 -3
- package/dist/index.js +173 -28
- package/dist/index.js.map +1 -1
- package/dist/{types-B2HEmyNq.d.cts → types-BmHuPfPS.d.cts} +33 -2
- package/dist/{types-DC68ZUuq.d.ts → types-D5CCaqcd.d.ts} +33 -2
- package/package.json +4 -4
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,17 @@
|
|
|
1
1
|
# @niscorp/vex
|
|
2
2
|
|
|
3
|
+
## 0.4.5
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- 637d412: **A vex read returns stamped JSON columns upgraded.** A JSON column whose values carry the grammar stamp they were written at in another JSON column of the row can now be named once on the engine, with a strata upgrader beside it: `createQueryEngine({ upgrader: await createUpgrader(grammars, { transform: evaluate }), upgradedColumns: { answers: { plan: { stamp: 'grammar', kind: 'acme.answers/plan' } } } })`. A kind can also be read from the row: `kind: { grammar: 'acme.notes', column: 'kind' }`. Every read that returns the column (a replay, a generated read, a reactive read's every answer, `engine.test`) hands its mapping and its caller the value upgraded, with the stamp set to the stamp the upgrade gives back. A read returns the column only as one of its own fields, with its stamp (and kind) column beside it; vex refuses any other read of it when the read is compiled. A row that does not upgrade refuses the read, or, where the column says `unreadable: 'omit'` or `'null'`, is left out or has that value set to null, with a line in `meta.warnings`. New exports: `UpgradedColumnsSchema`, `UNREADABLE_MODES`, and the types `UpgradedColumn`, `UpgradedColumns`, `ColumnUpgrader`.
|
|
8
|
+
|
|
9
|
+
**Moss takes them on the manifest.** `defineApp({ upgradedColumns })` hands them to the app's engine and to the one that answers intents, with the upgrader moss builds from nova's, Prism's and vex's grammars and the app's `grammars`. Only the app's own tables can be named. The boot is refused where a table named is a feature's, the artifactory's or the generation pointer's (their owners read them as stored), where a kind is in none of those grammars, where a column or its stamp is not a json column of the database, or where a seeded read does not compile: an app that names `upgradedColumns` has every seeded read compiled at boot and on a refresh, so a read of an upgraded column that vex refuses stops the boot rather than its first replay.
|
|
10
|
+
|
|
11
|
+
**The rulebook** says so under "Using Vex" and in rule 19.
|
|
12
|
+
|
|
13
|
+
**What to change:** nothing. An app or engine that names no `upgradedColumns` reads every column as stored, as before. No grammar, table, entry or cache format changes.
|
|
14
|
+
|
|
3
15
|
## 0.4.4
|
|
4
16
|
|
|
5
17
|
### Patch Changes
|
package/DOCS.md
CHANGED
|
@@ -15,6 +15,7 @@ In-depth usage guide for `@niscorp/vex`. For the high-level pitch see
|
|
|
15
15
|
- [The DSL](#the-dsl)
|
|
16
16
|
- [Scope policies](#scope-policies)
|
|
17
17
|
- [Grants and behaviors](#grants-and-behaviors)
|
|
18
|
+
- [Upgraded columns](#upgraded-columns)
|
|
18
19
|
- [Mutations](#mutations)
|
|
19
20
|
- [Caching](#caching)
|
|
20
21
|
- [Wiring the LLM agents](#wiring-the-llm-agents)
|
|
@@ -161,6 +162,8 @@ type QueryEngineConfig = {
|
|
|
161
162
|
mapToShape?: (rows, shape, intent?) => Promise<{ ir: CompiledIr; config: unknown; transformed: JsonValue }>;
|
|
162
163
|
embed?: (text, dimensions?) => Promise<number[]>; // text→vector for semantic filters
|
|
163
164
|
behaviors?: ScopeBehaviors; // which columns a scope key is bound to (see below)
|
|
165
|
+
upgradedColumns?: UpgradedColumns; // JSON columns a read returns upgraded (see below)
|
|
166
|
+
upgrader?: ColumnUpgrader; // strata's Upgrader; required with upgradedColumns
|
|
164
167
|
rows?: LiveRowsConfig; // the rows cache behind `refresh: 'reactive'`
|
|
165
168
|
config?: {
|
|
166
169
|
maxNestingDepth?: number; // default: 2
|
|
@@ -201,6 +204,7 @@ type QueryEngineConfig = {
|
|
|
201
204
|
`scopeBindings(behaviors, tables, scope)` returns that list for the given
|
|
202
205
|
tables: one entry per table, column and key, sorted, and only for keys
|
|
203
206
|
`scope` holds a value for.
|
|
207
|
+
- **`upgradedColumns`** and **`upgrader`**: see [Upgraded columns](#upgraded-columns).
|
|
204
208
|
- **`rows`** sizes the reactive rows cache — see
|
|
205
209
|
[Reactive reads](#reactive-reads-refresh).
|
|
206
210
|
- `defaultLimit` is applied when a DSL omits `limit`. `maxLimit` clamps a larger
|
|
@@ -863,6 +867,91 @@ entry declaring a reach no host can resolve is a 500, and a host returning
|
|
|
863
867
|
`undefined` is a 403. Falling back to the caller's own reach would be silent,
|
|
864
868
|
and silence is the whole risk — the read would answer, with too much.
|
|
865
869
|
|
|
870
|
+
## Upgraded columns
|
|
871
|
+
|
|
872
|
+
A JSON column may hold values written at an older grammar, with the stamp each
|
|
873
|
+
was written at in another JSON column of the same row: a strata document (rule
|
|
874
|
+
19 of the nisc rulebook). `upgradedColumns` names those columns, keyed by
|
|
875
|
+
table and column, and every read that returns one hands it to the mapping, and
|
|
876
|
+
to the caller, upgraded to the grammars of the engine's `upgrader`:
|
|
877
|
+
|
|
878
|
+
```typescript
|
|
879
|
+
import { createUpgrader } from '@niscorp/strata';
|
|
880
|
+
import { evaluate } from '@niscorp/prism';
|
|
881
|
+
|
|
882
|
+
const engine = createQueryEngine({
|
|
883
|
+
adapter,
|
|
884
|
+
upgrader: await createUpgrader(grammars, { transform: evaluate }),
|
|
885
|
+
upgradedColumns: {
|
|
886
|
+
// The kind is fixed for the column.
|
|
887
|
+
answers: { plan: { stamp: 'grammar', kind: 'acme.answers/plan', unreadable: 'omit' } },
|
|
888
|
+
// The kind is `<grammar>/<the row's value of column>`: `acme.notes/list` for a row whose `kind` is `list`.
|
|
889
|
+
notes: { body: { stamp: 'grammar', kind: { grammar: 'acme.notes', column: 'kind' } } },
|
|
890
|
+
},
|
|
891
|
+
});
|
|
892
|
+
```
|
|
893
|
+
|
|
894
|
+
`stamp` and `kind.column` name columns of the same table. `UpgradedColumnsSchema`
|
|
895
|
+
is the setting's schema, and `ColumnUpgrader` the part of strata's `Upgrader`
|
|
896
|
+
the engine uses (`upgrade`, `locate`, `stamp`).
|
|
897
|
+
|
|
898
|
+
**Checked at start.** `createQueryEngine` throws `invalid_request` when
|
|
899
|
+
`upgradedColumns` do not parse, when there is no `upgrader`, or when a fixed
|
|
900
|
+
`kind` (or the `grammar` of a kind read from the row) is not one the
|
|
901
|
+
upgrader's grammars declare. `introspect()` throws `execution_error` when a
|
|
902
|
+
column, its stamp or its kind column is not in the database, or when the
|
|
903
|
+
column or its stamp is not `json`/`jsonb`.
|
|
904
|
+
|
|
905
|
+
**What a read must say.** Every query is checked as it is compiled: a replay,
|
|
906
|
+
`engine.compile`, `engine.test` and the generator's own test runs. A seeded
|
|
907
|
+
entry is therefore checked when it is first replayed, unless its app compiles
|
|
908
|
+
it before (moss does, at boot).
|
|
909
|
+
|
|
910
|
+
- A read that returns an upgraded column returns its stamp column too, and for
|
|
911
|
+
a kind read from the row, that column. Otherwise: `invalid_dsl`, naming the
|
|
912
|
+
field to add.
|
|
913
|
+
- An upgraded column is returned only as one of the read's own fields
|
|
914
|
+
(`table.column` or `{ field, as }`). Named anywhere else (a subquery's fields
|
|
915
|
+
or computes, a `compute`, an `aggregate`, a `groupBy`), the read is refused,
|
|
916
|
+
because the value would arrive un-upgraded. A filter or a sort on it, a
|
|
917
|
+
subquery's included, is allowed and compares the stored value.
|
|
918
|
+
- Two values returned under one key (two fields, or a field and a compute or
|
|
919
|
+
aggregate of that name) are refused when the key is an upgraded column, its
|
|
920
|
+
stamp or its kind: give each its own key with `as`.
|
|
921
|
+
|
|
922
|
+
**What a read does.** After the rows come back and before the mapping, each
|
|
923
|
+
row's value is upgraded from the stamp in that row, and every key the stamp was
|
|
924
|
+
returned under is set to the stamp the upgrade gives back, so a caller that
|
|
925
|
+
upgrades the value again changes nothing. A null value is left as it is. This
|
|
926
|
+
holds for a replay, a generated read, every answer a reactive read delivers,
|
|
927
|
+
`engine.test` and the generator's test runs. The rows a reactive read shares
|
|
928
|
+
between callers are not changed. A write's returned rows are as stored.
|
|
929
|
+
|
|
930
|
+
**A row that does not upgrade.** A value does not upgrade when its stamp is
|
|
931
|
+
ahead of the upgrader's grammars (strata's `TOO_NEW`), a migration step fails
|
|
932
|
+
on it (`STEP_FAILED`), its kind is one no grammar declares (`UNKNOWN_KIND`),
|
|
933
|
+
its stamp is neither null nor an object of grammar names to whole numbers, or
|
|
934
|
+
its kind column is not text. The column's `unreadable` says what the read does
|
|
935
|
+
then (`UNREADABLE_MODES`):
|
|
936
|
+
|
|
937
|
+
- `'refuse'` (the default): the whole read fails. Strata's errors are answered
|
|
938
|
+
by `handleQuery` as a 500 with strata's code; a stamp or kind column that is
|
|
939
|
+
not one is vex's `execution_error`, a 400. A reactive read's refetch that
|
|
940
|
+
fails reports `rows.error`, and its followers keep their last answer.
|
|
941
|
+
- `'omit'`: the row is left out, and `meta.warnings` says which of the
|
|
942
|
+
database's rows (counting from 0) and why. A list can come back shorter than
|
|
943
|
+
its `limit`; the "may have been cut" warning counts the rows the database
|
|
944
|
+
returned. A read whose shape is an object answers the first row left, so a
|
|
945
|
+
row left out makes `$.result` the next one: use `'null'` or `'refuse'` where
|
|
946
|
+
that would mislead.
|
|
947
|
+
- `'null'`: the row stays with that value null, and `meta.warnings` says so.
|
|
948
|
+
This column does not change the stamp; another upgraded column of the row
|
|
949
|
+
that shares it may.
|
|
950
|
+
|
|
951
|
+
A reactive read delivers again when only its warnings change.
|
|
952
|
+
|
|
953
|
+
An engine given no `upgradedColumns` returns every column as stored.
|
|
954
|
+
|
|
866
955
|
## Mutations
|
|
867
956
|
|
|
868
957
|
Writes are first-class and deliberately narrower than reads: **no generation
|
package/README.md
CHANGED
|
@@ -96,6 +96,7 @@ engine.rows; // the reactive rows cache: stats(), stop()
|
|
|
96
96
|
| Writes | Mutation entries in the same cache — a closed grammar, replay-only by fingerprint, never generated; `lintMutation` checks a statement at seed |
|
|
97
97
|
| Seeding and freshness | `seedCache` turns authored `SeedEntry` / `SeedMutation` rows into protected cache entries; an entry's `refresh` is `'snapshot'` or `'reactive'` |
|
|
98
98
|
| Safety | Server-side scope policies, query analyzer (cartesian/nesting/index checks), parameterized SQL only |
|
|
99
|
+
| Upgraded columns | `upgradedColumns` + `upgrader` on the engine: a JSON column stamped in another column is returned upgraded by every read, before the mapping |
|
|
99
100
|
| Stored entries | `@niscorp/vex/migrations`: `VEX_SEQUENCE`, the strata sequence an entry is stamped with as a stored document |
|
|
100
101
|
| Its own table | The Postgres cache's table is a [strata](../strata/README.md) sequence (`cache.sequence`, `nisc.vex.cache`): `cache.init()` applies it through the ledger once, instead of `IF NOT EXISTS` on every boot. Requires a pool with `transaction` (the PGlite pool and `createPgPool`'s have one; a bare `pg.Pool` does not). |
|
|
101
102
|
|
|
@@ -1,5 +1,5 @@
|
|
|
1
|
-
import { a as QueryEngine,
|
|
2
|
-
import { V as VexHandlerConfig } from '../../handler-
|
|
1
|
+
import { a as QueryEngine, ab as ScopeValues } from '../../types-BmHuPfPS.cjs';
|
|
2
|
+
import { V as VexHandlerConfig } from '../../handler-CB0BqKsY.cjs';
|
|
3
3
|
import '../../query.schema-B-LVNfyX.cjs';
|
|
4
4
|
import 'zod';
|
|
5
5
|
import '@niscorp/prism';
|
|
@@ -1,5 +1,5 @@
|
|
|
1
|
-
import { a as QueryEngine,
|
|
2
|
-
import { V as VexHandlerConfig } from '../../handler-
|
|
1
|
+
import { a as QueryEngine, ab as ScopeValues } from '../../types-D5CCaqcd.js';
|
|
2
|
+
import { V as VexHandlerConfig } from '../../handler-O6XslV-S.js';
|
|
3
3
|
import '../../query.schema-B-LVNfyX.js';
|
|
4
4
|
import 'zod';
|
|
5
5
|
import '@niscorp/prism';
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
import { Env, Context, Hono } from 'hono';
|
|
2
|
-
import { a as QueryEngine,
|
|
3
|
-
import { M as MutationClient, W as WriteEvent, E as ExecuteRecord } from '../../handler-
|
|
2
|
+
import { a as QueryEngine, ab as ScopeValues, S as ScopePolicy } from '../../types-BmHuPfPS.cjs';
|
|
3
|
+
import { M as MutationClient, W as WriteEvent, E as ExecuteRecord } from '../../handler-CB0BqKsY.cjs';
|
|
4
4
|
import '../../query.schema-B-LVNfyX.cjs';
|
|
5
5
|
import 'zod';
|
|
6
6
|
import '@niscorp/prism';
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
import { Env, Context, Hono } from 'hono';
|
|
2
|
-
import { a as QueryEngine,
|
|
3
|
-
import { M as MutationClient, W as WriteEvent, E as ExecuteRecord } from '../../handler-
|
|
2
|
+
import { a as QueryEngine, ab as ScopeValues, S as ScopePolicy } from '../../types-D5CCaqcd.js';
|
|
3
|
+
import { M as MutationClient, W as WriteEvent, E as ExecuteRecord } from '../../handler-O6XslV-S.js';
|
|
4
4
|
import '../../query.schema-B-LVNfyX.js';
|
|
5
5
|
import 'zod';
|
|
6
6
|
import '@niscorp/prism';
|
package/dist/agent/index.d.cts
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import {
|
|
1
|
+
import { a5 as ScopeBinding, D as DatabaseSchema, p as GenerationCaller, G as GenerateDsl, u as MapToShape } from '../types-BmHuPfPS.cjs';
|
|
2
2
|
import { AgentDefinition, defineTool, SignalClient } from '@niscorp/cortex';
|
|
3
3
|
import { Q as Query } from '../query.schema-B-LVNfyX.cjs';
|
|
4
4
|
import '@niscorp/prism';
|
package/dist/agent/index.d.ts
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import {
|
|
1
|
+
import { a5 as ScopeBinding, D as DatabaseSchema, p as GenerationCaller, G as GenerateDsl, u as MapToShape } from '../types-D5CCaqcd.js';
|
|
2
2
|
import { AgentDefinition, defineTool, SignalClient } from '@niscorp/cortex';
|
|
3
3
|
import { Q as Query } from '../query.schema-B-LVNfyX.js';
|
|
4
4
|
import '@niscorp/prism';
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import {
|
|
1
|
+
import { ab as ScopeValues, S as ScopePolicy, D as DatabaseSchema, e as Row, M as MutationDefinition, v as Mutation, n as CoreMutation, a as QueryEngine, U as QueryResponse } from './types-BmHuPfPS.cjs';
|
|
2
2
|
import { Q as Query } from './query.schema-B-LVNfyX.cjs';
|
|
3
3
|
|
|
4
4
|
type MutationTx = {
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import {
|
|
1
|
+
import { ab as ScopeValues, S as ScopePolicy, D as DatabaseSchema, e as Row, M as MutationDefinition, v as Mutation, n as CoreMutation, a as QueryEngine, U as QueryResponse } from './types-D5CCaqcd.js';
|
|
2
2
|
import { Q as Query } from './query.schema-B-LVNfyX.js';
|
|
3
3
|
|
|
4
4
|
type MutationTx = {
|
package/dist/index.cjs
CHANGED
|
@@ -1742,6 +1742,128 @@ var createLiveRows = (config = {}, emit = () => {
|
|
|
1742
1742
|
};
|
|
1743
1743
|
return { read, follow, invalidate, stats, stop };
|
|
1744
1744
|
};
|
|
1745
|
+
var UNREADABLE_MODES = ["refuse", "omit", "null"];
|
|
1746
|
+
var UpgradedColumnSchema = zod.z.object({
|
|
1747
|
+
stamp: zod.z.string(),
|
|
1748
|
+
kind: zod.z.union([zod.z.string(), zod.z.object({ grammar: zod.z.string(), column: zod.z.string() }).strict()]),
|
|
1749
|
+
unreadable: zod.z.enum(UNREADABLE_MODES).optional()
|
|
1750
|
+
}).strict();
|
|
1751
|
+
var UpgradedColumnsSchema = zod.z.record(zod.z.string(), zod.z.record(zod.z.string(), UpgradedColumnSchema));
|
|
1752
|
+
var outputOf = (ref) => typeof ref === "string" ? { path: ref, key: ref.slice(ref.indexOf(".") + 1) } : { path: ref.field, key: ref.as };
|
|
1753
|
+
var outputsOf = (dsl, path) => (dsl.fields ?? []).map(outputOf).flatMap((out) => out.path === path ? [out.key] : []);
|
|
1754
|
+
var count = (value, path) => {
|
|
1755
|
+
if (value === path) return 1;
|
|
1756
|
+
if (Array.isArray(value)) return value.reduce((sum, item) => sum + count(item, path), 0);
|
|
1757
|
+
if (typeof value === "object" && value !== null) return Object.values(value).reduce((sum, item) => sum + count(item, path), 0);
|
|
1758
|
+
return 0;
|
|
1759
|
+
};
|
|
1760
|
+
var usesOutsideFields = (dsl, path, outer) => Object.entries(dsl).reduce((sum, [part, value]) => {
|
|
1761
|
+
if (part === "filter" || part === "sort" || outer && part === "fields") return sum;
|
|
1762
|
+
if (part !== "from") return sum + count(value, path);
|
|
1763
|
+
return sum + dsl.from.reduce((inner, source) => inner + (typeof source === "string" ? 0 : usesOutsideFields(source.query, path, false)), 0);
|
|
1764
|
+
}, 0);
|
|
1765
|
+
var selected = (dsl, path, upgraded) => {
|
|
1766
|
+
const [first, ...rest] = outputsOf(dsl, path);
|
|
1767
|
+
if (first === void 0) throw new VexError("invalid_dsl", `This read returns ${upgraded}, which is upgraded where it is read, without ${path}: add ${path} to its fields.`);
|
|
1768
|
+
return [first, ...rest];
|
|
1769
|
+
};
|
|
1770
|
+
var refuseSharedKeys = (dsl, upgrades) => {
|
|
1771
|
+
const sourcesByKey = /* @__PURE__ */ new Map();
|
|
1772
|
+
const note = (key, source) => {
|
|
1773
|
+
sourcesByKey.set(key, (sourcesByKey.get(key) ?? /* @__PURE__ */ new Set()).add(source));
|
|
1774
|
+
};
|
|
1775
|
+
for (const { path, key } of (dsl.fields ?? []).map(outputOf)) note(key, path);
|
|
1776
|
+
for (const key of [...Object.keys(dsl.compute ?? {}), ...Object.keys(dsl.aggregate ?? {})]) note(key, `the computed ${key}`);
|
|
1777
|
+
const used = upgrades.flatMap((upgrade) => [upgrade.key, ...upgrade.stampKeys, ...typeof upgrade.kind === "string" ? [] : [upgrade.kind.key]]);
|
|
1778
|
+
const shared = [...new Set(used)].filter((key) => (sourcesByKey.get(key)?.size ?? 0) > 1);
|
|
1779
|
+
if (shared.length > 0) {
|
|
1780
|
+
throw new VexError("invalid_dsl", `This read returns two values under one key (${shared.join(", ")}), and one is an upgraded column, its stamp or its kind: give each its own key with \`as\`.`);
|
|
1781
|
+
}
|
|
1782
|
+
};
|
|
1783
|
+
var declared = (columns) => Object.entries(columns).flatMap(([table, byColumn]) => Object.entries(byColumn).map(([column, upgraded]) => ({ table, column, upgraded })));
|
|
1784
|
+
var columnUpgrades = (dsl, columns) => {
|
|
1785
|
+
const upgrades = declared(columns).flatMap(({ table, column, upgraded }) => {
|
|
1786
|
+
const path = `${table}.${column}`;
|
|
1787
|
+
if (usesOutsideFields(dsl, path, true) > 0) {
|
|
1788
|
+
throw new VexError(
|
|
1789
|
+
"invalid_dsl",
|
|
1790
|
+
`${path} is upgraded where it is read, so a read returns it only as one of its own fields, never through a subquery, a compute, an aggregate or a groupBy. A filter or a sort on it compares the stored value.`
|
|
1791
|
+
);
|
|
1792
|
+
}
|
|
1793
|
+
const keys = outputsOf(dsl, path);
|
|
1794
|
+
if (keys.length === 0) return [];
|
|
1795
|
+
const stampKeys = selected(dsl, `${table}.${upgraded.stamp}`, path);
|
|
1796
|
+
const kind = typeof upgraded.kind === "string" ? upgraded.kind : { grammar: upgraded.kind.grammar, key: selected(dsl, `${table}.${upgraded.kind.column}`, path)[0] };
|
|
1797
|
+
return keys.map((key) => ({ path, key, stampKeys, kind, unreadable: upgraded.unreadable ?? "refuse" }));
|
|
1798
|
+
});
|
|
1799
|
+
refuseSharedKeys(dsl, upgrades);
|
|
1800
|
+
return upgrades;
|
|
1801
|
+
};
|
|
1802
|
+
var unknownKinds = (columns, upgrader) => declared(columns).flatMap(({ table, column, upgraded: { kind } }) => {
|
|
1803
|
+
if (typeof kind !== "string") return kind.grammar in upgrader.stamp ? [] : [`${table}.${column} names the grammar "${kind.grammar}"`];
|
|
1804
|
+
try {
|
|
1805
|
+
upgrader.locate({}, kind);
|
|
1806
|
+
return [];
|
|
1807
|
+
} catch {
|
|
1808
|
+
return [`${table}.${column} names the kind "${kind}"`];
|
|
1809
|
+
}
|
|
1810
|
+
});
|
|
1811
|
+
var mistypedColumns = (columns, schema) => declared(columns).flatMap(({ table, column, upgraded }) => {
|
|
1812
|
+
const fields = schema.entities.find((e) => e.name === table)?.fields ?? [];
|
|
1813
|
+
const json = (name) => {
|
|
1814
|
+
const field = fields.find((f) => f.name === name);
|
|
1815
|
+
if (field === void 0) return [`${table}.${name} is not in the database`];
|
|
1816
|
+
return ["json", "jsonb"].includes(field.type.toLowerCase()) ? [] : [`${table}.${name} is ${field.type}, not json`];
|
|
1817
|
+
};
|
|
1818
|
+
const { kind } = upgraded;
|
|
1819
|
+
const kindColumn = typeof kind === "string" || fields.some((f) => f.name === kind.column) ? [] : [`${table}.${kind.column} is not in the database`];
|
|
1820
|
+
return [...json(column), ...json(upgraded.stamp), ...kindColumn];
|
|
1821
|
+
});
|
|
1822
|
+
var StampSchema = zod.z.union([zod.z.null(), zod.z.undefined(), zod.z.record(zod.z.string(), zod.z.number().int().nonnegative())]);
|
|
1823
|
+
var stampOf = (value, path) => {
|
|
1824
|
+
const parsed = StampSchema.safeParse(value);
|
|
1825
|
+
if (!parsed.success) throw new VexError("execution_error", `The stamp beside ${path} is not a stamp: it must be null or an object of grammar names to whole numbers.`);
|
|
1826
|
+
return parsed.data ?? {};
|
|
1827
|
+
};
|
|
1828
|
+
var kindOf = (upgrade, row) => {
|
|
1829
|
+
if (typeof upgrade.kind === "string") return upgrade.kind;
|
|
1830
|
+
const name = row[upgrade.kind.key];
|
|
1831
|
+
if (typeof name !== "string") throw new VexError("execution_error", `The row's ${upgrade.kind.key} is not text, so it names no kind in ${upgrade.kind.grammar}.`);
|
|
1832
|
+
return `${upgrade.kind.grammar}/${name}`;
|
|
1833
|
+
};
|
|
1834
|
+
var reasonOf = (error) => {
|
|
1835
|
+
const message = error instanceof Error ? error.message.split("\n")[0] ?? "" : String(error);
|
|
1836
|
+
const code = error instanceof Error && "code" in error && typeof error.code === "string" ? `${error.code}: ` : "";
|
|
1837
|
+
return `${code}${message}`;
|
|
1838
|
+
};
|
|
1839
|
+
var upgradeRows = (rows, upgrades, upgrade) => {
|
|
1840
|
+
if (upgrades.length === 0) return { rows, warnings: [] };
|
|
1841
|
+
const warnings = [];
|
|
1842
|
+
const upgraded = rows.flatMap((row, index) => {
|
|
1843
|
+
const next = { ...row };
|
|
1844
|
+
const said = [];
|
|
1845
|
+
for (const column of upgrades) {
|
|
1846
|
+
const value = row[column.key];
|
|
1847
|
+
if (value === null || value === void 0) continue;
|
|
1848
|
+
try {
|
|
1849
|
+
const result = upgrade(value, { kind: kindOf(column, row), stamp: stampOf(row[column.stampKeys[0]], column.path) });
|
|
1850
|
+
next[column.key] = result.document;
|
|
1851
|
+
for (const stampKey of column.stampKeys) next[stampKey] = result.stamp;
|
|
1852
|
+
} catch (error) {
|
|
1853
|
+
if (column.unreadable === "refuse") throw error;
|
|
1854
|
+
if (column.unreadable === "omit") {
|
|
1855
|
+
warnings.push(`The database's row ${index} (counting from 0) was left out: its ${column.path} does not upgrade (${reasonOf(error)}).`);
|
|
1856
|
+
return [];
|
|
1857
|
+
}
|
|
1858
|
+
next[column.key] = null;
|
|
1859
|
+
said.push(`The database's row ${index} (counting from 0) has ${column.path} set to null: it does not upgrade (${reasonOf(error)}).`);
|
|
1860
|
+
}
|
|
1861
|
+
}
|
|
1862
|
+
warnings.push(...said);
|
|
1863
|
+
return [next];
|
|
1864
|
+
});
|
|
1865
|
+
return { rows: upgraded, warnings };
|
|
1866
|
+
};
|
|
1745
1867
|
var applySortContext = (dsl, context) => {
|
|
1746
1868
|
const sortBy = context["sortBy"];
|
|
1747
1869
|
if (typeof sortBy !== "string" || sortBy === "") return dsl;
|
|
@@ -1756,7 +1878,18 @@ var DEFAULT_STATEMENT_TIMEOUT_MS = 1e4;
|
|
|
1756
1878
|
var DEFAULT_UNSATISFIABLE_TTL_MS = 3e5;
|
|
1757
1879
|
var DEFAULT_MAX_PRESENCE_VARIANTS = 32;
|
|
1758
1880
|
var createQueryEngine = (engineConfig) => {
|
|
1759
|
-
const { adapter, scope: scopePolicy, generateDsl, mapToShape, embed, behaviors } = engineConfig;
|
|
1881
|
+
const { adapter, scope: scopePolicy, generateDsl, mapToShape, embed, behaviors, upgrader } = engineConfig;
|
|
1882
|
+
const parsedColumns = UpgradedColumnsSchema.safeParse(engineConfig.upgradedColumns ?? {});
|
|
1883
|
+
if (!parsedColumns.success) throw new VexError("invalid_request", `The engine's upgradedColumns do not parse: ${parsedColumns.error.message}`);
|
|
1884
|
+
const upgradedColumns = parsedColumns.data;
|
|
1885
|
+
if (engineConfig.upgradedColumns !== void 0 && upgrader === void 0) {
|
|
1886
|
+
throw new VexError("invalid_request", "The engine was given `upgradedColumns` and no `upgrader`: hand it strata's `createUpgrader(grammars, { transform: evaluate })`.");
|
|
1887
|
+
}
|
|
1888
|
+
const unknown = upgrader === void 0 ? [] : unknownKinds(upgradedColumns, upgrader);
|
|
1889
|
+
if (unknown.length > 0) {
|
|
1890
|
+
throw new VexError("invalid_request", `The upgradedColumns name what the upgrader's grammars do not declare: ${unknown.join("; ")}. Fix the name, or give the upgrader that grammar.`);
|
|
1891
|
+
}
|
|
1892
|
+
const upgradeOf = (stored, upgrades) => upgrader === void 0 ? { rows: stored, warnings: [] } : upgradeRows(stored, upgrades, upgrader.upgrade);
|
|
1760
1893
|
const emit = engineConfig.onEvent ?? (() => {
|
|
1761
1894
|
});
|
|
1762
1895
|
const maxNestingDepth = engineConfig.config?.maxNestingDepth ?? DEFAULT_MAX_NESTING_DEPTH;
|
|
@@ -1808,6 +1941,10 @@ var createQueryEngine = (engineConfig) => {
|
|
|
1808
1941
|
throw new VexError("execution_error", `The behaviors name columns the database does not have: ${missing.map((m) => `${m.entity}.${m.field}`).join(", ")}.`);
|
|
1809
1942
|
}
|
|
1810
1943
|
}
|
|
1944
|
+
const mistyped = mistypedColumns(upgradedColumns, schema);
|
|
1945
|
+
if (mistyped.length > 0) {
|
|
1946
|
+
throw new VexError("execution_error", `The upgradedColumns do not fit the database: ${mistyped.join("; ")}. Fix the name, or add the column in a migration.`);
|
|
1947
|
+
}
|
|
1811
1948
|
cachedSchema = schema;
|
|
1812
1949
|
cachedFingerprint = computeSchemaFingerprint(schema);
|
|
1813
1950
|
return schema;
|
|
@@ -1846,7 +1983,7 @@ var createQueryEngine = (engineConfig) => {
|
|
|
1846
1983
|
);
|
|
1847
1984
|
}
|
|
1848
1985
|
const compiled = adapter.compile(resolved);
|
|
1849
|
-
return { compiled, warnings: analysis.warnings, ...engineLimit !== void 0 ? { engineLimit } : {} };
|
|
1986
|
+
return { compiled, warnings: analysis.warnings, upgrades: columnUpgrades(processedDsl, upgradedColumns), ...engineLimit !== void 0 ? { engineLimit } : {} };
|
|
1850
1987
|
};
|
|
1851
1988
|
const cutSaid = /* @__PURE__ */ new Set();
|
|
1852
1989
|
const cutNote = (engineLimit, stated, rowCount, shape, fingerprint) => {
|
|
@@ -1863,12 +2000,12 @@ var createQueryEngine = (engineConfig) => {
|
|
|
1863
2000
|
const test = async (dsl, scopeValues) => {
|
|
1864
2001
|
try {
|
|
1865
2002
|
const testDsl = { ...pruneOptional(dsl, "all"), limit: 5 };
|
|
1866
|
-
const { compiled, warnings } = runPipeline(testDsl);
|
|
2003
|
+
const { compiled, warnings, upgrades } = runPipeline(testDsl);
|
|
1867
2004
|
const syntheticContext = buildValidationContext(compiled.contextContract);
|
|
1868
2005
|
const scope = scopeValues ?? {};
|
|
1869
2006
|
requireScope(compiled, scope);
|
|
1870
|
-
const rows = await executeQuery(compiled, syntheticContext, scope, adapter, embed);
|
|
1871
|
-
return { rows, warnings, errors: [] };
|
|
2007
|
+
const { rows, warnings: upgradeWarnings } = upgradeOf(await executeQuery(compiled, syntheticContext, scope, adapter, embed), upgrades);
|
|
2008
|
+
return { rows, warnings: [...warnings, ...upgradeWarnings], errors: [] };
|
|
1872
2009
|
} catch (err) {
|
|
1873
2010
|
const message = err instanceof Error ? err.message : String(err);
|
|
1874
2011
|
return { rows: [], warnings: [], errors: [message] };
|
|
@@ -1926,13 +2063,15 @@ var createQueryEngine = (engineConfig) => {
|
|
|
1926
2063
|
const callerOf = (policy, scope, tables) => ({
|
|
1927
2064
|
bindings: scopeBindings(behaviors ?? {}, tables, scope),
|
|
1928
2065
|
read: async (dsl) => {
|
|
1929
|
-
const
|
|
2066
|
+
const pruned = pruneOptional(dsl, "all");
|
|
2067
|
+
const { compiled, warnings, upgrades } = runPipeline(pruned, policy);
|
|
1930
2068
|
requireScope(compiled, scope);
|
|
1931
2069
|
if (compiled.paramSlots.some((slot) => slot.kind === "semantic")) {
|
|
1932
2070
|
return { rows: [], sql: compiled.sql, warnings: [...warnings, "Semantic params present \u2014 execution skipped, SQL structure validated only"] };
|
|
1933
2071
|
}
|
|
1934
2072
|
const params = compiled.paramSlots.map((slot) => slot.kind === "scope" ? scope[slot.key] : null);
|
|
1935
|
-
|
|
2073
|
+
const { rows, warnings: upgradeWarnings } = upgradeOf(await adapter.execute(compiled, params), upgrades);
|
|
2074
|
+
return { rows, sql: compiled.sql, warnings: [...warnings, ...upgradeWarnings] };
|
|
1936
2075
|
}
|
|
1937
2076
|
});
|
|
1938
2077
|
const generateMissingDsl = async (validRequest, flightKey, negKey, entities, policy, scope) => {
|
|
@@ -1984,9 +2123,10 @@ var createQueryEngine = (engineConfig) => {
|
|
|
1984
2123
|
};
|
|
1985
2124
|
return prism.execute(ir, source);
|
|
1986
2125
|
};
|
|
1987
|
-
const executeAndMap = async (compiled, validRequest, cachedIr, context, scope) => {
|
|
2126
|
+
const executeAndMap = async (compiled, validRequest, cachedIr, context, scope, upgrades) => {
|
|
1988
2127
|
const execStart = Date.now();
|
|
1989
|
-
const
|
|
2128
|
+
const stored = await executeQuery(compiled, context, scope, adapter, embed);
|
|
2129
|
+
const { rows, warnings: upgradeWarnings } = upgradeOf(stored, upgrades);
|
|
1990
2130
|
const executionMs = Date.now() - execStart;
|
|
1991
2131
|
emit({ type: "query.rows", count: rows.length, executionMs });
|
|
1992
2132
|
let result = rows;
|
|
@@ -2008,7 +2148,7 @@ var createQueryEngine = (engineConfig) => {
|
|
|
2008
2148
|
}
|
|
2009
2149
|
if (mappingMs !== void 0) emit({ type: "query.mapped", mappingMs });
|
|
2010
2150
|
}
|
|
2011
|
-
return { result,
|
|
2151
|
+
return { result, storedRowCount: stored.length, upgradeWarnings, executionMs, mappingMs, mappedIr, ...mappedConfig !== void 0 ? { mappedConfig } : {} };
|
|
2012
2152
|
};
|
|
2013
2153
|
const execute = async (request, options) => {
|
|
2014
2154
|
if (strata.exceedsDepth(request)) throw new VexError("invalid_request", `Invalid request: ${strata.depthRefusal()}`);
|
|
@@ -2057,7 +2197,7 @@ var createQueryEngine = (engineConfig) => {
|
|
|
2057
2197
|
}
|
|
2058
2198
|
const presence = presenceOf(validRequest.context);
|
|
2059
2199
|
const shaped = pruneOptional(applySortContext(dsl, validRequest.context), presence);
|
|
2060
|
-
const { compiled, warnings, engineLimit } = runPipeline(shaped, activePolicy, cached.seeded === true);
|
|
2200
|
+
const { compiled, warnings, upgrades, engineLimit } = runPipeline(shaped, activePolicy, cached.seeded === true);
|
|
2061
2201
|
noteVariant(cached.fingerprint, presenceSignature(dsl, presence), warnings);
|
|
2062
2202
|
emit({ type: "query.sql", sql: compiled.sql, warnings });
|
|
2063
2203
|
requireScope(compiled, scope);
|
|
@@ -2083,7 +2223,11 @@ var createQueryEngine = (engineConfig) => {
|
|
|
2083
2223
|
const reactiveIr = cached.cachedIr;
|
|
2084
2224
|
if (cached.hit && cached.refresh === "reactive" && (effectiveRequest.shape === void 0 || reactiveIr !== void 0)) {
|
|
2085
2225
|
const shape = effectiveRequest.shape;
|
|
2086
|
-
const shapeRows = (
|
|
2226
|
+
const shapeRows = (stored) => {
|
|
2227
|
+
const { rows: rows2, warnings: upgradeWarnings3 } = upgradeOf(stored, upgrades);
|
|
2228
|
+
const result3 = shape === void 0 || reactiveIr === void 0 ? rows2 : mapRows(rows2, shape, reactiveIr, context, scope);
|
|
2229
|
+
return { result: result3, upgradeWarnings: upgradeWarnings3 };
|
|
2230
|
+
};
|
|
2087
2231
|
const params = await resolveParams(compiled.paramSlots, context, scope, embed);
|
|
2088
2232
|
const key = canonicalHash([compiled.sql, params]);
|
|
2089
2233
|
const source = { tables: [...discoverEntities(shaped)], fetch: () => adapter.execute(compiled, params) };
|
|
@@ -2091,10 +2235,10 @@ var createQueryEngine = (engineConfig) => {
|
|
|
2091
2235
|
const rows = await live.read(key, source);
|
|
2092
2236
|
const executionMs2 = Date.now() - execStart;
|
|
2093
2237
|
emit({ type: "query.rows", count: rows.length, executionMs: executionMs2 });
|
|
2094
|
-
const result2 = shapeRows(rows);
|
|
2095
|
-
const metaOf = (
|
|
2096
|
-
const cut2 = cutNote(engineLimit, shaped.limit,
|
|
2097
|
-
const said2 = cut2 === void 0 ?
|
|
2238
|
+
const { result: result2, upgradeWarnings: upgradeWarnings2 } = shapeRows(rows);
|
|
2239
|
+
const metaOf = (rowCount, upgraded) => {
|
|
2240
|
+
const cut2 = cutNote(engineLimit, shaped.limit, rowCount, shape, validRequest.fingerprint);
|
|
2241
|
+
const said2 = [...warnings, ...upgraded, ...cut2 === void 0 ? [] : [cut2]];
|
|
2098
2242
|
return {
|
|
2099
2243
|
cache: {
|
|
2100
2244
|
hit: cached.hit,
|
|
@@ -2107,29 +2251,30 @@ var createQueryEngine = (engineConfig) => {
|
|
|
2107
2251
|
};
|
|
2108
2252
|
const { signal, onChange } = options ?? {};
|
|
2109
2253
|
if (signal !== void 0 && onChange !== void 0) {
|
|
2110
|
-
let lastHash = canonicalHash(result2);
|
|
2254
|
+
let lastHash = canonicalHash([result2, upgradeWarnings2]);
|
|
2111
2255
|
live.follow(key, source, rows, {
|
|
2112
2256
|
deliver: (next) => {
|
|
2113
|
-
const nextResult = shapeRows(next);
|
|
2114
|
-
const nextHash = canonicalHash(nextResult);
|
|
2257
|
+
const { result: nextResult, upgradeWarnings: nextUpgradeWarnings } = shapeRows(next);
|
|
2258
|
+
const nextHash = canonicalHash([nextResult, nextUpgradeWarnings]);
|
|
2115
2259
|
if (nextHash === lastHash) return;
|
|
2116
2260
|
lastHash = nextHash;
|
|
2117
|
-
onChange({ result: nextResult, meta: metaOf(next.length) });
|
|
2261
|
+
onChange({ result: nextResult, meta: metaOf(next.length, nextUpgradeWarnings) });
|
|
2118
2262
|
}
|
|
2119
2263
|
}, signal);
|
|
2120
2264
|
}
|
|
2121
2265
|
emit({ type: "query.done", totalMs: Date.now() - t0 });
|
|
2122
|
-
return { result: result2, meta: { ...metaOf(rows.length), timing: { executionMs: executionMs2 } } };
|
|
2266
|
+
return { result: result2, meta: { ...metaOf(rows.length, upgradeWarnings2), timing: { executionMs: executionMs2 } } };
|
|
2123
2267
|
}
|
|
2124
|
-
const { result,
|
|
2268
|
+
const { result, storedRowCount, upgradeWarnings, executionMs, mappingMs, mappedIr, mappedConfig } = await executeAndMap(
|
|
2125
2269
|
compiled,
|
|
2126
2270
|
effectiveRequest,
|
|
2127
2271
|
cached.cachedIr,
|
|
2128
2272
|
context,
|
|
2129
|
-
scope
|
|
2273
|
+
scope,
|
|
2274
|
+
upgrades
|
|
2130
2275
|
);
|
|
2131
|
-
const cut = cutNote(engineLimit, shaped.limit,
|
|
2132
|
-
const said = cut === void 0 ?
|
|
2276
|
+
const cut = cutNote(engineLimit, shaped.limit, storedRowCount, effectiveRequest.shape, validRequest.fingerprint);
|
|
2277
|
+
const said = [...warnings, ...upgradeWarnings, ...cut === void 0 ? [] : [cut]];
|
|
2133
2278
|
if (!cached.hit) {
|
|
2134
2279
|
const now = Date.now();
|
|
2135
2280
|
const entry = {
|
|
@@ -2332,8 +2477,8 @@ var introspectPostgres = async (pool, options) => {
|
|
|
2332
2477
|
const rowCountMap = /* @__PURE__ */ new Map();
|
|
2333
2478
|
for (const row of rowCountsResult.rows) {
|
|
2334
2479
|
const table = row["table_name"];
|
|
2335
|
-
const
|
|
2336
|
-
rowCountMap.set(table,
|
|
2480
|
+
const count2 = Number(row["row_count"] ?? 0);
|
|
2481
|
+
rowCountMap.set(table, count2);
|
|
2337
2482
|
}
|
|
2338
2483
|
const columnsByTable = /* @__PURE__ */ new Map();
|
|
2339
2484
|
for (const row of columnsResult.rows) {
|
|
@@ -4244,6 +4389,8 @@ exports.QueryRequestSchema = QueryRequestSchema;
|
|
|
4244
4389
|
exports.QuerySchema = QuerySchema;
|
|
4245
4390
|
exports.REFRESH_MODES = REFRESH_MODES;
|
|
4246
4391
|
exports.SCOPE_VERBS = SCOPE_VERBS;
|
|
4392
|
+
exports.UNREADABLE_MODES = UNREADABLE_MODES;
|
|
4393
|
+
exports.UpgradedColumnsSchema = UpgradedColumnsSchema;
|
|
4247
4394
|
exports.VexError = VexError;
|
|
4248
4395
|
exports.VexScopeError = VexScopeError;
|
|
4249
4396
|
exports.analyze = analyze;
|