@niscorp/prism 0.2.2 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +187 -0
- package/DESIGN.md +9 -8
- package/DOCS.md +68 -25
- package/README.md +35 -22
- package/dist/agent.cjs +4 -4
- package/dist/agent.js +2 -2
- package/dist/{chunk-26VYDHHC.cjs → chunk-66YSI4PX.cjs} +268 -97
- package/dist/chunk-66YSI4PX.cjs.map +1 -0
- package/dist/{chunk-O3OJAQWA.js → chunk-7VMFSXBM.js} +262 -98
- package/dist/chunk-7VMFSXBM.js.map +1 -0
- package/dist/{chunk-XRK55NPE.js → chunk-BKXCULZV.js} +5 -3
- package/dist/chunk-BKXCULZV.js.map +1 -0
- package/dist/{chunk-W32JXP33.cjs → chunk-OLZQK2ZF.cjs} +7 -5
- package/dist/chunk-OLZQK2ZF.cjs.map +1 -0
- package/dist/examples/index.cjs +25 -2
- package/dist/examples/index.cjs.map +1 -1
- package/dist/examples/index.d.cts +1 -1
- package/dist/examples/index.d.ts +1 -1
- package/dist/examples/index.js +25 -2
- package/dist/examples/index.js.map +1 -1
- package/dist/index.cjs +1224 -137
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +9 -6
- package/dist/index.d.ts +9 -6
- package/dist/index.js +1120 -26
- package/dist/index.js.map +1 -1
- package/dist/migrations/index.cjs +15 -7
- package/dist/migrations/index.cjs.map +1 -1
- package/dist/migrations/index.d.cts +0 -2
- package/dist/migrations/index.d.ts +0 -2
- package/dist/migrations/index.js +14 -2
- package/dist/migrations/index.js.map +1 -1
- package/dist/{profiles-C6r-x-Vj.d.cts → profiles-CPVfysxC.d.cts} +2 -1
- package/dist/{profiles-C6r-x-Vj.d.ts → profiles-CPVfysxC.d.ts} +2 -1
- package/package.json +4 -4
- package/dist/chunk-26VYDHHC.cjs.map +0 -1
- package/dist/chunk-O3OJAQWA.js.map +0 -1
- package/dist/chunk-V426VIRI.cjs +0 -1168
- package/dist/chunk-V426VIRI.cjs.map +0 -1
- package/dist/chunk-VC4PRB4H.js +0 -1086
- package/dist/chunk-VC4PRB4H.js.map +0 -1
- package/dist/chunk-W32JXP33.cjs.map +0 -1
- package/dist/chunk-XRK55NPE.js.map +0 -1
- package/dist/transform-DcVcOBzQ.d.cts +0 -3
- package/dist/transform-DcVcOBzQ.d.ts +0 -3
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,192 @@
|
|
|
1
1
|
# @niscorp/prism
|
|
2
2
|
|
|
3
|
+
## 0.3.0
|
|
4
|
+
|
|
5
|
+
### Minor Changes
|
|
6
|
+
|
|
7
|
+
- f834f3f: `evaluate` checks a config object once and only evaluates it after that; `prismTransform` is gone. **Breaking.**
|
|
8
|
+
|
|
9
|
+
BREAKING — approved by moccadroid, 2026-10-09: replace `prismTransform` with `evaluate` (`transform: evaluate`), and pass limits as `{ limits: … }`.
|
|
10
|
+
|
|
11
|
+
Prism had three ways to run a config, and the one a host was told to hand to its seam (`prismTransform`) was the one that reported worst, while the hosts on a hot path each wrapped `evaluate`, which checked the config on every call. There are two now, with one job each.
|
|
12
|
+
|
|
13
|
+
**`evaluate(config, source, options?)`** — for a config a program holds.
|
|
14
|
+
- The first time it is handed a config object it checks it and keeps its tree; the second time it prepares the tree as `compile` would; from then on a call only evaluates. Four fields picked and renamed, the same config object again: 3.5 µs before, 0.4 µs now. A nested choice on two fields: 18 µs before, 0.7 µs now. A config that is a new object on every call costs what it did.
|
|
15
|
+
- It is kept **by the object**: a config changed in place after its first call is not read again. Hand over a new object.
|
|
16
|
+
- Both arguments are `unknown`, so it is what a host's transform seam takes: `createTide({ transform: evaluate })`, `createUpgrader(grammars, { transform: evaluate })`.
|
|
17
|
+
- The third argument is `{ limits?, check? }`. Limits were the third argument themselves: `evaluate(config, source, { maxSteps: 100 })` is now `evaluate(config, source, { limits: { maxSteps: 100 } })`.
|
|
18
|
+
- `check: 'always'` is for writing a config, for tests and for tools: nothing is kept, the config is checked on every call, and the source must be plain JSON (a source holding `undefined`, a function, a `Date` or a non-finite number is refused with `E_TYPE`).
|
|
19
|
+
- An invalid config is a `PrismError` (`E_SCHEMA`) naming the part that is wrong, however deep the config. `prismTransform` threw zod's own error, and a `RangeError` for a config nested far past the limit.
|
|
20
|
+
|
|
21
|
+
`evaluateSafe` takes the same arguments.
|
|
22
|
+
|
|
23
|
+
**`compile` and `execute`** — for a config that is stored. Unchanged.
|
|
24
|
+
|
|
25
|
+
**`prismTransform` is removed**, from `@niscorp/prism` and from `@niscorp/prism/migrations`. What it added over `evaluate` was a check that the source is plain JSON; that check is now `check: 'always'`, and a seam's source is taken as it is.
|
|
26
|
+
|
|
27
|
+
**moss, nova, vex, loom, nisc, create-nisc and the cli move with it.** Each has Prism as a peer or a dependency, or has one of those that do, and a Prism at 0.3 is outside the range their last versions accept: an app takes them together. Nothing in vex, loom or the cli changed beyond that range.
|
|
28
|
+
|
|
29
|
+
moss's shell and nova's `$prism` binding called `evaluate` on every use and so paid the check each time: an endpoint's request and response under moss, and a `$prism` value on each draw and for each row of a list, are 8 to 45 times faster for the configs measured. tide's and strata's documentation name `evaluate` where they named `prismTransform`.
|
|
30
|
+
|
|
31
|
+
**What to change**
|
|
32
|
+
- `import { prismTransform } from '@niscorp/prism'` or `'@niscorp/prism/migrations'` → `import { evaluate } from '@niscorp/prism'`, and `transform: prismTransform` → `transform: evaluate`. Every app made by `create-nisc` has this in `src/dev/strata.ts`; the app's typecheck and `npm run strata` both stop on that line until it is changed.
|
|
33
|
+
- A third argument of limits becomes `{ limits: … }`.
|
|
34
|
+
- A wrapper that parsed the config before calling `evaluate` (`evaluate(ConfigSchema.parse(config), …)`) makes a new object on every call and so is checked on every call: hand `evaluate` the config itself.
|
|
35
|
+
- Code that changes a config object in place and evaluates it again must hand over a new object, or pass `{ check: 'always' }`.
|
|
36
|
+
- A host that relied on `prismTransform` refusing a source that is not plain JSON checks it itself, or passes `{ check: 'always' }`.
|
|
37
|
+
|
|
38
|
+
### Patch Changes
|
|
39
|
+
|
|
40
|
+
- 023bcf9: `compile` no longer reads a `$with` binding or a `$renameKeys` key that is named like an op as that op.
|
|
41
|
+
|
|
42
|
+
A `$with` binding and a `$renameKeys` map are keyed by names, and a name may start with `$`. The optimizer walked those records as nodes, so where a name was an op's, `execute` did not answer as `evaluate` does:
|
|
43
|
+
|
|
44
|
+
```ts
|
|
45
|
+
{ $with: { let: { $upper: 'x' }, value: { $var: '$upper' } } }
|
|
46
|
+
// evaluate: 'x'. execute: "Variable not found: $upper" — the binding had been folded to { $const: 'X' }.
|
|
47
|
+
|
|
48
|
+
{ $with: { let: { $ref: 2 }, value: { $var: '$ref' } } }
|
|
49
|
+
// evaluate: 2. compile threw TypeError: path.startsWith is not a function.
|
|
50
|
+
|
|
51
|
+
{ $renameKeys: { from: { $ref: '$' }, map: { $upper: 'x' } } } // over { $upper: 1 }
|
|
52
|
+
// evaluate: { x: 1 }. execute: { $upper: 1 } — the key was not renamed.
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
`execute` now answers all three as `evaluate` does, from a fresh IR and from one read back from storage. A config with no such name compiles to the same core and the same fingerprint as before. One with such a name compiles to a different core, so its fingerprint changes.
|
|
56
|
+
|
|
57
|
+
**What to change:** nothing. An IR compiled earlier from such a config and stored keeps its wrong answer until it is compiled again.
|
|
58
|
+
|
|
59
|
+
- 1db6df4: A plain object template is recognised before the ops are tried, not after: `execute` and `evaluate` are up to four times faster on configs that are templates.
|
|
60
|
+
|
|
61
|
+
A template such as `{ id: { $ref: '$.id' }, name: … }` is not an op, so the compiler attaches no handler to it, and the evaluator reached it only after asking about sixty times whether it was some op. It did that on every evaluation, in a compiled tree too, and once for each row where the template is the body of a `$map`. A template has no `$` key and every op needs one, so asking for the template first changes no answer.
|
|
62
|
+
|
|
63
|
+
Measured, Node 24, one order unless said:
|
|
64
|
+
|
|
65
|
+
| | before | now |
|
|
66
|
+
| -------------------------------------------- | ------- | ------- |
|
|
67
|
+
| `execute`, four fields picked and renamed | 1.32 µs | 0.31 µs |
|
|
68
|
+
| `execute`, a nested shape with mapped lines | 7.63 µs | 1.77 µs |
|
|
69
|
+
| `execute`, three strings built | 3.24 µs | 1.20 µs |
|
|
70
|
+
| `evaluate`, four fields | 4.15 µs | 3.17 µs |
|
|
71
|
+
| `execute`, a total for each of 10,000 orders | 30.4 ms | 19.8 ms |
|
|
72
|
+
|
|
73
|
+
A config whose time is in sorting, filtering or summing is as fast as it was.
|
|
74
|
+
|
|
75
|
+
**What to change:** nothing.
|
|
76
|
+
|
|
77
|
+
- 6d098e4: `execute` hands out a compiled config's constants frozen.
|
|
78
|
+
|
|
79
|
+
Every `execute` of an IR answers with the IR's own constants — a `$const` list or object, and anything the compiler folded because it did not depend on the source. A caller that wrote to one changed the IR, and so every later answer:
|
|
80
|
+
|
|
81
|
+
```ts
|
|
82
|
+
const ir = await compile({ tags: { $coalesce: [{ $ref: '$.tags' }, { $const: [] }] } });
|
|
83
|
+
const first = execute(ir, { tags: null });
|
|
84
|
+
first.tags.push('x');
|
|
85
|
+
execute(ir, { tags: null }); // { tags: ['x'] } — for the next caller too, whoever it was
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
Where an IR is kept and its answers go to different requests, as vex does with a query's mapping, that carried one request's change into the next. The constants are now frozen all the way down when a config is compiled, and again when an IR read back from storage first runs. `first.tags.push('x')` throws a `TypeError` where it is written.
|
|
89
|
+
|
|
90
|
+
`evaluate` is as it was: what it answers may be changed. So may everything `execute` builds for the call — a template's object, a `$map`'s list.
|
|
91
|
+
|
|
92
|
+
**What to change:** code that writes into an answer of `execute` (or of a vex read whose mapping has a constant list or object) now throws at that line, where it used to change later answers. Copy the part before changing it: `[...answer.tags, x]`.
|
|
93
|
+
|
|
94
|
+
- 8b4d97e: `$eq` and `$neq` no longer write both values out as JSON to compare two scalars.
|
|
95
|
+
|
|
96
|
+
Two values are equal when their JSON text is, and Prism compared that text for every pair, `'paid'` against `'paid'` included. The text is now only made for two different objects or arrays: the same value is equal, and a scalar is equal to nothing but itself. Every pair is answered as before, keys in another order included.
|
|
97
|
+
|
|
98
|
+
A filter on two fields over 10,000 rows, then a sort and a take: 3.7 ms before, 2.2 ms now. A rule of three comparisons on one row: 361 ns before, 207 ns now.
|
|
99
|
+
|
|
100
|
+
**What to change:** nothing.
|
|
101
|
+
|
|
102
|
+
- b084e30: `$sortBy` gives every key a place: an item whose key is `null`, a boolean, or a string among numbers is no longer left wherever the sort happened to put it.
|
|
103
|
+
|
|
104
|
+
`$sortBy` compared numbers with numbers and strings with strings. Any other pair of keys compared as equal, and an item that is equal to everything has no place in an order: where it ended up depended on which items it was compared with, so the same rows in another order came out differently.
|
|
105
|
+
|
|
106
|
+
Now, where the keys are of more than one kind: numbers first, then strings, then `false` and `true`, and `dir: "desc"` turns that around. An item whose key is `null` is last in both directions. Items with equal keys keep the order they came in.
|
|
107
|
+
|
|
108
|
+
A list whose keys are all numbers, or all strings, sorts exactly as it did.
|
|
109
|
+
|
|
110
|
+
**What to change:** nothing. A list that was sorted by a key that is sometimes `null` now has those items at the end.
|
|
111
|
+
|
|
112
|
+
- 663d724: Three things `execute` did on every call, or for every node, that it did not need to.
|
|
113
|
+
- An op's handler in a compiled tree was a function around the op, so every node was two calls. It is the op.
|
|
114
|
+
- A template built a set of its optional field names and caught errors for each field, whether or not it had a `__optional`. One without it, which is nearly every template, now just evaluates its fields.
|
|
115
|
+
- `execute` primed the path cache from the IR's path table on every call. It does so once for each IR, when it restores the IR's annotations.
|
|
116
|
+
|
|
117
|
+
Four fields picked and renamed: 305 ns before, 232 ns now. A nested shape with mapped lines: 1.56 µs and 1.29 µs. A total for each of 10,000 orders: 13.4 ms and 12.6 ms. Answers are as they were.
|
|
118
|
+
|
|
119
|
+
**What to change:** nothing.
|
|
120
|
+
|
|
121
|
+
- 6512d91: Four additions to Prism, and its grammar is at version 3.
|
|
122
|
+
- **`$toNumber`** — a number from a number or from numeric text, as a form, a CSV or an API often sends it: `{ "$toNumber": { "value": { "$ref": "$.price" } } }`. Text is read when it is a number written in digits (`"42"`, `"-3.5"`, `".5"`, `" 1e3 "`). Anything else — `""`, `"12 kg"`, `"1,234"`, `true`, `null` — throws `E_TYPE`, or answers `fallback` where one is given. No other op converts: `$add` on `"4"` is still an error.
|
|
123
|
+
- **`$mod`** — the remainder of a division: `{ "$mod": [{ "$ref": "$.row" }, 2] }`. It takes the sign of the divisor, so with a positive divisor it is never negative (`[-1, 5]` is `4`, where JavaScript's `%` gives `-1`). Throws `E_DIVISION_BY_ZERO` for a zero divisor.
|
|
124
|
+
- **`$round` takes `mode`** — `"floor"` rounds down and `"ceil"` up, at the same `digits`. Without it, it rounds to the nearest, as before.
|
|
125
|
+
- **`$replace` takes `all`** — `all: true` replaces every occurrence. Without it, the first, as before. `search` is text, not a pattern.
|
|
126
|
+
|
|
127
|
+
`MAPPING_OPS` includes `$mod` and `$toNumber`, and loom's Prism editor lists both under Math.
|
|
128
|
+
|
|
129
|
+
**The grammar.** `nisc.prism` goes from 2 to 3 with one migration that rewrites nothing: these are additions, and a reader at 2 must refuse a config that uses them. The same entry records the narrowing made earlier in this release — a `$ref` path that is not keys and indexes is refused — which the JSON Schema does not show.
|
|
130
|
+
|
|
131
|
+
**What to change:** an app that keeps Prism configs in its source moves its stamp: `npm run strata upgrade`, then `npm run strata verify`. There is nothing to edit; `strata.lock.json` goes to `nisc.prism 3`. A config written before parses to exactly what it did, so a compiled IR's fingerprint does not move.
|
|
132
|
+
|
|
133
|
+
- 7d682b9: A valid config is checked in microseconds: the node schema tries the one member a value can be before it walks its union.
|
|
134
|
+
|
|
135
|
+
`NodeSchema` is a union of 76 members that zod tries in order, and a plain object is the last. Each plain object in a config cost about a quarter of a millisecond to accept, so `evaluate`, `validate` and `compile` each spent about half a millisecond on a config of three operators before doing anything. A value's keys decide what it can be: one `$` key and nothing else is that op, no `$` key is a template. That member is now tried first.
|
|
136
|
+
|
|
137
|
+
On a config of three operators (Node 24, zod 4.6.5; the same on zod 4.2.0):
|
|
138
|
+
|
|
139
|
+
| | before | now |
|
|
140
|
+
| ------------------------ | ------ | ------ |
|
|
141
|
+
| `ConfigSchema.safeParse` | 505 µs | 1.9 µs |
|
|
142
|
+
| `evaluate` | 517 µs | 5.7 µs |
|
|
143
|
+
| `compile` | 652 µs | 79 µs |
|
|
144
|
+
|
|
145
|
+
A config that is refused is then walked as before, so what is refused, and every issue and path of the refusal, is unchanged; a refusal takes a few percent longer than it did (1.27 ms to 1.30–1.38 ms for one small refused config). `getConfigJsonSchema`, `getNodeJsonSchema` and `getProfileJsonSchema` return what they returned, and `NodeSchema` is still a zod union. `safeParseAsync` takes the walk alone and is as fast as it was.
|
|
146
|
+
|
|
147
|
+
The README, DOCS.md and DESIGN.md no longer say `execute` is "2-5x faster" than `evaluate`: it was 6 to 960 times before this change and is 1.7 to 65 times after it, by config.
|
|
148
|
+
|
|
149
|
+
**What to change:** nothing.
|
|
150
|
+
|
|
151
|
+
- d2e9d22: `$map`, `$filter`, `$reduce`, `$sortBy`, `$groupBy` and `$keyBy` make one scope for the loop, not one for each item.
|
|
152
|
+
|
|
153
|
+
Each of them built a new context and a new set of variables for every item, to set the loop's variable. They now make that once, as the loop's own copy, and set the variable in it for each item. What a body sees, and what is outside the loop, is as before: a variable of the same name outside is what it was when the loop is done, and a loop inside the body has a scope of its own.
|
|
154
|
+
|
|
155
|
+
Over 10,000 orders: a total for each, 20.3 ms before and 13.4 ms now; filter, sort and take ten, 2.2 ms and 1.2 ms; a count and a sum by country, 3.6 ms and 2.1 ms; a count, a revenue and twenty rows, 15.6 ms and 9.6 ms.
|
|
156
|
+
|
|
157
|
+
**What to change:** nothing.
|
|
158
|
+
|
|
159
|
+
- e0e8abc: A `$const` is returned as it is written, also where its data has a key named like a sugar op.
|
|
160
|
+
|
|
161
|
+
Before a config runs, the sugar ops (`$sum`, `$avg`, `$count`, `$min`, `$max`, `$pluck`, `$take`, `$drop`, `$match`, `$flatMap`) are rewritten to core ops. That pass walked every object, so it also rewrote what is not a node:
|
|
162
|
+
|
|
163
|
+
```ts
|
|
164
|
+
{ $const: { total: { $sum: 1 } } }
|
|
165
|
+
// answered { total: { $reduce: { … } } }, from evaluate and from execute alike
|
|
166
|
+
|
|
167
|
+
{ $with: { let: { $sum: [1, 2] }, value: { $var: '$sum' } } }
|
|
168
|
+
// "Variable not found: __acc" — the binding called $sum was rewritten as a sum
|
|
169
|
+
|
|
170
|
+
{ $renameKeys: { from: { $ref: '$' }, map: { $count: 'n' } } } // over { $count: 1 }
|
|
171
|
+
// { $count: 1 } — not renamed
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
A `$const`'s data, the names of a `$with`'s bindings and a `$renameKeys` map are now left alone; the three answer `{ total: { $sum: 1 } }`, `[1, 2]` and `{ n: 1 }`. Data that holds MongoDB-style operators, or a Prism config kept as data (as a strata migration writes one into a document), comes back as it was written.
|
|
175
|
+
|
|
176
|
+
A config with none of these compiles to the core and fingerprint it had. One that has them compiles differently, so its fingerprint changes.
|
|
177
|
+
|
|
178
|
+
**What to change:** nothing. An IR compiled earlier from such a config keeps its wrong constant until it is compiled again.
|
|
179
|
+
|
|
180
|
+
- 7fe1135: Two things Prism answered wrongly without a word are now refused.
|
|
181
|
+
|
|
182
|
+
**A path it does not read.** A `$ref` path is keys and indexes, as DOCS.md says: `$.rows[0].sku`. The rest of JSONPath — a wildcard `[*]`, a filter `[?(…)]`, a slice `[0:2]`, a negative index, a quoted key `['name']`, empty brackets, `..` — was accepted by the schema, and the parser gave up on it with no segments, which reads as `$`. So `{ "$ref": "$.rows[*].sku" }` answered the whole source, and `$..sku` was read as `$.sku`. Such a path is now refused when the config is checked: `validate`, `evaluate` and `compile` give `E_SCHEMA` at the `$ref`, "Not a path Prism reads: … A path is keys and indexes only …; for every item of a list use $map or $pluck." An IR that already holds one is refused when it runs.
|
|
183
|
+
|
|
184
|
+
**A sort key that is a list or an object.** `$sortBy` compares numbers and strings. For any other key it compared nothing, so `by: [a, b]` handed the list back in the order it came. It now throws `E_TYPE` and says what to write: sort by the second key, then by the first; the sort keeps the order of equal items.
|
|
185
|
+
|
|
186
|
+
`getConfigJsonSchema` and the grammar snapshot are unchanged: the path rule is the parser's, beside the published pattern, not in it.
|
|
187
|
+
|
|
188
|
+
**What to change:** a config with such a path or such a key was already answering wrongly; it now says so. Write the path with `$map` or `$pluck`, and the two-key sort as two sorts. Nothing in this repository, nisc-website, moccadroid-website or midas had one. Still as it was: a sort key that is `null`, a boolean, or a number beside a string compares as equal to everything, so such items keep no defined place.
|
|
189
|
+
|
|
3
190
|
## 0.2.2
|
|
4
191
|
|
|
5
192
|
### Patch Changes
|
package/DESIGN.md
CHANGED
|
@@ -45,12 +45,12 @@ Input Config (JSON)
|
|
|
45
45
|
```typescript
|
|
46
46
|
type EvalContext = {
|
|
47
47
|
source: JsonValue; // Root source data (immutable)
|
|
48
|
-
vars: Record<string, JsonValue>; // Scoped variables (
|
|
48
|
+
vars: Record<string, JsonValue>; // Scoped variables (a loop writes to its own copy only)
|
|
49
49
|
budget?: Budget; // What this evaluation may still cost (engine/budget.ts)
|
|
50
50
|
};
|
|
51
51
|
```
|
|
52
52
|
|
|
53
|
-
|
|
53
|
+
The context an op is given is never written to. A `$with` binding makes a new context via spread. A loop (`$map`, `$filter`, `$reduce`, `$sortBy`, `$groupBy`, `$keyBy`) makes one context of its own for the whole loop and sets its variable in that for each item (`ops/scope.ts`): a new one per item was a fifth of the time of a map over 10,000 rows. Nothing keeps a scope past the evaluation of one body, so ops are still pure — same input always produces same output.
|
|
54
54
|
|
|
55
55
|
---
|
|
56
56
|
|
|
@@ -109,14 +109,14 @@ type CompiledIr = {
|
|
|
109
109
|
stats: { nodeCount, opCount, maxDepth, optimizations };
|
|
110
110
|
};
|
|
111
111
|
tables: {
|
|
112
|
-
paths: string[]; //
|
|
112
|
+
paths: string[]; // every `$ref` string in the tree, a constant's data included: a listing, not read by execute
|
|
113
113
|
strings: string[]; // String literals
|
|
114
114
|
};
|
|
115
115
|
core: unknown; // Desugared, optimized config
|
|
116
116
|
};
|
|
117
117
|
```
|
|
118
118
|
|
|
119
|
-
`execute()` skips validation and desugaring — it
|
|
119
|
+
`execute()` skips validation and desugaring — it restores each node's handler and each `$ref`'s parsed path once per IR, and evaluates `core` directly. How much that saves over `evaluate()` depends on the config: `pnpm bench` runs five against each other (`test/optimize.bench.ts`), from under 2x for a short map to over 50x where the constants fold.
|
|
120
120
|
|
|
121
121
|
The fingerprint enables cache invalidation: store IRs by name, check the fingerprint to know if the config changed.
|
|
122
122
|
|
|
@@ -168,8 +168,7 @@ src/
|
|
|
168
168
|
│ ├── transform.schema.ts
|
|
169
169
|
│ └── sugar.schema.ts
|
|
170
170
|
├── engine/
|
|
171
|
-
│ ├── evaluate.ts # Dispatcher + evaluate / evaluateSafe
|
|
172
|
-
│ ├── transform.ts # prismTransform — Prism in the shape a host's transform seam takes
|
|
171
|
+
│ ├── evaluate.ts # Dispatcher + evaluate / evaluateSafe (what a config object was found to be is kept here)
|
|
173
172
|
│ ├── compile.ts # Config → CompiledIr
|
|
174
173
|
│ ├── optimize.ts # Compile-time passes (ref segments, handlers, constant folding)
|
|
175
174
|
│ ├── budget.ts # Limits: steps, string length, result size
|
|
@@ -195,7 +194,7 @@ src/
|
|
|
195
194
|
│ ├── index.ts
|
|
196
195
|
│ └── mapping-agent.ts # mappingAgent (Cortex agent definition)
|
|
197
196
|
├── migrations/
|
|
198
|
-
│ └── index.ts # @niscorp/prism/migrations — PRISM_SEQUENCE,
|
|
197
|
+
│ └── index.ts # @niscorp/prism/migrations — PRISM_SEQUENCE, PRISM_SCHEMAS
|
|
199
198
|
└── utils/
|
|
200
199
|
├── jsonpath.ts # JSONPath parser + cache
|
|
201
200
|
├── compare.ts # Deep equality, ordered comparison
|
|
@@ -231,7 +230,7 @@ and a mapping over a thousand rows would otherwise build a thousand of them.
|
|
|
231
230
|
|
|
232
231
|
4. **SHA256 fingerprint on IR.** Cache invalidation without version numbers. Same config = same fingerprint.
|
|
233
232
|
|
|
234
|
-
5. **`evaluate` + `evaluateSafe`.** Throwing for trusted code, Result type for untrusted input. Different call sites have different needs.
|
|
233
|
+
5. **`evaluate` + `evaluateSafe`.** Throwing for trusted code, Result type for untrusted input. Different call sites have different needs. Both keep what a config object was found to be, by the object: a host hands the same configs over on every request, and checking each again was the whole cost of a small transform. `check: 'always'` is the call that keeps nothing, for writing and testing a config. A separate function for the kept path (`prismTransform`) existed for a while and is gone: it was the one hosts were told to use and the one that reported worst.
|
|
235
234
|
|
|
236
235
|
6. **Ops receive `evaluate` as a parameter.** Breaks the circular import between ops and the dispatcher. Also enables mock-based unit testing of individual ops.
|
|
237
236
|
|
|
@@ -242,3 +241,5 @@ and a mapping over a thousand rows would otherwise build a thousand of them.
|
|
|
242
241
|
9. **`$case` uses `{ branches, else }`.** Clearer than a flat array mixing `{when,then}` and `{else}` objects. Simpler Zod schema.
|
|
243
242
|
|
|
244
243
|
10. **`compile()` is async.** Uses `crypto.subtle.digest` for SHA256, which is async in all runtimes. `evaluate()` stays sync because it doesn't need the fingerprint.
|
|
244
|
+
|
|
245
|
+
11. **The node union is tried by key before it is walked.** zod tries a union's 76 members in order, and a plain object is the last. One `$` key and nothing else can only be that op, and no `$` key can only be a template, so that member is tried first and a valid config never walks the union. An invalid one is walked as before, from the top, so a refusal is the union's own. The union stays a union: the JSON Schema, the profiles and loom read it as one.
|
package/DOCS.md
CHANGED
|
@@ -44,7 +44,7 @@ If `$.user.nickname` doesn't exist, the result is `{ "name": "Alice" }` — no e
|
|
|
44
44
|
|
|
45
45
|
### `$ref` — Read from source
|
|
46
46
|
|
|
47
|
-
Resolves a
|
|
47
|
+
Resolves a path against the source data. A path is keys and indexes: `.key` and `[index]`. The rest of JSONPath — a wildcard `[*]`, a filter `[?(…)]`, a slice, a quoted key, `..` — is refused when the config is checked (`E_SCHEMA`, "Not a path Prism reads"); for every item of a list use `$map` or `$pluck`.
|
|
48
48
|
|
|
49
49
|
```json
|
|
50
50
|
{ "$ref": "$.user.name" }
|
|
@@ -218,7 +218,7 @@ Source: `{ "tags": ["a", "b", "a", "c", "b"] }` → `["a", "b", "c"]`
|
|
|
218
218
|
}
|
|
219
219
|
```
|
|
220
220
|
|
|
221
|
-
`dir` defaults to `"asc"`.
|
|
221
|
+
`dir` defaults to `"asc"`. A sort key is a number or a string. Where the keys are of more than one kind, numbers come first, then strings, then `false` and `true`, and `"desc"` turns that around; an item whose key is `null` is last in both directions. Items with equal keys keep the order they came in. A key that is a list or an object is refused (`E_TYPE`): to sort by two keys, sort by the second and then by the first — the sort keeps the order of equal items.
|
|
222
222
|
|
|
223
223
|
---
|
|
224
224
|
|
|
@@ -247,11 +247,28 @@ All math ops take a `[left, right]` pair. Both operands are evaluated and must b
|
|
|
247
247
|
```
|
|
248
248
|
Throws `E_DIVISION_BY_ZERO` if the divisor is 0.
|
|
249
249
|
|
|
250
|
+
### `$mod`
|
|
251
|
+
```json
|
|
252
|
+
{ "$mod": [{ "$ref": "$.row" }, 2] }
|
|
253
|
+
```
|
|
254
|
+
The remainder of the first number divided by the second. It takes the sign of the **second** number, so with a positive divisor `n` the answer is always from `0` up to `n` — `{ "$mod": [-1, 5] }` is `4`, where JavaScript's `%` gives `-1`. Throws `E_DIVISION_BY_ZERO` if the divisor is 0.
|
|
255
|
+
|
|
250
256
|
### `$round`
|
|
251
257
|
```json
|
|
252
258
|
{ "$round": { "value": { "$const": 3.14159 }, "digits": 2 } }
|
|
253
259
|
```
|
|
254
|
-
Result: `3.14`. `digits` defaults to 0.
|
|
260
|
+
Result: `3.14`. `digits` defaults to 0. `mode` is `"nearest"` unless given: `"floor"` rounds down (toward negative infinity) and `"ceil"` rounds up, at the same `digits`.
|
|
261
|
+
|
|
262
|
+
```json
|
|
263
|
+
{ "$round": { "value": { "$div": [{ "$ref": "$.minutes" }, 60] }, "mode": "floor" } }
|
|
264
|
+
```
|
|
265
|
+
|
|
266
|
+
### `$toNumber` — A number from text
|
|
267
|
+
```json
|
|
268
|
+
{ "$toNumber": { "value": { "$ref": "$.price" } } }
|
|
269
|
+
{ "$toNumber": { "value": { "$ref": "$.discount" }, "fallback": 0 } }
|
|
270
|
+
```
|
|
271
|
+
A number is answered as it is. Text is read as a number when it is one written in digits, with spaces around it allowed: `"42"`, `"-3.5"`, `".5"`, `" 1e3 "`. Everything else — `""`, `"12 kg"`, `"1,234"`, `"0x10"`, `true`, `null`, a list — throws `E_TYPE`, or answers `fallback` where it is given. The other ops do not convert: `$add` on `"4"` is still an error.
|
|
255
272
|
|
|
256
273
|
---
|
|
257
274
|
|
|
@@ -314,10 +331,12 @@ Result: `"hello"`
|
|
|
314
331
|
```
|
|
315
332
|
Result: `["a", "b", "c"]`
|
|
316
333
|
|
|
317
|
-
### `$replace` — Replace
|
|
334
|
+
### `$replace` — Replace a substring
|
|
318
335
|
```json
|
|
319
336
|
{ "$replace": { "value": { "$ref": "$.text" }, "search": "world", "replacement": "there" } }
|
|
337
|
+
{ "$replace": { "value": { "$ref": "$.phone" }, "search": " ", "replacement": "", "all": true } }
|
|
320
338
|
```
|
|
339
|
+
The first occurrence of `search`, or every occurrence with `all: true`. `search` is plain text, not a pattern.
|
|
321
340
|
|
|
322
341
|
---
|
|
323
342
|
|
|
@@ -721,47 +740,71 @@ const ir = await compile(config, { name: 'user-transform', version: '1.0.0' });
|
|
|
721
740
|
// - Desugared config (sugar ops already resolved)
|
|
722
741
|
// - SHA256 fingerprint (for cache invalidation)
|
|
723
742
|
// - Stats (node count, op frequency, max depth, optimizations applied)
|
|
724
|
-
// - Tables (
|
|
743
|
+
// - Tables (every $ref string and string literal in the tree)
|
|
725
744
|
|
|
726
745
|
const result1 = execute(ir, source1); // No validation, no desugaring
|
|
727
|
-
const result2 = execute(ir, source2); //
|
|
746
|
+
const result2 = execute(ir, source2); // Each op's handler already attached
|
|
728
747
|
```
|
|
729
748
|
|
|
730
749
|
The IR is JSON-serializable — store it in a database, cache in Redis, send over a wire.
|
|
731
750
|
|
|
732
|
-
|
|
751
|
+
**A constant in an answer is frozen.** Every `execute` of an IR hands out the IR's own constants: a `$const` that is a list or an object, and anything the compiler worked out ahead of time because it did not depend on the source (`{ "$split": { "value": "a,b", "sep": "," } }` is a constant list). They are frozen, all the way down, so one caller cannot change what the next is given: `answer.tags.push(x)` on a list that came from `{ "$const": [] }` throws a `TypeError`. To change an answer, copy the part first (`[...answer.tags, x]`). Everything built for the call — a template's object, a `$map`'s list — is the caller's own. `evaluate` does not freeze: what it answers may be changed.
|
|
752
|
+
|
|
753
|
+
## evaluate
|
|
733
754
|
|
|
734
755
|
```typescript
|
|
735
|
-
|
|
756
|
+
evaluate(config: unknown, source: unknown, options?: { limits?: Partial<Limits>; check?: 'always' }): JsonValue
|
|
757
|
+
evaluateSafe(config, source, options?): { ok: true, data } | { ok: false, error }
|
|
736
758
|
```
|
|
737
759
|
|
|
738
|
-
|
|
760
|
+
A config and a source in, the answer out.
|
|
761
|
+
|
|
762
|
+
- **The config** is checked against the schema the first time its object is
|
|
763
|
+
handed over, and refused with a `PrismError` (`E_SCHEMA`) that names the part
|
|
764
|
+
that is wrong. Its tree is kept: the second call with the same object prepares
|
|
765
|
+
it as `compile` would (handlers attached, paths parsed, constants folded), and
|
|
766
|
+
every call after that only evaluates. It is kept by the object, so a config
|
|
767
|
+
changed in place after its first call is not read again — hand over a new
|
|
768
|
+
object. A config that is a new object on every call is checked on every call,
|
|
769
|
+
and is not prepared: that would cost more than it saves.
|
|
770
|
+
- **The source** is taken as it is. Nothing checks it.
|
|
771
|
+
- **The answer** is the caller's to change: a constant is handed out as a copy.
|
|
772
|
+
|
|
773
|
+
It is the function a host's transform seam takes. nova's shell (`transform`),
|
|
739
774
|
tide's engine (`transform`) and strata's upgrader (`{ transform }`) each run a
|
|
740
|
-
config through a function they are handed and know nothing of Prism
|
|
741
|
-
function to hand them.
|
|
775
|
+
config through a function they are handed and know nothing of Prism:
|
|
742
776
|
|
|
743
777
|
```typescript
|
|
744
|
-
import {
|
|
778
|
+
import { evaluate } from '@niscorp/prism';
|
|
745
779
|
|
|
746
|
-
const tide = createTide({ store, transform:
|
|
780
|
+
const tide = createTide({ store, transform: evaluate, effects });
|
|
747
781
|
```
|
|
748
782
|
|
|
749
|
-
|
|
750
|
-
each config object, so a config a host keeps and passes again is not parsed
|
|
751
|
-
again.
|
|
752
|
-
- **The source** must be plain JSON: `null`, strings, booleans, finite numbers,
|
|
753
|
-
and arrays and objects of those. One that holds `undefined`, a function or a
|
|
754
|
-
non-finite number is refused — `The source of a transform must be plain JSON.`
|
|
755
|
-
- **The result** is what `evaluate` answers.
|
|
783
|
+
### `check: 'always'`
|
|
756
784
|
|
|
757
|
-
|
|
758
|
-
|
|
759
|
-
|
|
785
|
+
```typescript
|
|
786
|
+
evaluate(config, source, { check: 'always' })
|
|
787
|
+
```
|
|
788
|
+
|
|
789
|
+
For writing a config, for tests and for tools. Nothing is kept and nothing is
|
|
790
|
+
taken on trust: the config is checked on this call whatever was checked before,
|
|
791
|
+
and the source must be plain JSON — `null`, strings, booleans, finite numbers,
|
|
792
|
+
and arrays and plain objects of those. One that holds `undefined`, a function,
|
|
793
|
+
a `Date` or a non-finite number is refused: `E_TYPE`, "The source must be plain
|
|
794
|
+
JSON."
|
|
795
|
+
|
|
796
|
+
### Which call
|
|
797
|
+
|
|
798
|
+
| You have | Call |
|
|
799
|
+
|---|---|
|
|
800
|
+
| a config your program holds, run again and again | `evaluate(config, source)` |
|
|
801
|
+
| a config being written, a test, a tool | `evaluate(config, source, { check: 'always' })` |
|
|
802
|
+
| a config that is stored, or that must run without its checks | `compile` once, `execute` the IR |
|
|
760
803
|
|
|
761
804
|
## Limits
|
|
762
805
|
|
|
763
|
-
`evaluate
|
|
764
|
-
`Partial<Limits>`, merged over `DEFAULT_LIMITS`:
|
|
806
|
+
`evaluate` and `evaluateSafe` take them as `options.limits`, `execute` as its
|
|
807
|
+
third argument: a `Partial<Limits>`, merged over `DEFAULT_LIMITS`:
|
|
765
808
|
|
|
766
809
|
| Limit | Default | Counts |
|
|
767
810
|
|-------|---------|--------|
|
package/README.md
CHANGED
|
@@ -49,14 +49,13 @@ const result = evaluate(
|
|
|
49
49
|
## API
|
|
50
50
|
|
|
51
51
|
```typescript
|
|
52
|
-
//
|
|
53
|
-
|
|
54
|
-
|
|
52
|
+
// A config and a source in, the answer out. A config object is checked the first
|
|
53
|
+
// time it is seen and only evaluated after that.
|
|
54
|
+
evaluate(config, source, options?) → JsonValue
|
|
55
|
+
evaluateSafe(config, source, options?) → { ok: true, data } | { ok: false, error }
|
|
56
|
+
// options: { limits?, check?: 'always' } — 'always' checks everything on every call
|
|
55
57
|
|
|
56
|
-
//
|
|
57
|
-
prismTransform(config: unknown, source: unknown) → unknown
|
|
58
|
-
|
|
59
|
-
// Compile once, execute many (2-5x faster for repeated configs)
|
|
58
|
+
// For a config that is stored: compile once, keep the IR, execute it
|
|
60
59
|
compile(config, options?) → Promise<CompiledIr>
|
|
61
60
|
execute(ir, source, limits?) → JsonValue
|
|
62
61
|
|
|
@@ -68,25 +67,40 @@ getNodeJsonSchema(target?) → object
|
|
|
68
67
|
getConfigJsonSchema(target?) → object
|
|
69
68
|
```
|
|
70
69
|
|
|
71
|
-
##
|
|
70
|
+
## Which call
|
|
71
|
+
|
|
72
|
+
**`evaluate(config, source)`** is the call for a config a program holds. The
|
|
73
|
+
first time it is handed a config object it checks it against the schema, and
|
|
74
|
+
refuses with the part that is wrong; the second time it prepares it as `compile`
|
|
75
|
+
would; from then on a call only evaluates. So a host that keeps its configs and
|
|
76
|
+
hands the same objects over on every request pays the check once. It is kept by
|
|
77
|
+
the object: a config changed in place after its first call is not read again —
|
|
78
|
+
hand over a new object.
|
|
72
79
|
|
|
80
|
+
Both arguments are `unknown`, so it is also what a host's transform seam takes.
|
|
73
81
|
nova's shell, tide's engine and strata's upgrader each run a config through a
|
|
74
|
-
|
|
75
|
-
Prism
|
|
82
|
+
function they are handed, `(config, source) => unknown`, and know nothing of
|
|
83
|
+
Prism:
|
|
76
84
|
|
|
77
85
|
```typescript
|
|
78
|
-
import {
|
|
86
|
+
import { evaluate } from '@niscorp/prism';
|
|
79
87
|
|
|
80
|
-
createTide({ store, transform:
|
|
81
|
-
createUpgrader(grammars, { transform:
|
|
88
|
+
createTide({ store, transform: evaluate, effects }); // @niscorp/tide
|
|
89
|
+
createUpgrader(grammars, { transform: evaluate }); // @niscorp/strata
|
|
82
90
|
```
|
|
83
91
|
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
92
|
+
A host that adds values of its own to the source first (the app's "today", the
|
|
93
|
+
session's principal) wraps it: `(config, source) => evaluate(config, { ...source, today })`.
|
|
94
|
+
|
|
95
|
+
**`evaluate(config, source, { check: 'always' })`** is for writing a config, for
|
|
96
|
+
tests and for tools. It keeps nothing: the config is checked on every call, and
|
|
97
|
+
so is the source, which must be plain JSON — one holding `undefined`, a
|
|
98
|
+
function, a `Date` or a non-finite number is refused.
|
|
99
|
+
|
|
100
|
+
**`compile` and `execute`** are for a config that is stored. The IR is JSON with
|
|
101
|
+
a fingerprint: keep it beside what it belongs to, and `execute` runs it without
|
|
102
|
+
checking anything again — in another process, on another day. It is the only
|
|
103
|
+
form that does not need the config object to be the same one.
|
|
90
104
|
|
|
91
105
|
## Examples
|
|
92
106
|
|
|
@@ -128,9 +142,8 @@ rules) — see [DOCS.md § Transform Operations](./DOCS.md#transform-operations)
|
|
|
128
142
|
They are what strata's document migrations are written in.
|
|
129
143
|
|
|
130
144
|
**`@niscorp/prism/migrations`** publishes Prism's own grammar — `PRISM_SEQUENCE`
|
|
131
|
-
(`nisc.prism`, kind `nisc.prism/config`), `PRISM_SCHEMAS
|
|
132
|
-
|
|
133
|
-
it): the same function as the main entry's.
|
|
145
|
+
(`nisc.prism`, kind `nisc.prism/config`), `PRISM_SCHEMAS`. The evaluator a
|
|
146
|
+
migration runs through is `evaluate`, from the main entry (strata injects it).
|
|
134
147
|
|
|
135
148
|
**The op set only ever grows.** Configs are stored — endpoint requests, vex
|
|
136
149
|
mappings, migrations — and a migration cannot be migrated by the language it is
|
package/dist/agent.cjs
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
'use strict';
|
|
2
2
|
|
|
3
|
-
var
|
|
4
|
-
var
|
|
3
|
+
var chunkOLZQK2ZF_cjs = require('./chunk-OLZQK2ZF.cjs');
|
|
4
|
+
var chunk66YSI4PX_cjs = require('./chunk-66YSI4PX.cjs');
|
|
5
5
|
var zod = require('zod');
|
|
6
6
|
var cortex = require('@niscorp/cortex');
|
|
7
7
|
|
|
@@ -21,7 +21,7 @@ Worked example (illustrative only \u2014 read the schema for op details):
|
|
|
21
21
|
INPUT { "sampleInput": { "first": "Ada", "last": "Lovelace" }, "targetShape": { "fullName": "" } }
|
|
22
22
|
data { "fullName": { "$interpolate": { "template": "{{f}} {{l}}", "values": { "f": { "$ref": "$.first" }, "l": { "$ref": "$.last" } } } } }`;
|
|
23
23
|
var OUTPUT_DOC = `OUTPUT SCHEMA \u2014 your output MUST validate against this JSON Schema:
|
|
24
|
-
${JSON.stringify(
|
|
24
|
+
${JSON.stringify(chunkOLZQK2ZF_cjs.getProfileJsonSchema(chunkOLZQK2ZF_cjs.MAPPING_OPS, "draft-7"))}`;
|
|
25
25
|
var mappingAgent = cortex.defineAgent({
|
|
26
26
|
id: "prism.mapping",
|
|
27
27
|
description: "Generates a Prism transformation config from a sample input row and a target shape.",
|
|
@@ -31,7 +31,7 @@ var mappingAgent = cortex.defineAgent({
|
|
|
31
31
|
// mapping uses, about 28% less prompt than the whole grammar
|
|
32
32
|
// (schemas/profiles.ts). The heading is cortex's own (context/schema-doc.ts),
|
|
33
33
|
// so the finish protocol's "OUTPUT SCHEMA" still names it.
|
|
34
|
-
output: { schema:
|
|
34
|
+
output: { schema: chunk66YSI4PX_cjs.ConfigSchema, doc: OUTPUT_DOC }
|
|
35
35
|
});
|
|
36
36
|
|
|
37
37
|
exports.MappingAgentInputSchema = MappingAgentInputSchema;
|
package/dist/agent.js
CHANGED
|
@@ -1,5 +1,5 @@
|
|
|
1
|
-
import { getProfileJsonSchema, MAPPING_OPS } from './chunk-
|
|
2
|
-
import { ConfigSchema } from './chunk-
|
|
1
|
+
import { getProfileJsonSchema, MAPPING_OPS } from './chunk-BKXCULZV.js';
|
|
2
|
+
import { ConfigSchema } from './chunk-7VMFSXBM.js';
|
|
3
3
|
import { z } from 'zod';
|
|
4
4
|
import { defineAgent } from '@niscorp/cortex';
|
|
5
5
|
|