@niscorp/solid 0.1.0 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +64 -0
- package/DESIGN.md +8 -5
- package/README.md +7 -5
- package/dist/index.cjs +18 -2
- package/dist/index.cjs.map +1 -1
- package/dist/index.js +18 -2
- package/dist/index.js.map +1 -1
- package/package.json +3 -2
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,64 @@
|
|
|
1
|
+
# @niscorp/solid
|
|
2
|
+
|
|
3
|
+
## 0.1.2
|
|
4
|
+
|
|
5
|
+
### Patch Changes
|
|
6
|
+
|
|
7
|
+
- 5923e38: `destroy()` no longer raises an unhandled rejection when nobody holds `final()`.
|
|
8
|
+
|
|
9
|
+
A stream's `final()` promise exists from the start, and `destroy()` rejects it. A stream read through `on` / `onFinal` / `onError` alone never takes that promise, so nothing could catch the rejection: in Node the process ended with `Error: [solid] stream destroyed`, exit code 1, and a browser logged an uncaught error. Every selection made with `select()` rejected a promise of its own the same way.
|
|
10
|
+
|
|
11
|
+
Both rejections are now caught where they are raised, as a strict failure's already was. A caller that holds `final()` — taken before or after `destroy()` — still sees it reject with `[solid] stream destroyed`.
|
|
12
|
+
|
|
13
|
+
**What to change:** nothing. A `stream.final().catch(() => {})` written only to keep `destroy()` quiet can go.
|
|
14
|
+
|
|
15
|
+
- 2815107: README and DESIGN say what `current()` holds. Outside `trust` mode, and where the schema states a value's JSON kind, a value of another kind is not written into it. An object the reply creates (a list row, an object where the initial value has `null` or nothing, a record entry) starts empty and fills in key by key, so it can lack required keys until it closes, with no error. `constraints: 'finalize'` reports a violation and leaves the value in place; in `strict` the stream fails with that value in it. What is written should be the JSON alone: a code fence around it is ignored, a `"`, a bracket or a brace in text before it is not. No code changed.
|
|
16
|
+
|
|
17
|
+
**What to change:** nothing.
|
|
18
|
+
|
|
19
|
+
- 885bc1e: README says what a rejected list element leaves behind. "Keep the prior valid one" holds where the list held a value at that position. Where it held nothing there, a rejected element that is followed by an accepted one leaves `undefined` at its position: `{"seats":["C4",5,"C6"]}` against `z.array(z.string())` with an empty initial list gives `['C4', undefined, 'C6']`, which the schema itself refuses. `onError` names the position, and `constraints: 'finalize'` reports the list again when it closes. A rejected element in last place leaves the list one short. Two tests now hold this. No code changed.
|
|
20
|
+
|
|
21
|
+
**What to change:** nothing. Code that maps over a streamed list in `recover` mode can meet `undefined` in it after an `onError` for that list.
|
|
22
|
+
|
|
23
|
+
- f9f1b26: `select(path).onFinal(listener)` added after the reply has ended is called, at once.
|
|
24
|
+
|
|
25
|
+
A selection learns that the reply ended from the root, once. A selection first made after that — `stream.select('widget')` called for the first time when the closing `}` had arrived, or after `close()` — was never told, so a listener given to its `onFinal` was never called: not at once, not after `close()`, not after a later write. Its `final()` did resolve, and the root's own `onFinal` was called at once in the same position.
|
|
26
|
+
|
|
27
|
+
Such a listener is now called at once with the part's value, as the root's is, and once.
|
|
28
|
+
|
|
29
|
+
Everything else is as it was. A listener added while the reply is still arriving is called when its part finishes — or, if the part had already finished when the selection was first made, by the next write, as before. One added from inside a root `onFinal` listener is called after it, as before.
|
|
30
|
+
|
|
31
|
+
**What to change:** nothing, with one thing to know. Where the reply had already ended, this listener used to be dropped and is now called inside the `onFinal()` call itself, before `onFinal()` has returned. A listener that calls its own unsubscribe — `const off = sel.onFinal(() => { off(); … })` — cannot name `off` yet at that moment and throws; that is already so for the root's `onFinal`, and for a selection made before the reply. Write `let off = () => {}; off = sel.onFinal(…)`, or do not unsubscribe a listener that is only ever called once.
|
|
32
|
+
|
|
33
|
+
- e3e0ff2: A value from `current()` or `on()` no longer changes after it was handed out.
|
|
34
|
+
|
|
35
|
+
When a piece of the reply ended right after a row's `{` or a list's `[`, the snapshot held the parser's own working object for that row or list — the one it goes on writing into. A value a consumer had kept, `{"rows":[{}]}` say, gained the row's keys as later pieces arrived, without `on()` saying so for that value. Fed a reply character by character, 4 of the 20 values delivered to `on()` changed after delivery.
|
|
36
|
+
|
|
37
|
+
A container the parser has only just opened is now copied into the snapshot. Nothing about sharing moves: across a reply fed character by character, in pieces ending right after an opener, and in one write, the same subtrees keep and change their references at every write as before, and every `on()` listener on the root and on selections is called the same number of times.
|
|
38
|
+
|
|
39
|
+
**What to change:** nothing.
|
|
40
|
+
|
|
41
|
+
- ed9d7f4: `write()` no longer throws on a string that begins outside any container.
|
|
42
|
+
|
|
43
|
+
Two ways led there. In `mode: 'trust'`, a quoted word in the text around the reply — `write('Here is the "best" option:\n')` — threw `TypeError: Cannot read properties of undefined (reading '0')`, and the reply after it was never read. And a stream whose schema is a bare `z.string()` threw the same error on `write('"hello"')`, in every mode.
|
|
44
|
+
|
|
45
|
+
Neither throws now. In `trust` the quoted word is passed over and the reply that follows is read as usual. A string root still receives nothing: its value stays the initial one, as a number or boolean root's already does, and nothing is reported — a stream reads a reply that is an object or an array.
|
|
46
|
+
|
|
47
|
+
What `recover` and `strict` do with text around the reply is unchanged.
|
|
48
|
+
|
|
49
|
+
**What to change:** nothing.
|
|
50
|
+
|
|
51
|
+
## 0.1.1
|
|
52
|
+
|
|
53
|
+
### Patch Changes
|
|
54
|
+
|
|
55
|
+
- fe30458: Every package exports its `package.json`, so an app can say which version of a package it runs by reading it from the package:
|
|
56
|
+
|
|
57
|
+
```ts
|
|
58
|
+
import prism from '@niscorp/prism/package.json' with { type: 'json' };
|
|
59
|
+
prism.version; // the version that is installed, not the range that asked for it
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
Until now the `exports` map hid it, and the only way to the version was a path into `node_modules`. `check:packages` installs the tarballs and reads every package's version this way.
|
|
63
|
+
|
|
64
|
+
**What to change:** nothing. An app that read a version by path can read it by name.
|
package/DESIGN.md
CHANGED
|
@@ -22,7 +22,7 @@ The streamed JSON is not the state. The state is a valid object that exists from
|
|
|
22
22
|
const stream = createStream({ schema, initial });
|
|
23
23
|
```
|
|
24
24
|
|
|
25
|
-
`initial` is validated against the Zod schema at construction. Every `current()` call returns
|
|
25
|
+
`initial` is validated against the Zod schema at construction. Every `current()` call returns an object — before, during, and after streaming — and a value of the wrong JSON kind is never written into it (see "The always-valid invariant" for where that check applies). The incoming JSON progressively overwrites fields in this object as they're parsed; an object the reply creates (a list row, an object where `initial` has `null` or nothing, a record entry) starts empty and fills in key by key.
|
|
26
26
|
|
|
27
27
|
This means consumers can render and act on the state immediately. There is no "waiting for valid JSON" phase.
|
|
28
28
|
|
|
@@ -210,7 +210,7 @@ Scaling ratio (20KB/1KB ms/KB): **1.4x** — effectively linear.
|
|
|
210
210
|
|
|
211
211
|
## The always-valid invariant
|
|
212
212
|
|
|
213
|
-
Solid's central promise:
|
|
213
|
+
Solid's central promise: outside `trust` mode, where the schema states a value's JSON kind, a value of another kind is never written into what `current()` returns. Not "will be right eventually" — right now, and after every write, regardless of what the LLM sends. It is a promise about kinds, not completeness: an object the reply creates fills in key by key (see Consequences).
|
|
214
214
|
|
|
215
215
|
### The problem it solves
|
|
216
216
|
|
|
@@ -234,16 +234,19 @@ With `constraints: 'finalize'`, the validator runs `subSchema.safeParse(currentV
|
|
|
234
234
|
|
|
235
235
|
### Modes
|
|
236
236
|
|
|
237
|
-
Three modes govern what happens on a violation:
|
|
237
|
+
Three modes govern what happens on a kind violation:
|
|
238
238
|
|
|
239
239
|
- **`trust`** — no validation. Today's pre-invariant behavior. Included as a debug escape hatch; discouraged in production.
|
|
240
240
|
- **`recover`** (default) — the bad value is skipped, the prior valid value stays in place, an error is emitted via `onError`, and the stream continues. One hallucinated field doesn't tank the rest of the response.
|
|
241
241
|
- **`strict`** — the bad value is skipped, an error is emitted once, the stream enters a terminal failed state. `current()` freezes at the last valid snapshot, further writes are no-ops, `final()` rejects. For when rendering wrong is worse than rendering nothing.
|
|
242
242
|
|
|
243
|
+
A constraint violation (phase 2) is found after the value has been written, so it is never skipped: in `recover` the value stays in `current()` and `final()` and the error is emitted; in `strict` the stream fails the same way, frozen with the violating value in it. `trust` runs neither phase.
|
|
244
|
+
|
|
243
245
|
### Consequences
|
|
244
246
|
|
|
245
|
-
- `current()`
|
|
246
|
-
- `current()` is
|
|
247
|
+
- A value of the wrong JSON kind is never written into `current()` (in `recover` and `strict`, for the schema kinds the walker checks — see Sub-schema walking), and every field `initial` holds keeps a value.
|
|
248
|
+
- An object the reply creates starts empty and fills in key by key. Until it closes, `current()` can lack keys the schema requires, and no error is raised for that; a complete reply that never sends a required key of such an object leaves `final()` without it (reported only under `constraints: 'finalize'`).
|
|
249
|
+
- `current()` is *not* always constraint-valid, mid-stream or at the end. `constraints: 'finalize'` reports a violation when the field closes; it does not take the value back.
|
|
247
250
|
- No throwing from `write()`. All error surfaces go through `onError` + `final()` rejection. Signal adapters and other producer loops can write without try/catch.
|
|
248
251
|
- Selected streams see only errors at-or-below their path. `select('widget').onError(...)` ignores errors on sibling fields.
|
|
249
252
|
|
package/README.md
CHANGED
|
@@ -11,7 +11,7 @@ Solid takes a different approach:
|
|
|
11
11
|
- **Incremental parser** — processes only new characters. No re-scanning.
|
|
12
12
|
- **Structural sharing** — snapshots reuse unchanged subtree references. No deep cloning.
|
|
13
13
|
- **Reference equality** — change detection is `===`, not deep comparison. O(1).
|
|
14
|
-
- **Always-valid invariant** — the parser kind-checks every value against the schema at the moment its type is known
|
|
14
|
+
- **Always-valid invariant** — the parser kind-checks every value against the schema at the moment its type is known, so a value of the wrong JSON kind is never written into `current()`, whatever the LLM sends. (`trust` mode switches the check off, and a field whose schema is `z.lazy`, an intersection or a pipe is not kind-checked.)
|
|
15
15
|
|
|
16
16
|
The result: linear scaling, 44x faster than naive repair+parse on a 10 KB payload, a clean subscription API that tells you exactly when each subtree finalizes, and a structural contract that holds even when the model hallucinates the wrong type for a field.
|
|
17
17
|
|
|
@@ -82,7 +82,7 @@ const result = await stream.final();
|
|
|
82
82
|
|
|
83
83
|
## Validation modes
|
|
84
84
|
|
|
85
|
-
Every stream enforces the structural invariant:
|
|
85
|
+
Every stream outside `trust` mode enforces the structural invariant: where the kind check applies, a value of the wrong JSON kind is not written into `current()`. When the LLM sends a value whose JSON kind doesn't match (a string where a number was expected, an array where an object lives, a field not in the schema), solid reacts based on `mode`:
|
|
86
86
|
|
|
87
87
|
```typescript
|
|
88
88
|
createStream({ schema, initial, mode: 'recover' }) // default
|
|
@@ -96,6 +96,8 @@ createStream({ schema, initial, mode: 'trust' })
|
|
|
96
96
|
| `strict` | Enter a terminal failed state on first violation. `current()` freezes, further writes are no-ops, `final()` rejects. |
|
|
97
97
|
| `trust` | No validation. Escape hatch for debugging or for producers you fully control. Discouraged. |
|
|
98
98
|
|
|
99
|
+
In a list, the prior value is whatever the list held at that position. Where it held nothing there, a rejected element that is followed by an accepted one leaves `undefined` at its position: `{"seats":["C4",5,"C6"]}` against `z.array(z.string())` with an empty initial list gives `['C4', undefined, 'C6']`, which the schema itself refuses. `onError` names the position (`seats.1`), and `constraints: 'finalize'` reports the list again when it closes. A rejected element in last place leaves the list one short instead.
|
|
100
|
+
|
|
99
101
|
### Constraint validation
|
|
100
102
|
|
|
101
103
|
Kind checks catch the structural foot-guns (map-over-a-string, add-to-a-string). For constraint checks — `.min`, `.max`, `.regex`, `.email`, `.int`, `.refine` — opt into `constraints: 'finalize'`:
|
|
@@ -104,13 +106,13 @@ Kind checks catch the structural foot-guns (map-over-a-string, add-to-a-string).
|
|
|
104
106
|
createStream({ schema, initial, constraints: 'finalize' })
|
|
105
107
|
```
|
|
106
108
|
|
|
107
|
-
This runs the sub-schema `safeParse` at the exact moment each field closes in the stream, so partial strings never trip constraints they'll eventually satisfy. Violations emit `onError` with `phase: 'finalize'`.
|
|
109
|
+
This runs the sub-schema `safeParse` at the exact moment each field closes in the stream, so partial strings never trip constraints they'll eventually satisfy. Violations emit `onError` with `phase: 'finalize'`. A violation is reported, not undone: the value stays in `current()` and `final()`. In `strict` mode it also fails the stream, which stays frozen with that value in it.
|
|
108
110
|
|
|
109
111
|
## How it works
|
|
110
112
|
|
|
111
|
-
1. **Base object** — validated against your Zod schema at construction. This is the starting state. Every field has a value from the start.
|
|
113
|
+
1. **Base object** — validated against your Zod schema at construction. This is the starting state. Every field it holds has a value from the start. An object the reply creates (a list row, an object where the base has `null` or nothing, a record entry) has no base: it starts empty and gets its keys as they arrive.
|
|
112
114
|
|
|
113
|
-
2. **Incremental parsing** — each `write(chunk)` feeds characters into a state machine that tracks JSON structure, extracts values, and emits structural events. No `JSON.parse`, no buffer re-scanning.
|
|
115
|
+
2. **Incremental parsing** — each `write(chunk)` feeds characters into a state machine that tracks JSON structure, extracts values, and emits structural events. No `JSON.parse`, no buffer re-scanning. What is written should be the JSON value and nothing else. A code fence around it is ignored, and so is text with no double quote, bracket or brace in it. A `"` in text before the JSON is read as a string where the root belongs: an `onError` (`[solid] <root>: expected object, got string`) in `recover`, a failed stream in `strict`. A bracket or brace in text before the JSON is read as the root itself: the stream ends there, on the initial value or, in `strict`, sometimes as a failure.
|
|
114
116
|
|
|
115
117
|
3. **Value-open validation** — the moment the parser detects a value's JSON kind (`{`, `[`, `"`, digit, `t`/`f`/`n`), it checks it against the schema. If the kind doesn't match (string where number expected, array where object lives), the parser enters **skip-mode** — consuming the entire bad subtree without writing, dirtying, or emitting anything. The prior valid value stays in place.
|
|
116
118
|
|
package/dist/index.cjs
CHANGED
|
@@ -108,6 +108,7 @@ var createIncrementalParser = (base, opts = {}) => {
|
|
|
108
108
|
}
|
|
109
109
|
};
|
|
110
110
|
const setValue = (value) => {
|
|
111
|
+
if (containerStack.length === 0) return;
|
|
111
112
|
const container = topContainer();
|
|
112
113
|
let existing;
|
|
113
114
|
if (topContainerType() === "object") {
|
|
@@ -466,6 +467,13 @@ var createIncrementalParser = (base, opts = {}) => {
|
|
|
466
467
|
isDirty = false;
|
|
467
468
|
return { changed: true, value };
|
|
468
469
|
};
|
|
470
|
+
const detachValue = (value) => {
|
|
471
|
+
if (Array.isArray(value)) return value.map(detachValue);
|
|
472
|
+
if (typeof value === "object" && value !== null) {
|
|
473
|
+
return Object.fromEntries(Object.entries(value).map(([key, child]) => [key, detachValue(child)]));
|
|
474
|
+
}
|
|
475
|
+
return value;
|
|
476
|
+
};
|
|
469
477
|
const shareStructure = (source, previous, path) => {
|
|
470
478
|
const dirtyKeys = dirtyParents.get(path);
|
|
471
479
|
if (!dirtyKeys) return previous;
|
|
@@ -476,7 +484,7 @@ var createIncrementalParser = (base, opts = {}) => {
|
|
|
476
484
|
const key = String(i);
|
|
477
485
|
if (dirtyKeys.has(key)) {
|
|
478
486
|
const childPath = path === "" ? key : `${path}.${key}`;
|
|
479
|
-
result[i] = dirtyParents.has(childPath) ? shareStructure(source[i], prevArr[i], childPath) : source[i];
|
|
487
|
+
result[i] = dirtyParents.has(childPath) ? shareStructure(source[i], prevArr[i], childPath) : detachValue(source[i]);
|
|
480
488
|
} else {
|
|
481
489
|
result[i] = i < prevArr.length ? prevArr[i] : source[i];
|
|
482
490
|
}
|
|
@@ -490,7 +498,7 @@ var createIncrementalParser = (base, opts = {}) => {
|
|
|
490
498
|
for (const key of Object.keys(sObj)) {
|
|
491
499
|
if (dirtyKeys.has(key)) {
|
|
492
500
|
const childPath = path === "" ? key : `${path}.${key}`;
|
|
493
|
-
result[key] = dirtyParents.has(childPath) ? shareStructure(sObj[key], pObj[key], childPath) : sObj[key];
|
|
501
|
+
result[key] = dirtyParents.has(childPath) ? shareStructure(sObj[key], pObj[key], childPath) : detachValue(sObj[key]);
|
|
494
502
|
} else {
|
|
495
503
|
result[key] = key in pObj ? pObj[key] : sObj[key];
|
|
496
504
|
}
|
|
@@ -731,6 +739,7 @@ var createSelectedStream = (deps) => {
|
|
|
731
739
|
const onFinal = (listener) => {
|
|
732
740
|
if (isDestroyed) return () => {
|
|
733
741
|
};
|
|
742
|
+
if (!finalState.resolved && deps.hasRootFinalized()) resolveFinal(project());
|
|
734
743
|
if (finalState.resolved) {
|
|
735
744
|
listener(finalState.value);
|
|
736
745
|
return () => {
|
|
@@ -761,6 +770,8 @@ var createSelectedStream = (deps) => {
|
|
|
761
770
|
errorListeners.clear();
|
|
762
771
|
pendingImmediateFires.length = 0;
|
|
763
772
|
if (!finalState.resolved) {
|
|
773
|
+
finalState.promise.catch(() => {
|
|
774
|
+
});
|
|
764
775
|
finalState.reject(new Error("[solid] stream destroyed"));
|
|
765
776
|
}
|
|
766
777
|
};
|
|
@@ -1058,6 +1069,7 @@ var createStream = (options) => {
|
|
|
1058
1069
|
let isDestroyed = false;
|
|
1059
1070
|
let isFailed = false;
|
|
1060
1071
|
let failureError = null;
|
|
1072
|
+
let hasNotifiedFinal = false;
|
|
1061
1073
|
const listeners = /* @__PURE__ */ new Set();
|
|
1062
1074
|
const finalListeners = /* @__PURE__ */ new Set();
|
|
1063
1075
|
const errorListeners = /* @__PURE__ */ new Set();
|
|
@@ -1122,6 +1134,7 @@ var createStream = (options) => {
|
|
|
1122
1134
|
for (const sub of finalizeSubscribers) {
|
|
1123
1135
|
sub();
|
|
1124
1136
|
}
|
|
1137
|
+
hasNotifiedFinal = true;
|
|
1125
1138
|
};
|
|
1126
1139
|
const write = (chunk) => {
|
|
1127
1140
|
if (isClosed || isDestroyed || isFailed) return;
|
|
@@ -1164,6 +1177,8 @@ var createStream = (options) => {
|
|
|
1164
1177
|
errorSubscribers.clear();
|
|
1165
1178
|
pendingImmediateFires.length = 0;
|
|
1166
1179
|
if (!finalState.resolved) {
|
|
1180
|
+
finalState.promise.catch(() => {
|
|
1181
|
+
});
|
|
1167
1182
|
finalState.reject(new Error("[solid] stream destroyed"));
|
|
1168
1183
|
}
|
|
1169
1184
|
for (const [, selected] of selectedStreams) {
|
|
@@ -1220,6 +1235,7 @@ var createStream = (options) => {
|
|
|
1220
1235
|
getRootValue: () => currentValue,
|
|
1221
1236
|
isTerminal: () => isClosed || isDestroyed,
|
|
1222
1237
|
isPathFinal: () => tracker.isFinal(path),
|
|
1238
|
+
hasRootFinalized: () => hasNotifiedFinal,
|
|
1223
1239
|
onRootChange: (listener) => {
|
|
1224
1240
|
changeSubscribers.add(listener);
|
|
1225
1241
|
return () => {
|