@zakkster/lite-stream 1.1.0 → 1.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +142 -3
- package/README.md +58 -69
- package/Stream.d.ts +13 -0
- package/Stream.js +36 -19
- package/llms.txt +40 -20
- package/package.json +11 -7
- package/ROADMAP.md +0 -239
package/CHANGELOG.md
CHANGED
|
@@ -7,12 +7,149 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
---
|
|
9
9
|
|
|
10
|
+
## 1.2.0 -- 2026-09-02
|
|
11
|
+
|
|
12
|
+
**The law's gate, able to fail.** A torture harness that proves the package's
|
|
13
|
+
own claims -- including the zero-steady-state-allocation sentence that has been
|
|
14
|
+
unverified since 1.0.0 -- plus the drift guards that keep docs, exports, and
|
|
15
|
+
ASCII honest, a `VERSION` export, and the LS-11 packaging cleanup. No behavior
|
|
16
|
+
change: the only `Stream.js` code change is the additive `VERSION` const.
|
|
17
|
+
|
|
18
|
+
### Added
|
|
19
|
+
|
|
20
|
+
- **Torture harness** -- `test/torture.mjs` + `test/torture/s0-s8`, run by
|
|
21
|
+
`npm run torture` (`node --expose-gc test/torture.mjs`): metamorphic laws
|
|
22
|
+
(s0), degenerate values and hostile iterators (s1), async-iteration protocol
|
|
23
|
+
conformance (s2), API abuse with a decided policy per case (s3), seeded
|
|
24
|
+
fuzz-vs-oracle (s4), the allocation gate (s5, `@zakkster/lite-gc-profiler`,
|
|
25
|
+
`maxMajor` 0 with per-scenario byte budgets), a retention soak (s6,
|
|
26
|
+
`@zakkster/lite-leak` WeakRef census), the StreamQuery conformance corpus and
|
|
27
|
+
TimeoutError duck-pact (s7), and the controls tier (s8). Prints `ok`, exits
|
|
28
|
+
0/1; `TORTURE_SEED` replays a failure; `STREAM_TORTURE_BREAK=<s0..s8|1>` arms
|
|
29
|
+
each tier's injected breakage so every tier's control exits non-zero.
|
|
30
|
+
- **Scripts** -- `torture`, `torture:control` (all-tiers break lane), and
|
|
31
|
+
`test:floor`. `verify` gains `torture`. **LS-09 amendment:** the floor lane
|
|
32
|
+
runs `npm test` ONLY against the pinned peer -- `@zakkster/lite-leak` peers
|
|
33
|
+
`@zakkster/lite-signal >= 1.5.0`, so pinning `1.2.2` alongside it would put
|
|
34
|
+
lite-leak outside its supported peer range; torture (which imports lite-leak)
|
|
35
|
+
runs against the resolved latest only. Fail closed: never run a tool outside
|
|
36
|
+
its peer range.
|
|
37
|
+
- **`VERSION` export** (LS-05) -- a string that always equals the installed
|
|
38
|
+
`package.json` version. Declared in `Stream.d.ts`; documented in llms.txt and
|
|
39
|
+
the README API reference with no embedded version literal. Never re-exported
|
|
40
|
+
by consumers (suite ruling). The `Stream.d.ts` `TimeoutError` type declaration
|
|
41
|
+
was also added (the shipped runtime export had no type; the surface-guard
|
|
42
|
+
requires three-way parity).
|
|
43
|
+
- **`test/09-guards.test.mjs`** (5 tests) -- ascii-guard (every `files[]` +
|
|
44
|
+
`test/`/`bench/`/`demo/` byte is printable ASCII or LF, empty allowlist) and
|
|
45
|
+
surface-guard (runtime exports == llms.txt import list == `Stream.d.ts` value
|
|
46
|
+
exports; `VERSION` string-equals `package.json` version; the declared peer is
|
|
47
|
+
present in both docs). Each family carries an inline failing control.
|
|
48
|
+
- **devDependencies** -- `@zakkster/lite-leak ^1.10.0` and
|
|
49
|
+
`@zakkster/lite-gc-profiler ^1.16.0` (torture only; not runtime deps).
|
|
50
|
+
|
|
51
|
+
### Changed
|
|
52
|
+
|
|
53
|
+
- **`files[]` drops `ROADMAP.md`** (LS-11) -- the roadmap is a working doc, not
|
|
54
|
+
a shipped artifact. llms.txt's Files section and the CHANGELOG [1.1.0]
|
|
55
|
+
forward-reference were reworded so tarball readers are not pointed at an
|
|
56
|
+
unshipped file.
|
|
57
|
+
- **`test` / `test:gc` globs scoped to `test/*.test.mjs`** so the `node:test`
|
|
58
|
+
runner does not discover the torture files.
|
|
59
|
+
|
|
60
|
+
### Fixed
|
|
61
|
+
|
|
62
|
+
- Docs and packaging only; no runtime behavior changed.
|
|
63
|
+
|
|
64
|
+
### Known issues
|
|
65
|
+
|
|
66
|
+
- **LS-13** (S1, opened this release, fix targeted for 1.3.0) -- a `next()`
|
|
67
|
+
result object whose `done` or `value` is a throwing getter escapes the pump's
|
|
68
|
+
`.then` onFulfilled uncaught: it surfaces as an unhandledRejection and
|
|
69
|
+
`fromAsyncIterable` never reaches a terminal state (it should transition to
|
|
70
|
+
the error state). Torture tier s1 reproduces it with a bounded sentinel and a
|
|
71
|
+
scoped rejection capture and prints it as `TODO(LS3, LS-13)` in the gate
|
|
72
|
+
output; it does not fail the gate. This refutes the "iterator throws -> error
|
|
73
|
+
state" triplet leg for the throwing-getter shape specifically.
|
|
74
|
+
|
|
75
|
+
### Testing
|
|
76
|
+
|
|
77
|
+
- **90 tests** across `test/01-*` through `test/09-*` (up from 85), 0 fail,
|
|
78
|
+
0 skip under `npm test` and `npm run test:gc`.
|
|
79
|
+
- `npm run torture` -> `ok`, ~1.8s wall (node v26.3.1, macOS arm64,
|
|
80
|
+
2026-09-02); zero steady-state majors in every s5 scenario.
|
|
81
|
+
- `npm run test:floor` -> FLOOR-PASS (`npm test` 90/0 against
|
|
82
|
+
`@zakkster/lite-signal@1.2.2`) and LATEST-PASS (`npm test` 90/0 + torture ok
|
|
83
|
+
against the resolved `@zakkster/lite-signal@1.5.0`). The `^1.2.0` peer floor
|
|
84
|
+
is now a tested fact, not a claim.
|
|
85
|
+
- s5 budget doctrine: per-scenario `maxBytesPerOp` budgets (32 B/op for the
|
|
86
|
+
caught-up toAsyncIterable and pipeToSignal steady pump; 16 B/op for the
|
|
87
|
+
by-design fromAsyncIterable state-object and waiter-heavy paths), each ~3x the
|
|
88
|
+
measured 3-pass peak; the global `maxMajor` rule never widens.
|
|
89
|
+
|
|
90
|
+
---
|
|
91
|
+
|
|
92
|
+
## 1.1.1 -- 2026-09-01
|
|
93
|
+
|
|
94
|
+
**Truth patch. No API change, no behavior change; the `Stream.js` diff is
|
|
95
|
+
comment-only.** Every shipped sentence the runnable probes could refute is
|
|
96
|
+
corrected, and the previously dark GC tier now gates on the default test run.
|
|
97
|
+
|
|
98
|
+
### Fixed
|
|
99
|
+
|
|
100
|
+
- **dispose-teardown claim (LS-01)** -- llms.txt, README, the `Stream.js`
|
|
101
|
+
`fromAsyncIterable` docstring, and the `package.json` description all
|
|
102
|
+
claimed the pump detects a disposed result signal on the next `sig.set()`
|
|
103
|
+
and cleans up silently. It never did: `set()` after `dispose()` is a
|
|
104
|
+
silent no-op in lite-signal 1.2.2 and 1.5.0, so the pump keeps pulling and
|
|
105
|
+
neither `iterator.return()` nor any callback fires. Corrected to the true
|
|
106
|
+
behavior -- AbortSignal or natural completion are the only stop mechanisms.
|
|
107
|
+
The defensive `catch` around `set()` and its comment were re-framed
|
|
108
|
+
accordingly.
|
|
109
|
+
- **phantom contract doc + session narrative (LS-06)** -- three references
|
|
110
|
+
to a non-existent `CONTRACT-toAsyncIterable-1.1.md` (in `Stream.js` and
|
|
111
|
+
`test/07-to-async-iterable-rich.test.mjs`) and the "Session A/Session B"
|
|
112
|
+
development narrative were removed; the locked 1.1 semantics now point at
|
|
113
|
+
CHANGELOG.md [1.1.0] and llms.txt.
|
|
114
|
+
- **pump deferral comment (LS-07)** -- the comment claimed a microtask
|
|
115
|
+
deferral that is not in the code. `pump()` is called directly; the stack
|
|
116
|
+
stays bounded because the continuation runs from a `.then()` handler.
|
|
117
|
+
- **perf numbers (LS-08)** -- the shipped ops/s figures named no node
|
|
118
|
+
version, date, or machine and no longer matched a current run. Re-measured
|
|
119
|
+
once with provenance; llms.txt (Performance) is now the single source of
|
|
120
|
+
truth and README cites it. Four 1.1 `toAsyncIterable` bench scenarios
|
|
121
|
+
(`latest-1to1-x10`, `latest-churn-100`, `filter-timeout`, `multi-waiter-x3`)
|
|
122
|
+
were removed from the docs as unreproducible -- no runnable source for them
|
|
123
|
+
exists in the repo; only the six scenarios `bench/bench.mjs` runs are cited.
|
|
124
|
+
|
|
125
|
+
### Added
|
|
126
|
+
|
|
127
|
+
- **`test/08-dispose-behavior.test.mjs`** -- 6 tests pinning the TRUE dispose
|
|
128
|
+
semantics (LS-01: disposing a result/target signal does not stop the pump;
|
|
129
|
+
LS-04: a pending `next()` on a disposed source never settles, `timeout` is
|
|
130
|
+
the escape hatch).
|
|
131
|
+
- **`--expose-gc` on the default `npm test`** (LS-02) -- the 8 GC-tier tests
|
|
132
|
+
were dark under the default gate (71 run, 8 skipped); they now run on every
|
|
133
|
+
`npm test` (85 total, 0 skipped on Node 20+). `npm run test:gc` is retained
|
|
134
|
+
as an alias.
|
|
135
|
+
- **no-lockfile policy** (LS-10) -- one sentence in the README Testing
|
|
136
|
+
strategy documents why no `package-lock.json` is committed and how the peer
|
|
137
|
+
floor is verified.
|
|
138
|
+
|
|
139
|
+
### Testing
|
|
140
|
+
|
|
141
|
+
- 85 tests across `test/01-*` through `test/08-*` (up from 79 in 1.1.0). All
|
|
142
|
+
pass under `npm test` and `npm run test:gc`; per-scenario bench numbers with
|
|
143
|
+
provenance live in llms.txt (no ops/s digits are pinned in this changelog).
|
|
144
|
+
|
|
145
|
+
---
|
|
146
|
+
|
|
10
147
|
## 1.1.0 -- 2026-07-10
|
|
11
148
|
|
|
12
149
|
**Additive `toAsyncIterable` enrichment.** Six additions, no breaking
|
|
13
|
-
changes. All 1.0.0 callers continue to work without modification.
|
|
14
|
-
|
|
15
|
-
|
|
150
|
+
changes. All 1.0.0 callers continue to work without modification. Each
|
|
151
|
+
addition below records the triggering signal that justified it; the
|
|
152
|
+
`Added` and `Changed` entries in this section stand on their own.
|
|
16
153
|
|
|
17
154
|
### Added
|
|
18
155
|
|
|
@@ -148,6 +285,8 @@ All three paths handled structurally; abort listeners always removed.
|
|
|
148
285
|
Additionally: if the consumer disposes the result signal externally, the
|
|
149
286
|
pump detects on next `sig.set()` and tears down silently (no `onError` --
|
|
150
287
|
this is consumer-initiated cleanup).
|
|
288
|
+
(Superseded in 1.1.1: this claim was never true -- see [1.1.1] and ROADMAP
|
|
289
|
+
LS-01.)
|
|
151
290
|
|
|
152
291
|
### Performance (Node v22, --expose-gc, 500ms runs)
|
|
153
292
|
|
package/README.md
CHANGED
|
@@ -116,10 +116,11 @@ The abort listener is registered with `addEventListener("abort", ...)` and
|
|
|
116
116
|
**always removed** on any of the three paths -- no AbortSignal accumulates
|
|
117
117
|
dangling listeners across stream lifecycles.
|
|
118
118
|
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
119
|
+
Disposing the result signal does NOT stop the pump: the iterator keeps
|
|
120
|
+
pulling. AbortSignal or natural completion are the only stop mechanisms.
|
|
121
|
+
`iterator.return()` is never called and no callback fires, because
|
|
122
|
+
`lite-signal`'s `set()` after `dispose()` is a silent no-op -- the pump never
|
|
123
|
+
learns of the disposal. Abort `opts.signal` to stop early. See ROADMAP LS-01.
|
|
123
124
|
|
|
124
125
|
## API reference
|
|
125
126
|
|
|
@@ -182,6 +183,8 @@ Use `pipeToSignal` when:
|
|
|
182
183
|
- You want a `stop` fn instead of an `AbortController` for cleanup
|
|
183
184
|
|
|
184
185
|
**Does NOT dispose the target signal.** The caller owns its lifetime.
|
|
186
|
+
Disposing the target signal does NOT stop the pump: call the returned stop
|
|
187
|
+
fn or abort `opts.signal`.
|
|
185
188
|
|
|
186
189
|
### toAsyncIterable
|
|
187
190
|
|
|
@@ -246,6 +249,16 @@ Treat this as the secondary API for classic "consume signal changes"
|
|
|
246
249
|
patterns; for the streaming pipeline case (paginated APIs, SSE, pubsub)
|
|
247
250
|
the forward direction is what most consumers want.
|
|
248
251
|
|
|
252
|
+
### VERSION
|
|
253
|
+
|
|
254
|
+
```ts
|
|
255
|
+
VERSION: string
|
|
256
|
+
```
|
|
257
|
+
|
|
258
|
+
A string that always equals the installed `package.json` version. Consumers
|
|
259
|
+
do NOT re-export it (suite ruling); the surface-guard asserts the equality on
|
|
260
|
+
every test run.
|
|
261
|
+
|
|
249
262
|
## Modes: latest vs buffer
|
|
250
263
|
|
|
251
264
|
Pick `"latest"` when you only care about the most recent value:
|
|
@@ -305,15 +318,8 @@ aborts on detach, `toAsyncIterable` catches the abort and calls
|
|
|
305
318
|
## Zero-GC hot paths
|
|
306
319
|
|
|
307
320
|
`lite-stream`'s hot paths -- per-yield state allocation and per-pull abort
|
|
308
|
-
checks -- minimize per-op allocation.
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
```
|
|
312
|
-
from-latest-10 178K ops/s 0.07 B/op
|
|
313
|
-
from-buffer-20-drop 128K ops/s 0.13 B/op
|
|
314
|
-
abort-cycle 84K ops/s 0.71 B/op
|
|
315
|
-
pipe-to-signal 235K ops/s 0.10 B/op
|
|
316
|
-
```
|
|
321
|
+
checks -- minimize per-op allocation. Current numbers with provenance live in
|
|
322
|
+
`llms.txt` (Performance); re-run `npm run bench` on your target.
|
|
317
323
|
|
|
318
324
|
Per-yield, `"latest"` mode allocates exactly one wrapper state object. The
|
|
319
325
|
underlying signal node is reused via lite-signal's pool. `"buffer"` mode
|
|
@@ -471,8 +477,9 @@ const ws = new WebSocket("wss://logs.example.com/ingest");
|
|
|
471
477
|
- **Sync `Iterable<T>`**: accepted via `Symbol.iterator`. Pulls happen
|
|
472
478
|
microtask-asynchronously even for sync sources, so caller's `effect()`
|
|
473
479
|
can wire up before the first yield.
|
|
474
|
-
- **Consumer disposes the signal mid-stream**: pump
|
|
475
|
-
`
|
|
480
|
+
- **Consumer disposes the signal mid-stream**: the pump keeps pulling.
|
|
481
|
+
`iterator.return()` is not called and no callback fires; abort
|
|
482
|
+
`opts.signal` to stop.
|
|
476
483
|
- **Buffer mode snapshot array**: a fresh array per yield. Past observers
|
|
477
484
|
hold stable, non-mutated references.
|
|
478
485
|
- **Disposing the signal multiple times**: lite-signal's `dispose` is
|
|
@@ -481,62 +488,15 @@ const ws = new WebSocket("wss://logs.example.com/ingest");
|
|
|
481
488
|
|
|
482
489
|
## Benchmarks
|
|
483
490
|
|
|
484
|
-
Run via `npm run bench` (requires `--expose-gc`).
|
|
485
|
-
|
|
486
|
-
|
|
487
|
-
[1] fromAsyncIterable latest mode -- 10-item async gen
|
|
488
|
-
from-latest-10 178,554 ops/s retained: 0.07 B/op
|
|
489
|
-
|
|
490
|
-
[2] fromAsyncIterable buffer mode -- 20-item gen, maxBuffer 5
|
|
491
|
-
from-buffer-20-drop 128,630 ops/s retained: 0.13 B/op
|
|
492
|
-
|
|
493
|
-
[3] Abort cycle -- infinite gen, abort after first yield
|
|
494
|
-
abort-cycle 84,102 ops/s retained: 0.71 B/op
|
|
495
|
-
|
|
496
|
-
[4] pipeToSignal -- pump existing signal directly
|
|
497
|
-
pipe-to-signal 235,376 ops/s retained: 0.10 B/op
|
|
498
|
-
|
|
499
|
-
[5] toAsyncIterable -- 10 signal updates, consumer drains in order
|
|
500
|
-
to-async-iterable 42,480 ops/s retained: ~0 B/op
|
|
501
|
-
|
|
502
|
-
[6] toAsyncIterable -- producer overflow, drop-oldest
|
|
503
|
-
to-async-iterable-overflow 62,164 ops/s retained: ~25 B/op
|
|
504
|
-
|
|
505
|
-
--- 1.1 additions ---
|
|
506
|
-
|
|
507
|
-
[7] toAsyncIterable mode:latest -- 10 write/next pairs per cycle
|
|
508
|
-
to-async-iterable-latest-1to1-x10 119,278 ops/s retained: 0.12 B/op
|
|
509
|
-
|
|
510
|
-
[8] toAsyncIterable mode:latest -- 100 writes coalesced, single read
|
|
511
|
-
to-async-iterable-latest-churn-100 60,674 ops/s retained: 0.28 B/op
|
|
512
|
-
|
|
513
|
-
[9] toAsyncIterable filter + timeout -- consumer wins before timeout
|
|
514
|
-
to-async-iterable-filter-timeout 141,080 ops/s retained: -11.80 B/op
|
|
515
|
-
|
|
516
|
-
[10] toAsyncIterable multi-waiter -- 3 concurrent next() calls
|
|
517
|
-
to-async-iterable-multi-waiter-x3 120,020 ops/s retained: -0.01 B/op
|
|
518
|
-
```
|
|
519
|
-
|
|
520
|
-
(Node v22, 150ms warmup, 500ms runs.)
|
|
521
|
-
|
|
522
|
-
The full lifecycle (construct, pull, settle, dispose) of `fromAsyncIterable`
|
|
523
|
-
in latest mode runs at 178K ops/s with sub-byte/op retained heap. Per-pull
|
|
524
|
-
allocation is dominated by the iterator's own microtask cost, not by
|
|
525
|
-
lite-stream's wrapper state.
|
|
526
|
-
|
|
527
|
-
1.1's `toAsyncIterable mode:"latest"` amortizes to ~2.4M ops/s per operation
|
|
528
|
-
(divide the reported cycle rate by the inner loop count). The filter+timeout
|
|
529
|
-
scenario `[9]` runs at 141K ops/s with NEGATIVE retained heap -- the
|
|
530
|
-
timer-clear discipline on `iter.return()` lets V8's subsequent GC pass
|
|
531
|
-
reclaim more than the pre-run measurement. The multi-waiter scenario `[10]`
|
|
532
|
-
confirms the 1.0.0 latent-bug fix (single-slot `pendingResolve` overwriting
|
|
533
|
-
the first resolver) works cleanly under bench load.
|
|
491
|
+
Run via `npm run bench` (requires `--expose-gc`). Current numbers with
|
|
492
|
+
provenance live in `llms.txt` (Performance); re-run `npm run bench` on your
|
|
493
|
+
target.
|
|
534
494
|
|
|
535
495
|
## Testing strategy
|
|
536
496
|
|
|
537
497
|
### Tier 1 -- behavior (unit tests, fast)
|
|
538
498
|
|
|
539
|
-
|
|
499
|
+
90 tests across `test/01-*` through `test/09-*`:
|
|
540
500
|
|
|
541
501
|
- `01-from-async-iterable-latest.test.mjs` -- state shape, lifecycle,
|
|
542
502
|
Iterable acceptance variants, pre-aborted, subscriber observability
|
|
@@ -552,6 +512,15 @@ the first resolver) works cleanly under bench load.
|
|
|
552
512
|
filter (including throwing-filter no-writer-surface discipline), timeout
|
|
553
513
|
+ TimeoutError, Symbol.asyncDispose (Node 20+ guarded), multi-waiter
|
|
554
514
|
queue, `overflowCount` alias, 4K structural cleanup cycles
|
|
515
|
+
- `08-dispose-behavior.test.mjs` -- pins the TRUE dispose semantics
|
|
516
|
+
(LS-01: disposing a result/target signal does NOT stop the pump;
|
|
517
|
+
LS-04: a pending `next()` on a disposed source never settles, `timeout`
|
|
518
|
+
is the only escape hatch)
|
|
519
|
+
- `09-guards.test.mjs` -- drift guards: ascii-guard (every shipped +
|
|
520
|
+
test/bench/demo byte is printable ASCII or LF) and surface-guard (runtime
|
|
521
|
+
exports == llms.txt == Stream.d.ts, VERSION == package.json version, the
|
|
522
|
+
declared peer is present in both docs). Each family carries an inline
|
|
523
|
+
failing control proving the guard can fail.
|
|
555
524
|
|
|
556
525
|
Run via `npm test`.
|
|
557
526
|
|
|
@@ -562,12 +531,32 @@ ceilings at 2 MB for the 1.0.0 paths (2K latest, 1K buffer, 1K abort) and
|
|
|
562
531
|
the 1.1 additions (5K `mode:"latest"` resolve, 3K timeout, 1K buffer
|
|
563
532
|
fill+drain, 2K `mode:"latest"` churn, 2K filter-throw).
|
|
564
533
|
|
|
565
|
-
|
|
534
|
+
The GC tier now runs under the default `npm test` (the script carries
|
|
535
|
+
`--expose-gc`), so these budgets gate on every run; `npm run test:gc` is
|
|
536
|
+
retained as an alias.
|
|
566
537
|
|
|
567
538
|
### Tier 3 -- performance (measured throughput)
|
|
568
539
|
|
|
569
|
-
`bench/bench.mjs` --
|
|
570
|
-
|
|
540
|
+
`bench/bench.mjs` -- six scenarios; throughput and B/op retained. Run via
|
|
541
|
+
`npm run bench`.
|
|
542
|
+
|
|
543
|
+
### Tier 4 -- torture (retention + alloc gate + controls)
|
|
544
|
+
|
|
545
|
+
`npm run torture` (`node --expose-gc test/torture.mjs`) runs tiers s0-s8
|
|
546
|
+
sequentially -- metamorphic laws, degenerate values, protocol conformance,
|
|
547
|
+
API abuse, seeded fuzz-vs-oracle, the s5 per-scenario allocation gate
|
|
548
|
+
(maxMajor 0), a lite-leak retention soak, the StreamQuery conformance corpus,
|
|
549
|
+
and the controls tier. It prints `ok` and exits 0 on success.
|
|
550
|
+
`STREAM_TORTURE_BREAK=<s0..s8|1>` arms one tier's (or every tier's) injected
|
|
551
|
+
breakage so the control lane exits non-zero; `npm run torture:control` runs
|
|
552
|
+
the all-tiers form. `npm run test:floor` runs `npm test` against the peer
|
|
553
|
+
floor `@zakkster/lite-signal@1.2.2` and then `npm test` + torture against the
|
|
554
|
+
resolved latest, printing both verdicts.
|
|
555
|
+
|
|
556
|
+
No `package-lock.json` is committed: devDependencies float within their
|
|
557
|
+
stated ranges, and correctness against the peer floor is proven by running
|
|
558
|
+
`npm run test:floor` against `@zakkster/lite-signal@1.2.2` rather than by
|
|
559
|
+
pinning a lockfile.
|
|
571
560
|
|
|
572
561
|
## What this is not
|
|
573
562
|
|
package/Stream.d.ts
CHANGED
|
@@ -135,3 +135,16 @@ export function toAsyncIterable<T>(
|
|
|
135
135
|
sig: Signal<T> | Computed<T>,
|
|
136
136
|
opts?: ToAsyncIterableOptions
|
|
137
137
|
): AsyncIterable<T> & { readonly droppedCount: number };
|
|
138
|
+
|
|
139
|
+
/**
|
|
140
|
+
* Thrown when the `timeout` option on `toAsyncIterable` elapses. Structurally
|
|
141
|
+
* identical to `@zakkster/lite-await`'s TimeoutError (`e.name === "TimeoutError"`
|
|
142
|
+
* duck-checks work across both packages); not imported -- lite-stream stays zero-dep.
|
|
143
|
+
*/
|
|
144
|
+
export declare class TimeoutError extends Error {
|
|
145
|
+
readonly name: "TimeoutError";
|
|
146
|
+
readonly timeout: number;
|
|
147
|
+
}
|
|
148
|
+
|
|
149
|
+
/** Package version string; always equals the installed package.json version. */
|
|
150
|
+
export declare const VERSION: string;
|
package/Stream.js
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
// @zakkster/lite-stream 1.
|
|
1
|
+
// @zakkster/lite-stream 1.2.0
|
|
2
2
|
//
|
|
3
3
|
// Zero-GC bridge between async iterators and @zakkster/lite-signal. The
|
|
4
4
|
// multi-shot dual of lite-await's fromPromise: project an async source of N
|
|
@@ -22,7 +22,8 @@
|
|
|
22
22
|
//
|
|
23
23
|
// 1.1 added `mode: "latest"`, `filter`, `timeout`, `Symbol.asyncDispose`,
|
|
24
24
|
// multi-waiter FIFO queue, and `overflowCount` on `toAsyncIterable`. See
|
|
25
|
-
//
|
|
25
|
+
// CHANGELOG.md [1.1.0] and llms.txt (toAsyncIterable section) for the full
|
|
26
|
+
// 1.1 story.
|
|
26
27
|
//
|
|
27
28
|
// Copyright (c) 2026 Zahary Shinikchiev <shinikchiev@yahoo.com>
|
|
28
29
|
// MIT License
|
|
@@ -40,9 +41,10 @@ import { signal as _signal } from "@zakkster/lite-signal";
|
|
|
40
41
|
* duck-check via `e.name === "TimeoutError"` across both packages -- but
|
|
41
42
|
* NOT imported from lite-await, to preserve lite-stream's zero-dep story.
|
|
42
43
|
*
|
|
43
|
-
*
|
|
44
|
-
*
|
|
45
|
-
*
|
|
44
|
+
* A structural twin of lite-await's TimeoutError, deliberately not imported
|
|
45
|
+
* (zero-dep by design), carrying `name === "TimeoutError"` and a `.timeout`
|
|
46
|
+
* field (the elapsed deadline in ms). The duck-check is suite-wide vocabulary;
|
|
47
|
+
* renaming it would be a breaking change everywhere it is consumed.
|
|
46
48
|
*/
|
|
47
49
|
class TimeoutError extends Error {
|
|
48
50
|
constructor(timeoutMs) {
|
|
@@ -145,11 +147,11 @@ function closeIterator(iter) {
|
|
|
145
147
|
* - iterator throws
|
|
146
148
|
* - opts.signal aborts (the iterator's return() is called best-effort)
|
|
147
149
|
*
|
|
148
|
-
*
|
|
149
|
-
*
|
|
150
|
-
*
|
|
151
|
-
*
|
|
152
|
-
*
|
|
150
|
+
* Disposing the result signal does NOT stop the pump: the iterator keeps
|
|
151
|
+
* pulling. AbortSignal or natural completion are the only stop mechanisms.
|
|
152
|
+
* A disposed signal's set() is a silent no-op, so the pump never learns of
|
|
153
|
+
* the disposal; an infinite source becomes an unbounded background loop.
|
|
154
|
+
* Pass opts.signal and abort it to stop early. See ROADMAP LS-01.
|
|
153
155
|
*
|
|
154
156
|
* @template T
|
|
155
157
|
* @param {AsyncIterable<T> | AsyncIterator<T> | Iterable<T>} source
|
|
@@ -301,9 +303,11 @@ function fromAsyncIterable(source, opts) {
|
|
|
301
303
|
}
|
|
302
304
|
const v = result.value;
|
|
303
305
|
// Build the new state from lastState (never sig.peek), then
|
|
304
|
-
// commit to both lastState and sig.
|
|
305
|
-
//
|
|
306
|
-
//
|
|
306
|
+
// commit to both lastState and sig. The try/catch around set()
|
|
307
|
+
// is defensive against a peer whose set() throws; on a throw we
|
|
308
|
+
// bail and tear down the iterator rather than pull into the
|
|
309
|
+
// void. Disposal is NOT the trigger -- set() after dispose() is
|
|
310
|
+
// a silent no-op in lite-signal 1.2.2 and 1.5.0 (see LS-01).
|
|
307
311
|
let newState;
|
|
308
312
|
if (mode === "latest") {
|
|
309
313
|
newState = {
|
|
@@ -341,16 +345,22 @@ function fromAsyncIterable(source, opts) {
|
|
|
341
345
|
try {
|
|
342
346
|
sig.set(newState);
|
|
343
347
|
} catch (_e) {
|
|
344
|
-
//
|
|
345
|
-
//
|
|
346
|
-
//
|
|
348
|
+
// Defensive against a peer signal whose set() throws. This
|
|
349
|
+
// is NOT disposal detection: set() after dispose() is a
|
|
350
|
+
// silent no-op in lite-signal 1.2.2 and 1.5.0 (probe P0),
|
|
351
|
+
// so this branch has never fired against any published
|
|
352
|
+
// peer. If it ever does, tear down without firing onError.
|
|
347
353
|
stopped = true;
|
|
348
354
|
cleanup();
|
|
349
355
|
closeIterator(iter);
|
|
350
356
|
return;
|
|
351
357
|
}
|
|
352
|
-
//
|
|
353
|
-
//
|
|
358
|
+
// pump() is called DIRECTLY here -- no explicit deferral at
|
|
359
|
+
// this site. The stack is already unwound because this
|
|
360
|
+
// continuation runs from a .then() handler, i.e. a fresh
|
|
361
|
+
// microtask. Proven safe: 1,000,000 synchronous values drained
|
|
362
|
+
// in ~80ms with no stack overflow (probe P4, node v26.3.1,
|
|
363
|
+
// 2026-09-01).
|
|
354
364
|
pump();
|
|
355
365
|
},
|
|
356
366
|
function (err) {
|
|
@@ -420,6 +430,8 @@ function makeErrorState(prevState, err, mode) {
|
|
|
420
430
|
* - You want a stop fn instead of an AbortController for cleanup
|
|
421
431
|
*
|
|
422
432
|
* NOTE: this does NOT dispose the target signal. The caller owns its lifetime.
|
|
433
|
+
* Disposing the target signal does NOT stop the pump: call the returned
|
|
434
|
+
* stop fn or abort opts.signal. See ROADMAP LS-01.
|
|
423
435
|
*
|
|
424
436
|
* @template T
|
|
425
437
|
* @param {AsyncIterable<T> | AsyncIterator<T>} source
|
|
@@ -557,7 +569,8 @@ const ASYNC_DISPOSE = (typeof Symbol !== "undefined" && Symbol.asyncDispose)
|
|
|
557
569
|
* where Promise.all([iter.next(), iter.next()]) silently lost the first
|
|
558
570
|
* resolver.
|
|
559
571
|
*
|
|
560
|
-
* See
|
|
572
|
+
* See CHANGELOG.md [1.1.0] and llms.txt (toAsyncIterable section) for the
|
|
573
|
+
* full locked semantics.
|
|
561
574
|
*
|
|
562
575
|
* @template T
|
|
563
576
|
* @param {import("@zakkster/lite-signal").Signal<T> | import("@zakkster/lite-signal").Computed<T>} sig
|
|
@@ -860,3 +873,7 @@ export {
|
|
|
860
873
|
toAsyncIterable,
|
|
861
874
|
TimeoutError
|
|
862
875
|
};
|
|
876
|
+
|
|
877
|
+
// Always equals the installed package.json version (single source of truth is
|
|
878
|
+
// package.json; the /release drill bumps both sites in the same commit).
|
|
879
|
+
export const VERSION = "1.2.0";
|
package/llms.txt
CHANGED
|
@@ -14,7 +14,7 @@ Peer dep: `@zakkster/lite-signal ^1.2.0`.
|
|
|
14
14
|
|
|
15
15
|
## Imports
|
|
16
16
|
|
|
17
|
-
import { fromAsyncIterable, pipeToSignal, toAsyncIterable, TimeoutError } from "@zakkster/lite-stream";
|
|
17
|
+
import { fromAsyncIterable, pipeToSignal, toAsyncIterable, TimeoutError, VERSION } from "@zakkster/lite-stream";
|
|
18
18
|
|
|
19
19
|
## Exports
|
|
20
20
|
|
|
@@ -61,7 +61,9 @@ const stop = pipeToSignal(source, targetSig, {
|
|
|
61
61
|
});
|
|
62
62
|
```
|
|
63
63
|
|
|
64
|
-
Does NOT dispose the target signal. The caller owns its lifetime.
|
|
64
|
+
Does NOT dispose the target signal. The caller owns its lifetime. Disposing
|
|
65
|
+
the target signal does NOT stop the pump: call the returned stop fn or abort
|
|
66
|
+
opts.signal.
|
|
65
67
|
|
|
66
68
|
### `toAsyncIterable(sig, opts?) -> AsyncIterable<T> & { droppedCount, overflowCount }`
|
|
67
69
|
|
|
@@ -118,6 +120,12 @@ Structurally identical to `@zakkster/lite-await`'s TimeoutError so
|
|
|
118
120
|
`e.name === "TimeoutError"` duck-checks work across both packages. Not
|
|
119
121
|
imported from lite-await -- lite-stream stays zero-dep.
|
|
120
122
|
|
|
123
|
+
### `VERSION`
|
|
124
|
+
|
|
125
|
+
String, always equals the installed package.json version. Not re-exported
|
|
126
|
+
by consumers (suite ruling); the surface-guard asserts the equality on
|
|
127
|
+
every test run.
|
|
128
|
+
|
|
121
129
|
## Cleanup termination triplet
|
|
122
130
|
|
|
123
131
|
`fromAsyncIterable` ends on exactly three paths, all structural:
|
|
@@ -129,9 +137,13 @@ imported from lite-await -- lite-stream stays zero-dep.
|
|
|
129
137
|
The abort listener is always removed on any of the three paths -- no
|
|
130
138
|
AbortSignal accumulates dangling listeners.
|
|
131
139
|
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
140
|
+
Disposing the result signal does NOT stop the pump: the iterator keeps
|
|
141
|
+
pulling. AbortSignal or natural completion are the only stop mechanisms.
|
|
142
|
+
`iterator.return()` is never called and neither `onError` nor `onDone` ever
|
|
143
|
+
fire, because lite-signal's `set()` after `dispose()` is a silent no-op (in
|
|
144
|
+
1.2.2 and 1.5.0), so the pump never learns of the disposal. For an infinite
|
|
145
|
+
source (SSE, a live feed) this turns the stream into an unbounded background
|
|
146
|
+
loop. Abort `opts.signal` to stop early. See ROADMAP LS-01.
|
|
135
147
|
|
|
136
148
|
## Key invariants
|
|
137
149
|
|
|
@@ -243,36 +255,44 @@ states: {
|
|
|
243
255
|
true}` per the async iteration protocol; only `return()`'s own returned
|
|
244
256
|
Promise carries the value.
|
|
245
257
|
- DO NOT dispose the source signal while an iterable is still consuming
|
|
246
|
-
from it. Call `iter.return()` first
|
|
247
|
-
|
|
258
|
+
from it. Call `iter.return()` first. A pending `next()` at the moment of
|
|
259
|
+
disposal never settles -- it neither resolves nor rejects; the only escape
|
|
260
|
+
hatch is the `timeout` option, which rejects the pending `next()` with
|
|
261
|
+
`TimeoutError` once its deadline elapses. See ROADMAP LS-04.
|
|
248
262
|
|
|
249
263
|
## Performance
|
|
250
264
|
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
|
|
265
|
+
Measured 2026-09-01 on node v26.3.1 (macOS arm64, laptop class), 150ms
|
|
266
|
+
warmup / 500ms runs, via npm run bench. Your numbers will differ -- re-run
|
|
267
|
+
npm run bench on your target.
|
|
268
|
+
|
|
269
|
+
The six scenarios `bench/bench.mjs` actually runs (ops/s, retained B/op):
|
|
270
|
+
|
|
271
|
+
- `from-latest-10`: 68,156 ops/s, 0.3218 B/op
|
|
272
|
+
- `from-buffer-20-drop`: 61,960 ops/s, 0.4715 B/op
|
|
273
|
+
- `abort-cycle`: 29,078 ops/s, 1.5132 B/op
|
|
274
|
+
- `pipe-to-signal`: 50,098 ops/s, 0.0853 B/op
|
|
275
|
+
- `to-async-iterable`: 198,948 ops/s, -8.4923 B/op
|
|
276
|
+
- `to-async-iterable-overflow`: 208,790 ops/s, 0.1013 B/op
|
|
254
277
|
|
|
255
|
-
|
|
256
|
-
-
|
|
257
|
-
-
|
|
258
|
-
|
|
259
|
-
- `filter + timeout`: 141K ops/s, negative retained (the timer-clear
|
|
260
|
-
discipline on `iter.return()` helps V8 reclaim more than baseline)
|
|
261
|
-
- multi-waiter x3 concurrent .next(): 120K cycles/s
|
|
278
|
+
The negative B/op on `to-async-iterable` is a GC-timing artifact (the
|
|
279
|
+
post-run gc() reclaims more than the pre-run baseline held); it is not a
|
|
280
|
+
zero-alloc guarantee. Per-scenario numbers only -- there is no "sub-byte
|
|
281
|
+
across all scenarios" claim.
|
|
262
282
|
|
|
263
283
|
Per-yield, `fromAsyncIterable` latest mode allocates one wrapper state
|
|
264
284
|
object; buffer mode adds one snapshot array. `toAsyncIterable` `mode:
|
|
265
285
|
"latest"` allocates zero on the steady-state hot path when the consumer
|
|
266
|
-
is caught up; `mode: "buffer"` also zero when no waiter is pending
|
|
286
|
+
is caught up; `mode: "buffer"` also zero when no waiter is pending
|
|
287
|
+
(gated: test/torture.mjs s5, per-scenario budgets).
|
|
267
288
|
|
|
268
289
|
## Files
|
|
269
290
|
|
|
270
|
-
- `Stream.js` -- single-file ESM implementation (~
|
|
291
|
+
- `Stream.js` -- single-file ESM implementation (~875 lines)
|
|
271
292
|
- `Stream.d.ts` -- TypeScript types with discriminated state union
|
|
272
293
|
- `README.md` -- full docs with integration recipes
|
|
273
294
|
- `llms.txt` -- this file
|
|
274
295
|
- `CHANGELOG.md` -- version history
|
|
275
|
-
- `ROADMAP.md` -- deferred features
|
|
276
296
|
|
|
277
297
|
## Author
|
|
278
298
|
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@zakkster/lite-stream",
|
|
3
|
-
"version": "1.
|
|
4
|
-
"description": "Zero-GC bridge between async iterators and @zakkster/lite-signal. Project async streams (paginated APIs, SSE, network frames, pubsub topics) into signals. Bounded buffering with explicit overflow diagnostics, structural cleanup on three termination paths (iterator done,
|
|
3
|
+
"version": "1.2.0",
|
|
4
|
+
"description": "Zero-GC bridge between async iterators and @zakkster/lite-signal. Project async streams (paginated APIs, SSE, network frames, pubsub topics) into signals. Bounded buffering with explicit overflow diagnostics, structural cleanup on three termination paths (iterator done, iterator throws, AbortSignal abort). The multi-shot dual of lite-await's fromPromise.",
|
|
5
5
|
"author": "Zahary Shinikchiev <shinikchiev@yahoo.com>",
|
|
6
6
|
"license": "MIT",
|
|
7
7
|
"type": "module",
|
|
@@ -22,14 +22,16 @@
|
|
|
22
22
|
"README.md",
|
|
23
23
|
"llms.txt",
|
|
24
24
|
"LICENSE.txt",
|
|
25
|
-
"CHANGELOG.md"
|
|
26
|
-
"ROADMAP.md"
|
|
25
|
+
"CHANGELOG.md"
|
|
27
26
|
],
|
|
28
27
|
"scripts": {
|
|
29
|
-
"test": "node --test --test-reporter=spec",
|
|
30
|
-
"test:gc": "node --expose-gc --test --test-reporter=spec",
|
|
28
|
+
"test": "node --expose-gc --test --test-reporter=spec test/*.test.mjs",
|
|
29
|
+
"test:gc": "node --expose-gc --test --test-reporter=spec test/*.test.mjs",
|
|
31
30
|
"bench": "node --expose-gc bench/bench.mjs",
|
|
32
|
-
"
|
|
31
|
+
"torture": "node --expose-gc test/torture.mjs",
|
|
32
|
+
"torture:control": "for t in s0 s1 s2 s3 s4 s5 s6 s7 s8; do if STREAM_TORTURE_BREAK=$t node --expose-gc test/torture.mjs >/dev/null 2>&1; then echo \"control $t FAILED-TO-FAIL\"; exit 1; else echo \"control $t failed-as-expected\"; fi; done",
|
|
33
|
+
"test:floor": "echo '== floor lane: pin @zakkster/lite-signal@1.2.2 (torture NOT run here -- lite-leak peers lite-signal>=1.5.0, LS-09 amendment) ==' && npm i --no-save --no-package-lock @zakkster/lite-signal@1.2.2 --legacy-peer-deps && npm test && echo 'FLOOR-PASS: npm test green @ lite-signal@1.2.2' && echo '== latest lane: restore @zakkster/lite-signal@^1.2.0 ==' && npm i --no-save --no-package-lock @zakkster/lite-signal@^1.2.0 && npm test && npm run torture && echo 'LATEST-PASS: npm test + torture green @ lite-signal latest'",
|
|
34
|
+
"verify": "npm test && npm run test:gc && npm run bench && npm run torture"
|
|
33
35
|
},
|
|
34
36
|
"keywords": [
|
|
35
37
|
"stream",
|
|
@@ -66,6 +68,8 @@
|
|
|
66
68
|
"@zakkster/lite-signal": "^1.2.0"
|
|
67
69
|
},
|
|
68
70
|
"devDependencies": {
|
|
71
|
+
"@zakkster/lite-gc-profiler": "^1.16.0",
|
|
72
|
+
"@zakkster/lite-leak": "^1.10.0",
|
|
69
73
|
"@zakkster/lite-signal": "^1.2.2"
|
|
70
74
|
}
|
|
71
75
|
}
|
package/ROADMAP.md
DELETED
|
@@ -1,239 +0,0 @@
|
|
|
1
|
-
# @zakkster/lite-stream Roadmap
|
|
2
|
-
|
|
3
|
-
Features deliberately deferred from 1.0 with the rationale. Each is gated
|
|
4
|
-
on a specific triggering signal from a real consumer rather than
|
|
5
|
-
speculative inclusion.
|
|
6
|
-
|
|
7
|
-
## Triggering signals
|
|
8
|
-
|
|
9
|
-
This roadmap exists because of the same discipline that drove
|
|
10
|
-
`lite-signal`'s and `lite-statechart`'s development: ship the minimum that
|
|
11
|
-
solves the actual problem, defer everything else until a concrete consumer
|
|
12
|
-
demonstrates the need with a real code site. Nothing in this file is
|
|
13
|
-
"planned for 1.x" -- everything is "considered, deferred, waiting for a
|
|
14
|
-
triggering signal."
|
|
15
|
-
|
|
16
|
-
---
|
|
17
|
-
|
|
18
|
-
## Shipped in 1.1.0 -- toAsyncIterable enrichment
|
|
19
|
-
|
|
20
|
-
The six additions below landed together in 1.1.0. Each is documented with
|
|
21
|
-
the triggering signal that justified moving it out of "deferred" status.
|
|
22
|
-
|
|
23
|
-
### `mode: "latest" | "buffer"` (default `"buffer"`)
|
|
24
|
-
**Triggering signal:** vocabulary mismatch with `@zakkster/lite-query`'s
|
|
25
|
-
`streamQuery`, which already offers `mode: "latest"` on its
|
|
26
|
-
`fromAsyncIterable` side. Bridging a signal INTO streamQuery via
|
|
27
|
-
`toAsyncIterable` needed a matching option. Twitch overlay reactive-state
|
|
28
|
-
use ("show current frame, not the backlog") is the second consumer.
|
|
29
|
-
|
|
30
|
-
### `filter` option
|
|
31
|
-
**Triggering signal:** per-value gating was previously done with async-
|
|
32
|
-
generator wrappers around `toAsyncIterable`. In hot-path use (reactive
|
|
33
|
-
state filtered by predicate) the wrapper adds an allocation per yield.
|
|
34
|
-
Inlining the filter into the subscription callback removes the wrapper
|
|
35
|
-
cost.
|
|
36
|
-
|
|
37
|
-
### `timeout` option + new `TimeoutError` export
|
|
38
|
-
**Triggering signal:** consumers wrap `toAsyncIterable` with external
|
|
39
|
-
`AbortController` + `setTimeout` boilerplate today. Native `timeout` opt
|
|
40
|
-
saves ~10 lines per usage. `TimeoutError` is structurally identical to
|
|
41
|
-
lite-await's class (`e.name === "TimeoutError"` duck-checks work across
|
|
42
|
-
both packages), NOT imported from lite-await -- zero-dep story preserved.
|
|
43
|
-
|
|
44
|
-
### `Symbol.asyncDispose` on Node 20+
|
|
45
|
-
**Triggering signal:** Node 20 shipped, `await using` is idiomatic modern
|
|
46
|
-
async-iterator ergonomics. Absent on older runtimes; `iter.return()`
|
|
47
|
-
remains the portable path.
|
|
48
|
-
|
|
49
|
-
### Multi-waiter FIFO queue (latent bug fix)
|
|
50
|
-
**Triggering signal:** correctness. `Promise.all([iter.next(),
|
|
51
|
-
iter.next()])` silently lost the first resolver in 1.0.0 (single-slot
|
|
52
|
-
`pendingResolve`). Async generators queue concurrent .next() calls; ours
|
|
53
|
-
now does too. Invisible fix for correct callers; unblocks
|
|
54
|
-
`Promise.all`-style consumers.
|
|
55
|
-
|
|
56
|
-
### `overflowCount` getter (alias for `droppedCount`)
|
|
57
|
-
**Triggering signal:** vocabulary consistency with `mode: "latest"`.
|
|
58
|
-
"Dropped" reads as FIFO-specific; "overflow" is mode-neutral.
|
|
59
|
-
`droppedCount` continues to work identically in both modes.
|
|
60
|
-
|
|
61
|
-
## 1.1.0 candidates
|
|
62
|
-
|
|
63
|
-
### Concurrent mapping (`mapLimit`)
|
|
64
|
-
|
|
65
|
-
```js
|
|
66
|
-
const live = fromAsyncIterable(urls, {
|
|
67
|
-
transform: async (url, ctx) => {
|
|
68
|
-
return await fetch(url, { signal: ctx.signal }).then((r) => r.json());
|
|
69
|
-
},
|
|
70
|
-
concurrency: 4 // up to 4 in-flight transforms
|
|
71
|
-
});
|
|
72
|
-
```
|
|
73
|
-
|
|
74
|
-
A consumer-side pattern where each yielded value needs an async
|
|
75
|
-
transformation, and the user wants bounded concurrency. Currently doable
|
|
76
|
-
by composing async generators externally, but if multiple consumers (helix
|
|
77
|
-
batch enrichment, EBS request fan-out) end up writing the same pattern,
|
|
78
|
-
add it.
|
|
79
|
-
|
|
80
|
-
**Triggering signal:** Two or more consumers in the ecosystem writing
|
|
81
|
-
identical "transform-with-concurrency-limit" wrappers around
|
|
82
|
-
`fromAsyncIterable`.
|
|
83
|
-
|
|
84
|
-
---
|
|
85
|
-
|
|
86
|
-
### Reverse-direction buffering overflow policies
|
|
87
|
-
|
|
88
|
-
`toAsyncIterable` in `mode: "buffer"` currently only does drop-oldest on
|
|
89
|
-
overflow. Some consumers might want:
|
|
90
|
-
|
|
91
|
-
- `overflow: "drop-newest"` -- reject incoming when full
|
|
92
|
-
- `overflow: "throw"` -- iterator throws on overflow
|
|
93
|
-
- `overflow: "block"` -- producer-side signal back-channel (requires
|
|
94
|
-
consumer cooperation; significant API change)
|
|
95
|
-
|
|
96
|
-
**Not to be confused with `mode: "latest"`** (shipped in 1.1), which is a
|
|
97
|
-
different backpressure MODEL -- single-slot overwrite semantics rather
|
|
98
|
-
than a bounded FIFO with a different overflow policy. See the "Shipped in
|
|
99
|
-
1.1.0" section above.
|
|
100
|
-
|
|
101
|
-
**Why deferred from 1.0:** Drop-oldest is the only mode any real consumer
|
|
102
|
-
in the roadmap actually wants. Adding alternatives speculatively grows the
|
|
103
|
-
API surface for no measurable benefit.
|
|
104
|
-
|
|
105
|
-
**Triggering signal:** A consumer demonstrating that drop-oldest causes
|
|
106
|
-
observable correctness issues for their use case.
|
|
107
|
-
|
|
108
|
-
---
|
|
109
|
-
|
|
110
|
-
### `mergeIterables(iterA, iterB, ...)`
|
|
111
|
-
|
|
112
|
-
Merge multiple async iterators into one signal, interleaving values as
|
|
113
|
-
they arrive. Useful for "watch both helix and pubsub for follower
|
|
114
|
-
changes" patterns.
|
|
115
|
-
|
|
116
|
-
**Why deferred from 1.0:** Doable today by composing iterators externally
|
|
117
|
-
(an async generator that races multiple sources). Built-in support cleans
|
|
118
|
-
up the cancellation story (one AbortSignal cancels all sources) but adds
|
|
119
|
-
complexity.
|
|
120
|
-
|
|
121
|
-
**Triggering signal:** First Twitch consumer that needs to merge >= 2
|
|
122
|
-
async sources into a single reactive surface with shared cancellation.
|
|
123
|
-
|
|
124
|
-
---
|
|
125
|
-
|
|
126
|
-
### `splitSignal(sig, predicate) -> [matchingSig, restSig]`
|
|
127
|
-
|
|
128
|
-
The dual of `mergeIterables`: split a signal's value stream into two based
|
|
129
|
-
on a predicate, exposing both halves as new signals.
|
|
130
|
-
|
|
131
|
-
**Why deferred from 1.0:** No concrete consumer in the roadmap needs this.
|
|
132
|
-
Speculative.
|
|
133
|
-
|
|
134
|
-
**Triggering signal:** A consumer demonstrating a real need to fan-out a
|
|
135
|
-
single async source into two reactive surfaces.
|
|
136
|
-
|
|
137
|
-
---
|
|
138
|
-
|
|
139
|
-
## 1.2.0+ candidates (smaller, lower priority)
|
|
140
|
-
|
|
141
|
-
- **`fromEventTarget(target, eventName)`** -- a wrapper that turns DOM
|
|
142
|
-
EventTarget events into an async iterable feeding a signal. Doable
|
|
143
|
-
today via async generators, but ergonomically common. Add if the
|
|
144
|
-
pattern recurs across browser-facing consumers (e.g. window resize
|
|
145
|
-
observers, pointer event streams).
|
|
146
|
-
|
|
147
|
-
- **`backpressure: "block"` for pipeToSignal** -- await the target
|
|
148
|
-
signal's effects before pulling the next value. Requires effect-
|
|
149
|
-
completion semantics from lite-signal, which don't exist today.
|
|
150
|
-
|
|
151
|
-
- **`opts.equals` for diff-mode signals** -- by default the underlying
|
|
152
|
-
signal uses lite-signal's default `Object.is`. Allow opt-in to a custom
|
|
153
|
-
equality predicate for buffer-mode snapshot arrays (where Object.is
|
|
154
|
-
always reports them as distinct).
|
|
155
|
-
|
|
156
|
-
- **Diagnostic surface** -- `live.stats() -> { yields, drops, errors,
|
|
157
|
-
startedAt, settledAt }` for telemetry. Useful but no consumer asking
|
|
158
|
-
for it yet.
|
|
159
|
-
|
|
160
|
-
---
|
|
161
|
-
|
|
162
|
-
## Deferred indefinitely
|
|
163
|
-
|
|
164
|
-
### Full reactive stream combinators
|
|
165
|
-
|
|
166
|
-
`map`, `filter`, `take`, `skip`, `debounce`, `throttle`, etc. -- the
|
|
167
|
-
ReactiveX / RxJS surface area.
|
|
168
|
-
|
|
169
|
-
**Why:** Once you have async iterators, all of these are five-line async
|
|
170
|
-
generators. Adding them to lite-stream would create two ways to do the
|
|
171
|
-
same thing, where the second way (combinator API) hides the source code
|
|
172
|
-
and makes the cancellation model harder to reason about. The async
|
|
173
|
-
generator form is more inspectable and composes naturally with all of
|
|
174
|
-
JS's existing async tooling.
|
|
175
|
-
|
|
176
|
-
If a consumer wants `debounce`, they write:
|
|
177
|
-
|
|
178
|
-
```js
|
|
179
|
-
async function* debounce(source, ms, signal) {
|
|
180
|
-
let pending;
|
|
181
|
-
for await (const v of source) {
|
|
182
|
-
if (pending) clearTimeout(pending);
|
|
183
|
-
pending = await new Promise((r) => {
|
|
184
|
-
setTimeout(() => r(v), ms);
|
|
185
|
-
signal?.addEventListener("abort", () => r(undefined));
|
|
186
|
-
});
|
|
187
|
-
if (pending !== undefined) yield pending;
|
|
188
|
-
}
|
|
189
|
-
}
|
|
190
|
-
```
|
|
191
|
-
|
|
192
|
-
Not as pretty as `source.pipe(debounce(500))`, but inspectable, debuggable,
|
|
193
|
-
and composes with everything else in the JS async ecosystem.
|
|
194
|
-
|
|
195
|
-
---
|
|
196
|
-
|
|
197
|
-
### `propagate: "always"` mode
|
|
198
|
-
|
|
199
|
-
A mode where every yielded value triggers an effect run even if
|
|
200
|
-
`Object.is`-equal to the previous. Currently the underlying signal's
|
|
201
|
-
default equality (`Object.is`) is sufficient for all known consumers --
|
|
202
|
-
our wrapper state object is fresh per yield, so dedup never kicks in.
|
|
203
|
-
|
|
204
|
-
**Why deferred:** No consumer wants force-propagate semantics in
|
|
205
|
-
practice; the wrapper state shape already guarantees per-yield
|
|
206
|
-
propagation.
|
|
207
|
-
|
|
208
|
-
---
|
|
209
|
-
|
|
210
|
-
## Non-goals
|
|
211
|
-
|
|
212
|
-
These will never ship in `lite-stream`:
|
|
213
|
-
|
|
214
|
-
- **A standalone scheduler.** Microtask / setImmediate decisions stay
|
|
215
|
-
inside the pump. The library does not expose a scheduling API.
|
|
216
|
-
- **A retry/backoff policy.** Consumers compose with `lite-await`'s
|
|
217
|
-
`withTimeout`, `withAbort`, and a plain `for` loop. Every Twitch SDK
|
|
218
|
-
endpoint has different retry semantics; a single policy would be
|
|
219
|
-
opinionated wrong for half of them.
|
|
220
|
-
- **WebSocket / EventSource adapters.** Those belong in the SDK that
|
|
221
|
-
consumes them (lite-twitch-ebs, lite-twitch-pubsub), not in lite-stream.
|
|
222
|
-
- **Concurrent multi-iterator orchestration beyond `mergeIterables`** --
|
|
223
|
-
use `lite-await`'s `allOf` / `anyOf` / `raceOf` for that.
|
|
224
|
-
|
|
225
|
-
---
|
|
226
|
-
|
|
227
|
-
## Versioning policy
|
|
228
|
-
|
|
229
|
-
`@zakkster/lite-stream` follows strict semver. 1.x is a stable API; only
|
|
230
|
-
additive changes (new exports, new opts fields, new state-shape fields)
|
|
231
|
-
ship in minor releases. Breaking changes wait for 2.0. Patch releases are
|
|
232
|
-
bug fixes only.
|
|
233
|
-
|
|
234
|
-
Each addition will land with:
|
|
235
|
-
|
|
236
|
-
1. A concrete triggering consumer in the `@zakkster/lite-*` ecosystem
|
|
237
|
-
2. Tests covering the full lifecycle (happy + all three cleanup paths)
|
|
238
|
-
3. A bench measuring throughput + retained heap
|
|
239
|
-
4. A line item in CHANGELOG.md with a hot-path-cost note
|