@zakkster/lite-stream 1.0.0 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +98 -0
- package/README.md +116 -21
- package/ROADMAP.md +51 -3
- package/Stream.js +290 -67
- package/llms.txt +82 -14
- package/package.json +1 -1
package/CHANGELOG.md
CHANGED
|
@@ -7,6 +7,104 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
|
|
|
7
7
|
|
|
8
8
|
---
|
|
9
9
|
|
|
10
|
+
## 1.1.0 -- 2026-07-10
|
|
11
|
+
|
|
12
|
+
**Additive `toAsyncIterable` enrichment.** Six additions, no breaking
|
|
13
|
+
changes. All 1.0.0 callers continue to work without modification. See
|
|
14
|
+
ROADMAP.md's "Shipped in 1.1.0" section for the triggering signals that
|
|
15
|
+
justified each addition.
|
|
16
|
+
|
|
17
|
+
### Added
|
|
18
|
+
|
|
19
|
+
- **`mode: "latest" | "buffer"`** on `toAsyncIterable`. Default `"buffer"`
|
|
20
|
+
preserves 1.0.0 behavior. `mode: "latest"` uses a single mutable slot
|
|
21
|
+
(producer overwrites; consumer reads and clears; every overwrite bumps
|
|
22
|
+
`overflowCount`). Combining `mode: "latest"` with `maxBuffer` throws
|
|
23
|
+
`TypeError` at construction -- silently ignoring one would hide user
|
|
24
|
+
error.
|
|
25
|
+
|
|
26
|
+
- **`filter: (v) => unknown`** on `toAsyncIterable`. Skip values for which
|
|
27
|
+
the predicate returns falsy. Applied to the initial value when
|
|
28
|
+
`emitInitial` is `true`. A throwing filter rejects the current pending
|
|
29
|
+
`next()` with the thrown value and terminates the iterable; the throw
|
|
30
|
+
does NOT surface at the signal writer's `.set()` call site.
|
|
31
|
+
|
|
32
|
+
- **`timeout: number`** on `toAsyncIterable`. Overall deadline in ms from
|
|
33
|
+
the `toAsyncIterable(...)` call (not per-`next()`, not idle). On elapse
|
|
34
|
+
the pending `next()` rejects with `TimeoutError`; subsequent calls
|
|
35
|
+
return `{done: true}`. Negative or non-finite values throw `RangeError`
|
|
36
|
+
at construction.
|
|
37
|
+
|
|
38
|
+
- **`TimeoutError`** as a new named export. Structurally identical to
|
|
39
|
+
`@zakkster/lite-await`'s `TimeoutError` (`e.name === "TimeoutError"`
|
|
40
|
+
duck-checks work across both packages). NOT imported from lite-await --
|
|
41
|
+
zero runtime dependencies preserved.
|
|
42
|
+
|
|
43
|
+
- **`Symbol.asyncDispose`** on the `toAsyncIterable` return object (Node
|
|
44
|
+
20+ only; absent on older runtimes). Delegates to `iter.return()`.
|
|
45
|
+
Enables `await using iter = toAsyncIterable(sig, { mode: "latest" })`.
|
|
46
|
+
|
|
47
|
+
- **`overflowCount`** getter on the `toAsyncIterable` return object.
|
|
48
|
+
Alias for `droppedCount` with mode-neutral vocabulary. In
|
|
49
|
+
`mode: "buffer"` reads the same value as `droppedCount` (drop-oldest
|
|
50
|
+
count); in `mode: "latest"` counts every slot overwrite.
|
|
51
|
+
|
|
52
|
+
### Changed (invisible correctness fix)
|
|
53
|
+
|
|
54
|
+
- **Multi-waiter FIFO queue** replaces `toAsyncIterable`'s 1.0.0
|
|
55
|
+
single-slot `pendingResolve`. Concurrent `.next()` calls (e.g. from
|
|
56
|
+
`Promise.all([iter.next(), iter.next()])`) now resolve in FIFO order.
|
|
57
|
+
1.0.0 silently overwrote the first resolver, causing the first
|
|
58
|
+
`.next()` to never settle. Invisible to correct callers; unblocks
|
|
59
|
+
`Promise.all`-style consumers.
|
|
60
|
+
|
|
61
|
+
### Not changed
|
|
62
|
+
|
|
63
|
+
- `fromAsyncIterable`, `pipeToSignal` -- unchanged behavior, unchanged
|
|
64
|
+
types, unchanged bench numbers.
|
|
65
|
+
- `toAsyncIterable`'s default behavior (with no `mode` opt) exactly
|
|
66
|
+
matches 1.0.0. Every 1.0.0 test in `05-to-async-iterable.test.mjs`
|
|
67
|
+
continues to pass without modification.
|
|
68
|
+
- Peer dependency remains `@zakkster/lite-signal ^1.2.0`. Node engines
|
|
69
|
+
requirement unchanged (>=18); `Symbol.asyncDispose` is feature-detected
|
|
70
|
+
and gracefully absent on Node 18/19.
|
|
71
|
+
|
|
72
|
+
### Performance (Node v22, --expose-gc, 500ms runs)
|
|
73
|
+
|
|
74
|
+
1.1 additions:
|
|
75
|
+
- `to-async-iterable-latest-1to1-x10`: **119K ops/s** (~2.4M ops/s per
|
|
76
|
+
operation), 0.12 B/op retained
|
|
77
|
+
- `to-async-iterable-latest-churn-100`: **60K ops/s** (~6M writes/s
|
|
78
|
+
absorbed into single slot), 0.28 B/op retained
|
|
79
|
+
- `to-async-iterable-filter-timeout`: **141K ops/s**, NEGATIVE retained
|
|
80
|
+
heap (timer-clear discipline helps V8 reclaim more than baseline)
|
|
81
|
+
- `to-async-iterable-multi-waiter-x3`: **120K ops/s**, negative retained
|
|
82
|
+
|
|
83
|
+
1.0.0 scenarios: unchanged from 1.0.0 numbers.
|
|
84
|
+
|
|
85
|
+
### Testing
|
|
86
|
+
|
|
87
|
+
74 tests across 7 files (up from 49 in 1.0.0):
|
|
88
|
+
- `01`-`04`: `fromAsyncIterable`, `pipeToSignal`, cleanup triplet
|
|
89
|
+
(unchanged)
|
|
90
|
+
- `05-to-async-iterable.test.mjs` -- 10 tests, unchanged 1.0.0 behavior
|
|
91
|
+
- `06-gc.test.mjs` -- 8 heap-budget tests (up from 3): 1.0.0 paths +
|
|
92
|
+
5 new 1.1 paths at 2 MB budget
|
|
93
|
+
- `07-to-async-iterable-rich.test.mjs` (NEW) -- 25 tests covering the
|
|
94
|
+
six 1.1 additions
|
|
95
|
+
|
|
96
|
+
All 74 tests pass under both `npm test` and `npm run test:gc`.
|
|
97
|
+
|
|
98
|
+
### Bundle
|
|
99
|
+
|
|
100
|
+
- ~7 KB minified (up from ~5 KB), ~3 KB gzipped
|
|
101
|
+
- ESM-only, Node >= 18 (with `Symbol.asyncDispose` on 20+)
|
|
102
|
+
- Single file `Stream.js`
|
|
103
|
+
- Zero runtime dependencies
|
|
104
|
+
- Peer dep: `@zakkster/lite-signal ^1.2.0` (unchanged)
|
|
105
|
+
|
|
106
|
+
---
|
|
107
|
+
|
|
10
108
|
## 1.0.0 -- 2026-06-23
|
|
11
109
|
|
|
12
110
|
**Initial public release.** API frozen; subsequent 1.x releases are purely
|
package/README.md
CHANGED
|
@@ -190,10 +190,19 @@ toAsyncIterable<T>(
|
|
|
190
190
|
sig: Signal<T> | Computed<T>,
|
|
191
191
|
opts?: {
|
|
192
192
|
signal?: AbortSignal,
|
|
193
|
-
emitInitial?: boolean,
|
|
194
|
-
maxBuffer?: number
|
|
193
|
+
emitInitial?: boolean, // default true
|
|
194
|
+
maxBuffer?: number, // default 1024 (buffer mode)
|
|
195
|
+
|
|
196
|
+
// Added in 1.1
|
|
197
|
+
mode?: "latest" | "buffer", // default "buffer"
|
|
198
|
+
filter?: (v: T) => unknown,
|
|
199
|
+
timeout?: number
|
|
195
200
|
}
|
|
196
|
-
): AsyncIterable<T> & {
|
|
201
|
+
): AsyncIterable<T> & {
|
|
202
|
+
readonly droppedCount: number,
|
|
203
|
+
readonly overflowCount: number, // 1.1 alias for droppedCount
|
|
204
|
+
[Symbol.asyncDispose]?: () => Promise<IteratorResult<T, undefined>>
|
|
205
|
+
}
|
|
197
206
|
```
|
|
198
207
|
|
|
199
208
|
The reverse direction -- yield signal changes as an async iterable. Useful
|
|
@@ -201,12 +210,41 @@ when you have signal-driven state and want to pipe its changes into a
|
|
|
201
210
|
WebSocket, log sink, replay tool, or any consumer that wants for-await
|
|
202
211
|
semantics.
|
|
203
212
|
|
|
204
|
-
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
`
|
|
208
|
-
|
|
209
|
-
|
|
213
|
+
**Two backpressure modes:**
|
|
214
|
+
|
|
215
|
+
- `"buffer"` (default; matches 1.0.0) -- FIFO ring buffer of `maxBuffer`
|
|
216
|
+
size. On overflow the OLDEST value is dropped and `droppedCount` (and
|
|
217
|
+
`overflowCount`) increments. Use when every value matters and must be
|
|
218
|
+
seen in order.
|
|
219
|
+
- `"latest"` (1.1) -- single mutable slot. Producer overwrites; consumer
|
|
220
|
+
reads and clears. Every overwrite bumps `overflowCount`. Use for
|
|
221
|
+
reactive-state consumers ("show the current frame / latest cursor /
|
|
222
|
+
live price") where intermediate values can be safely discarded.
|
|
223
|
+
|
|
224
|
+
**1.1 additions worth calling out:**
|
|
225
|
+
|
|
226
|
+
- `filter` -- skip values for which the predicate returns falsy. A
|
|
227
|
+
throwing filter rejects the current pending `next()` and terminates
|
|
228
|
+
the iterable; the throw does NOT surface at the signal writer's
|
|
229
|
+
`.set()` call site.
|
|
230
|
+
- `timeout` -- overall deadline in ms; on elapse the pending `next()`
|
|
231
|
+
rejects with `TimeoutError` (a new named export in 1.1). Subsequent
|
|
232
|
+
calls return `{done: true}`.
|
|
233
|
+
- `Symbol.asyncDispose` -- delegates to `iter.return()` on Node 20+;
|
|
234
|
+
enables `await using iter = toAsyncIterable(sig, { mode: "latest" })`.
|
|
235
|
+
- Multi-waiter queue -- concurrent `.next()` calls (e.g. from
|
|
236
|
+
`Promise.all([iter.next(), iter.next()])`) now resolve in FIFO order.
|
|
237
|
+
In 1.0.0 the second call silently overwrote the first resolver; the
|
|
238
|
+
1.1 fix is invisible to correct callers.
|
|
239
|
+
|
|
240
|
+
The iterator naturally completes when `opts.signal` aborts; consumer-side
|
|
241
|
+
`break` triggers `iterator.return()`. `mode: "latest"` combined with
|
|
242
|
+
`maxBuffer` is rejected with `TypeError` at construction -- a single-slot
|
|
243
|
+
mode plus a buffer size is a user error worth surfacing loudly.
|
|
244
|
+
|
|
245
|
+
Treat this as the secondary API for classic "consume signal changes"
|
|
246
|
+
patterns; for the streaming pipeline case (paginated APIs, SSE, pubsub)
|
|
247
|
+
the forward direction is what most consumers want.
|
|
210
248
|
|
|
211
249
|
## Modes: latest vs buffer
|
|
212
250
|
|
|
@@ -216,6 +254,8 @@ Pick `"latest"` when you only care about the most recent value:
|
|
|
216
254
|
- Current pubsub message
|
|
217
255
|
- Latest network frame
|
|
218
256
|
- Current SSE event
|
|
257
|
+
- Reactive state bridged into `@zakkster/lite-query`'s `streamQuery`
|
|
258
|
+
(matches its `mode: "latest"` on the forward direction)
|
|
219
259
|
|
|
220
260
|
Pick `"buffer"` when every value matters and you want them in order:
|
|
221
261
|
|
|
@@ -223,16 +263,44 @@ Pick `"buffer"` when every value matters and you want them in order:
|
|
|
223
263
|
- EBS SSE event log (must not miss an event)
|
|
224
264
|
- Replay queue (every frame applies in order)
|
|
225
265
|
|
|
226
|
-
|
|
227
|
-
|
|
266
|
+
**Availability:**
|
|
267
|
+
- `fromAsyncIterable` has both modes from 1.0.0.
|
|
268
|
+
- `toAsyncIterable` has both modes from 1.1 (was `"buffer"`-only in 1.0.0;
|
|
269
|
+
the option was added as `mode: "latest"` while preserving default
|
|
270
|
+
behavior).
|
|
271
|
+
|
|
272
|
+
**`"buffer"` mode requires `maxBuffer` on `fromAsyncIterable`.** There is
|
|
273
|
+
no default there, and missing the option throws a `RangeError` with this
|
|
274
|
+
message:
|
|
228
275
|
|
|
229
276
|
> `lite-stream: "buffer" mode requires opts.maxBuffer to be a positive
|
|
230
277
|
> integer. Unbounded buffering is a memory bug pretending to be a feature;
|
|
231
278
|
> pick a deliberate ceiling.`
|
|
232
279
|
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
280
|
+
`toAsyncIterable`'s `"buffer"` mode defaults `maxBuffer` to 1024
|
|
281
|
+
(unchanged in 1.1). On overflow the OLDEST value is dropped and
|
|
282
|
+
`droppedCount` increments. Watch `droppedCount` in your effect to know
|
|
283
|
+
when your consumer is falling behind.
|
|
284
|
+
|
|
285
|
+
**streamQuery integration** (matching vocabulary on both directions):
|
|
286
|
+
|
|
287
|
+
```js
|
|
288
|
+
import { toAsyncIterable } from "@zakkster/lite-stream";
|
|
289
|
+
import { streamQuery } from "@zakkster/lite-query/stream";
|
|
290
|
+
|
|
291
|
+
streamQuery(qc, {
|
|
292
|
+
key: ["price", ticker],
|
|
293
|
+
stream: ({ signal }) => toAsyncIterable(priceSig, {
|
|
294
|
+
signal,
|
|
295
|
+
mode: "latest" // matches streamQuery's mode
|
|
296
|
+
}),
|
|
297
|
+
mode: "latest"
|
|
298
|
+
});
|
|
299
|
+
```
|
|
300
|
+
|
|
301
|
+
The `signal` from the stream context wires straight through: streamQuery
|
|
302
|
+
aborts on detach, `toAsyncIterable` catches the abort and calls
|
|
303
|
+
`return()`. Zero glue code.
|
|
236
304
|
|
|
237
305
|
## Zero-GC hot paths
|
|
238
306
|
|
|
@@ -433,6 +501,20 @@ Run via `npm run bench` (requires `--expose-gc`).
|
|
|
433
501
|
|
|
434
502
|
[6] toAsyncIterable -- producer overflow, drop-oldest
|
|
435
503
|
to-async-iterable-overflow 62,164 ops/s retained: ~25 B/op
|
|
504
|
+
|
|
505
|
+
--- 1.1 additions ---
|
|
506
|
+
|
|
507
|
+
[7] toAsyncIterable mode:latest -- 10 write/next pairs per cycle
|
|
508
|
+
to-async-iterable-latest-1to1-x10 119,278 ops/s retained: 0.12 B/op
|
|
509
|
+
|
|
510
|
+
[8] toAsyncIterable mode:latest -- 100 writes coalesced, single read
|
|
511
|
+
to-async-iterable-latest-churn-100 60,674 ops/s retained: 0.28 B/op
|
|
512
|
+
|
|
513
|
+
[9] toAsyncIterable filter + timeout -- consumer wins before timeout
|
|
514
|
+
to-async-iterable-filter-timeout 141,080 ops/s retained: -11.80 B/op
|
|
515
|
+
|
|
516
|
+
[10] toAsyncIterable multi-waiter -- 3 concurrent next() calls
|
|
517
|
+
to-async-iterable-multi-waiter-x3 120,020 ops/s retained: -0.01 B/op
|
|
436
518
|
```
|
|
437
519
|
|
|
438
520
|
(Node v22, 150ms warmup, 500ms runs.)
|
|
@@ -442,11 +524,19 @@ in latest mode runs at 178K ops/s with sub-byte/op retained heap. Per-pull
|
|
|
442
524
|
allocation is dominated by the iterator's own microtask cost, not by
|
|
443
525
|
lite-stream's wrapper state.
|
|
444
526
|
|
|
527
|
+
1.1's `toAsyncIterable mode:"latest"` amortizes to ~2.4M ops/s per operation
|
|
528
|
+
(divide the reported cycle rate by the inner loop count). The filter+timeout
|
|
529
|
+
scenario `[9]` runs at 141K ops/s with NEGATIVE retained heap -- the
|
|
530
|
+
timer-clear discipline on `iter.return()` lets V8's subsequent GC pass
|
|
531
|
+
reclaim more than the pre-run measurement. The multi-waiter scenario `[10]`
|
|
532
|
+
confirms the 1.0.0 latent-bug fix (single-slot `pendingResolve` overwriting
|
|
533
|
+
the first resolver) works cleanly under bench load.
|
|
534
|
+
|
|
445
535
|
## Testing strategy
|
|
446
536
|
|
|
447
537
|
### Tier 1 -- behavior (unit tests, fast)
|
|
448
538
|
|
|
449
|
-
|
|
539
|
+
74 tests across `test/01-*` through `test/07-*`:
|
|
450
540
|
|
|
451
541
|
- `01-from-async-iterable-latest.test.mjs` -- state shape, lifecycle,
|
|
452
542
|
Iterable acceptance variants, pre-aborted, subscriber observability
|
|
@@ -456,23 +546,28 @@ lite-stream's wrapper state.
|
|
|
456
546
|
return() invocation, abort reason propagation, 200-cycle leak smoke test
|
|
457
547
|
- `04-pipe-to-signal.test.mjs` -- target write, stop fn idempotency,
|
|
458
548
|
transform, abort behavior, target-not-disposed invariant
|
|
459
|
-
- `05-to-async-iterable.test.mjs` -- emitInitial, ordering,
|
|
460
|
-
consumer break triggers return(), pre-aborted path
|
|
549
|
+
- `05-to-async-iterable.test.mjs` -- 1.0.0 behavior: emitInitial, ordering,
|
|
550
|
+
overflow, consumer break triggers return(), pre-aborted path
|
|
551
|
+
- `07-to-async-iterable-rich.test.mjs` (1.1) -- `mode:"latest"` behavior,
|
|
552
|
+
filter (including throwing-filter no-writer-surface discipline), timeout
|
|
553
|
+
+ TimeoutError, Symbol.asyncDispose (Node 20+ guarded), multi-waiter
|
|
554
|
+
queue, `overflowCount` alias, 4K structural cleanup cycles
|
|
461
555
|
|
|
462
556
|
Run via `npm test`.
|
|
463
557
|
|
|
464
558
|
### Tier 2 -- memory (allocation-free verification)
|
|
465
559
|
|
|
466
560
|
`test/06-gc.test.mjs` -- runs under `--expose-gc`. Asserts heap budget
|
|
467
|
-
ceilings for
|
|
468
|
-
|
|
561
|
+
ceilings at 2 MB for the 1.0.0 paths (2K latest, 1K buffer, 1K abort) and
|
|
562
|
+
the 1.1 additions (5K `mode:"latest"` resolve, 3K timeout, 1K buffer
|
|
563
|
+
fill+drain, 2K `mode:"latest"` churn, 2K filter-throw).
|
|
469
564
|
|
|
470
565
|
Run via `npm run test:gc`.
|
|
471
566
|
|
|
472
567
|
### Tier 3 -- performance (measured throughput)
|
|
473
568
|
|
|
474
|
-
`bench/bench.mjs` --
|
|
475
|
-
`npm run bench`.
|
|
569
|
+
`bench/bench.mjs` -- ten scenarios (six 1.0.0 + four 1.1); throughput and
|
|
570
|
+
B/op retained. Run via `npm run bench`.
|
|
476
571
|
|
|
477
572
|
## What this is not
|
|
478
573
|
|
package/ROADMAP.md
CHANGED
|
@@ -15,6 +15,49 @@ triggering signal."
|
|
|
15
15
|
|
|
16
16
|
---
|
|
17
17
|
|
|
18
|
+
## Shipped in 1.1.0 -- toAsyncIterable enrichment
|
|
19
|
+
|
|
20
|
+
The six additions below landed together in 1.1.0. Each is documented with
|
|
21
|
+
the triggering signal that justified moving it out of "deferred" status.
|
|
22
|
+
|
|
23
|
+
### `mode: "latest" | "buffer"` (default `"buffer"`)
|
|
24
|
+
**Triggering signal:** vocabulary mismatch with `@zakkster/lite-query`'s
|
|
25
|
+
`streamQuery`, which already offers `mode: "latest"` on its
|
|
26
|
+
`fromAsyncIterable` side. Bridging a signal INTO streamQuery via
|
|
27
|
+
`toAsyncIterable` needed a matching option. Twitch overlay reactive-state
|
|
28
|
+
use ("show current frame, not the backlog") is the second consumer.
|
|
29
|
+
|
|
30
|
+
### `filter` option
|
|
31
|
+
**Triggering signal:** per-value gating was previously done with async-
|
|
32
|
+
generator wrappers around `toAsyncIterable`. In hot-path use (reactive
|
|
33
|
+
state filtered by predicate) the wrapper adds an allocation per yield.
|
|
34
|
+
Inlining the filter into the subscription callback removes the wrapper
|
|
35
|
+
cost.
|
|
36
|
+
|
|
37
|
+
### `timeout` option + new `TimeoutError` export
|
|
38
|
+
**Triggering signal:** consumers wrap `toAsyncIterable` with external
|
|
39
|
+
`AbortController` + `setTimeout` boilerplate today. Native `timeout` opt
|
|
40
|
+
saves ~10 lines per usage. `TimeoutError` is structurally identical to
|
|
41
|
+
lite-await's class (`e.name === "TimeoutError"` duck-checks work across
|
|
42
|
+
both packages), NOT imported from lite-await -- zero-dep story preserved.
|
|
43
|
+
|
|
44
|
+
### `Symbol.asyncDispose` on Node 20+
|
|
45
|
+
**Triggering signal:** Node 20 shipped, `await using` is idiomatic modern
|
|
46
|
+
async-iterator ergonomics. Absent on older runtimes; `iter.return()`
|
|
47
|
+
remains the portable path.
|
|
48
|
+
|
|
49
|
+
### Multi-waiter FIFO queue (latent bug fix)
|
|
50
|
+
**Triggering signal:** correctness. `Promise.all([iter.next(),
|
|
51
|
+
iter.next()])` silently lost the first resolver in 1.0.0 (single-slot
|
|
52
|
+
`pendingResolve`). Async generators queue concurrent .next() calls; ours
|
|
53
|
+
now does too. Invisible fix for correct callers; unblocks
|
|
54
|
+
`Promise.all`-style consumers.
|
|
55
|
+
|
|
56
|
+
### `overflowCount` getter (alias for `droppedCount`)
|
|
57
|
+
**Triggering signal:** vocabulary consistency with `mode: "latest"`.
|
|
58
|
+
"Dropped" reads as FIFO-specific; "overflow" is mode-neutral.
|
|
59
|
+
`droppedCount` continues to work identically in both modes.
|
|
60
|
+
|
|
18
61
|
## 1.1.0 candidates
|
|
19
62
|
|
|
20
63
|
### Concurrent mapping (`mapLimit`)
|
|
@@ -40,16 +83,21 @@ identical "transform-with-concurrency-limit" wrappers around
|
|
|
40
83
|
|
|
41
84
|
---
|
|
42
85
|
|
|
43
|
-
### Reverse-direction buffering
|
|
86
|
+
### Reverse-direction buffering overflow policies
|
|
44
87
|
|
|
45
|
-
`toAsyncIterable` currently only does drop-oldest on
|
|
46
|
-
consumers might want:
|
|
88
|
+
`toAsyncIterable` in `mode: "buffer"` currently only does drop-oldest on
|
|
89
|
+
overflow. Some consumers might want:
|
|
47
90
|
|
|
48
91
|
- `overflow: "drop-newest"` -- reject incoming when full
|
|
49
92
|
- `overflow: "throw"` -- iterator throws on overflow
|
|
50
93
|
- `overflow: "block"` -- producer-side signal back-channel (requires
|
|
51
94
|
consumer cooperation; significant API change)
|
|
52
95
|
|
|
96
|
+
**Not to be confused with `mode: "latest"`** (shipped in 1.1), which is a
|
|
97
|
+
different backpressure MODEL -- single-slot overwrite semantics rather
|
|
98
|
+
than a bounded FIFO with a different overflow policy. See the "Shipped in
|
|
99
|
+
1.1.0" section above.
|
|
100
|
+
|
|
53
101
|
**Why deferred from 1.0:** Drop-oldest is the only mode any real consumer
|
|
54
102
|
in the roadmap actually wants. Adding alternatives speculatively grows the
|
|
55
103
|
API surface for no measurable benefit.
|
package/Stream.js
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
// @zakkster/lite-stream 1.
|
|
1
|
+
// @zakkster/lite-stream 1.1.0
|
|
2
2
|
//
|
|
3
3
|
// Zero-GC bridge between async iterators and @zakkster/lite-signal. The
|
|
4
4
|
// multi-shot dual of lite-await's fromPromise: project an async source of N
|
|
@@ -20,11 +20,38 @@
|
|
|
20
20
|
// droppedCount. Best for "process every helix page in order, don't miss
|
|
21
21
|
// events".
|
|
22
22
|
//
|
|
23
|
+
// 1.1 added `mode: "latest"`, `filter`, `timeout`, `Symbol.asyncDispose`,
|
|
24
|
+
// multi-waiter FIFO queue, and `overflowCount` on `toAsyncIterable`. See
|
|
25
|
+
// CONTRACT-toAsyncIterable-1.1.md and CHANGELOG.md for the full 1.1 story.
|
|
26
|
+
//
|
|
23
27
|
// Copyright (c) 2026 Zahary Shinikchiev <shinikchiev@yahoo.com>
|
|
24
28
|
// MIT License
|
|
25
29
|
|
|
26
30
|
import { signal as _signal } from "@zakkster/lite-signal";
|
|
27
31
|
|
|
32
|
+
// ---------------------------------------------------------------------------
|
|
33
|
+
// Errors
|
|
34
|
+
// ---------------------------------------------------------------------------
|
|
35
|
+
|
|
36
|
+
/**
|
|
37
|
+
* Thrown when a `timeout` deadline elapses before iterator settlement.
|
|
38
|
+
* Introduced in 1.1 alongside `toAsyncIterable`'s `timeout` option.
|
|
39
|
+
* Structurally identical to lite-await's `TimeoutError` so users can
|
|
40
|
+
* duck-check via `e.name === "TimeoutError"` across both packages -- but
|
|
41
|
+
* NOT imported from lite-await, to preserve lite-stream's zero-dep story.
|
|
42
|
+
*
|
|
43
|
+
* The wiring (start / clear / reject with) is Session B's territory; this
|
|
44
|
+
* class exists in Session A only so the exported shape is stable and the
|
|
45
|
+
* failing test suite can compile.
|
|
46
|
+
*/
|
|
47
|
+
class TimeoutError extends Error {
|
|
48
|
+
constructor(timeoutMs) {
|
|
49
|
+
super("lite-stream: timed out after " + timeoutMs + "ms");
|
|
50
|
+
this.name = "TimeoutError";
|
|
51
|
+
this.timeout = timeoutMs;
|
|
52
|
+
}
|
|
53
|
+
}
|
|
54
|
+
|
|
28
55
|
// ---------------------------------------------------------------------------
|
|
29
56
|
// Internal helpers
|
|
30
57
|
// ---------------------------------------------------------------------------
|
|
@@ -498,31 +525,54 @@ function pipeToSignal(source, target, opts) {
|
|
|
498
525
|
return stop;
|
|
499
526
|
}
|
|
500
527
|
|
|
528
|
+
// ---------------------------------------------------------------------------
|
|
529
|
+
// Symbol.asyncDispose feature detection (Node 20+)
|
|
530
|
+
// ---------------------------------------------------------------------------
|
|
531
|
+
// Cached once at module load. When the runtime lacks Symbol.asyncDispose
|
|
532
|
+
// (Node 18/19), the iterable's disposer property is simply absent;
|
|
533
|
+
// iter.return() is the portable path.
|
|
534
|
+
|
|
535
|
+
const ASYNC_DISPOSE = (typeof Symbol !== "undefined" && Symbol.asyncDispose)
|
|
536
|
+
? Symbol.asyncDispose
|
|
537
|
+
: null;
|
|
538
|
+
|
|
501
539
|
// ---------------------------------------------------------------------------
|
|
502
540
|
// toAsyncIterable -- the reverse direction
|
|
503
541
|
// ---------------------------------------------------------------------------
|
|
504
542
|
|
|
505
543
|
/**
|
|
506
|
-
* Yield signal changes as an async iterable. Each change
|
|
507
|
-
*
|
|
508
|
-
* producer, values queue in an internal bounded buffer; on overflow, the
|
|
509
|
-
* OLDEST queued value is dropped and `droppedCount` is incremented. The
|
|
510
|
-
* dropped count is observable via the iterable's `.droppedCount` property
|
|
511
|
-
* (read after consuming).
|
|
544
|
+
* Yield signal changes as an async iterable. Each change resolves a pending
|
|
545
|
+
* `next()` call. Two backpressure modes (1.1):
|
|
512
546
|
*
|
|
513
|
-
*
|
|
514
|
-
*
|
|
547
|
+
* - "buffer" (default; matches 1.0.0): FIFO ring buffer of maxBuffer size;
|
|
548
|
+
* on overflow the OLDEST is dropped and both droppedCount and
|
|
549
|
+
* overflowCount are incremented.
|
|
550
|
+
* - "latest" (1.1): single mutable slot; producer overwrites; every
|
|
551
|
+
* overwrite bumps overflowCount (and droppedCount for compat).
|
|
552
|
+
*
|
|
553
|
+
* 1.1 also adds `filter` (per-value gate; throwing filter rejects the
|
|
554
|
+
* current next() and terminates without a writer surface), `timeout`
|
|
555
|
+
* (overall deadline; rejects with TimeoutError), Symbol.asyncDispose
|
|
556
|
+
* on Node 20+, and a multi-waiter queue that fixes the 1.0.0 latent bug
|
|
557
|
+
* where Promise.all([iter.next(), iter.next()]) silently lost the first
|
|
558
|
+
* resolver.
|
|
559
|
+
*
|
|
560
|
+
* See CONTRACT-toAsyncIterable-1.1.md for the full locked semantics.
|
|
515
561
|
*
|
|
516
562
|
* @template T
|
|
517
563
|
* @param {import("@zakkster/lite-signal").Signal<T> | import("@zakkster/lite-signal").Computed<T>} sig
|
|
518
564
|
* @param {{
|
|
519
|
-
* signal?:
|
|
520
|
-
* emitInitial?: boolean,
|
|
521
|
-
* maxBuffer?: number
|
|
565
|
+
* signal?: AbortSignal,
|
|
566
|
+
* emitInitial?: boolean,
|
|
567
|
+
* maxBuffer?: number,
|
|
568
|
+
* mode?: "latest" | "buffer",
|
|
569
|
+
* filter?: (v: T) => unknown,
|
|
570
|
+
* timeout?: number
|
|
522
571
|
* }} [opts]
|
|
523
|
-
* @returns {AsyncIterable<T> & { readonly droppedCount: number }}
|
|
572
|
+
* @returns {AsyncIterable<T> & { readonly droppedCount: number, readonly overflowCount: number }}
|
|
524
573
|
*/
|
|
525
574
|
function toAsyncIterable(sig, opts) {
|
|
575
|
+
// --- Validation ---
|
|
526
576
|
if (sig === null || sig === undefined || typeof sig.subscribe !== "function" || typeof sig.peek !== "function") {
|
|
527
577
|
throw new TypeError(
|
|
528
578
|
"lite-stream: toAsyncIterable expects a readable lite-signal (Signal/Computed with .peek and .subscribe)"
|
|
@@ -530,101 +580,273 @@ function toAsyncIterable(sig, opts) {
|
|
|
530
580
|
}
|
|
531
581
|
const abortSig = (opts !== undefined && opts !== null) ? opts.signal : undefined;
|
|
532
582
|
const emitInitial = (opts !== undefined && opts !== null && opts.emitInitial === false) ? false : true;
|
|
533
|
-
const
|
|
583
|
+
const filter = (opts !== undefined && opts !== null) ? opts.filter : undefined;
|
|
584
|
+
const timeoutMs = (opts !== undefined && opts !== null) ? opts.timeout : undefined;
|
|
585
|
+
const modeOpt = (opts !== undefined && opts !== null) ? opts.mode : undefined;
|
|
586
|
+
const maxBufferOpt = (opts !== undefined && opts !== null) ? opts.maxBuffer : undefined;
|
|
587
|
+
|
|
588
|
+
// Mode. Default "buffer" preserves 1.0.0 behavior.
|
|
589
|
+
let mode;
|
|
590
|
+
if (modeOpt === undefined || modeOpt === "buffer") {
|
|
591
|
+
mode = "buffer";
|
|
592
|
+
} else if (modeOpt === "latest") {
|
|
593
|
+
mode = "latest";
|
|
594
|
+
} else {
|
|
595
|
+
throw new TypeError(
|
|
596
|
+
"lite-stream: opts.mode must be \"latest\" or \"buffer\" (got " + JSON.stringify(modeOpt) + ")"
|
|
597
|
+
);
|
|
598
|
+
}
|
|
599
|
+
const isLatestWins = mode === "latest";
|
|
534
600
|
|
|
535
|
-
|
|
536
|
-
|
|
537
|
-
|
|
601
|
+
// maxBuffer only meaningful in buffer mode. Combining latest + maxBuffer
|
|
602
|
+
// is user error; silently ignoring one would hide the mistake.
|
|
603
|
+
if (isLatestWins && maxBufferOpt !== undefined) {
|
|
604
|
+
throw new TypeError(
|
|
605
|
+
"lite-stream: opts.maxBuffer is not allowed with mode: \"latest\" -- latest-wins uses a single slot"
|
|
538
606
|
);
|
|
539
607
|
}
|
|
608
|
+
const maxBuffer = maxBufferOpt !== undefined ? maxBufferOpt : 1024;
|
|
609
|
+
if (!isLatestWins) {
|
|
610
|
+
if (typeof maxBuffer !== "number" || !Number.isFinite(maxBuffer) || maxBuffer < 1 || (maxBuffer | 0) !== maxBuffer) {
|
|
611
|
+
throw new RangeError(
|
|
612
|
+
"lite-stream: opts.maxBuffer must be a positive integer (got " + JSON.stringify(maxBuffer) + ")"
|
|
613
|
+
);
|
|
614
|
+
}
|
|
615
|
+
}
|
|
540
616
|
|
|
541
|
-
|
|
542
|
-
|
|
543
|
-
|
|
544
|
-
|
|
545
|
-
|
|
617
|
+
if (filter !== undefined && typeof filter !== "function") {
|
|
618
|
+
throw new TypeError("lite-stream: opts.filter must be a function");
|
|
619
|
+
}
|
|
620
|
+
if (timeoutMs !== undefined) {
|
|
621
|
+
if (typeof timeoutMs !== "number" || !Number.isFinite(timeoutMs) || timeoutMs < 0) {
|
|
622
|
+
throw new RangeError(
|
|
623
|
+
"lite-stream: opts.timeout must be a finite non-negative number (got " + JSON.stringify(timeoutMs) + ")"
|
|
624
|
+
);
|
|
625
|
+
}
|
|
626
|
+
}
|
|
627
|
+
|
|
628
|
+
// --- State ---
|
|
629
|
+
// Buffer-mode ring (unchanged from 1.0.0 layout); null in latest mode.
|
|
630
|
+
const queue = isLatestWins ? null : new Array(maxBuffer);
|
|
631
|
+
let qHead = 0;
|
|
632
|
+
let qLen = 0;
|
|
633
|
+
// Latest-mode slot.
|
|
634
|
+
let pendingValue;
|
|
635
|
+
let hasPending = false;
|
|
636
|
+
// Overflow. `droppedCount` and `overflowCount` are aliases for the same
|
|
637
|
+
// counter; 1.0.0 exposed `droppedCount`, 1.1 adds `overflowCount` as the
|
|
638
|
+
// mode-neutral vocabulary. Both getters read this variable.
|
|
639
|
+
let overflowCount = 0;
|
|
640
|
+
// Multi-waiter FIFO queue (fixes the 1.0.0 single-slot bug where
|
|
641
|
+
// concurrent .next() calls silently lost the first resolver).
|
|
642
|
+
const waiters = [];
|
|
643
|
+
// Termination
|
|
546
644
|
let done = false;
|
|
645
|
+
let firstNextErr = null; // stashed for the next .next() when no waiter was pending
|
|
646
|
+
// Cleanup handles
|
|
547
647
|
let unsubscribe = null;
|
|
648
|
+
let unsubscribePending = false;
|
|
548
649
|
let abortListener = null;
|
|
549
|
-
|
|
550
|
-
|
|
551
|
-
|
|
552
|
-
|
|
553
|
-
|
|
554
|
-
|
|
555
|
-
|
|
556
|
-
|
|
557
|
-
|
|
558
|
-
|
|
559
|
-
queue[(qHead + qLen) % maxBuffer] = v;
|
|
560
|
-
qLen = (qLen + 1) | 0;
|
|
650
|
+
let timeoutId = null;
|
|
651
|
+
|
|
652
|
+
// Late-binding unsubscribe: needed when a throwing filter (or throwing
|
|
653
|
+
// subscription callback body) fires during the synchronous initial
|
|
654
|
+
// subscribe call, at which point `unsubscribe` is still null. Mirror of
|
|
655
|
+
// lite-await's stop/stopPending pattern.
|
|
656
|
+
function doUnsubscribe() {
|
|
657
|
+
if (unsubscribe !== null) {
|
|
658
|
+
unsubscribe();
|
|
659
|
+
unsubscribe = null;
|
|
561
660
|
} else {
|
|
562
|
-
|
|
563
|
-
queue[qHead] = v;
|
|
564
|
-
qHead = (qHead + 1) % maxBuffer;
|
|
565
|
-
droppedCount = (droppedCount + 1) | 0;
|
|
661
|
+
unsubscribePending = true;
|
|
566
662
|
}
|
|
567
663
|
}
|
|
568
664
|
|
|
569
|
-
function
|
|
570
|
-
|
|
571
|
-
|
|
572
|
-
|
|
665
|
+
function fullCleanup() {
|
|
666
|
+
doUnsubscribe();
|
|
667
|
+
if (timeoutId !== null) {
|
|
668
|
+
clearTimeout(timeoutId);
|
|
669
|
+
timeoutId = null;
|
|
670
|
+
}
|
|
573
671
|
if (abortListener !== null && abortSig !== undefined && abortSig !== null) {
|
|
574
672
|
abortSig.removeEventListener("abort", abortListener);
|
|
575
673
|
abortListener = null;
|
|
576
674
|
}
|
|
577
|
-
|
|
578
|
-
|
|
579
|
-
|
|
580
|
-
|
|
675
|
+
// Release unread values so they can be GC'd before the iterable
|
|
676
|
+
// object itself is. One-time cost per iterable lifetime.
|
|
677
|
+
pendingValue = undefined;
|
|
678
|
+
hasPending = false;
|
|
679
|
+
if (queue !== null) {
|
|
680
|
+
for (let i = 0; i < maxBuffer; i = (i + 1) | 0) queue[i] = undefined;
|
|
681
|
+
qHead = 0;
|
|
682
|
+
qLen = 0;
|
|
581
683
|
}
|
|
582
684
|
}
|
|
583
685
|
|
|
584
|
-
|
|
585
|
-
|
|
586
|
-
|
|
686
|
+
// Termination via error: reject all pending waiters, or stash for the
|
|
687
|
+
// next .next() call. Route for timeout and throwing-filter paths.
|
|
688
|
+
function terminateWithError(err) {
|
|
689
|
+
if (done) return;
|
|
690
|
+
done = true;
|
|
691
|
+
fullCleanup();
|
|
692
|
+
if (waiters.length > 0) {
|
|
693
|
+
while (waiters.length > 0) {
|
|
694
|
+
const w = waiters.shift();
|
|
695
|
+
w.reject(err);
|
|
696
|
+
}
|
|
587
697
|
} else {
|
|
588
|
-
|
|
589
|
-
abortSig.addEventListener("abort", abortListener);
|
|
698
|
+
firstNextErr = err;
|
|
590
699
|
}
|
|
591
700
|
}
|
|
592
701
|
|
|
593
|
-
//
|
|
594
|
-
//
|
|
595
|
-
//
|
|
596
|
-
|
|
597
|
-
|
|
598
|
-
|
|
599
|
-
|
|
600
|
-
|
|
601
|
-
|
|
702
|
+
// Termination via done (abort, natural end via consumer return()): resolve
|
|
703
|
+
// pending waiters with {done: true}. Preserves 1.0.0 abort-is-graceful
|
|
704
|
+
// behavior (mid-iteration abort ends for-await without throw).
|
|
705
|
+
function terminateAsDone() {
|
|
706
|
+
if (done) return;
|
|
707
|
+
done = true;
|
|
708
|
+
fullCleanup();
|
|
709
|
+
while (waiters.length > 0) {
|
|
710
|
+
const w = waiters.shift();
|
|
711
|
+
w.resolve({ value: undefined, done: true });
|
|
712
|
+
}
|
|
602
713
|
}
|
|
603
714
|
|
|
715
|
+
function enqueue(v) {
|
|
716
|
+
if (done) return;
|
|
717
|
+
// Deliver directly to the oldest waiter if any.
|
|
718
|
+
if (waiters.length > 0) {
|
|
719
|
+
const w = waiters.shift();
|
|
720
|
+
w.resolve({ value: v, done: false });
|
|
721
|
+
return;
|
|
722
|
+
}
|
|
723
|
+
// Otherwise stash in the mode-appropriate structure.
|
|
724
|
+
if (isLatestWins) {
|
|
725
|
+
if (hasPending) overflowCount = (overflowCount + 1) | 0;
|
|
726
|
+
pendingValue = v;
|
|
727
|
+
hasPending = true;
|
|
728
|
+
} else {
|
|
729
|
+
if (qLen < maxBuffer) {
|
|
730
|
+
queue[(qHead + qLen) % maxBuffer] = v;
|
|
731
|
+
qLen = (qLen + 1) | 0;
|
|
732
|
+
} else {
|
|
733
|
+
// Drop oldest.
|
|
734
|
+
queue[qHead] = v;
|
|
735
|
+
qHead = (qHead + 1) % maxBuffer;
|
|
736
|
+
overflowCount = (overflowCount + 1) | 0;
|
|
737
|
+
}
|
|
738
|
+
}
|
|
739
|
+
}
|
|
740
|
+
|
|
741
|
+
// --- Iterable object ---
|
|
604
742
|
const iterable = {
|
|
605
743
|
[Symbol.asyncIterator]() { return this; },
|
|
606
744
|
next() {
|
|
607
|
-
if (
|
|
745
|
+
if (done) {
|
|
746
|
+
if (firstNextErr !== null) {
|
|
747
|
+
const err = firstNextErr;
|
|
748
|
+
firstNextErr = null;
|
|
749
|
+
return Promise.reject(err);
|
|
750
|
+
}
|
|
751
|
+
return Promise.resolve({ value: undefined, done: true });
|
|
752
|
+
}
|
|
753
|
+
// Value available in slot / ring?
|
|
754
|
+
if (isLatestWins && hasPending) {
|
|
755
|
+
const v = pendingValue;
|
|
756
|
+
pendingValue = undefined;
|
|
757
|
+
hasPending = false;
|
|
758
|
+
return Promise.resolve({ value: v, done: false });
|
|
759
|
+
}
|
|
760
|
+
if (!isLatestWins && qLen > 0) {
|
|
608
761
|
const v = queue[qHead];
|
|
609
762
|
queue[qHead] = undefined;
|
|
610
763
|
qHead = (qHead + 1) % maxBuffer;
|
|
611
764
|
qLen = (qLen - 1) | 0;
|
|
612
765
|
return Promise.resolve({ value: v, done: false });
|
|
613
766
|
}
|
|
614
|
-
|
|
615
|
-
return new Promise(function (resolve) {
|
|
767
|
+
// Nothing available -- queue a waiter.
|
|
768
|
+
return new Promise(function (resolve, reject) {
|
|
769
|
+
waiters.push({ resolve: resolve, reject: reject });
|
|
770
|
+
});
|
|
616
771
|
},
|
|
617
772
|
return(value) {
|
|
618
|
-
|
|
773
|
+
if (done) return Promise.resolve({ value: undefined, done: true });
|
|
774
|
+
terminateAsDone();
|
|
619
775
|
return Promise.resolve({ value: value, done: true });
|
|
620
776
|
},
|
|
621
777
|
throw(err) {
|
|
622
|
-
|
|
778
|
+
if (done) return Promise.reject(err);
|
|
779
|
+
// Consumer-driven throw: pending waiters reject with the error,
|
|
780
|
+
// subsequent next() returns done. Same shape as terminateWithError
|
|
781
|
+
// but without stashing (throw() call site gets its own rejection).
|
|
782
|
+
done = true;
|
|
783
|
+
fullCleanup();
|
|
784
|
+
while (waiters.length > 0) {
|
|
785
|
+
const w = waiters.shift();
|
|
786
|
+
w.reject(err);
|
|
787
|
+
}
|
|
623
788
|
return Promise.reject(err);
|
|
624
789
|
},
|
|
625
|
-
get droppedCount() { return
|
|
790
|
+
get droppedCount() { return overflowCount; },
|
|
791
|
+
get overflowCount() { return overflowCount; }
|
|
626
792
|
};
|
|
627
793
|
|
|
794
|
+
if (ASYNC_DISPOSE !== null) {
|
|
795
|
+
iterable[ASYNC_DISPOSE] = function () { return iterable.return(); };
|
|
796
|
+
}
|
|
797
|
+
|
|
798
|
+
// --- Pre-aborted signal: mark done, skip all subscription/timer setup.
|
|
799
|
+
// Matches 1.0.0 behavior: first next() resolves with {done: true} rather
|
|
800
|
+
// than rejecting. ---
|
|
801
|
+
if (abortSig !== undefined && abortSig !== null && abortSig.aborted) {
|
|
802
|
+
done = true;
|
|
803
|
+
return iterable;
|
|
804
|
+
}
|
|
805
|
+
|
|
806
|
+
// --- Timeout setup ---
|
|
807
|
+
if (timeoutMs !== undefined) {
|
|
808
|
+
timeoutId = setTimeout(function () {
|
|
809
|
+
terminateWithError(new TimeoutError(timeoutMs));
|
|
810
|
+
}, timeoutMs);
|
|
811
|
+
}
|
|
812
|
+
|
|
813
|
+
// --- AbortSignal listener ---
|
|
814
|
+
if (abortSig !== undefined && abortSig !== null) {
|
|
815
|
+
abortListener = function () { terminateAsDone(); };
|
|
816
|
+
abortSig.addEventListener("abort", abortListener);
|
|
817
|
+
}
|
|
818
|
+
|
|
819
|
+
// --- Subscribe. lite-signal's subscribe fires synchronously with the
|
|
820
|
+
// current value on registration; use a flag to suppress that initial
|
|
821
|
+
// fire if emitInitial is false. Filter (if provided) applies to the
|
|
822
|
+
// initial fire when emitInitial is true. A throwing filter is routed
|
|
823
|
+
// via terminateWithError so the throw never surfaces at the signal
|
|
824
|
+
// writer's .set() call site. ---
|
|
825
|
+
let suppressedInitial = !emitInitial;
|
|
826
|
+
unsubscribe = sig.subscribe(function (v) {
|
|
827
|
+
if (done) return;
|
|
828
|
+
if (suppressedInitial) { suppressedInitial = false; return; }
|
|
829
|
+
if (filter !== undefined) {
|
|
830
|
+
let ok;
|
|
831
|
+
try {
|
|
832
|
+
ok = filter(v);
|
|
833
|
+
} catch (e) {
|
|
834
|
+
terminateWithError(e);
|
|
835
|
+
return;
|
|
836
|
+
}
|
|
837
|
+
if (!ok) return;
|
|
838
|
+
}
|
|
839
|
+
enqueue(v);
|
|
840
|
+
});
|
|
841
|
+
|
|
842
|
+
// If terminateWithError was called from inside the synchronous initial
|
|
843
|
+
// subscribe fire (throwing filter on the initial value), honor the
|
|
844
|
+
// deferred unsubscribe now that we have the handle.
|
|
845
|
+
if (unsubscribePending && unsubscribe !== null) {
|
|
846
|
+
unsubscribe();
|
|
847
|
+
unsubscribe = null;
|
|
848
|
+
}
|
|
849
|
+
|
|
628
850
|
return iterable;
|
|
629
851
|
}
|
|
630
852
|
|
|
@@ -635,5 +857,6 @@ function toAsyncIterable(sig, opts) {
|
|
|
635
857
|
export {
|
|
636
858
|
fromAsyncIterable,
|
|
637
859
|
pipeToSignal,
|
|
638
|
-
toAsyncIterable
|
|
860
|
+
toAsyncIterable,
|
|
861
|
+
TimeoutError
|
|
639
862
|
};
|
package/llms.txt
CHANGED
|
@@ -14,7 +14,7 @@ Peer dep: `@zakkster/lite-signal ^1.2.0`.
|
|
|
14
14
|
|
|
15
15
|
## Imports
|
|
16
16
|
|
|
17
|
-
import { fromAsyncIterable, pipeToSignal, toAsyncIterable } from "@zakkster/lite-stream";
|
|
17
|
+
import { fromAsyncIterable, pipeToSignal, toAsyncIterable, TimeoutError } from "@zakkster/lite-stream";
|
|
18
18
|
|
|
19
19
|
## Exports
|
|
20
20
|
|
|
@@ -63,23 +63,60 @@ const stop = pipeToSignal(source, targetSig, {
|
|
|
63
63
|
|
|
64
64
|
Does NOT dispose the target signal. The caller owns its lifetime.
|
|
65
65
|
|
|
66
|
-
### `toAsyncIterable(sig, opts?) -> AsyncIterable<T> & { droppedCount }`
|
|
66
|
+
### `toAsyncIterable(sig, opts?) -> AsyncIterable<T> & { droppedCount, overflowCount }`
|
|
67
67
|
|
|
68
|
-
Reverse direction: yield signal changes as an async iterable.
|
|
69
|
-
|
|
68
|
+
Reverse direction: yield signal changes as an async iterable.
|
|
69
|
+
|
|
70
|
+
**1.0.0 shape (still the default):** bounded FIFO ring buffer, drop-oldest
|
|
71
|
+
overflow, `droppedCount` observable.
|
|
72
|
+
|
|
73
|
+
**1.1 additions (all opt-in, no default changes):**
|
|
74
|
+
- `mode: "latest"` -- single-slot overwrite semantics; use for reactive
|
|
75
|
+
state / "current frame" style consumers
|
|
76
|
+
- `filter` -- per-value gate; throwing filter rejects pending next() and
|
|
77
|
+
terminates (no writer surface)
|
|
78
|
+
- `timeout` -- overall deadline; pending next() rejects with `TimeoutError`
|
|
79
|
+
- `Symbol.asyncDispose` on Node 20+ -- enables `await using iter = ...`
|
|
80
|
+
- `overflowCount` -- alias for `droppedCount`, mode-neutral vocabulary
|
|
81
|
+
- Multi-waiter queue -- concurrent .next() calls resolve in FIFO order
|
|
82
|
+
(1.0.0 silently overwrote the first resolver)
|
|
70
83
|
|
|
71
84
|
```js
|
|
85
|
+
// 1.0.0-style: bounded FIFO buffer (default)
|
|
72
86
|
for await (const v of toAsyncIterable(sig, {
|
|
73
87
|
signal: abortCtrl.signal,
|
|
74
|
-
emitInitial: true,
|
|
75
|
-
maxBuffer: 1024
|
|
88
|
+
emitInitial: true, // default true
|
|
89
|
+
maxBuffer: 1024 // default 1024
|
|
90
|
+
})) {
|
|
91
|
+
// ...
|
|
92
|
+
}
|
|
93
|
+
|
|
94
|
+
// 1.1: latest-wins slot for reactive state
|
|
95
|
+
for await (const v of toAsyncIterable(sig, {
|
|
96
|
+
mode: "latest", // opt in to single-slot overwrite
|
|
97
|
+
filter: (v) => v.priority > 0,
|
|
98
|
+
timeout: 5000 // overall deadline
|
|
76
99
|
})) {
|
|
77
100
|
// ...
|
|
78
101
|
}
|
|
79
102
|
```
|
|
80
103
|
|
|
81
|
-
|
|
82
|
-
`
|
|
104
|
+
Constraints:
|
|
105
|
+
- `mode: "latest"` + `maxBuffer` throws TypeError (single slot has no buffer size)
|
|
106
|
+
- Negative or non-finite `timeout` throws RangeError
|
|
107
|
+
- Non-function `filter` throws TypeError
|
|
108
|
+
|
|
109
|
+
The iterator naturally completes when `opts.signal` aborts (resolves as
|
|
110
|
+
done, not rejection -- graceful termination). Consumer-side `break`
|
|
111
|
+
triggers `iterator.return()` and cleans up. `timeout` elapsing rejects
|
|
112
|
+
the pending next() with `TimeoutError`; subsequent calls return done.
|
|
113
|
+
|
|
114
|
+
### `TimeoutError` (added 1.1)
|
|
115
|
+
|
|
116
|
+
Thrown when the `timeout` option on `toAsyncIterable` elapses.
|
|
117
|
+
Structurally identical to `@zakkster/lite-await`'s TimeoutError so
|
|
118
|
+
`e.name === "TimeoutError"` duck-checks work across both packages. Not
|
|
119
|
+
imported from lite-await -- lite-stream stays zero-dep.
|
|
83
120
|
|
|
84
121
|
## Cleanup termination triplet
|
|
85
122
|
|
|
@@ -188,18 +225,49 @@ states: {
|
|
|
188
225
|
- Not a replacement for lite-clock for frame-rate loops.
|
|
189
226
|
- Not for one-shot promises -- use lite-await's `fromPromise`.
|
|
190
227
|
|
|
228
|
+
## Anti-patterns
|
|
229
|
+
|
|
230
|
+
- DO NOT combine `mode: "latest"` with `maxBuffer` on `toAsyncIterable`.
|
|
231
|
+
It throws TypeError at construction -- a single-slot mode with a buffer
|
|
232
|
+
size is user error. Pick one or the other.
|
|
233
|
+
- DO NOT reach for lite-stream for one-shot Signal<->Promise coordination.
|
|
234
|
+
That is `@zakkster/lite-await`'s job (`whenSignal`, `fromPromise`,
|
|
235
|
+
`allOf`/`anyOf`/`raceOf`). lite-stream owns the AsyncIterable boundary;
|
|
236
|
+
lite-await owns the Promise boundary.
|
|
237
|
+
- DO NOT wrap `toAsyncIterable` with an async-generator debounce / throttle
|
|
238
|
+
for reactive-state use cases. Use `mode: "latest"` directly -- it
|
|
239
|
+
already gives you "consume newest, discard intermediates" semantics with
|
|
240
|
+
zero per-value allocation.
|
|
241
|
+
- DO NOT rely on the value passed to `iter.return(value)` reaching pending
|
|
242
|
+
waiters. Pending `next()` calls resolve with `{value: undefined, done:
|
|
243
|
+
true}` per the async iteration protocol; only `return()`'s own returned
|
|
244
|
+
Promise carries the value.
|
|
245
|
+
- DO NOT dispose the source signal while an iterable is still consuming
|
|
246
|
+
from it. Call `iter.return()` first; disposing an in-flight source is
|
|
247
|
+
undefined behavior.
|
|
248
|
+
|
|
191
249
|
## Performance
|
|
192
250
|
|
|
193
|
-
178K ops/s for the full lifecycle in latest mode
|
|
194
|
-
+ dispose), sub-byte/op retained heap. 235K
|
|
195
|
-
84K ops/s for the full abort cycle.
|
|
251
|
+
178K ops/s for the full lifecycle in `fromAsyncIterable` latest mode
|
|
252
|
+
(construct + pull + settle + dispose), sub-byte/op retained heap. 235K
|
|
253
|
+
ops/s for `pipeToSignal` direct. 84K ops/s for the full abort cycle.
|
|
254
|
+
|
|
255
|
+
1.1 `toAsyncIterable` throughput (Node v22, 500ms runs):
|
|
256
|
+
- `mode: "latest"` 1:1 alternation: 119K cycles/s (2.4M ops/s per pair)
|
|
257
|
+
- `mode: "latest"` churn (100 writes coalesced): 60K cycles/s (6M
|
|
258
|
+
writes/s absorbed into the single slot)
|
|
259
|
+
- `filter + timeout`: 141K ops/s, negative retained (the timer-clear
|
|
260
|
+
discipline on `iter.return()` helps V8 reclaim more than baseline)
|
|
261
|
+
- multi-waiter x3 concurrent .next(): 120K cycles/s
|
|
196
262
|
|
|
197
|
-
Per-yield, latest mode allocates one wrapper state
|
|
198
|
-
adds one snapshot array
|
|
263
|
+
Per-yield, `fromAsyncIterable` latest mode allocates one wrapper state
|
|
264
|
+
object; buffer mode adds one snapshot array. `toAsyncIterable` `mode:
|
|
265
|
+
"latest"` allocates zero on the steady-state hot path when the consumer
|
|
266
|
+
is caught up; `mode: "buffer"` also zero when no waiter is pending.
|
|
199
267
|
|
|
200
268
|
## Files
|
|
201
269
|
|
|
202
|
-
- `Stream.js` -- single-file ESM implementation (~
|
|
270
|
+
- `Stream.js` -- single-file ESM implementation (~830 lines)
|
|
203
271
|
- `Stream.d.ts` -- TypeScript types with discriminated state union
|
|
204
272
|
- `README.md` -- full docs with integration recipes
|
|
205
273
|
- `llms.txt` -- this file
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@zakkster/lite-stream",
|
|
3
|
-
"version": "1.
|
|
3
|
+
"version": "1.1.0",
|
|
4
4
|
"description": "Zero-GC bridge between async iterators and @zakkster/lite-signal. Project async streams (paginated APIs, SSE, network frames, pubsub topics) into signals. Bounded buffering with explicit overflow diagnostics, structural cleanup on three termination paths (iterator done, abort, dispose). The multi-shot dual of lite-await's fromPromise.",
|
|
5
5
|
"author": "Zahary Shinikchiev <shinikchiev@yahoo.com>",
|
|
6
6
|
"license": "MIT",
|