@zakkster/lite-stream 1.0.0 → 1.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -7,6 +7,104 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ---
9
9
 
10
+ ## 1.1.0 -- 2026-07-10
11
+
12
+ **Additive `toAsyncIterable` enrichment.** Six additions, no breaking
13
+ changes. All 1.0.0 callers continue to work without modification. See
14
+ ROADMAP.md's "Shipped in 1.1.0" section for the triggering signals that
15
+ justified each addition.
16
+
17
+ ### Added
18
+
19
+ - **`mode: "latest" | "buffer"`** on `toAsyncIterable`. Default `"buffer"`
20
+ preserves 1.0.0 behavior. `mode: "latest"` uses a single mutable slot
21
+ (producer overwrites; consumer reads and clears; every overwrite bumps
22
+ `overflowCount`). Combining `mode: "latest"` with `maxBuffer` throws
23
+ `TypeError` at construction -- silently ignoring one would hide user
24
+ error.
25
+
26
+ - **`filter: (v) => unknown`** on `toAsyncIterable`. Skip values for which
27
+ the predicate returns falsy. Applied to the initial value when
28
+ `emitInitial` is `true`. A throwing filter rejects the current pending
29
+ `next()` with the thrown value and terminates the iterable; the throw
30
+ does NOT surface at the signal writer's `.set()` call site.
31
+
32
+ - **`timeout: number`** on `toAsyncIterable`. Overall deadline in ms from
33
+ the `toAsyncIterable(...)` call (not per-`next()`, not idle). On elapse
34
+ the pending `next()` rejects with `TimeoutError`; subsequent calls
35
+ return `{done: true}`. Negative or non-finite values throw `RangeError`
36
+ at construction.
37
+
38
+ - **`TimeoutError`** as a new named export. Structurally identical to
39
+ `@zakkster/lite-await`'s `TimeoutError` (`e.name === "TimeoutError"`
40
+ duck-checks work across both packages). NOT imported from lite-await --
41
+ zero runtime dependencies preserved.
42
+
43
+ - **`Symbol.asyncDispose`** on the `toAsyncIterable` return object (Node
44
+ 20+ only; absent on older runtimes). Delegates to `iter.return()`.
45
+ Enables `await using iter = toAsyncIterable(sig, { mode: "latest" })`.
46
+
47
+ - **`overflowCount`** getter on the `toAsyncIterable` return object.
48
+ Alias for `droppedCount` with mode-neutral vocabulary. In
49
+ `mode: "buffer"` reads the same value as `droppedCount` (drop-oldest
50
+ count); in `mode: "latest"` counts every slot overwrite.
51
+
52
+ ### Changed (invisible correctness fix)
53
+
54
+ - **Multi-waiter FIFO queue** replaces `toAsyncIterable`'s 1.0.0
55
+ single-slot `pendingResolve`. Concurrent `.next()` calls (e.g. from
56
+ `Promise.all([iter.next(), iter.next()])`) now resolve in FIFO order.
57
+ 1.0.0 silently overwrote the first resolver, causing the first
58
+ `.next()` to never settle. Invisible to correct callers; unblocks
59
+ `Promise.all`-style consumers.
60
+
61
+ ### Not changed
62
+
63
+ - `fromAsyncIterable`, `pipeToSignal` -- unchanged behavior, unchanged
64
+ types, unchanged bench numbers.
65
+ - `toAsyncIterable`'s default behavior (with no `mode` opt) exactly
66
+ matches 1.0.0. Every 1.0.0 test in `05-to-async-iterable.test.mjs`
67
+ continues to pass without modification.
68
+ - Peer dependency remains `@zakkster/lite-signal ^1.2.0`. Node engines
69
+ requirement unchanged (>=18); `Symbol.asyncDispose` is feature-detected
70
+ and gracefully absent on Node 18/19.
71
+
72
+ ### Performance (Node v22, --expose-gc, 500ms runs)
73
+
74
+ 1.1 additions:
75
+ - `to-async-iterable-latest-1to1-x10`: **119K ops/s** (~2.4M ops/s per
76
+ operation), 0.12 B/op retained
77
+ - `to-async-iterable-latest-churn-100`: **60K ops/s** (~6M writes/s
78
+ absorbed into single slot), 0.28 B/op retained
79
+ - `to-async-iterable-filter-timeout`: **141K ops/s**, NEGATIVE retained
80
+ heap (timer-clear discipline helps V8 reclaim more than baseline)
81
+ - `to-async-iterable-multi-waiter-x3`: **120K ops/s**, negative retained
82
+
83
+ 1.0.0 scenarios: unchanged from 1.0.0 numbers.
84
+
85
+ ### Testing
86
+
87
+ 74 tests across 7 files (up from 49 in 1.0.0):
88
+ - `01`-`04`: `fromAsyncIterable`, `pipeToSignal`, cleanup triplet
89
+ (unchanged)
90
+ - `05-to-async-iterable.test.mjs` -- 10 tests, unchanged 1.0.0 behavior
91
+ - `06-gc.test.mjs` -- 8 heap-budget tests (up from 3): 1.0.0 paths +
92
+ 5 new 1.1 paths at 2 MB budget
93
+ - `07-to-async-iterable-rich.test.mjs` (NEW) -- 25 tests covering the
94
+ six 1.1 additions
95
+
96
+ All 74 tests pass under both `npm test` and `npm run test:gc`.
97
+
98
+ ### Bundle
99
+
100
+ - ~7 KB minified (up from ~5 KB), ~3 KB gzipped
101
+ - ESM-only, Node >= 18 (with `Symbol.asyncDispose` on 20+)
102
+ - Single file `Stream.js`
103
+ - Zero runtime dependencies
104
+ - Peer dep: `@zakkster/lite-signal ^1.2.0` (unchanged)
105
+
106
+ ---
107
+
10
108
  ## 1.0.0 -- 2026-06-23
11
109
 
12
110
  **Initial public release.** API frozen; subsequent 1.x releases are purely
package/README.md CHANGED
@@ -190,10 +190,19 @@ toAsyncIterable<T>(
190
190
  sig: Signal<T> | Computed<T>,
191
191
  opts?: {
192
192
  signal?: AbortSignal,
193
- emitInitial?: boolean, // default true
194
- maxBuffer?: number // default 1024
193
+ emitInitial?: boolean, // default true
194
+ maxBuffer?: number, // default 1024 (buffer mode)
195
+
196
+ // Added in 1.1
197
+ mode?: "latest" | "buffer", // default "buffer"
198
+ filter?: (v: T) => unknown,
199
+ timeout?: number
195
200
  }
196
- ): AsyncIterable<T> & { readonly droppedCount: number }
201
+ ): AsyncIterable<T> & {
202
+ readonly droppedCount: number,
203
+ readonly overflowCount: number, // 1.1 alias for droppedCount
204
+ [Symbol.asyncDispose]?: () => Promise<IteratorResult<T, undefined>>
205
+ }
197
206
  ```
198
207
 
199
208
  The reverse direction -- yield signal changes as an async iterable. Useful
@@ -201,12 +210,41 @@ when you have signal-driven state and want to pipe its changes into a
201
210
  WebSocket, log sink, replay tool, or any consumer that wants for-await
202
211
  semantics.
203
212
 
204
- Internal queue is bounded (default 1024). On overflow, the OLDEST value is
205
- dropped and `iterable.droppedCount` increments. The iterator naturally
206
- completes when `opts.signal` aborts; consumer-side `break` triggers
207
- `iterator.return()`.
208
-
209
- Treat this as the secondary API. Most consumers want the forward direction.
213
+ **Two backpressure modes:**
214
+
215
+ - `"buffer"` (default; matches 1.0.0) -- FIFO ring buffer of `maxBuffer`
216
+ size. On overflow the OLDEST value is dropped and `droppedCount` (and
217
+ `overflowCount`) increments. Use when every value matters and must be
218
+ seen in order.
219
+ - `"latest"` (1.1) -- single mutable slot. Producer overwrites; consumer
220
+ reads and clears. Every overwrite bumps `overflowCount`. Use for
221
+ reactive-state consumers ("show the current frame / latest cursor /
222
+ live price") where intermediate values can be safely discarded.
223
+
224
+ **1.1 additions worth calling out:**
225
+
226
+ - `filter` -- skip values for which the predicate returns falsy. A
227
+ throwing filter rejects the current pending `next()` and terminates
228
+ the iterable; the throw does NOT surface at the signal writer's
229
+ `.set()` call site.
230
+ - `timeout` -- overall deadline in ms; on elapse the pending `next()`
231
+ rejects with `TimeoutError` (a new named export in 1.1). Subsequent
232
+ calls return `{done: true}`.
233
+ - `Symbol.asyncDispose` -- delegates to `iter.return()` on Node 20+;
234
+ enables `await using iter = toAsyncIterable(sig, { mode: "latest" })`.
235
+ - Multi-waiter queue -- concurrent `.next()` calls (e.g. from
236
+ `Promise.all([iter.next(), iter.next()])`) now resolve in FIFO order.
237
+ In 1.0.0 the second call silently overwrote the first resolver; the
238
+ 1.1 fix is invisible to correct callers.
239
+
240
+ The iterator naturally completes when `opts.signal` aborts; consumer-side
241
+ `break` triggers `iterator.return()`. `mode: "latest"` combined with
242
+ `maxBuffer` is rejected with `TypeError` at construction -- a single-slot
243
+ mode plus a buffer size is a user error worth surfacing loudly.
244
+
245
+ Treat this as the secondary API for classic "consume signal changes"
246
+ patterns; for the streaming pipeline case (paginated APIs, SSE, pubsub)
247
+ the forward direction is what most consumers want.
210
248
 
211
249
  ## Modes: latest vs buffer
212
250
 
@@ -216,6 +254,8 @@ Pick `"latest"` when you only care about the most recent value:
216
254
  - Current pubsub message
217
255
  - Latest network frame
218
256
  - Current SSE event
257
+ - Reactive state bridged into `@zakkster/lite-query`'s `streamQuery`
258
+ (matches its `mode: "latest"` on the forward direction)
219
259
 
220
260
  Pick `"buffer"` when every value matters and you want them in order:
221
261
 
@@ -223,16 +263,44 @@ Pick `"buffer"` when every value matters and you want them in order:
223
263
  - EBS SSE event log (must not miss an event)
224
264
  - Replay queue (every frame applies in order)
225
265
 
226
- **`"buffer"` mode requires `maxBuffer`.** There is no default, and missing
227
- the option throws a `RangeError` with this message:
266
+ **Availability:**
267
+ - `fromAsyncIterable` has both modes from 1.0.0.
268
+ - `toAsyncIterable` has both modes from 1.1 (was `"buffer"`-only in 1.0.0;
269
+ the option was added as `mode: "latest"` while preserving default
270
+ behavior).
271
+
272
+ **`"buffer"` mode requires `maxBuffer` on `fromAsyncIterable`.** There is
273
+ no default there, and missing the option throws a `RangeError` with this
274
+ message:
228
275
 
229
276
  > `lite-stream: "buffer" mode requires opts.maxBuffer to be a positive
230
277
  > integer. Unbounded buffering is a memory bug pretending to be a feature;
231
278
  > pick a deliberate ceiling.`
232
279
 
233
- On overflow, the OLDEST value is dropped and `droppedCount` increments.
234
- Watch `droppedCount` in your effect to know when your consumer is falling
235
- behind.
280
+ `toAsyncIterable`'s `"buffer"` mode defaults `maxBuffer` to 1024
281
+ (unchanged in 1.1). On overflow the OLDEST value is dropped and
282
+ `droppedCount` increments. Watch `droppedCount` in your effect to know
283
+ when your consumer is falling behind.
284
+
285
+ **streamQuery integration** (matching vocabulary on both directions):
286
+
287
+ ```js
288
+ import { toAsyncIterable } from "@zakkster/lite-stream";
289
+ import { streamQuery } from "@zakkster/lite-query/stream";
290
+
291
+ streamQuery(qc, {
292
+ key: ["price", ticker],
293
+ stream: ({ signal }) => toAsyncIterable(priceSig, {
294
+ signal,
295
+ mode: "latest" // matches streamQuery's mode
296
+ }),
297
+ mode: "latest"
298
+ });
299
+ ```
300
+
301
+ The `signal` from the stream context wires straight through: streamQuery
302
+ aborts on detach, `toAsyncIterable` catches the abort and calls
303
+ `return()`. Zero glue code.
236
304
 
237
305
  ## Zero-GC hot paths
238
306
 
@@ -433,6 +501,20 @@ Run via `npm run bench` (requires `--expose-gc`).
433
501
 
434
502
  [6] toAsyncIterable -- producer overflow, drop-oldest
435
503
  to-async-iterable-overflow 62,164 ops/s retained: ~25 B/op
504
+
505
+ --- 1.1 additions ---
506
+
507
+ [7] toAsyncIterable mode:latest -- 10 write/next pairs per cycle
508
+ to-async-iterable-latest-1to1-x10 119,278 ops/s retained: 0.12 B/op
509
+
510
+ [8] toAsyncIterable mode:latest -- 100 writes coalesced, single read
511
+ to-async-iterable-latest-churn-100 60,674 ops/s retained: 0.28 B/op
512
+
513
+ [9] toAsyncIterable filter + timeout -- consumer wins before timeout
514
+ to-async-iterable-filter-timeout 141,080 ops/s retained: -11.80 B/op
515
+
516
+ [10] toAsyncIterable multi-waiter -- 3 concurrent next() calls
517
+ to-async-iterable-multi-waiter-x3 120,020 ops/s retained: -0.01 B/op
436
518
  ```
437
519
 
438
520
  (Node v22, 150ms warmup, 500ms runs.)
@@ -442,11 +524,19 @@ in latest mode runs at 178K ops/s with sub-byte/op retained heap. Per-pull
442
524
  allocation is dominated by the iterator's own microtask cost, not by
443
525
  lite-stream's wrapper state.
444
526
 
527
+ 1.1's `toAsyncIterable mode:"latest"` amortizes to ~2.4M ops/s per operation
528
+ (divide the reported cycle rate by the inner loop count). The filter+timeout
529
+ scenario `[9]` runs at 141K ops/s with NEGATIVE retained heap -- the
530
+ timer-clear discipline on `iter.return()` lets V8's subsequent GC pass
531
+ reclaim more than the pre-run measurement. The multi-waiter scenario `[10]`
532
+ confirms the 1.0.0 latent-bug fix (single-slot `pendingResolve` overwriting
533
+ the first resolver) works cleanly under bench load.
534
+
445
535
  ## Testing strategy
446
536
 
447
537
  ### Tier 1 -- behavior (unit tests, fast)
448
538
 
449
- 49 tests across `test/01-*` through `test/06-*`:
539
+ 74 tests across `test/01-*` through `test/07-*`:
450
540
 
451
541
  - `01-from-async-iterable-latest.test.mjs` -- state shape, lifecycle,
452
542
  Iterable acceptance variants, pre-aborted, subscriber observability
@@ -456,23 +546,28 @@ lite-stream's wrapper state.
456
546
  return() invocation, abort reason propagation, 200-cycle leak smoke test
457
547
  - `04-pipe-to-signal.test.mjs` -- target write, stop fn idempotency,
458
548
  transform, abort behavior, target-not-disposed invariant
459
- - `05-to-async-iterable.test.mjs` -- emitInitial, ordering, overflow,
460
- consumer break triggers return(), pre-aborted path
549
+ - `05-to-async-iterable.test.mjs` -- 1.0.0 behavior: emitInitial, ordering,
550
+ overflow, consumer break triggers return(), pre-aborted path
551
+ - `07-to-async-iterable-rich.test.mjs` (1.1) -- `mode:"latest"` behavior,
552
+ filter (including throwing-filter no-writer-surface discipline), timeout
553
+ + TimeoutError, Symbol.asyncDispose (Node 20+ guarded), multi-waiter
554
+ queue, `overflowCount` alias, 4K structural cleanup cycles
461
555
 
462
556
  Run via `npm test`.
463
557
 
464
558
  ### Tier 2 -- memory (allocation-free verification)
465
559
 
466
560
  `test/06-gc.test.mjs` -- runs under `--expose-gc`. Asserts heap budget
467
- ceilings for 2K latest cycles (< 2 MB), 1K buffer cycles (< 2 MB), and
468
- 1K abort cycles (< 2 MB).
561
+ ceilings at 2 MB for the 1.0.0 paths (2K latest, 1K buffer, 1K abort) and
562
+ the 1.1 additions (5K `mode:"latest"` resolve, 3K timeout, 1K buffer
563
+ fill+drain, 2K `mode:"latest"` churn, 2K filter-throw).
469
564
 
470
565
  Run via `npm run test:gc`.
471
566
 
472
567
  ### Tier 3 -- performance (measured throughput)
473
568
 
474
- `bench/bench.mjs` -- six scenarios; throughput and B/op retained. Run via
475
- `npm run bench`.
569
+ `bench/bench.mjs` -- ten scenarios (six 1.0.0 + four 1.1); throughput and
570
+ B/op retained. Run via `npm run bench`.
476
571
 
477
572
  ## What this is not
478
573
 
package/ROADMAP.md CHANGED
@@ -15,6 +15,49 @@ triggering signal."
15
15
 
16
16
  ---
17
17
 
18
+ ## Shipped in 1.1.0 -- toAsyncIterable enrichment
19
+
20
+ The six additions below landed together in 1.1.0. Each is documented with
21
+ the triggering signal that justified moving it out of "deferred" status.
22
+
23
+ ### `mode: "latest" | "buffer"` (default `"buffer"`)
24
+ **Triggering signal:** vocabulary mismatch with `@zakkster/lite-query`'s
25
+ `streamQuery`, which already offers `mode: "latest"` on its
26
+ `fromAsyncIterable` side. Bridging a signal INTO streamQuery via
27
+ `toAsyncIterable` needed a matching option. Twitch overlay reactive-state
28
+ use ("show current frame, not the backlog") is the second consumer.
29
+
30
+ ### `filter` option
31
+ **Triggering signal:** per-value gating was previously done with async-
32
+ generator wrappers around `toAsyncIterable`. In hot-path use (reactive
33
+ state filtered by predicate) the wrapper adds an allocation per yield.
34
+ Inlining the filter into the subscription callback removes the wrapper
35
+ cost.
36
+
37
+ ### `timeout` option + new `TimeoutError` export
38
+ **Triggering signal:** consumers wrap `toAsyncIterable` with external
39
+ `AbortController` + `setTimeout` boilerplate today. Native `timeout` opt
40
+ saves ~10 lines per usage. `TimeoutError` is structurally identical to
41
+ lite-await's class (`e.name === "TimeoutError"` duck-checks work across
42
+ both packages), NOT imported from lite-await -- zero-dep story preserved.
43
+
44
+ ### `Symbol.asyncDispose` on Node 20+
45
+ **Triggering signal:** Node 20 shipped, `await using` is idiomatic modern
46
+ async-iterator ergonomics. Absent on older runtimes; `iter.return()`
47
+ remains the portable path.
48
+
49
+ ### Multi-waiter FIFO queue (latent bug fix)
50
+ **Triggering signal:** correctness. `Promise.all([iter.next(),
51
+ iter.next()])` silently lost the first resolver in 1.0.0 (single-slot
52
+ `pendingResolve`). Async generators queue concurrent .next() calls; ours
53
+ now does too. Invisible fix for correct callers; unblocks
54
+ `Promise.all`-style consumers.
55
+
56
+ ### `overflowCount` getter (alias for `droppedCount`)
57
+ **Triggering signal:** vocabulary consistency with `mode: "latest"`.
58
+ "Dropped" reads as FIFO-specific; "overflow" is mode-neutral.
59
+ `droppedCount` continues to work identically in both modes.
60
+
18
61
  ## 1.1.0 candidates
19
62
 
20
63
  ### Concurrent mapping (`mapLimit`)
@@ -40,16 +83,21 @@ identical "transform-with-concurrency-limit" wrappers around
40
83
 
41
84
  ---
42
85
 
43
- ### Reverse-direction buffering modes
86
+ ### Reverse-direction buffering overflow policies
44
87
 
45
- `toAsyncIterable` currently only does drop-oldest on overflow. Some
46
- consumers might want:
88
+ `toAsyncIterable` in `mode: "buffer"` currently only does drop-oldest on
89
+ overflow. Some consumers might want:
47
90
 
48
91
  - `overflow: "drop-newest"` -- reject incoming when full
49
92
  - `overflow: "throw"` -- iterator throws on overflow
50
93
  - `overflow: "block"` -- producer-side signal back-channel (requires
51
94
  consumer cooperation; significant API change)
52
95
 
96
+ **Not to be confused with `mode: "latest"`** (shipped in 1.1), which is a
97
+ different backpressure MODEL -- single-slot overwrite semantics rather
98
+ than a bounded FIFO with a different overflow policy. See the "Shipped in
99
+ 1.1.0" section above.
100
+
53
101
  **Why deferred from 1.0:** Drop-oldest is the only mode any real consumer
54
102
  in the roadmap actually wants. Adding alternatives speculatively grows the
55
103
  API surface for no measurable benefit.
package/Stream.js CHANGED
@@ -1,4 +1,4 @@
1
- // @zakkster/lite-stream 1.0.0
1
+ // @zakkster/lite-stream 1.1.0
2
2
  //
3
3
  // Zero-GC bridge between async iterators and @zakkster/lite-signal. The
4
4
  // multi-shot dual of lite-await's fromPromise: project an async source of N
@@ -20,11 +20,38 @@
20
20
  // droppedCount. Best for "process every helix page in order, don't miss
21
21
  // events".
22
22
  //
23
+ // 1.1 added `mode: "latest"`, `filter`, `timeout`, `Symbol.asyncDispose`,
24
+ // multi-waiter FIFO queue, and `overflowCount` on `toAsyncIterable`. See
25
+ // CONTRACT-toAsyncIterable-1.1.md and CHANGELOG.md for the full 1.1 story.
26
+ //
23
27
  // Copyright (c) 2026 Zahary Shinikchiev <shinikchiev@yahoo.com>
24
28
  // MIT License
25
29
 
26
30
  import { signal as _signal } from "@zakkster/lite-signal";
27
31
 
32
+ // ---------------------------------------------------------------------------
33
+ // Errors
34
+ // ---------------------------------------------------------------------------
35
+
36
+ /**
37
+ * Thrown when a `timeout` deadline elapses before iterator settlement.
38
+ * Introduced in 1.1 alongside `toAsyncIterable`'s `timeout` option.
39
+ * Structurally identical to lite-await's `TimeoutError` so users can
40
+ * duck-check via `e.name === "TimeoutError"` across both packages -- but
41
+ * NOT imported from lite-await, to preserve lite-stream's zero-dep story.
42
+ *
43
+ * The wiring (start / clear / reject with) is Session B's territory; this
44
+ * class exists in Session A only so the exported shape is stable and the
45
+ * failing test suite can compile.
46
+ */
47
+ class TimeoutError extends Error {
48
+ constructor(timeoutMs) {
49
+ super("lite-stream: timed out after " + timeoutMs + "ms");
50
+ this.name = "TimeoutError";
51
+ this.timeout = timeoutMs;
52
+ }
53
+ }
54
+
28
55
  // ---------------------------------------------------------------------------
29
56
  // Internal helpers
30
57
  // ---------------------------------------------------------------------------
@@ -498,31 +525,54 @@ function pipeToSignal(source, target, opts) {
498
525
  return stop;
499
526
  }
500
527
 
528
+ // ---------------------------------------------------------------------------
529
+ // Symbol.asyncDispose feature detection (Node 20+)
530
+ // ---------------------------------------------------------------------------
531
+ // Cached once at module load. When the runtime lacks Symbol.asyncDispose
532
+ // (Node 18/19), the iterable's disposer property is simply absent;
533
+ // iter.return() is the portable path.
534
+
535
+ const ASYNC_DISPOSE = (typeof Symbol !== "undefined" && Symbol.asyncDispose)
536
+ ? Symbol.asyncDispose
537
+ : null;
538
+
501
539
  // ---------------------------------------------------------------------------
502
540
  // toAsyncIterable -- the reverse direction
503
541
  // ---------------------------------------------------------------------------
504
542
 
505
543
  /**
506
- * Yield signal changes as an async iterable. Each change to the signal
507
- * resolves a pending `next()` call. If the consumer is slower than the
508
- * producer, values queue in an internal bounded buffer; on overflow, the
509
- * OLDEST queued value is dropped and `droppedCount` is incremented. The
510
- * dropped count is observable via the iterable's `.droppedCount` property
511
- * (read after consuming).
544
+ * Yield signal changes as an async iterable. Each change resolves a pending
545
+ * `next()` call. Two backpressure modes (1.1):
512
546
  *
513
- * The iterator naturally completes when `opts.signal` aborts; iterator.return()
514
- * resolves cleanly.
547
+ * - "buffer" (default; matches 1.0.0): FIFO ring buffer of maxBuffer size;
548
+ * on overflow the OLDEST is dropped and both droppedCount and
549
+ * overflowCount are incremented.
550
+ * - "latest" (1.1): single mutable slot; producer overwrites; every
551
+ * overwrite bumps overflowCount (and droppedCount for compat).
552
+ *
553
+ * 1.1 also adds `filter` (per-value gate; throwing filter rejects the
554
+ * current next() and terminates without a writer surface), `timeout`
555
+ * (overall deadline; rejects with TimeoutError), Symbol.asyncDispose
556
+ * on Node 20+, and a multi-waiter queue that fixes the 1.0.0 latent bug
557
+ * where Promise.all([iter.next(), iter.next()]) silently lost the first
558
+ * resolver.
559
+ *
560
+ * See CONTRACT-toAsyncIterable-1.1.md for the full locked semantics.
515
561
  *
516
562
  * @template T
517
563
  * @param {import("@zakkster/lite-signal").Signal<T> | import("@zakkster/lite-signal").Computed<T>} sig
518
564
  * @param {{
519
- * signal?: AbortSignal,
520
- * emitInitial?: boolean, // default true: yield the current value first
521
- * maxBuffer?: number // default 1024
565
+ * signal?: AbortSignal,
566
+ * emitInitial?: boolean,
567
+ * maxBuffer?: number,
568
+ * mode?: "latest" | "buffer",
569
+ * filter?: (v: T) => unknown,
570
+ * timeout?: number
522
571
  * }} [opts]
523
- * @returns {AsyncIterable<T> & { readonly droppedCount: number }}
572
+ * @returns {AsyncIterable<T> & { readonly droppedCount: number, readonly overflowCount: number }}
524
573
  */
525
574
  function toAsyncIterable(sig, opts) {
575
+ // --- Validation ---
526
576
  if (sig === null || sig === undefined || typeof sig.subscribe !== "function" || typeof sig.peek !== "function") {
527
577
  throw new TypeError(
528
578
  "lite-stream: toAsyncIterable expects a readable lite-signal (Signal/Computed with .peek and .subscribe)"
@@ -530,101 +580,273 @@ function toAsyncIterable(sig, opts) {
530
580
  }
531
581
  const abortSig = (opts !== undefined && opts !== null) ? opts.signal : undefined;
532
582
  const emitInitial = (opts !== undefined && opts !== null && opts.emitInitial === false) ? false : true;
533
- const maxBuffer = (opts !== undefined && opts !== null && opts.maxBuffer !== undefined) ? opts.maxBuffer : 1024;
583
+ const filter = (opts !== undefined && opts !== null) ? opts.filter : undefined;
584
+ const timeoutMs = (opts !== undefined && opts !== null) ? opts.timeout : undefined;
585
+ const modeOpt = (opts !== undefined && opts !== null) ? opts.mode : undefined;
586
+ const maxBufferOpt = (opts !== undefined && opts !== null) ? opts.maxBuffer : undefined;
587
+
588
+ // Mode. Default "buffer" preserves 1.0.0 behavior.
589
+ let mode;
590
+ if (modeOpt === undefined || modeOpt === "buffer") {
591
+ mode = "buffer";
592
+ } else if (modeOpt === "latest") {
593
+ mode = "latest";
594
+ } else {
595
+ throw new TypeError(
596
+ "lite-stream: opts.mode must be \"latest\" or \"buffer\" (got " + JSON.stringify(modeOpt) + ")"
597
+ );
598
+ }
599
+ const isLatestWins = mode === "latest";
534
600
 
535
- if (typeof maxBuffer !== "number" || !Number.isFinite(maxBuffer) || maxBuffer < 1 || (maxBuffer | 0) !== maxBuffer) {
536
- throw new RangeError(
537
- "lite-stream: opts.maxBuffer must be a positive integer (got " + JSON.stringify(maxBuffer) + ")"
601
+ // maxBuffer only meaningful in buffer mode. Combining latest + maxBuffer
602
+ // is user error; silently ignoring one would hide the mistake.
603
+ if (isLatestWins && maxBufferOpt !== undefined) {
604
+ throw new TypeError(
605
+ "lite-stream: opts.maxBuffer is not allowed with mode: \"latest\" -- latest-wins uses a single slot"
538
606
  );
539
607
  }
608
+ const maxBuffer = maxBufferOpt !== undefined ? maxBufferOpt : 1024;
609
+ if (!isLatestWins) {
610
+ if (typeof maxBuffer !== "number" || !Number.isFinite(maxBuffer) || maxBuffer < 1 || (maxBuffer | 0) !== maxBuffer) {
611
+ throw new RangeError(
612
+ "lite-stream: opts.maxBuffer must be a positive integer (got " + JSON.stringify(maxBuffer) + ")"
613
+ );
614
+ }
615
+ }
540
616
 
541
- const queue = new Array(maxBuffer);
542
- let qHead = 0;
543
- let qLen = 0;
544
- let droppedCount = 0;
545
- let pendingResolve = null;
617
+ if (filter !== undefined && typeof filter !== "function") {
618
+ throw new TypeError("lite-stream: opts.filter must be a function");
619
+ }
620
+ if (timeoutMs !== undefined) {
621
+ if (typeof timeoutMs !== "number" || !Number.isFinite(timeoutMs) || timeoutMs < 0) {
622
+ throw new RangeError(
623
+ "lite-stream: opts.timeout must be a finite non-negative number (got " + JSON.stringify(timeoutMs) + ")"
624
+ );
625
+ }
626
+ }
627
+
628
+ // --- State ---
629
+ // Buffer-mode ring (unchanged from 1.0.0 layout); null in latest mode.
630
+ const queue = isLatestWins ? null : new Array(maxBuffer);
631
+ let qHead = 0;
632
+ let qLen = 0;
633
+ // Latest-mode slot.
634
+ let pendingValue;
635
+ let hasPending = false;
636
+ // Overflow. `droppedCount` and `overflowCount` are aliases for the same
637
+ // counter; 1.0.0 exposed `droppedCount`, 1.1 adds `overflowCount` as the
638
+ // mode-neutral vocabulary. Both getters read this variable.
639
+ let overflowCount = 0;
640
+ // Multi-waiter FIFO queue (fixes the 1.0.0 single-slot bug where
641
+ // concurrent .next() calls silently lost the first resolver).
642
+ const waiters = [];
643
+ // Termination
546
644
  let done = false;
645
+ let firstNextErr = null; // stashed for the next .next() when no waiter was pending
646
+ // Cleanup handles
547
647
  let unsubscribe = null;
648
+ let unsubscribePending = false;
548
649
  let abortListener = null;
549
-
550
- function enqueue(v) {
551
- if (done) return;
552
- if (pendingResolve !== null) {
553
- const r = pendingResolve;
554
- pendingResolve = null;
555
- r({ value: v, done: false });
556
- return;
557
- }
558
- if (qLen < maxBuffer) {
559
- queue[(qHead + qLen) % maxBuffer] = v;
560
- qLen = (qLen + 1) | 0;
650
+ let timeoutId = null;
651
+
652
+ // Late-binding unsubscribe: needed when a throwing filter (or throwing
653
+ // subscription callback body) fires during the synchronous initial
654
+ // subscribe call, at which point `unsubscribe` is still null. Mirror of
655
+ // lite-await's stop/stopPending pattern.
656
+ function doUnsubscribe() {
657
+ if (unsubscribe !== null) {
658
+ unsubscribe();
659
+ unsubscribe = null;
561
660
  } else {
562
- // Drop oldest.
563
- queue[qHead] = v;
564
- qHead = (qHead + 1) % maxBuffer;
565
- droppedCount = (droppedCount + 1) | 0;
661
+ unsubscribePending = true;
566
662
  }
567
663
  }
568
664
 
569
- function teardown() {
570
- if (done) return;
571
- done = true;
572
- if (unsubscribe !== null) { unsubscribe(); unsubscribe = null; }
665
+ function fullCleanup() {
666
+ doUnsubscribe();
667
+ if (timeoutId !== null) {
668
+ clearTimeout(timeoutId);
669
+ timeoutId = null;
670
+ }
573
671
  if (abortListener !== null && abortSig !== undefined && abortSig !== null) {
574
672
  abortSig.removeEventListener("abort", abortListener);
575
673
  abortListener = null;
576
674
  }
577
- if (pendingResolve !== null) {
578
- const r = pendingResolve;
579
- pendingResolve = null;
580
- r({ value: undefined, done: true });
675
+ // Release unread values so they can be GC'd before the iterable
676
+ // object itself is. One-time cost per iterable lifetime.
677
+ pendingValue = undefined;
678
+ hasPending = false;
679
+ if (queue !== null) {
680
+ for (let i = 0; i < maxBuffer; i = (i + 1) | 0) queue[i] = undefined;
681
+ qHead = 0;
682
+ qLen = 0;
581
683
  }
582
684
  }
583
685
 
584
- if (abortSig !== undefined && abortSig !== null) {
585
- if (abortSig.aborted) {
586
- done = true;
686
+ // Termination via error: reject all pending waiters, or stash for the
687
+ // next .next() call. Route for timeout and throwing-filter paths.
688
+ function terminateWithError(err) {
689
+ if (done) return;
690
+ done = true;
691
+ fullCleanup();
692
+ if (waiters.length > 0) {
693
+ while (waiters.length > 0) {
694
+ const w = waiters.shift();
695
+ w.reject(err);
696
+ }
587
697
  } else {
588
- abortListener = function () { teardown(); };
589
- abortSig.addEventListener("abort", abortListener);
698
+ firstNextErr = err;
590
699
  }
591
700
  }
592
701
 
593
- // Subscribe to signal changes. lite-signal's subscribe fires once
594
- // synchronously with the current value when called; we use a flag to
595
- // suppress that initial fire if emitInitial is false.
596
- let suppressedInitial = !emitInitial;
597
- if (!done) {
598
- unsubscribe = sig.subscribe(function (v) {
599
- if (suppressedInitial) { suppressedInitial = false; return; }
600
- enqueue(v);
601
- });
702
+ // Termination via done (abort, natural end via consumer return()): resolve
703
+ // pending waiters with {done: true}. Preserves 1.0.0 abort-is-graceful
704
+ // behavior (mid-iteration abort ends for-await without throw).
705
+ function terminateAsDone() {
706
+ if (done) return;
707
+ done = true;
708
+ fullCleanup();
709
+ while (waiters.length > 0) {
710
+ const w = waiters.shift();
711
+ w.resolve({ value: undefined, done: true });
712
+ }
602
713
  }
603
714
 
715
+ function enqueue(v) {
716
+ if (done) return;
717
+ // Deliver directly to the oldest waiter if any.
718
+ if (waiters.length > 0) {
719
+ const w = waiters.shift();
720
+ w.resolve({ value: v, done: false });
721
+ return;
722
+ }
723
+ // Otherwise stash in the mode-appropriate structure.
724
+ if (isLatestWins) {
725
+ if (hasPending) overflowCount = (overflowCount + 1) | 0;
726
+ pendingValue = v;
727
+ hasPending = true;
728
+ } else {
729
+ if (qLen < maxBuffer) {
730
+ queue[(qHead + qLen) % maxBuffer] = v;
731
+ qLen = (qLen + 1) | 0;
732
+ } else {
733
+ // Drop oldest.
734
+ queue[qHead] = v;
735
+ qHead = (qHead + 1) % maxBuffer;
736
+ overflowCount = (overflowCount + 1) | 0;
737
+ }
738
+ }
739
+ }
740
+
741
+ // --- Iterable object ---
604
742
  const iterable = {
605
743
  [Symbol.asyncIterator]() { return this; },
606
744
  next() {
607
- if (qLen > 0) {
745
+ if (done) {
746
+ if (firstNextErr !== null) {
747
+ const err = firstNextErr;
748
+ firstNextErr = null;
749
+ return Promise.reject(err);
750
+ }
751
+ return Promise.resolve({ value: undefined, done: true });
752
+ }
753
+ // Value available in slot / ring?
754
+ if (isLatestWins && hasPending) {
755
+ const v = pendingValue;
756
+ pendingValue = undefined;
757
+ hasPending = false;
758
+ return Promise.resolve({ value: v, done: false });
759
+ }
760
+ if (!isLatestWins && qLen > 0) {
608
761
  const v = queue[qHead];
609
762
  queue[qHead] = undefined;
610
763
  qHead = (qHead + 1) % maxBuffer;
611
764
  qLen = (qLen - 1) | 0;
612
765
  return Promise.resolve({ value: v, done: false });
613
766
  }
614
- if (done) return Promise.resolve({ value: undefined, done: true });
615
- return new Promise(function (resolve) { pendingResolve = resolve; });
767
+ // Nothing available -- queue a waiter.
768
+ return new Promise(function (resolve, reject) {
769
+ waiters.push({ resolve: resolve, reject: reject });
770
+ });
616
771
  },
617
772
  return(value) {
618
- teardown();
773
+ if (done) return Promise.resolve({ value: undefined, done: true });
774
+ terminateAsDone();
619
775
  return Promise.resolve({ value: value, done: true });
620
776
  },
621
777
  throw(err) {
622
- teardown();
778
+ if (done) return Promise.reject(err);
779
+ // Consumer-driven throw: pending waiters reject with the error,
780
+ // subsequent next() returns done. Same shape as terminateWithError
781
+ // but without stashing (throw() call site gets its own rejection).
782
+ done = true;
783
+ fullCleanup();
784
+ while (waiters.length > 0) {
785
+ const w = waiters.shift();
786
+ w.reject(err);
787
+ }
623
788
  return Promise.reject(err);
624
789
  },
625
- get droppedCount() { return droppedCount; }
790
+ get droppedCount() { return overflowCount; },
791
+ get overflowCount() { return overflowCount; }
626
792
  };
627
793
 
794
+ if (ASYNC_DISPOSE !== null) {
795
+ iterable[ASYNC_DISPOSE] = function () { return iterable.return(); };
796
+ }
797
+
798
+ // --- Pre-aborted signal: mark done, skip all subscription/timer setup.
799
+ // Matches 1.0.0 behavior: first next() resolves with {done: true} rather
800
+ // than rejecting. ---
801
+ if (abortSig !== undefined && abortSig !== null && abortSig.aborted) {
802
+ done = true;
803
+ return iterable;
804
+ }
805
+
806
+ // --- Timeout setup ---
807
+ if (timeoutMs !== undefined) {
808
+ timeoutId = setTimeout(function () {
809
+ terminateWithError(new TimeoutError(timeoutMs));
810
+ }, timeoutMs);
811
+ }
812
+
813
+ // --- AbortSignal listener ---
814
+ if (abortSig !== undefined && abortSig !== null) {
815
+ abortListener = function () { terminateAsDone(); };
816
+ abortSig.addEventListener("abort", abortListener);
817
+ }
818
+
819
+ // --- Subscribe. lite-signal's subscribe fires synchronously with the
820
+ // current value on registration; use a flag to suppress that initial
821
+ // fire if emitInitial is false. Filter (if provided) applies to the
822
+ // initial fire when emitInitial is true. A throwing filter is routed
823
+ // via terminateWithError so the throw never surfaces at the signal
824
+ // writer's .set() call site. ---
825
+ let suppressedInitial = !emitInitial;
826
+ unsubscribe = sig.subscribe(function (v) {
827
+ if (done) return;
828
+ if (suppressedInitial) { suppressedInitial = false; return; }
829
+ if (filter !== undefined) {
830
+ let ok;
831
+ try {
832
+ ok = filter(v);
833
+ } catch (e) {
834
+ terminateWithError(e);
835
+ return;
836
+ }
837
+ if (!ok) return;
838
+ }
839
+ enqueue(v);
840
+ });
841
+
842
+ // If terminateWithError was called from inside the synchronous initial
843
+ // subscribe fire (throwing filter on the initial value), honor the
844
+ // deferred unsubscribe now that we have the handle.
845
+ if (unsubscribePending && unsubscribe !== null) {
846
+ unsubscribe();
847
+ unsubscribe = null;
848
+ }
849
+
628
850
  return iterable;
629
851
  }
630
852
 
@@ -635,5 +857,6 @@ function toAsyncIterable(sig, opts) {
635
857
  export {
636
858
  fromAsyncIterable,
637
859
  pipeToSignal,
638
- toAsyncIterable
860
+ toAsyncIterable,
861
+ TimeoutError
639
862
  };
package/llms.txt CHANGED
@@ -14,7 +14,7 @@ Peer dep: `@zakkster/lite-signal ^1.2.0`.
14
14
 
15
15
  ## Imports
16
16
 
17
- import { fromAsyncIterable, pipeToSignal, toAsyncIterable } from "@zakkster/lite-stream";
17
+ import { fromAsyncIterable, pipeToSignal, toAsyncIterable, TimeoutError } from "@zakkster/lite-stream";
18
18
 
19
19
  ## Exports
20
20
 
@@ -63,23 +63,60 @@ const stop = pipeToSignal(source, targetSig, {
63
63
 
64
64
  Does NOT dispose the target signal. The caller owns its lifetime.
65
65
 
66
- ### `toAsyncIterable(sig, opts?) -> AsyncIterable<T> & { droppedCount }`
66
+ ### `toAsyncIterable(sig, opts?) -> AsyncIterable<T> & { droppedCount, overflowCount }`
67
67
 
68
- Reverse direction: yield signal changes as an async iterable. Bounded
69
- internal queue (default 1024) with drop-oldest overflow.
68
+ Reverse direction: yield signal changes as an async iterable.
69
+
70
+ **1.0.0 shape (still the default):** bounded FIFO ring buffer, drop-oldest
71
+ overflow, `droppedCount` observable.
72
+
73
+ **1.1 additions (all opt-in, no default changes):**
74
+ - `mode: "latest"` -- single-slot overwrite semantics; use for reactive
75
+ state / "current frame" style consumers
76
+ - `filter` -- per-value gate; throwing filter rejects pending next() and
77
+ terminates (no writer surface)
78
+ - `timeout` -- overall deadline; pending next() rejects with `TimeoutError`
79
+ - `Symbol.asyncDispose` on Node 20+ -- enables `await using iter = ...`
80
+ - `overflowCount` -- alias for `droppedCount`, mode-neutral vocabulary
81
+ - Multi-waiter queue -- concurrent .next() calls resolve in FIFO order
82
+ (1.0.0 silently overwrote the first resolver)
70
83
 
71
84
  ```js
85
+ // 1.0.0-style: bounded FIFO buffer (default)
72
86
  for await (const v of toAsyncIterable(sig, {
73
87
  signal: abortCtrl.signal,
74
- emitInitial: true, // default true
75
- maxBuffer: 1024 // default 1024
88
+ emitInitial: true, // default true
89
+ maxBuffer: 1024 // default 1024
90
+ })) {
91
+ // ...
92
+ }
93
+
94
+ // 1.1: latest-wins slot for reactive state
95
+ for await (const v of toAsyncIterable(sig, {
96
+ mode: "latest", // opt in to single-slot overwrite
97
+ filter: (v) => v.priority > 0,
98
+ timeout: 5000 // overall deadline
76
99
  })) {
77
100
  // ...
78
101
  }
79
102
  ```
80
103
 
81
- The iterator naturally completes when `opts.signal` aborts. Consumer-side
82
- `break` triggers `iterator.return()` and cleans up the subscription.
104
+ Constraints:
105
+ - `mode: "latest"` + `maxBuffer` throws TypeError (single slot has no buffer size)
106
+ - Negative or non-finite `timeout` throws RangeError
107
+ - Non-function `filter` throws TypeError
108
+
109
+ The iterator naturally completes when `opts.signal` aborts (resolves as
110
+ done, not rejection -- graceful termination). Consumer-side `break`
111
+ triggers `iterator.return()` and cleans up. `timeout` elapsing rejects
112
+ the pending next() with `TimeoutError`; subsequent calls return done.
113
+
114
+ ### `TimeoutError` (added 1.1)
115
+
116
+ Thrown when the `timeout` option on `toAsyncIterable` elapses.
117
+ Structurally identical to `@zakkster/lite-await`'s TimeoutError so
118
+ `e.name === "TimeoutError"` duck-checks work across both packages. Not
119
+ imported from lite-await -- lite-stream stays zero-dep.
83
120
 
84
121
  ## Cleanup termination triplet
85
122
 
@@ -188,18 +225,49 @@ states: {
188
225
  - Not a replacement for lite-clock for frame-rate loops.
189
226
  - Not for one-shot promises -- use lite-await's `fromPromise`.
190
227
 
228
+ ## Anti-patterns
229
+
230
+ - DO NOT combine `mode: "latest"` with `maxBuffer` on `toAsyncIterable`.
231
+ It throws TypeError at construction -- a single-slot mode with a buffer
232
+ size is user error. Pick one or the other.
233
+ - DO NOT reach for lite-stream for one-shot Signal<->Promise coordination.
234
+ That is `@zakkster/lite-await`'s job (`whenSignal`, `fromPromise`,
235
+ `allOf`/`anyOf`/`raceOf`). lite-stream owns the AsyncIterable boundary;
236
+ lite-await owns the Promise boundary.
237
+ - DO NOT wrap `toAsyncIterable` with an async-generator debounce / throttle
238
+ for reactive-state use cases. Use `mode: "latest"` directly -- it
239
+ already gives you "consume newest, discard intermediates" semantics with
240
+ zero per-value allocation.
241
+ - DO NOT rely on the value passed to `iter.return(value)` reaching pending
242
+ waiters. Pending `next()` calls resolve with `{value: undefined, done:
243
+ true}` per the async iteration protocol; only `return()`'s own returned
244
+ Promise carries the value.
245
+ - DO NOT dispose the source signal while an iterable is still consuming
246
+ from it. Call `iter.return()` first; disposing an in-flight source is
247
+ undefined behavior.
248
+
191
249
  ## Performance
192
250
 
193
- 178K ops/s for the full lifecycle in latest mode (construct + pull + settle
194
- + dispose), sub-byte/op retained heap. 235K ops/s for pipeToSignal direct.
195
- 84K ops/s for the full abort cycle.
251
+ 178K ops/s for the full lifecycle in `fromAsyncIterable` latest mode
252
+ (construct + pull + settle + dispose), sub-byte/op retained heap. 235K
253
+ ops/s for `pipeToSignal` direct. 84K ops/s for the full abort cycle.
254
+
255
+ 1.1 `toAsyncIterable` throughput (Node v22, 500ms runs):
256
+ - `mode: "latest"` 1:1 alternation: 119K cycles/s (2.4M ops/s per pair)
257
+ - `mode: "latest"` churn (100 writes coalesced): 60K cycles/s (6M
258
+ writes/s absorbed into the single slot)
259
+ - `filter + timeout`: 141K ops/s, negative retained (the timer-clear
260
+ discipline on `iter.return()` helps V8 reclaim more than baseline)
261
+ - multi-waiter x3 concurrent .next(): 120K cycles/s
196
262
 
197
- Per-yield, latest mode allocates one wrapper state object. Buffer mode
198
- adds one snapshot array per yield. Ring beneath is fixed pre-allocated.
263
+ Per-yield, `fromAsyncIterable` latest mode allocates one wrapper state
264
+ object; buffer mode adds one snapshot array. `toAsyncIterable` `mode:
265
+ "latest"` allocates zero on the steady-state hot path when the consumer
266
+ is caught up; `mode: "buffer"` also zero when no waiter is pending.
199
267
 
200
268
  ## Files
201
269
 
202
- - `Stream.js` -- single-file ESM implementation (~600 lines)
270
+ - `Stream.js` -- single-file ESM implementation (~830 lines)
203
271
  - `Stream.d.ts` -- TypeScript types with discriminated state union
204
272
  - `README.md` -- full docs with integration recipes
205
273
  - `llms.txt` -- this file
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@zakkster/lite-stream",
3
- "version": "1.0.0",
3
+ "version": "1.1.0",
4
4
  "description": "Zero-GC bridge between async iterators and @zakkster/lite-signal. Project async streams (paginated APIs, SSE, network frames, pubsub topics) into signals. Bounded buffering with explicit overflow diagnostics, structural cleanup on three termination paths (iterator done, abort, dispose). The multi-shot dual of lite-await's fromPromise.",
5
5
  "author": "Zahary Shinikchiev <shinikchiev@yahoo.com>",
6
6
  "license": "MIT",