@zakkster/lite-stream 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +85 -0
- package/LICENSE.txt +21 -0
- package/README.md +521 -0
- package/ROADMAP.md +191 -0
- package/Stream.d.ts +137 -0
- package/Stream.js +639 -0
- package/llms.txt +212 -0
- package/package.json +71 -0
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to `@zakkster/lite-stream` are documented in this file.
|
|
4
|
+
|
|
5
|
+
The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
|
|
6
|
+
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
|
+
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
## 1.0.0 -- 2026-06-23
|
|
11
|
+
|
|
12
|
+
**Initial public release.** API frozen; subsequent 1.x releases are purely
|
|
13
|
+
additive.
|
|
14
|
+
|
|
15
|
+
### Added
|
|
16
|
+
|
|
17
|
+
- **`fromAsyncIterable(source, opts?)`** -- drive a signal from an async
|
|
18
|
+
iterator. Two modes:
|
|
19
|
+
- `"latest"` (default): signal value is `{ value, count, done, error }`.
|
|
20
|
+
The most recent yielded item wins; cheapest path.
|
|
21
|
+
- `"buffer"`: signal value is `{ values, count, droppedCount, done, error }`.
|
|
22
|
+
Bounded ring buffer (newest-last). REQUIRES `maxBuffer`. Overflow drops
|
|
23
|
+
oldest and increments `droppedCount`. Unbounded buffering is rejected
|
|
24
|
+
with a `RangeError`.
|
|
25
|
+
|
|
26
|
+
Accepts `AsyncIterable`, `AsyncIterator` (just `.next()`), or sync
|
|
27
|
+
`Iterable` as a convenience.
|
|
28
|
+
|
|
29
|
+
- **`pipeToSignal(source, target, opts?)`** -- lower-level companion that
|
|
30
|
+
pumps an existing writable signal from an async iterator. The signal's
|
|
31
|
+
value is replaced directly (no wrapper). Returns an idempotent stop fn.
|
|
32
|
+
Does NOT dispose the target signal.
|
|
33
|
+
|
|
34
|
+
- **`toAsyncIterable(sig, opts?)`** -- reverse direction. Yield signal
|
|
35
|
+
changes as an async iterable. Bounded internal queue (default 1024) with
|
|
36
|
+
drop-oldest overflow. `iterable.droppedCount` is observable. Consumer
|
|
37
|
+
`break` triggers `iterator.return()`. Pre-aborted AbortSignal short-
|
|
38
|
+
circuits with `done: true`.
|
|
39
|
+
|
|
40
|
+
### Cleanup termination triplet
|
|
41
|
+
|
|
42
|
+
All three paths handled structurally; abort listeners always removed.
|
|
43
|
+
|
|
44
|
+
1. Iterator natural completion (`{ done: true }`) -> done state, `onDone`
|
|
45
|
+
fires.
|
|
46
|
+
2. Iterator throws -> error state with the thrown value, `onError` fires.
|
|
47
|
+
3. AbortSignal aborts -> `iter.return()` called best-effort, error state
|
|
48
|
+
with `AbortError`, `onError` fires.
|
|
49
|
+
|
|
50
|
+
Additionally: if the consumer disposes the result signal externally, the
|
|
51
|
+
pump detects on next `sig.set()` and tears down silently (no `onError` --
|
|
52
|
+
this is consumer-initiated cleanup).
|
|
53
|
+
|
|
54
|
+
### Performance (Node v22, --expose-gc, 500ms runs)
|
|
55
|
+
|
|
56
|
+
- `from-latest-10`: **178K ops/s**, 0.07 B/op retained
|
|
57
|
+
- `from-buffer-20-drop`: **128K ops/s**, 0.13 B/op retained
|
|
58
|
+
- `abort-cycle`: **84K ops/s**, 0.71 B/op retained
|
|
59
|
+
- `pipe-to-signal`: **235K ops/s**, 0.10 B/op retained
|
|
60
|
+
- `to-async-iterable`: **42K ops/s**, ~0 B/op retained
|
|
61
|
+
- `to-async-iterable-overflow`: **62K ops/s**, ~25 B/op retained
|
|
62
|
+
|
|
63
|
+
### Testing
|
|
64
|
+
|
|
65
|
+
49 tests across 6 files:
|
|
66
|
+
- `01-from-async-iterable-latest.test.mjs` (11 tests)
|
|
67
|
+
- `02-from-async-iterable-buffer.test.mjs` (8 tests)
|
|
68
|
+
- `03-cleanup-triplet.test.mjs` (7 tests including a 200-cycle leak smoke)
|
|
69
|
+
- `04-pipe-to-signal.test.mjs` (10 tests)
|
|
70
|
+
- `05-to-async-iterable.test.mjs` (10 tests)
|
|
71
|
+
- `06-gc.test.mjs` (3 heap-budget tests under `--expose-gc`)
|
|
72
|
+
|
|
73
|
+
All 49 tests pass under both `npm test` and `npm run test:gc`.
|
|
74
|
+
|
|
75
|
+
### Bundle
|
|
76
|
+
|
|
77
|
+
- ~5 KB minified, ~2 KB gzipped
|
|
78
|
+
- ESM-only, Node >= 18
|
|
79
|
+
- Single file `Stream.js`
|
|
80
|
+
- Zero runtime dependencies
|
|
81
|
+
- Peer dep: `@zakkster/lite-signal ^1.2.0`
|
|
82
|
+
|
|
83
|
+
### License
|
|
84
|
+
|
|
85
|
+
MIT. Copyright (c) 2026 Zahary Shinikchiev.
|
package/LICENSE.txt
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Zahary Shinikchiev <shinikchiev@yahoo.com>
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,521 @@
|
|
|
1
|
+
# @zakkster/lite-stream
|
|
2
|
+
|
|
3
|
+
[](https://www.npmjs.com/package/@zakkster/lite-stream)
|
|
4
|
+

|
|
5
|
+
[](https://github.com/sponsors/PeshoVurtoleta)
|
|
6
|
+
[](https://bundlephobia.com/result?p=@zakkster/lite-stream)
|
|
7
|
+
[](https://www.npmjs.com/package/@zakkster/lite-stream)
|
|
8
|
+
[](https://www.npmjs.com/package/@zakkster/lite-stream)
|
|
9
|
+
[](https://github.com/PeshoVurtoleta/lite-signal)
|
|
10
|
+

|
|
11
|
+

|
|
12
|
+
[](https://opensource.org/licenses/MIT)
|
|
13
|
+
|
|
14
|
+
Zero-GC bridge between async iterators and
|
|
15
|
+
[`@zakkster/lite-signal`](https://www.npmjs.com/package/@zakkster/lite-signal).
|
|
16
|
+
The multi-shot dual of [`lite-await`](https://www.npmjs.com/package/@zakkster/lite-await)'s
|
|
17
|
+
`fromPromise`: project an async source of N values (paginated APIs, SSE
|
|
18
|
+
streams, network frame queues, pubsub topics) into a signal-shaped reactive
|
|
19
|
+
surface, with bounded buffering and structural cleanup on every termination
|
|
20
|
+
path.
|
|
21
|
+
|
|
22
|
+
## Why this exists
|
|
23
|
+
|
|
24
|
+
The hand-rolled version of "drive a signal from an async iterator" is a
|
|
25
|
+
five-line for-await loop that leaks the iterator forever the moment the
|
|
26
|
+
consumer navigates away, the AbortSignal aborts, or the parent component
|
|
27
|
+
unmounts. Every place in the `@zakkster/lite-*` ecosystem that handles a
|
|
28
|
+
multi-shot async source -- helix pagination, EBS server-sent events,
|
|
29
|
+
pubsub topic subscriptions, rollback input-frame queues -- would otherwise
|
|
30
|
+
write that broken loop. This library is the one correct implementation.
|
|
31
|
+
|
|
32
|
+
```
|
|
33
|
+
+-------------------------+ +-----------------+ +-----------------+
|
|
34
|
+
| AsyncIterable<T> | | lite-stream | | Signal<State> |
|
|
35
|
+
| (helix pagination, | -----> | fromAsync- | -----> | (your effect() |
|
|
36
|
+
| EBS SSE, pubsub, | | Iterable | | reads this) |
|
|
37
|
+
| rollback frames) | +-----------------+ +-----------------+
|
|
38
|
+
+-------------------------+ |
|
|
39
|
+
| three structural cleanup paths:
|
|
40
|
+
| (a) iterator done -> done: true
|
|
41
|
+
| (b) iterator throws -> error: ...
|
|
42
|
+
| (c) AbortSignal abort -> iter.return()
|
|
43
|
+
v
|
|
44
|
+
no leaks. no listeners outliving the stream.
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
## Install
|
|
48
|
+
|
|
49
|
+
```sh
|
|
50
|
+
npm install @zakkster/lite-stream @zakkster/lite-signal
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
`@zakkster/lite-signal` is a peer dependency (^1.2.0). No other runtime deps.
|
|
54
|
+
|
|
55
|
+
## Quick start
|
|
56
|
+
|
|
57
|
+
```js
|
|
58
|
+
import { fromAsyncIterable } from "@zakkster/lite-stream";
|
|
59
|
+
import { effect } from "@zakkster/lite-signal";
|
|
60
|
+
|
|
61
|
+
const ctrl = new AbortController();
|
|
62
|
+
|
|
63
|
+
const live = fromAsyncIterable(networkFrames(), {
|
|
64
|
+
signal: ctrl.signal,
|
|
65
|
+
onDone: () => console.log("frame stream ended"),
|
|
66
|
+
onError: (e) => console.error("frame stream error", e)
|
|
67
|
+
});
|
|
68
|
+
|
|
69
|
+
effect(() => {
|
|
70
|
+
const s = live();
|
|
71
|
+
if (s.error) renderError(s.error);
|
|
72
|
+
else if (s.done) renderEndCard();
|
|
73
|
+
else if (s.value) renderFrame(s.value);
|
|
74
|
+
});
|
|
75
|
+
|
|
76
|
+
// Later -- consumer navigates away:
|
|
77
|
+
ctrl.abort(); // iterator.return() called; signal settles; no leaks
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
## Table of contents
|
|
81
|
+
|
|
82
|
+
- [The cleanup termination triplet](#the-cleanup-termination-triplet)
|
|
83
|
+
- [API reference](#api-reference)
|
|
84
|
+
- [fromAsyncIterable](#fromasynciterable)
|
|
85
|
+
- [pipeToSignal](#pipetosignal)
|
|
86
|
+
- [toAsyncIterable](#toasynciterable)
|
|
87
|
+
- [Modes: latest vs buffer](#modes-latest-vs-buffer)
|
|
88
|
+
- [Zero-GC hot paths](#zero-gc-hot-paths)
|
|
89
|
+
- [Integration recipes](#integration-recipes)
|
|
90
|
+
- [Edge cases pinned down](#edge-cases-pinned-down)
|
|
91
|
+
- [Benchmarks](#benchmarks)
|
|
92
|
+
- [Testing strategy](#testing-strategy)
|
|
93
|
+
- [What this is not](#what-this-is-not)
|
|
94
|
+
- [Ecosystem](#ecosystem)
|
|
95
|
+
|
|
96
|
+
## The cleanup termination triplet
|
|
97
|
+
|
|
98
|
+
A stream-to-signal bridge ends on exactly three paths. `lite-stream` handles
|
|
99
|
+
all three structurally:
|
|
100
|
+
|
|
101
|
+
1. **Iterator natural completion** -- the source yields `{ done: true }`. The
|
|
102
|
+
signal state settles with `done: true`. `onDone` fires once. No further
|
|
103
|
+
state mutations occur.
|
|
104
|
+
|
|
105
|
+
2. **Iterator throws** -- the source rejects a pull or throws synchronously
|
|
106
|
+
from `next()`. The signal state settles with `error: <thrown>` and
|
|
107
|
+
`done: true`. `onError` fires once with the thrown value. The last
|
|
108
|
+
successfully yielded value is preserved.
|
|
109
|
+
|
|
110
|
+
3. **AbortSignal aborts** -- the caller-provided `opts.signal` enters the
|
|
111
|
+
aborted state. `iter.return()` is called best-effort to give the
|
|
112
|
+
generator a chance to clean up. The signal state settles with
|
|
113
|
+
`error: <AbortError>` and `done: true`. `onError` fires.
|
|
114
|
+
|
|
115
|
+
The abort listener is registered with `addEventListener("abort", ...)` and
|
|
116
|
+
**always removed** on any of the three paths -- no AbortSignal accumulates
|
|
117
|
+
dangling listeners across stream lifecycles.
|
|
118
|
+
|
|
119
|
+
If the consumer disposes the result signal via `lite-signal`'s `dispose()`,
|
|
120
|
+
the pump detects the disposal on its next attempted `sig.set()`, tears down
|
|
121
|
+
the iterator silently (no `onError` fired -- this is consumer-initiated
|
|
122
|
+
cleanup, not a stream error), and exits.
|
|
123
|
+
|
|
124
|
+
## API reference
|
|
125
|
+
|
|
126
|
+
### fromAsyncIterable
|
|
127
|
+
|
|
128
|
+
```ts
|
|
129
|
+
fromAsyncIterable<T>(
|
|
130
|
+
source: AsyncIterable<T> | AsyncIterator<T> | Iterable<T>,
|
|
131
|
+
opts?: {
|
|
132
|
+
mode?: "latest" | "buffer", // default "latest"
|
|
133
|
+
maxBuffer?: number, // REQUIRED if mode = "buffer"
|
|
134
|
+
initial?: T, // initial signal value
|
|
135
|
+
signal?: AbortSignal, // abort to stop
|
|
136
|
+
onError?: (err: unknown) => void,
|
|
137
|
+
onDone?: () => void
|
|
138
|
+
}
|
|
139
|
+
): Signal<State<T>>
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
Drive a signal from an async iterator. Returns a Signal whose value is a
|
|
143
|
+
tagged state object reflecting the iterator's lifecycle.
|
|
144
|
+
|
|
145
|
+
State shape varies by mode:
|
|
146
|
+
|
|
147
|
+
```ts
|
|
148
|
+
// "latest" mode
|
|
149
|
+
{ value: T | undefined, count: number, done: boolean, error: unknown }
|
|
150
|
+
|
|
151
|
+
// "buffer" mode
|
|
152
|
+
{ values: T[], count: number, droppedCount: number, done: boolean, error: unknown }
|
|
153
|
+
```
|
|
154
|
+
|
|
155
|
+
`count` increments on every yielded value (regardless of whether it survives
|
|
156
|
+
buffering). `done` flips to `true` on any of the three termination paths.
|
|
157
|
+
`error` is `undefined` on natural completion, populated otherwise.
|
|
158
|
+
|
|
159
|
+
### pipeToSignal
|
|
160
|
+
|
|
161
|
+
```ts
|
|
162
|
+
pipeToSignal<T>(
|
|
163
|
+
source: AsyncIterable<T> | AsyncIterator<T>,
|
|
164
|
+
target: Signal<T>,
|
|
165
|
+
opts?: {
|
|
166
|
+
signal?: AbortSignal,
|
|
167
|
+
onError?: (err: unknown) => void,
|
|
168
|
+
onDone?: () => void,
|
|
169
|
+
transform?: (value: T) => T
|
|
170
|
+
}
|
|
171
|
+
): () => void // stop fn (idempotent)
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
Lower-level companion: pump an existing writable signal from an async
|
|
175
|
+
iterator. The signal's value is replaced directly with each yielded value
|
|
176
|
+
(no `{ value, count, ... }` wrapper). Returns a `stop` function.
|
|
177
|
+
|
|
178
|
+
Use `pipeToSignal` when:
|
|
179
|
+
|
|
180
|
+
- You already have a signal you want to drive
|
|
181
|
+
- You don't need the lifecycle metadata wrapper
|
|
182
|
+
- You want a `stop` fn instead of an `AbortController` for cleanup
|
|
183
|
+
|
|
184
|
+
**Does NOT dispose the target signal.** The caller owns its lifetime.
|
|
185
|
+
|
|
186
|
+
### toAsyncIterable
|
|
187
|
+
|
|
188
|
+
```ts
|
|
189
|
+
toAsyncIterable<T>(
|
|
190
|
+
sig: Signal<T> | Computed<T>,
|
|
191
|
+
opts?: {
|
|
192
|
+
signal?: AbortSignal,
|
|
193
|
+
emitInitial?: boolean, // default true
|
|
194
|
+
maxBuffer?: number // default 1024
|
|
195
|
+
}
|
|
196
|
+
): AsyncIterable<T> & { readonly droppedCount: number }
|
|
197
|
+
```
|
|
198
|
+
|
|
199
|
+
The reverse direction -- yield signal changes as an async iterable. Useful
|
|
200
|
+
when you have signal-driven state and want to pipe its changes into a
|
|
201
|
+
WebSocket, log sink, replay tool, or any consumer that wants for-await
|
|
202
|
+
semantics.
|
|
203
|
+
|
|
204
|
+
Internal queue is bounded (default 1024). On overflow, the OLDEST value is
|
|
205
|
+
dropped and `iterable.droppedCount` increments. The iterator naturally
|
|
206
|
+
completes when `opts.signal` aborts; consumer-side `break` triggers
|
|
207
|
+
`iterator.return()`.
|
|
208
|
+
|
|
209
|
+
Treat this as the secondary API. Most consumers want the forward direction.
|
|
210
|
+
|
|
211
|
+
## Modes: latest vs buffer
|
|
212
|
+
|
|
213
|
+
Pick `"latest"` when you only care about the most recent value:
|
|
214
|
+
|
|
215
|
+
- Live cursor positions
|
|
216
|
+
- Current pubsub message
|
|
217
|
+
- Latest network frame
|
|
218
|
+
- Current SSE event
|
|
219
|
+
|
|
220
|
+
Pick `"buffer"` when every value matters and you want them in order:
|
|
221
|
+
|
|
222
|
+
- Helix paginated results (each page must be processed)
|
|
223
|
+
- EBS SSE event log (must not miss an event)
|
|
224
|
+
- Replay queue (every frame applies in order)
|
|
225
|
+
|
|
226
|
+
**`"buffer"` mode requires `maxBuffer`.** There is no default, and missing
|
|
227
|
+
the option throws a `RangeError` with this message:
|
|
228
|
+
|
|
229
|
+
> `lite-stream: "buffer" mode requires opts.maxBuffer to be a positive
|
|
230
|
+
> integer. Unbounded buffering is a memory bug pretending to be a feature;
|
|
231
|
+
> pick a deliberate ceiling.`
|
|
232
|
+
|
|
233
|
+
On overflow, the OLDEST value is dropped and `droppedCount` increments.
|
|
234
|
+
Watch `droppedCount` in your effect to know when your consumer is falling
|
|
235
|
+
behind.
|
|
236
|
+
|
|
237
|
+
## Zero-GC hot paths
|
|
238
|
+
|
|
239
|
+
`lite-stream`'s hot paths -- per-yield state allocation and per-pull abort
|
|
240
|
+
checks -- minimize per-op allocation. Measured retention is sub-byte per op
|
|
241
|
+
across all scenarios:
|
|
242
|
+
|
|
243
|
+
```
|
|
244
|
+
from-latest-10 178K ops/s 0.07 B/op
|
|
245
|
+
from-buffer-20-drop 128K ops/s 0.13 B/op
|
|
246
|
+
abort-cycle 84K ops/s 0.71 B/op
|
|
247
|
+
pipe-to-signal 235K ops/s 0.10 B/op
|
|
248
|
+
```
|
|
249
|
+
|
|
250
|
+
Per-yield, `"latest"` mode allocates exactly one wrapper state object. The
|
|
251
|
+
underlying signal node is reused via lite-signal's pool. `"buffer"` mode
|
|
252
|
+
adds one snapshot array per yield (so consumer effects see a stable
|
|
253
|
+
non-mutated array view at each tick) -- the ring beneath it is a fixed
|
|
254
|
+
pre-allocated array reused across all yields.
|
|
255
|
+
|
|
256
|
+
The only per-op allocations beyond intrinsic iterator cost:
|
|
257
|
+
|
|
258
|
+
- One wrapper state object per yield (`"latest"`) or wrapper + snapshot
|
|
259
|
+
array (`"buffer"`)
|
|
260
|
+
- One AbortListener function and registration on the AbortSignal (once per
|
|
261
|
+
stream lifetime, not per yield)
|
|
262
|
+
- The Promise chain from `iter.next().then(...)` itself
|
|
263
|
+
|
|
264
|
+
V8's escape analysis covers the wrapper object closures in most JIT modes.
|
|
265
|
+
|
|
266
|
+
## Integration recipes
|
|
267
|
+
|
|
268
|
+
### lite-twitch-helix paginated endpoint
|
|
269
|
+
|
|
270
|
+
```js
|
|
271
|
+
import { fromAsyncIterable } from "@zakkster/lite-stream";
|
|
272
|
+
import { effect } from "@zakkster/lite-signal";
|
|
273
|
+
|
|
274
|
+
async function* getFollowers(channelId, signal) {
|
|
275
|
+
let cursor = "";
|
|
276
|
+
while (true) {
|
|
277
|
+
const res = await fetch(
|
|
278
|
+
"https://api.twitch.tv/helix/channels/followers"
|
|
279
|
+
+ "?broadcaster_id=" + channelId
|
|
280
|
+
+ (cursor ? "&after=" + cursor : ""),
|
|
281
|
+
{ signal, headers: { Authorization: bearer, "Client-Id": clientId } }
|
|
282
|
+
);
|
|
283
|
+
const { data, pagination } = await res.json();
|
|
284
|
+
yield data; // yield one page at a time
|
|
285
|
+
cursor = pagination?.cursor;
|
|
286
|
+
if (!cursor) return;
|
|
287
|
+
}
|
|
288
|
+
}
|
|
289
|
+
|
|
290
|
+
const ctrl = new AbortController();
|
|
291
|
+
const pages = fromAsyncIterable(getFollowers(channelId, ctrl.signal), {
|
|
292
|
+
mode: "buffer",
|
|
293
|
+
maxBuffer: 100, // 100 pages = 10K followers max
|
|
294
|
+
signal: ctrl.signal
|
|
295
|
+
});
|
|
296
|
+
|
|
297
|
+
effect(() => {
|
|
298
|
+
const s = pages();
|
|
299
|
+
if (s.droppedCount > 0) console.warn("dropped pages:", s.droppedCount);
|
|
300
|
+
renderFollowerList(s.values.flat());
|
|
301
|
+
if (s.done && !s.error) renderDoneCard(s.count);
|
|
302
|
+
});
|
|
303
|
+
```
|
|
304
|
+
|
|
305
|
+
### lite-twitch-ebs server-sent events
|
|
306
|
+
|
|
307
|
+
```js
|
|
308
|
+
import { fromAsyncIterable } from "@zakkster/lite-stream";
|
|
309
|
+
|
|
310
|
+
async function* sseEvents(url, signal) {
|
|
311
|
+
const res = await fetch(url, { signal });
|
|
312
|
+
const reader = res.body.getReader();
|
|
313
|
+
const decoder = new TextDecoder();
|
|
314
|
+
let buffer = "";
|
|
315
|
+
while (true) {
|
|
316
|
+
const { done, value } = await reader.read();
|
|
317
|
+
if (done) return;
|
|
318
|
+
buffer += decoder.decode(value, { stream: true });
|
|
319
|
+
for (const line of buffer.split("\n\n").slice(0, -1)) {
|
|
320
|
+
yield parseSSE(line);
|
|
321
|
+
}
|
|
322
|
+
buffer = buffer.split("\n\n").slice(-1)[0];
|
|
323
|
+
}
|
|
324
|
+
}
|
|
325
|
+
|
|
326
|
+
const ctrl = new AbortController();
|
|
327
|
+
const events = fromAsyncIterable(sseEvents("/ebs/events", ctrl.signal), {
|
|
328
|
+
mode: "latest", // only newest matters
|
|
329
|
+
signal: ctrl.signal
|
|
330
|
+
});
|
|
331
|
+
```
|
|
332
|
+
|
|
333
|
+
### lite-rollback input-frame queue
|
|
334
|
+
|
|
335
|
+
```js
|
|
336
|
+
import { fromAsyncIterable } from "@zakkster/lite-stream";
|
|
337
|
+
|
|
338
|
+
async function* gamepadFrames(signal) {
|
|
339
|
+
while (!signal.aborted) {
|
|
340
|
+
await nextAnimationFrame();
|
|
341
|
+
yield sampleGamepad();
|
|
342
|
+
}
|
|
343
|
+
}
|
|
344
|
+
|
|
345
|
+
const frames = fromAsyncIterable(gamepadFrames(ctrl.signal), {
|
|
346
|
+
mode: "buffer",
|
|
347
|
+
maxBuffer: 60, // 1 second at 60fps
|
|
348
|
+
signal: ctrl.signal
|
|
349
|
+
});
|
|
350
|
+
|
|
351
|
+
effect(() => {
|
|
352
|
+
const s = frames();
|
|
353
|
+
for (const frame of s.values) applyInputFrame(frame);
|
|
354
|
+
if (s.droppedCount) console.warn("rollback drops:", s.droppedCount);
|
|
355
|
+
});
|
|
356
|
+
```
|
|
357
|
+
|
|
358
|
+
### lite-twitch-pubsub topic subscription
|
|
359
|
+
|
|
360
|
+
```js
|
|
361
|
+
import { pipeToSignal } from "@zakkster/lite-stream";
|
|
362
|
+
import { signal } from "@zakkster/lite-signal";
|
|
363
|
+
|
|
364
|
+
const subs = signal(null);
|
|
365
|
+
|
|
366
|
+
const ctrl = new AbortController();
|
|
367
|
+
const stop = pipeToSignal(subscribeToTopic("channel-subscribe-events.v1"), subs, {
|
|
368
|
+
signal: ctrl.signal,
|
|
369
|
+
transform: (raw) => JSON.parse(raw.data),
|
|
370
|
+
onError: (e) => console.error("pubsub error", e)
|
|
371
|
+
});
|
|
372
|
+
|
|
373
|
+
// Later: ctrl.abort() OR stop() -- either ends the pipe.
|
|
374
|
+
```
|
|
375
|
+
|
|
376
|
+
### Reverse direction: pipe signal changes into a WebSocket
|
|
377
|
+
|
|
378
|
+
```js
|
|
379
|
+
import { toAsyncIterable } from "@zakkster/lite-stream";
|
|
380
|
+
|
|
381
|
+
const ctrl = new AbortController();
|
|
382
|
+
const ws = new WebSocket("wss://logs.example.com/ingest");
|
|
383
|
+
|
|
384
|
+
(async () => {
|
|
385
|
+
for await (const event of toAsyncIterable(loggerSig, { signal: ctrl.signal })) {
|
|
386
|
+
if (ws.readyState === WebSocket.OPEN) {
|
|
387
|
+
ws.send(JSON.stringify(event));
|
|
388
|
+
}
|
|
389
|
+
}
|
|
390
|
+
})();
|
|
391
|
+
```
|
|
392
|
+
|
|
393
|
+
## Edge cases pinned down
|
|
394
|
+
|
|
395
|
+
- **Pre-aborted AbortSignal**: the iterator is never started. The signal
|
|
396
|
+
settles synchronously with an `AbortError`. No pulls.
|
|
397
|
+
- **Synchronous throw from `iter.next()`**: caught and surfaced via the
|
|
398
|
+
same error path as async rejections.
|
|
399
|
+
- **Iterator yields `null`/`undefined`**: passed through as-is. `lite-stream`
|
|
400
|
+
doesn't interpret values; only `{ done: true }` terminates.
|
|
401
|
+
- **Iterator returns a non-object from `next()`**: treated as a protocol
|
|
402
|
+
error; settles with a `TypeError`.
|
|
403
|
+
- **Sync `Iterable<T>`**: accepted via `Symbol.iterator`. Pulls happen
|
|
404
|
+
microtask-asynchronously even for sync sources, so caller's `effect()`
|
|
405
|
+
can wire up before the first yield.
|
|
406
|
+
- **Consumer disposes the signal mid-stream**: pump detects on next
|
|
407
|
+
`sig.set()`, tears down the iterator silently. No `onError` fired.
|
|
408
|
+
- **Buffer mode snapshot array**: a fresh array per yield. Past observers
|
|
409
|
+
hold stable, non-mutated references.
|
|
410
|
+
- **Disposing the signal multiple times**: lite-signal's `dispose` is
|
|
411
|
+
idempotent; lite-stream tolerates concurrent abort + dispose without
|
|
412
|
+
double-firing callbacks.
|
|
413
|
+
|
|
414
|
+
## Benchmarks
|
|
415
|
+
|
|
416
|
+
Run via `npm run bench` (requires `--expose-gc`).
|
|
417
|
+
|
|
418
|
+
```
|
|
419
|
+
[1] fromAsyncIterable latest mode -- 10-item async gen
|
|
420
|
+
from-latest-10 178,554 ops/s retained: 0.07 B/op
|
|
421
|
+
|
|
422
|
+
[2] fromAsyncIterable buffer mode -- 20-item gen, maxBuffer 5
|
|
423
|
+
from-buffer-20-drop 128,630 ops/s retained: 0.13 B/op
|
|
424
|
+
|
|
425
|
+
[3] Abort cycle -- infinite gen, abort after first yield
|
|
426
|
+
abort-cycle 84,102 ops/s retained: 0.71 B/op
|
|
427
|
+
|
|
428
|
+
[4] pipeToSignal -- pump existing signal directly
|
|
429
|
+
pipe-to-signal 235,376 ops/s retained: 0.10 B/op
|
|
430
|
+
|
|
431
|
+
[5] toAsyncIterable -- 10 signal updates, consumer drains in order
|
|
432
|
+
to-async-iterable 42,480 ops/s retained: ~0 B/op
|
|
433
|
+
|
|
434
|
+
[6] toAsyncIterable -- producer overflow, drop-oldest
|
|
435
|
+
to-async-iterable-overflow 62,164 ops/s retained: ~25 B/op
|
|
436
|
+
```
|
|
437
|
+
|
|
438
|
+
(Node v22, 150ms warmup, 500ms runs.)
|
|
439
|
+
|
|
440
|
+
The full lifecycle (construct, pull, settle, dispose) of `fromAsyncIterable`
|
|
441
|
+
in latest mode runs at 178K ops/s with sub-byte/op retained heap. Per-pull
|
|
442
|
+
allocation is dominated by the iterator's own microtask cost, not by
|
|
443
|
+
lite-stream's wrapper state.
|
|
444
|
+
|
|
445
|
+
## Testing strategy
|
|
446
|
+
|
|
447
|
+
### Tier 1 -- behavior (unit tests, fast)
|
|
448
|
+
|
|
449
|
+
49 tests across `test/01-*` through `test/06-*`:
|
|
450
|
+
|
|
451
|
+
- `01-from-async-iterable-latest.test.mjs` -- state shape, lifecycle,
|
|
452
|
+
Iterable acceptance variants, pre-aborted, subscriber observability
|
|
453
|
+
- `02-from-async-iterable-buffer.test.mjs` -- ring buffer correctness,
|
|
454
|
+
drop-oldest, snapshot freshness, maxBuffer validation, helix shape
|
|
455
|
+
- `03-cleanup-triplet.test.mjs` -- all three termination paths, iterator
|
|
456
|
+
return() invocation, abort reason propagation, 200-cycle leak smoke test
|
|
457
|
+
- `04-pipe-to-signal.test.mjs` -- target write, stop fn idempotency,
|
|
458
|
+
transform, abort behavior, target-not-disposed invariant
|
|
459
|
+
- `05-to-async-iterable.test.mjs` -- emitInitial, ordering, overflow,
|
|
460
|
+
consumer break triggers return(), pre-aborted path
|
|
461
|
+
|
|
462
|
+
Run via `npm test`.
|
|
463
|
+
|
|
464
|
+
### Tier 2 -- memory (allocation-free verification)
|
|
465
|
+
|
|
466
|
+
`test/06-gc.test.mjs` -- runs under `--expose-gc`. Asserts heap budget
|
|
467
|
+
ceilings for 2K latest cycles (< 2 MB), 1K buffer cycles (< 2 MB), and
|
|
468
|
+
1K abort cycles (< 2 MB).
|
|
469
|
+
|
|
470
|
+
Run via `npm run test:gc`.
|
|
471
|
+
|
|
472
|
+
### Tier 3 -- performance (measured throughput)
|
|
473
|
+
|
|
474
|
+
`bench/bench.mjs` -- six scenarios; throughput and B/op retained. Run via
|
|
475
|
+
`npm run bench`.
|
|
476
|
+
|
|
477
|
+
## What this is not
|
|
478
|
+
|
|
479
|
+
- **Not a full reactive stream library.** No `map`, `filter`, `merge`, or
|
|
480
|
+
any other combinator surface. Compose iterators yourself with async
|
|
481
|
+
generators -- `for await (const x of source) { if (pred(x)) yield x; }` --
|
|
482
|
+
and pass the composed iterator to `fromAsyncIterable`.
|
|
483
|
+
- **Not a backpressure protocol.** Async iterators in JS don't have a way
|
|
484
|
+
to tell the producer "wait, I'm full." The buffer mode's `maxBuffer` is a
|
|
485
|
+
ceiling on in-flight values; overflow drops oldest. If you need
|
|
486
|
+
bidirectional backpressure, your source needs to expose that surface.
|
|
487
|
+
- **Not a replacement for `lite-clock`**. For frame-rate driven loops, use
|
|
488
|
+
`lite-clock`. Use `lite-stream` for one-way async sources where the
|
|
489
|
+
source's pace is external.
|
|
490
|
+
- **Not for one-shot promises.** Use `lite-await`'s `fromPromise` for that.
|
|
491
|
+
`lite-stream` is the multi-shot dual; the cleanup model is fundamentally
|
|
492
|
+
different.
|
|
493
|
+
|
|
494
|
+
## Ecosystem
|
|
495
|
+
|
|
496
|
+
`lite-stream` composes with the rest of the `@zakkster/lite-*` family:
|
|
497
|
+
|
|
498
|
+
- **`@zakkster/lite-signal`** -- the reactive signal core. Required peer.
|
|
499
|
+
- **`@zakkster/lite-await`** -- single-shot async primitives
|
|
500
|
+
(`whenSignal`, `withTimeout`, `withAbort`, `fromPromise`). Pair
|
|
501
|
+
`withTimeout` with each `next()` call to add per-pull timeouts to your
|
|
502
|
+
iterator.
|
|
503
|
+
- **`@zakkster/lite-statechart`** -- finite state machines. Use
|
|
504
|
+
`ctx.signal` (1.1+) from a state's entry action to drive a `lite-stream`
|
|
505
|
+
pump that auto-aborts on state transitions.
|
|
506
|
+
- **`@zakkster/lite-twitch`** (upcoming) -- Twitch Extension SDK. Helix
|
|
507
|
+
pagination, EBS SSE, pubsub topics all consume `lite-stream`.
|
|
508
|
+
|
|
509
|
+
## Bundle
|
|
510
|
+
|
|
511
|
+
```
|
|
512
|
+
~5 KB minified (gzipped: ~2 KB)
|
|
513
|
+
ESM-only, Node >= 18
|
|
514
|
+
Single file: Stream.js
|
|
515
|
+
Peer dep: @zakkster/lite-signal ^1.2.0
|
|
516
|
+
Zero runtime deps.
|
|
517
|
+
```
|
|
518
|
+
|
|
519
|
+
## License
|
|
520
|
+
|
|
521
|
+
MIT (c) 2026 Zahary Shinikchiev. See [LICENSE.txt](./LICENSE.txt).
|