@twilio/video-node-sdk 1.0.0-beta.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md ADDED
@@ -0,0 +1,409 @@
1
+ This SDK is currently in beta. It supports Linux x86-64 and macOS 26+ on x86-64.
2
+ See the [README](README.md) for details.
3
+
4
+ # 1.0.0-beta.1 (September 25, 2026)
5
+
6
+ ## Breaking Changes
7
+
8
+ ### `frameId` is no longer accepted on published frames
9
+
10
+ `VideoFrameInput.frameId` and `AudioFrameInput.frameId` are removed. They were
11
+ typed and documented as "carried for tracing", but the publish path never read
12
+ them, so the value was silently discarded. `frameId` on _received_ frames is
13
+ unaffected: it is SDK-generated and monotonic per track.
14
+
15
+ ```js
16
+ // Before - frameId was accepted and dropped
17
+ track.write({ ...frame, frameId: n });
18
+
19
+ // After - carry your own correlation outside the frame
20
+ track.write(frame);
21
+ ```
22
+
23
+ ### `FrameStream` is no longer exported
24
+
25
+ It is the internal queue behind `frames()`. `frames()` returns an
26
+ `AsyncIterableIterator`, which is the whole contract; exporting the class made
27
+ its internal constructor part of the public API. `MAX_QUEUE_CEILING` is still
28
+ exported.
29
+
30
+ ### `UnsupportedPlatformError` is now thrown
31
+
32
+ Importing the SDK on a platform the addon is not built for, with no local build
33
+ present, throws `UnsupportedPlatformError` naming the platform, instead of a
34
+ `NativeBindingLoadError` advising a build that cannot succeed. A local build in
35
+ `build/Release` or `build/Debug` is loaded first, so contributors who compiled
36
+ the addon themselves on an unlisted platform are not blocked; an installed
37
+ package never has one, because `files` excludes `build/`.
38
+
39
+ `TwilioErrorClass`, the constructor shape shared by every error subclass, is now
40
+ exported as a type, and every exported error class exposes the static `code`
41
+ that shape requires - `NativeBindingLoadError` included.
42
+
43
+ ### `frames()` replaces `onFrame()`
44
+
45
+ `RemoteVideoTrack.onFrame()` and `removeFrameCallback()` are removed. Receive
46
+ frames by async iteration instead. Awaiting each frame is what applies
47
+ backpressure; a callback could not.
48
+
49
+ ```js
50
+ // Before
51
+ track.onFrame(frame => process(frame));
52
+ track.removeFrameCallback();
53
+
54
+ // After
55
+ for await (const frame of track.frames()) {
56
+ await process(frame);
57
+ frame.close?.();
58
+ }
59
+ // The loop ends on unsubscribe or Room disconnect. `break` releases the track.
60
+ ```
61
+
62
+ Each track supports a single receiver. Fan out inside your own loop if several
63
+ consumers need the same stream.
64
+
65
+ ### `RemoteDataTrack` messages are an event
66
+
67
+ ```js
68
+ // Before
69
+ track.onMessage(data => handle(data));
70
+ track.removeMessageCallback();
71
+
72
+ // After
73
+ track.on('message', (data, track) => handle(data, track));
74
+ ```
75
+
76
+ Any `EventEmitter` registration method works: `on`, `once`, `addListener`,
77
+ `prependListener`, `prependOnceListener`.
78
+
79
+ ### Timestamps are microseconds as `number`, not nanoseconds as `bigint`
80
+
81
+ `timestampNs` and `captureTimestampNs` become `timestamp` and
82
+ `captureTimestamp`, in microseconds. Microsecond resolution stays exact in a JS
83
+ number for roughly 285 years, and the engine reports microseconds natively, so
84
+ this removes a conversion in both directions and lets frame timing be done with
85
+ ordinary arithmetic.
86
+
87
+ ```js
88
+ // Before
89
+ const ms = Number(frame.timestampNs / 1_000_000n);
90
+ track.write({ ..., timestampNs: process.hrtime.bigint() });
91
+
92
+ // After
93
+ const ms = frame.timestamp / 1000;
94
+ track.write({ ... }); // omit `timestamp` and the SDK stamps "now"
95
+ ```
96
+
97
+ ### `VideoFrameInput` is planar
98
+
99
+ Input now uses the same shape as the delivered `VideoFrame`, so a received frame
100
+ can be republished without reshaping.
101
+
102
+ ```js
103
+ // Before
104
+ track.write({
105
+ y,
106
+ u,
107
+ v, // Buffers
108
+ yStride,
109
+ uStride,
110
+ vStride,
111
+ width,
112
+ height,
113
+ });
114
+
115
+ // After
116
+ track.write({
117
+ format: 'I420', // optional
118
+ width,
119
+ height,
120
+ y: { data: y, stride: yStride, width, height },
121
+ u: { data: u, stride: uStride, width: width / 2, height: height / 2 },
122
+ v: { data: v, stride: vStride, width: width / 2, height: height / 2 },
123
+ });
124
+ ```
125
+
126
+ ### `LocalDataTrack.send()` returns a promise, and enforces the 64 KB limit
127
+
128
+ `send()` previously returned `void` and silently discarded a message larger than
129
+ `kMaxMessageSize`. It now throws a `RangeError` for an oversize message, and
130
+ returns a promise describing the outcome. The promise **always resolves**, so a
131
+ fire-and-forget `send()` cannot produce an unhandled rejection. A send still in
132
+ flight when the track is destroyed resolves with `ok: false` rather than being
133
+ left pending.
134
+
135
+ ```js
136
+ const result = await track.send('hello');
137
+ if (!result.ok) console.warn('send failed:', result.error);
138
+ ```
139
+
140
+ ### Audio publish is bounded, and rejects rather than sheds
141
+
142
+ `LocalAudioTrack.write()` previously buffered up to **45 seconds** of audio and
143
+ always returned `true`. The queue is now bounded (~500 ms by default,
144
+ configurable through `source.maxQueue` in 10 ms chunks) and a write is accepted
145
+ only if it fits whole in the remaining space. When it does not fit, nothing is
146
+ buffered and `write()` returns `false`. A producer running at real-time cadence
147
+ is unaffected; one running faster now learns it is outrunning the wire instead
148
+ of silently accumulating latency.
149
+
150
+ A single `write()` larger than `maxQueue` can never fit and is always rejected,
151
+ so size `maxQueue` to the largest burst you intend to publish. A caller that
152
+ handed `write()` a whole utterance at once previously had the front of it
153
+ discarded silently; it now gets `false` and can resize the queue or split the
154
+ write.
155
+
156
+ ### `source.mode` and `source.drop` are removed from audio track options
157
+
158
+ `RawAudioSourceOptions` no longer accepts `mode` or `drop`. Both were validated
159
+ and then discarded: the publish queue lives in the process-wide audio device,
160
+ which has one policy and cannot express a per-track one. `source.maxQueue` is
161
+ kept and now documents that it binds that shared device, so the most recent
162
+ value passed to `createLocalAudioTrack` applies to every local audio track in
163
+ the process. `mode`, `maxQueue` and `drop` on the receive side (`frames()`) are
164
+ unchanged and remain per-track.
165
+
166
+ ### `AudioFrameInput.timestamp` is observability-only
167
+
168
+ Audio publish is FIFO: the audio device emits queued samples on its own 10 ms
169
+ cadence. The `timestamp` on a published audio frame feeds
170
+ `WriteStats.lastTimestamp` and `WriteStats.timestampRegressions` and does not
171
+ change what is sent or when. Video publish still carries the timestamp through
172
+ to the encoded frame. This documents existing behavior; nothing changed in the
173
+ publish path.
174
+
175
+ ### The package ships linux-x64 and darwin-x64 prebuilds
176
+
177
+ The darwin-x64 prebuild requires macOS 26 or later and an x64 Node, so on Apple Silicon Node
178
+ must run under Rosetta. npm cannot check the macOS version, so on an older macOS the install
179
+ succeeds and loading the addon fails.
180
+
181
+ ### `trackSubscribed` and `trackUnsubscribed` pass a `RemoteTrackPublication`
182
+
183
+ Listeners receive `(track, publication)` on a `RemoteParticipant` and
184
+ `(track, publication, participant)` on a `Room`, matching twilio-video.js. Update any listener
185
+ that took `(track, participant)` on a Room. On `trackUnsubscribed` the publication reports
186
+ `isSubscribed: false` and its `track` is `undefined`, as documented for
187
+ `RemoteTrackPublication`; the unsubscribed track is the event's first argument.
188
+
189
+ ### Remote track events carry a `RemoteTrackPublication`
190
+
191
+ These four events passed a plain `{ trackSid, trackName }` record (plus `isSubscribed` on the
192
+ state events). They now pass the same `RemoteTrackPublication` the track collections return,
193
+ matching twilio-video.js and the `trackSubscribed` events. `trackSubscriptionFailed` carries one
194
+ too, in place of its own `{ trackSid, trackName, kind }` record. `RemoteTrackPublishEvent`,
195
+ `RemoteTrackStateEvent` and `RemoteTrackSubscriptionFailedEvent` are no longer exported.
196
+
197
+ ```js
198
+ // Before - a record, with no way to reach the track
199
+ participant.on('trackPublished', pub => console.log(pub.trackSid));
200
+
201
+ // After - a publication, with kind, enabled state and the track once subscribed
202
+ participant.on('trackPublished', pub => {
203
+ console.log(pub.trackSid, pub.kind, pub.isTrackEnabled, pub.isSubscribed);
204
+ if (pub.kind === 'video' && pub.track) pub.track.frames();
205
+ });
206
+ ```
207
+
208
+ ### An event listener that throws is no longer ignored
209
+
210
+ The error is reported as an `uncaughtException`, and events queued behind it are retained, so
211
+ an application that handles `uncaughtException` still receives them. Previously such an error
212
+ produced no output at all, which was indistinguishable from an event that was never emitted. An
213
+ application relying on that behavior will start seeing `uncaughtException`.
214
+
215
+ ### `disconnected` passes the Room first
216
+
217
+ Listeners receive `(room, error?)` instead of `(error?)`, matching twilio-video.js.
218
+
219
+ ```js
220
+ // Before
221
+ room.on('disconnected', error => { ... });
222
+
223
+ // After
224
+ room.on('disconnected', (room, error) => { ... });
225
+ ```
226
+
227
+ ### `trackPublicationFailed` passes the track that failed
228
+
229
+ Listeners receive `(error, localTrack?)` instead of `(error)`. `localTrack` is the instance
230
+ passed to `publishTrack()`, or `undefined` when the failure cannot be attributed to a track
231
+ this participant still holds.
232
+
233
+ ```js
234
+ // Before
235
+ localParticipant.on('trackPublicationFailed', error => { ... });
236
+
237
+ // After
238
+ localParticipant.on('trackPublicationFailed', (error, localTrack) => { ... });
239
+ ```
240
+
241
+ ## Features
242
+
243
+ - `RemoteVideoTrack.getStats()` / `RemoteAudioTrack.getStats()` return
244
+ `DeliveryStats`: frames delivered, frames dropped, queue depth and the
245
+ configured bound. Drops are counted in both the JS policy queue and the native
246
+ transfer queue, so the number is the total the consumer never saw.
247
+ - `LocalVideoTrack.getWriteStats()` / `LocalAudioTrack.getWriteStats()` return
248
+ `WriteStats` for the publish direction.
249
+ - A coalesced `frameDropped(count, sinceLastUs)` event on remote media tracks.
250
+ - `frames(options)` takes `mode` (`'latest'` | `'queue'`), `maxQueue` and `drop`
251
+ (`'oldest'` | `'newest'`). Defaults are media-aware: video keeps only the
252
+ newest frame, audio buffers a little to smooth jitter. `maxQueue` is capped so
253
+ a misconfiguration cannot exhaust memory.
254
+ - The published type declarations no longer carry the SDK's `@internal`
255
+ members. `_native`, `_attachFrameSink`, `_end` and the rest were annotated
256
+ `@internal` but still landed in `index.d.ts`, so they appeared in consumer
257
+ autocomplete as though they were API. None was documented or supported.
258
+ - `VideoFrame.close()` / `AudioFrame.close()` release the buffers promptly.
259
+ Optional - the frame is an owned copy and GC reclaims it. Idempotent; reading
260
+ plane data after `close()` throws.
261
+ - `CreateLocalVideoTrackOptions.source` pins the frame size, so a mismatched
262
+ frame is rejected rather than silently rescaled.
263
+ `CreateLocalAudioTrackOptions.source` bounds the publish queue.
264
+ - `ConnectOptions.connectionTimeout` (default 30s) rejects with
265
+ `RoomConnectTimeoutError`. There was previously no timeout at any layer, so a
266
+ wedged connect hung the caller indefinitely.
267
+ - One error subclass per known Twilio code, 26 in all, plus typed SDK-local
268
+ errors (`NativeBindingLoadError`, `UnsupportedPlatformError`,
269
+ `RoomConnectTimeoutError`, `DataTrackSendError`). Previously only 5 of the 26
270
+ codes had a class.
271
+
272
+ ## Bug Fixes
273
+
274
+ - `VideoFrame.frameId` now advances. It was taken from libwebrtc's
275
+ `VideoFrame::id()`, which is not populated on the receive path and read `0`
276
+ for every frame, so gap and drop detection was impossible despite being the
277
+ documented purpose of the field. It is now an SDK-generated monotonic
278
+ per-track counter, matching what the audio path already did.
279
+ - Frames dropped at the native-to-JS boundary are counted instead of discarded
280
+ silently. The transfer queue always had a fixed depth of 5 with a drop-oldest
281
+ policy; nothing reported it.
282
+ - `trackSubscribed` is now emitted for tracks a remote participant was already publishing when
283
+ they joined. The observer the media engine calls is installed as soon as the participant
284
+ connects, rather than one thread hop later, which is after the subscription has already
285
+ completed. Rejoining participants were the most affected: the first join often reported its
286
+ tracks and later ones silently did not.
287
+ - `trackUnsubscribed` is now emitted for every track a participant was still publishing when they
288
+ disconnected. It could previously be lost because it and `participantDisconnected` were
289
+ delivered on independent internal queues with no guaranteed order between them.
290
+ - Fixed several cases where a `RemoteParticipant` stopped receiving further track events after
291
+ the SDK built a second internal wrapper for it, which happened on reading `room.participants`
292
+ or `room.dominantSpeaker`, on a reconnect, or for a participant who was already in the Room
293
+ when the local participant connected. Reading `room.participants` even once after a
294
+ participant connected could permanently stop delivery of `trackSubscribed`,
295
+ `trackUnsubscribed`, and related events for that participant, with no error.
296
+ - Fixed the equivalent issue for `RemoteDataTrack`: reading a participant's `dataTracks` (or
297
+ `tracks`) a second time could stop message delivery to a `RemoteDataTrack` object obtained
298
+ from an earlier read. Every independently obtained `RemoteDataTrack` for the same underlying
299
+ track now receives messages, matching how multiple `RemoteVideoTrack`/`RemoteAudioTrack`
300
+ objects for the same track already each receive frames.
301
+ - A participant is no longer kept in memory for the rest of the Room's lifetime after they
302
+ disconnect. Their cached wrapper was previously released only as a side effect of a later
303
+ `room.participants` read; an application that only listens to events, without reading that
304
+ getter, retained every departed participant until the Room itself was disposed.
305
+ - `participantDisconnected` now arrives after that participant's final `trackUnsubscribed`
306
+ events, and is emitted from the Room's own event queue so it stays in order with Room events
307
+ raised before it, such as `dominantSpeakerChanged`. It is not ordered against a Room event
308
+ raised in the same moment: a Room that ends as a participant leaves can report `disconnected`
309
+ first, and an application that calls `room.dispose()` from that handler may not receive the
310
+ `participantDisconnected` at all.
311
+ - `participantDisconnected` now reports the same `RemoteParticipant` object the application
312
+ already received from `participantConnected` or `room.participants`. A `room.participants`
313
+ read while the participant was leaving dropped the cached instance, so the event carried a
314
+ second one: a listener attached to the original never saw the disconnect, and that original
315
+ was never disposed.
316
+ - Fixed a crash during teardown when an event listener disposes the Room it is handling an
317
+ event for. The internal event queue continued to touch its own state after the listener
318
+ returned, which the `dispose()` had already freed.
319
+ - A `RemoteDataTrack` no longer delivers a message after `trackUnsubscribed` has been emitted
320
+ for it. A message that was already on its way to the JS thread when the track was
321
+ unsubscribed is now dropped, matching the documented contract.
322
+ - Two Rooms in the same process subscribed to the same published data track now each receive
323
+ messages. They were keyed by Track SID, which is shared between them, so only the first Room
324
+ received messages and tearing that Room down stopped delivery for the other one.
325
+ - Data track observers are no longer retained for the process's lifetime. They were released
326
+ only when a track was explicitly unsubscribed, so any other teardown, such as disposing a
327
+ Room mid-call, left them behind.
328
+
329
+ ## Documentation
330
+
331
+ ### API reference on GitHub Pages
332
+
333
+ The API reference for each release is published at https://twilio.github.io/twilio-video-node/,
334
+ which opens the latest version.
335
+
336
+ ### `SECURITY-PRIVACY.md` ships in the package
337
+
338
+ It describes what the SDK does with decoded media and which obligations remain with the
339
+ application.
340
+
341
+ # 1.0.0-preview.3 (September 2, 2026)
342
+
343
+ ## Documentation
344
+
345
+ - Added an API reference generated with TypeDoc, covering every exported symbol. Build it
346
+ locally with `npm run docs`.
347
+
348
+ ## Breaking Changes
349
+
350
+ - `VideoCodec` only accepts `'VP8'`, the only video codec this SDK supports in Group Rooms.
351
+ `'H264'` and `'VP9'` are removed from the type and now throw
352
+ `TypeError: Unknown video codec: <name>` from `connect()` instead of being silently accepted.
353
+
354
+ - `AudioCodec` only accepts `'opus'` and `'PCMU'`, the only audio codecs this SDK supports in
355
+ Group Rooms. `'PCMA'` and `'G722'` are removed from the type and now throw
356
+ `TypeError: Unknown audio codec: <name>` from `connect()`.
357
+
358
+ - `LocalDataTrack.maxPacketLifeTime` and `maxRetransmits` are now `number | null`, reporting
359
+ `null` when unset instead of `65535`. Check for `null`, or use `reliable`. Both now report the
360
+ value passed to `createLocalDataTrack()`, so a limit of `65535` reads back as `65535`.
361
+
362
+ - `ErrorCode.TRACK_NAME_TOO_LONG` is renamed to `ErrorCode.TRACK_NAME_INVALID`. The code it maps
363
+ to, `53301`, is Twilio's `TrackNameInvalid`. `ErrorCode.TRACK_NAME_TOO_LONG` now maps to
364
+ `53302`, the code that condition actually reports.
365
+
366
+ - `createLocalDataTrack()` validates delivery options. `maxPacketLifeTime` and `maxRetransmits`
367
+ must be integers in `[0, 65535]` (`RangeError`, replacing the plain `Error` for negatives);
368
+ `ordered` must be a boolean (`TypeError`). Invalid values were previously coerced.
369
+
370
+ ## Features
371
+
372
+ - `RemoteDataTrack` exposes `maxPacketLifeTime` and `maxRetransmits`, so a subscriber can tell
373
+ how the publisher configured delivery. A publisher's limit of `65535` reads back as `null`,
374
+ since a subscribed track reports it the same way it reports an unset limit.
375
+
376
+ - `LocalDataTrackOptions.maxPacketLifeTime` and `maxRetransmits` accept `null`, so a value read
377
+ off a track can be passed back into `createLocalDataTrack()`.
378
+
379
+ - `trackSubscriptionFailed` passes a `RemoteTrackSubscriptionFailedEvent` (`trackSid`,
380
+ `trackName`, `kind`) after the error, so a listener can tell which publication failed.
381
+ Existing single-argument listeners are unaffected.
382
+
383
+ ## Bug Fixes
384
+
385
+ - Fixed a crash (`SIGSEGV`) during teardown of a Room that ended remotely. A remote track's
386
+ frame sink was freed while still registered with the underlying WebRTC track, so the next
387
+ frame delivered on an SDK-internal thread wrote through freed memory. Affected consumers
388
+ using `RemoteAudioTrack.onFrame` or `RemoteVideoTrack.onFrame`.
389
+
390
+ - Data track options set to `undefined` are treated as unset. `{ maxRetransmits: undefined }`
391
+ previously failed with `A number was expected`.
392
+
393
+ - Joining a Room that already had participants in it no longer misses their track
394
+ events. `trackSubscribed`, `trackEnabled`, `trackDisabled`, `trackPublished`, and
395
+ `trackUnpublished` were never emitted for participants who were already present.
396
+ Participants who joined later were unaffected.
397
+
398
+ - Room now re-emits `trackSubscriptionFailed`. It was the only `RemoteParticipant` track
399
+ event the Room did not forward, so `room` listeners never saw subscription failures.
400
+
401
+ # 1.0.0-preview.2 (July 23, 2026)
402
+
403
+ ## Breaking Changes
404
+
405
+ - Room no longer emits a `connected` event. The underlying native signal was always
406
+ one-shot and consumed internally to resolve `connect()`'s promise before the Room
407
+ was returned, so a `room.on('connected', ...)` listener could never fire. TypeScript
408
+ consumers with such a listener will see a compile error and should remove it;
409
+ plain JavaScript consumers are unaffected.
@@ -0,0 +1,204 @@
1
+ # Frame contract
2
+
3
+ The precise behavior of the raw-frame API: buffer ownership, backpressure and
4
+ drop semantics, and the invariants the publish and receive paths guarantee.
5
+ [README.md](README.md) covers usage; this document covers the guarantees.
6
+
7
+ ## Buffer ownership
8
+
9
+ **Publish.** `write()` copies every buffer synchronously before it returns. The
10
+ caller may reuse or free `y.data`, `u.data`, `v.data` and `pcm` immediately.
11
+ There is no borrow, and no lifetime obligation.
12
+
13
+ **Receive.** A delivered frame is an owned copy in JavaScript memory. The SDK
14
+ keeps no reference to it after delivery, so a frame may be held across `await`,
15
+ transferred to a `worker_thread`, or batched, with no restriction.
16
+
17
+ `close()` is an optional hint that releases the plane buffers immediately rather
18
+ than waiting for garbage collection. It is not required for correctness. It is
19
+ idempotent, and reading plane data (`y`/`u`/`v`, or `pcm`) after `close()`
20
+ throws. Non-plane metadata stays readable.
21
+
22
+ ## When `write()` returns `false`
23
+
24
+ `false` always means the frame was not sent. It never means "buffer full, wait
25
+ for drain" - that is the Node `Writable` convention, and tracks are deliberately
26
+ not presented as Node streams.
27
+
28
+ | Kind | `false` means |
29
+ | ----- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
30
+ | Video | libwebrtc's adapter rejected the frame: most often the encoder sink has not attached yet (a write before `connect()` resolves), and otherwise rate-limiting or a rejected resolution. |
31
+ | Audio | The write did not fit in the bounded publish queue. Nothing was buffered: a rejected write is never partially published. |
32
+
33
+ Invalid input does not return `false`; it throws `TypeError` or `RangeError`.
34
+ The distinction is deliberate: `false` is a runtime condition to react to, an
35
+ exception is a programming error to fix.
36
+
37
+ ## Backpressure and drops
38
+
39
+ Every frame queue is bounded. Blocking the media engine and buffering without
40
+ limit are both rejected: blocking corrupts reconnection and statistics, and at
41
+ roughly 41 MB/s per 720p30 video track, unbounded buffering is an eventual
42
+ out-of-memory failure. A full receive queue drops; a full audio publish queue
43
+ rejects the write, because the publisher is a caller that can be told `false`
44
+ while a decoder thread cannot.
45
+
46
+ ### Where frames are dropped
47
+
48
+ There are two queues in the receive path, and `DeliveryStats.framesDropped` is
49
+ the **sum** across both, so it is the total the consumer never saw.
50
+
51
+ | Queue | Location | Policy |
52
+ | --------------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
53
+ | Native transfer queue | Before the native-to-JS handoff | Bounded to the resolved `maxQueue`, drop-oldest. Bounding it here is what stops a `latest` consumer holding a deep queue of ~1.3 MB frames in native memory. |
54
+ | JS policy queue | After the handoff | The `mode`/`maxQueue`/`drop` policy given to `frames()`. |
55
+
56
+ Publish is different. Video publish is synchronous - `write()` hands the frame
57
+ straight to the encoder - so there is no SDK-side send queue at all, and
58
+ `sendQueueDepth`/`maxQueue` are always `0`. Audio publish has a real queue,
59
+ drained one 10 ms chunk at a time, so its depth and bound are meaningful.
60
+
61
+ The audio publish queue rejects rather than sheds. A `write()` is accepted only
62
+ if it fits whole within `source.maxQueue` (10 ms chunks, default 50, so ~500 ms);
63
+ otherwise nothing is buffered and `write()` returns `false`. A single `write()`
64
+ larger than the bound can never fit and is always rejected, so size `maxQueue`
65
+ to the largest burst the application intends to publish. Rejecting rather than
66
+ shedding keeps `false` meaning "not sent" on both paths, and keeps
67
+ `WriteStats.framesWritten` an exact count of what the SDK accepted.
68
+
69
+ ### Receive policy
70
+
71
+ The rest of this section is the **receive** path: the options `frames()` takes.
72
+ The publish bound above is separate and is configured through
73
+ `source.maxQueue`.
74
+
75
+ `mode: 'latest'` keeps only the newest frame. `mode: 'queue'` buffers up to
76
+ `maxQueue` and then applies `drop`: `'oldest'` sheds from the front, `'newest'`
77
+ sheds the arriving frame.
78
+
79
+ Defaults are media-aware, because the cost of loss differs by kind:
80
+
81
+ | Kind | `mode` | `maxQueue` | Why |
82
+ | ----- | -------- | ---------- | ------------------------------------------------------------------- |
83
+ | Video | `latest` | 1 | A stale frame has no value for live inference. |
84
+ | Audio | `queue` | 10 | A small buffer smooths receiver jitter; gaps degrade transcription. |
85
+
86
+ These are counts of frames, not 10 ms chunks; the publish-side `source.maxQueue`
87
+ is the one measured in chunks.
88
+
89
+ `maxQueue` is capped at 1024 so a misconfiguration cannot exhaust memory.
90
+
91
+ **Ordering is preserved.** In `queue` mode, frames are delivered in arrival
92
+ order. Dropping removes frames from the sequence; it never reorders the ones
93
+ that survive.
94
+
95
+ A frame handed directly to a waiting consumer is never queued, so no policy
96
+ applies to it and it cannot be dropped.
97
+
98
+ ## Observability
99
+
100
+ Every drop is counted. Nothing is discarded silently anywhere in either path.
101
+
102
+ | Signal | Direction | Shape |
103
+ | ----------------------- | --------- | ---------------------------------------------------------------------------------------------------------------------- |
104
+ | `track.getStats()` | Receive | `DeliveryStats`: `framesDelivered`, `framesDropped`, `queueDepth`, `maxQueue`, `lastTimestamp?` |
105
+ | `track.getWriteStats()` | Publish | `WriteStats`: `framesWritten`, `framesDropped`, `sendQueueDepth`, `maxQueue`, `lastTimestamp?`, `timestampRegressions` |
106
+ | `frameDropped` event | Receive | `(count, sinceLastUs)`, coalesced over 500 ms |
107
+
108
+ Receive stats remain readable after the receiver ends, reporting the final
109
+ numbers, so a summary can be taken during teardown.
110
+
111
+ There is no separate underrun signal. An idle track is indistinguishable from a
112
+ stalled one at this layer; use `framesDelivered` against wall-clock time, or the
113
+ WebRTC statistics from `Room.getStats()`, to tell them apart.
114
+
115
+ Audio publish counters split by scope. `framesWritten`, `framesDropped`,
116
+ `lastTimestamp` and `timestampRegressions` count one track's own `write()`
117
+ calls. `sendQueueDepth` and `maxQueue` describe the audio device, which is
118
+ shared by every local audio track in the process; `source.maxQueue` binds that
119
+ same shared device, so the most recent value passed to `createLocalAudioTrack`
120
+ applies to all of them. With one local audio track - the normal case - the
121
+ distinction does not arise.
122
+
123
+ ## Timestamps
124
+
125
+ `timestamp` is a plain `number` of **microseconds**, in both directions.
126
+ Microsecond resolution stays exact in a JS number for roughly 285 years, and the
127
+ engine reports microseconds natively, so no wider integer type is needed.
128
+
129
+ Omitting `timestamp` on publish stamps the frame "now", which is correct for an
130
+ application-paced live source.
131
+
132
+ The two publish paths use the supplied `timestamp` differently. Video carries it
133
+ through to the encoded frame. Audio publish is strictly FIFO - the audio device
134
+ emits queued samples on its own 10 ms cadence - so an audio `timestamp` is
135
+ observability-only: it feeds `WriteStats.lastTimestamp` and
136
+ `WriteStats.timestampRegressions` and does not alter what is sent or when.
137
+
138
+ **Non-monotonic timestamps are accepted and counted, not rejected.** A timestamp
139
+ that does not advance past the previous one increments
140
+ `WriteStats.timestampRegressions`. Rejecting would break a legitimate producer
141
+ that restarts, such as a looping file source; silently reordering or discarding
142
+ would hide a real problem. A rising count on a live source means the supplied
143
+ timestamps are wrong, which shows up downstream as jitter.
144
+
145
+ `frameId` is an SDK-generated monotonic per-track counter on received frames.
146
+ Use it for gap and drop detection. There is no `frameId` on input: correlate a
147
+ published frame by its `timestamp`.
148
+
149
+ ## Publish invariants
150
+
151
+ - **Audio format is fixed**: PCM S16LE, 48 kHz, mono. `AudioFrameInput` exposes
152
+ no sample-rate or channel fields, because the engine's encoder path is wired
153
+ for exactly this and anything else would be silently wrong rather than
154
+ resampled. Resampling is the application's responsibility.
155
+ - **Video format is fixed**: I420. `format` is optional on input but rejected if
156
+ present and not `'I420'`.
157
+ - **No dynamic resize.** Where a track was created with a `source`, a frame whose
158
+ dimensions disagree is rejected rather than rescaled. Without a `source`, the
159
+ encoder follows what is written.
160
+ - **Dimensions must be positive and even.** I420 chroma is subsampled by two in
161
+ each dimension, so an odd dimension has no well-defined chroma plane size.
162
+ - **Strides must be at least the plane width.** Padding beyond that is fine and
163
+ is preserved through the copy.
164
+
165
+ ## Execution model and threading
166
+
167
+ Frames are delivered to the event loop as ready, application-owned objects.
168
+ Delivery is serial and in order per track.
169
+
170
+ Decoded frames arrive on native decoder threads. The bulk plane copy out of
171
+ libwebrtc happens there, off the event loop; the copy into JS buffers currently
172
+ happens on the loop. Whether that second copy moves off-loop is an internal
173
+ optimization that can land without an API change.
174
+
175
+ The SDK cannot stop an application blocking the loop with synchronous per-frame
176
+ work. Heavy per-frame CPU work belongs in `worker_threads` or in an asynchronous
177
+ service call; frames are structured to transfer to a worker without copying.
178
+
179
+ A track supports a **single receiver**. Calling `frames()` while a receiver is
180
+ active throws. An application that needs several consumers fans out inside its
181
+ own loop. The iterator ends on unsubscribe and on Room disconnect or dispose, so
182
+ a `for await` loop exits on its own rather than hanging.
183
+
184
+ ## Data track
185
+
186
+ `send()` accepts a string or a `Buffer`. Messages larger than **64 KB**
187
+ (`kMaxMessageSize`) are rejected synchronously with a `RangeError` and never
188
+ transmitted; the limit is counted in UTF-8 bytes, not characters.
189
+
190
+ `send()` returns a promise that **always resolves**, never rejects, so a
191
+ fire-and-forget send cannot produce an unhandled rejection. It resolves to
192
+ `{ ok, messageId, error? }` once the engine has processed the send.
193
+
194
+ If the track is destroyed while a send is still in flight, the outstanding
195
+ promise resolves with `ok: false` and an `error` naming the teardown, rather
196
+ than being left pending.
197
+
198
+ ## ABI and compatibility
199
+
200
+ The addon is built with `node-addon-api` (N-API), so it is ABI-stable across
201
+ Node versions within the N-API version it targets, and does not need rebuilding
202
+ for each Node release. `engines.node` records the supported range. The supported
203
+ platform for the beta is Linux x86-64; the binary is x86-64 only, with no arm64
204
+ build. See [DEVELOPER_GUIDE.md](DEVELOPER_GUIDE.md) for building from source.
package/LICENSE.md ADDED
@@ -0,0 +1,3 @@
1
+ ## LICENSE
2
+
3
+ Twilio Programmable Video for Node.js is distributed under TWILIO-TOS. https://www.twilio.com/legal/tos