@stinkycomputing/sesame-api-client 1.5.1 → 1.5.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,433 +1,433 @@
1
- # @stinkycomputing/sesame-api-client
2
-
3
- TypeScript client library for the Sesame video production server. Provides type-safe protobuf definitions, a WebSocket RPC client with automatic reconnection, and helpers for building command lists, parsing data frames, and sending remote control messages.
4
-
5
- ## Install
6
-
7
- ```bash
8
- npm install @stinkycomputing/sesame-api-client
9
- ```
10
-
11
- Works in Node.js and the browser. The browser entry point bundles `events` and other Node built-ins - no polyfills needed.
12
-
13
- ---
14
-
15
- ## Quick Start
16
-
17
- ```typescript
18
- import { SesameClient, CommandList } from '@stinkycomputing/sesame-api-client';
19
-
20
- const client = new SesameClient(9000); // port, or a full ws:// URL
21
-
22
- const cl = new CommandList();
23
- cl.sourceAdd('cam1', { file: { url: '/path/to/video.mp4' } });
24
- cl.compositorAdd('main', 1920, 1080, false);
25
- cl.nodeAdd('cam1-node', {
26
- compositorId: 'main',
27
- parentAddress: '',
28
- nodeType: 'source',
29
- });
30
-
31
- await client.execute(cl);
32
- ```
33
-
34
- ---
35
-
36
- ## SesameClient
37
-
38
- ```typescript
39
- const client = new SesameClient(9000);
40
- // or with a custom URL:
41
- const client = new SesameClient('ws://192.168.1.10:9000/api');
42
- ```
43
-
44
- The client auto-connects and auto-reconnects. Listen for connection state changes:
45
-
46
- ```typescript
47
- client.on('connected', () => console.log('ready'));
48
- client.on('disconnected', () => console.log('lost connection'));
49
- ```
50
-
51
- | Member | Description |
52
- |--------|-------------|
53
- | `client.execute(cl)` | Send a `CommandList` to the server |
54
- | `client.isConnected()` | Current connection state |
55
- | `client.disconnect()` | Close the connection |
56
- | `client.status` | [`StatusApi`](#statusapi) instance |
57
- | `client.recorder` | [`RecorderApi`](#recorderapi) instance |
58
- | `client.jobs` | `JobsApi` instance |
59
-
60
- ---
61
-
62
- ## Command List
63
-
64
- `CommandList` builds a batch of operations sent in a single `execute` call. All methods are chainable and accept an optional `timeOffsetMs` to schedule commands relative to the current frame.
65
-
66
- ### Sources
67
-
68
- ```typescript
69
- const cl = new CommandList();
70
-
71
- // File source
72
- cl.sourceAdd('clip1', { file: { url: '/clips/a.mp4' } });
73
-
74
- // Decklink capture
75
- cl.sourceAdd('sdi-in', { decklink: { deviceIndex: 0, videoFormat: DecklinkVideoFormat.HD_1080i_50 } });
76
-
77
- // WebSocket ingest
78
- cl.sourceAdd('ws-in', { websocket: { channel: 'my-channel' } });
79
-
80
- // MoQ ingest
81
- cl.sourceAdd('moq-in', { moq: { url: 'https://relay.example', broadcast: 'feed', key: '' } });
82
-
83
- cl.sourceRemove('clip1');
84
- ```
85
-
86
- ### Compositor
87
-
88
- ```typescript
89
- cl.compositorAdd('main', 1920, 1080, false);
90
-
91
- cl.nodeAdd('bg', { compositorId: 'main', parentAddress: '', nodeType: 'source' });
92
- cl.nodeAdd('fg', { compositorId: 'main', parentAddress: '', nodeType: 'source' });
93
-
94
- cl.nodeRemove('main', 'fg');
95
- cl.compositorRemove('main');
96
- ```
97
-
98
- ### Outputs
99
-
100
- ```typescript
101
- cl.outputAdd('stream', {
102
- encoder: { width: 1920, height: 1080, fps: 50, preset: 0, bitrateKbps: 8000 },
103
- websocket: { channel: 'live-out' },
104
- });
105
-
106
- cl.outputUpdate('stream', { /* updated config */ });
107
- cl.outputRemove('stream');
108
- ```
109
-
110
- ### Audio Mixers
111
-
112
- ```typescript
113
- import { sesame } from '@stinkycomputing/sesame-api-client';
114
- const { AUDIO_MIXER_CHANNEL_TYPE_STEREO } = sesame.v1.audio.AudioMixerChannelType;
115
-
116
- cl.audioMixerAdd('mix1', {
117
- outputType: AUDIO_MIXER_CHANNEL_TYPE_STEREO,
118
- order: 0,
119
- channels: [
120
- { id: 'ch1', sourceId: 'cam1', channelType: AUDIO_MIXER_CHANNEL_TYPE_STEREO,
121
- sourceChannels: [0, 1], level: 1.0, pan: 0.0, plugins: [] },
122
- ],
123
- });
124
-
125
- cl.audioMixerChannelAdd('mix1', { id: 'ch2', sourceId: 'cam2', /* ... */ });
126
- cl.audioMixerChannelRemove('mix1', 'ch2');
127
- cl.audioMixerRemove('mix1');
128
- ```
129
-
130
- ### Properties
131
-
132
- Property names are type-checked against a generated registry - TypeScript will autocomplete valid names and value shapes.
133
-
134
- ```typescript
135
- // Compositor property (typed)
136
- cl.propertySet({ compositorId: 'main' }, 'cam1-node', 'transform/position', { vecValue: { r: 100, g: 200 } });
137
-
138
- // Audio mixer property
139
- cl.propertySet({ audioMixerId: 'mix1' }, 'ch1', 'level', { floatValue: 0.8 });
140
-
141
- // Source property
142
- cl.propertySet({ sourceId: 'cam1' }, '', 'decoder/buffer_frames', { intValue: 4 });
143
- ```
144
-
145
- ### Transport Commands
146
-
147
- ```typescript
148
- cl.transportCommand('cam1', { type: 'play' });
149
- cl.transportCommand('cam1', { type: 'pause' });
150
- cl.transportCommand('cam1', { type: 'seek', value: 5_000_000 }); // microseconds
151
- cl.transportCommand('cam1', { type: 'next_playlist_clip' });
152
- ```
153
-
154
- Available types: `play`, `pause`, `stop`, `seek`, `seek_usertime`, `jog`, `live`, `set_recorder`, `load_playlist`, `eject_playlist`, `load_clip`, `set_metadata`, `take`, `pre_roll`, `post_roll`, `next_playlist_clip`, `skip_next_playlist_clip`, `transitions_disabled`, `set_scrubbing`.
155
-
156
- ### Playlists
157
-
158
- ```typescript
159
- cl.playlistLoad('cam1', {
160
- id: 'my-playlist',
161
- duration: 0,
162
- clips: [
163
- { id: 1, recorderId: 'recorder1', speed: 1, transitionTimeUs: 0,
164
- startTimeUs: 0, endTimeUs: 10_000_000, audioRouting: [] },
165
- ],
166
- });
167
-
168
- cl.playlistEject('cam1');
169
- ```
170
-
171
- ### Animations
172
-
173
- ```typescript
174
- import { CommandList, keyframe, EaseKind, sesame } from '@stinkycomputing/sesame-api-client';
175
- const { HOLD } = sesame.v1.compositor.AnimationChannelEvaluationMode;
176
-
177
- cl.propertyAnimate(
178
- { compositorId: 'main' }, 'cam1-node', 'opacity', HOLD, HOLD,
179
- [
180
- keyframe(0, { floatValue: 0.0 }),
181
- keyframe(500_000, { floatValue: 1.0 }, EaseKind.QUADRATIC_INOUT),
182
- keyframe(1_000_000, { floatValue: 0.5 }, EaseKind.CUBIC_IN, EaseKind.CUBIC_OUT),
183
- ],
184
- );
185
- ```
186
-
187
- `keyframe(timeUs, value, easingIn?, easingOut?)` - time in microseconds.
188
-
189
- ### Metadata Bindings
190
-
191
- Wire output tracks to node capabilities for remote state / control. See [Data Frames](#data-frames) for the other side.
192
-
193
- ```typescript
194
- cl.setMetadataBindings([
195
- // Push AudioMixerStatus from 'mix1' out on the 'audio-state' track of output 'ws-out'
196
- { sourceId: 'mix1', metadataId: 'audio-state', outputId: 'ws-out', trackName: 'audio-state' },
197
- // Route incoming 'audio-ctrl' track on source 'ws-in' → mixer 'mix1'
198
- { sourceId: 'ws-in', trackName: 'audio-ctrl', targetId: 'mix1', metadataId: 'audio-control' },
199
- ]);
200
- ```
201
-
202
- ---
203
-
204
- ## Data Frames
205
-
206
- `FRAME_TYPE_DATA` frames carry typed protobuf payloads over output WebSocket/MoQ tracks. Use `parseDataFrame` to decode incoming frames without manually inspecting `data_type`.
207
-
208
- ### Receiving (output stream → client)
209
-
210
- ```typescript
211
- import { WireProtocol, parseDataFrame } from '@stinkycomputing/sesame-api-client';
212
-
213
- ws.addEventListener('message', async (evt) => {
214
- const bytes = new Uint8Array(await evt.data.arrayBuffer());
215
- const frame = WireProtocol.parse(bytes);
216
- const data = parseDataFrame(frame);
217
- if (!data) return; // not a DATA frame
218
-
219
- switch (data.dataType) {
220
- case 'audio-state':
221
- // data.status: sesame.v1.audio.AudioMixerStatus
222
- console.log(data.trackName, data.status.id, data.status.vu);
223
- break;
224
- case 'transport-status':
225
- // data.event: sesame.v1.status.TransportEvent
226
- console.log(data.event.state, data.event.positionUs);
227
- break;
228
- case 'json':
229
- console.log(data.json); // parsed JS object
230
- break;
231
- }
232
- });
233
- ```
234
-
235
- | `dataType` | Payload type | Direction |
236
- |------------|-------------|-----------|
237
- | `'json'` | `unknown` (parsed JSON) | server → client |
238
- | `'transport-status'` | `sesame.v1.status.TransportEvent` | server → client |
239
- | `'audio-state'` | `sesame.v1.audio.AudioMixerStatus` | server → client |
240
- | `'transport-control'` | `sesame.v1.sources.TransportControlRequest` | client → server |
241
- | `'audio-control'` | `sesame.v1.audio.AudioControlRequest` | client → server |
242
-
243
- ### Sending Remote Control (source stream → server)
244
-
245
- Send control frames over a WebSocket **source** connection (inbound to the server). The server resolves the target node from the metadata binding - no ID is needed in the payload.
246
-
247
- ```typescript
248
- import { buildAudioControlFrame, buildTransportControlFrame, sesame } from '@stinkycomputing/sesame-api-client';
249
-
250
- // Adjust a mixer channel level and pan
251
- ws.send(buildAudioControlFrame('audio-ctrl', {
252
- channels: [
253
- { channelId: 'ch1', level: 0.8, pan: -0.2 },
254
- ],
255
- }));
256
-
257
- // Adjust a plugin parameter on a channel
258
- ws.send(buildAudioControlFrame('audio-ctrl', {
259
- channels: [{
260
- channelId: 'ch1',
261
- plugins: [{ pluginId: 'eq1', params: [{ id: 3, floatValue: 200.0 }] }],
262
- }],
263
- }));
264
-
265
- // Queue a transport command on a bound source
266
- ws.send(buildTransportControlFrame('transport-ctrl', {
267
- cmdType: sesame.v1.sources.SourceTransportCommandType.SOURCE_TRANSPORT_CMD_PLAY,
268
- }));
269
- ```
270
-
271
- `buildAudioControlFrame(trackName, request)` and `buildTransportControlFrame(trackName, request)` both return a ready-to-send `Uint8Array`.
272
-
273
- ### Sending Binary Data
274
-
275
- Send opaque binary payloads on a named metadata track:
276
-
277
- ```typescript
278
- import { buildBinaryDataFrame } from '@stinkycomputing/sesame-api-client';
279
-
280
- const payload = new Uint8Array([0xF0, 0x15, 0x31, /* ... */ 0xF7]);
281
- ws.send(buildBinaryDataFrame('my-binary-track', payload));
282
- ```
283
-
284
- ---
285
-
286
- ## StatusApi
287
-
288
- Accessed via `client.status`.
289
-
290
- ```typescript
291
- const status = await client.status.getStatus();
292
- // status: sesame.v1.status.Status - full server state snapshot
293
-
294
- const devices = await client.status.getIODevices();
295
- // devices.devices: list of available capture/output hardware
296
- ```
297
-
298
- ---
299
-
300
- ## RecorderApi
301
-
302
- Accessed via `client.recorder`.
303
-
304
- ```typescript
305
- // Add a clip to a recorder
306
- await client.recorder.addClip('recorder1', {
307
- id: 1, path: '/recordings/clip1.mp4', durationUs: 30_000_000,
308
- });
309
-
310
- // Copy, remove, clear
311
- await client.recorder.copyClip('recorder1', 1, 2);
312
- await client.recorder.removeClip('recorder1', 1);
313
- await client.recorder.removeAllClips('recorder1');
314
-
315
- // Fetch all clips
316
- const clips = await client.recorder.getClips('recorder1');
317
-
318
- // Verify clip files on disk (returns list of error strings)
319
- const errors = await client.recorder.verifyClips('recorder1');
320
-
321
- // Set clip metadata
322
- await client.recorder.setMetadata('recorder1', { /* ITransportMetadata */ });
323
- ```
324
-
325
- ---
326
-
327
- ## Event Subscriptions
328
-
329
- Subscribe to server-pushed event topics via `addSubscription`:
330
-
331
- ```typescript
332
- import { sesame } from '@stinkycomputing/sesame-api-client';
333
- const { EVENT_TOPIC_TRANSPORT, EVENT_TOPIC_ERROR } = sesame.v1.common.EventTopic;
334
-
335
- client.addSubscription(EVENT_TOPIC_TRANSPORT, (topic, event) => {
336
- const transport = event.transportEvent;
337
- console.log(transport?.state, transport?.positionUs);
338
- });
339
-
340
- client.addSubscription(EVENT_TOPIC_ERROR, (topic, event) => {
341
- console.error(event.errorEvent?.message);
342
- });
343
-
344
- // Unsubscribe when done
345
- client.removeSubscription(EVENT_TOPIC_TRANSPORT, myCallback);
346
- ```
347
-
348
- Available topics: `EVENT_TOPIC_TRANSPORT`, `EVENT_TOPIC_ERROR`, `EVENT_TOPIC_CALLBACK`, `EVENT_TOPIC_JOB`, `EVENT_TOPIC_RECORDER`, `EVENT_TOPIC_METADATA`.
349
-
350
- ---
351
-
352
- ## Wire Protocol
353
-
354
- Every WebSocket message is framed as:
355
-
356
- ```
357
- [4-byte LE header_size][FrameHeader protobuf][payload bytes]
358
- ```
359
-
360
- `WireProtocol.serialize` / `WireProtocol.parse` handle the framing layer:
361
-
362
- ```typescript
363
- import { WireProtocol, FrameType, sesame } from '@stinkycomputing/sesame-api-client';
364
-
365
- // Build a frame manually
366
- const bytes = WireProtocol.serialize(
367
- { type: FrameType.FRAME_TYPE_RPC },
368
- rpcPayload,
369
- );
370
-
371
- // Parse an incoming frame
372
- const { valid, header, payload } = WireProtocol.parse(incoming);
373
- if (valid && header.type === FrameType.FRAME_TYPE_DATA) {
374
- // use parseDataFrame() instead - see Data Frames above
375
- }
376
- ```
377
-
378
- Frame types: `FRAME_TYPE_RPC`, `FRAME_TYPE_VIDEO`, `FRAME_TYPE_AUDIO`, `FRAME_TYPE_MUXED`, `FRAME_TYPE_DECODER_DATA`, `FRAME_TYPE_DATA`.
379
-
380
- ---
381
-
382
- ## Protobuf Modules
383
-
384
- All protobuf types are accessible via the `sesame` namespace:
385
-
386
- ```typescript
387
- import { sesame } from '@stinkycomputing/sesame-api-client';
388
- ```
389
-
390
- | Module | Key contents |
391
- |--------|-------------|
392
- | `sesame.v1.wire` | `FrameHeader`, `FrameType`, `DataType`, `MediaCodecData` |
393
- | `sesame.v1.common` | `Empty`, `Vec4`, `PropValue`, `EventTopic`, `PropertyDomain` |
394
- | `sesame.v1.sources` | Source configs, `SourceTransportCommand`, `TransportControlRequest` |
395
- | `sesame.v1.outputs` | Output configs, `EncoderConfig` |
396
- | `sesame.v1.compositor` | Scene graph, nodes, properties, animations, `KeyFrame` |
397
- | `sesame.v1.audio` | `AudioMixerConfig`, `AudioMixerStatus`, `AudioControlRequest` |
398
- | `sesame.v1.recorder` | `RecorderClip`, `PlaylistItem`, `TransitionType` |
399
- | `sesame.v1.jobs` | Background export/import jobs |
400
- | `sesame.v1.status` | `Status`, `Event`, `TransportEvent`, `SubscriptionRequest` |
401
- | `sesame.v1.commands` | `CommandList`, `CommandListItem`, `MetadataBinding` |
402
- | `sesame.v1.rpc` | `Message`, `Request`, `Response`, `Event` envelope |
403
-
404
- For the complete field and enum reference see **[Protocol Reference](docs/protocol-reference.md)**.
405
-
406
- ---
407
-
408
- ## Bundling
409
-
410
- All dependencies (`events`, `long`, `protobufjs`) are pure JS - no native modules. Bundles cleanly with esbuild, Vite, webpack, or similar without any special `external` config.
411
-
412
- ---
413
-
414
- ## Publishing
415
-
416
- ```bash
417
- # Bump version in package.json, then:
418
- pnpm run publish:api-client # stable
419
- pnpm run publish:api-client:alpha # pre-release (--tag alpha)
420
- ```
421
-
422
- Or from this package directly:
423
-
424
- ```bash
425
- npm run build
426
- npm publish --access public [--tag alpha]
427
- ```
428
-
429
- ---
430
-
431
- ## License
432
-
433
- MIT
1
+ # @stinkycomputing/sesame-api-client
2
+
3
+ TypeScript client library for the Sesame video production server. Provides type-safe protobuf definitions, a WebSocket RPC client with automatic reconnection, and helpers for building command lists, parsing data frames, and sending remote control messages.
4
+
5
+ ## Install
6
+
7
+ ```bash
8
+ npm install @stinkycomputing/sesame-api-client
9
+ ```
10
+
11
+ Works in Node.js and the browser. The browser entry point bundles `events` and other Node built-ins - no polyfills needed.
12
+
13
+ ---
14
+
15
+ ## Quick Start
16
+
17
+ ```typescript
18
+ import { SesameClient, CommandList } from '@stinkycomputing/sesame-api-client';
19
+
20
+ const client = new SesameClient(9000); // port, or a full ws:// URL
21
+
22
+ const cl = new CommandList();
23
+ cl.sourceAdd('cam1', { file: { url: '/path/to/video.mp4' } });
24
+ cl.compositorAdd('main', 1920, 1080, false);
25
+ cl.nodeAdd('cam1-node', {
26
+ compositorId: 'main',
27
+ parentAddress: '',
28
+ nodeType: 'source',
29
+ });
30
+
31
+ await client.execute(cl);
32
+ ```
33
+
34
+ ---
35
+
36
+ ## SesameClient
37
+
38
+ ```typescript
39
+ const client = new SesameClient(9000);
40
+ // or with a custom URL:
41
+ const client = new SesameClient('ws://192.168.1.10:9000/api');
42
+ ```
43
+
44
+ The client auto-connects and auto-reconnects. Listen for connection state changes:
45
+
46
+ ```typescript
47
+ client.on('connected', () => console.log('ready'));
48
+ client.on('disconnected', () => console.log('lost connection'));
49
+ ```
50
+
51
+ | Member | Description |
52
+ |--------|-------------|
53
+ | `client.execute(cl)` | Send a `CommandList` to the server |
54
+ | `client.isConnected()` | Current connection state |
55
+ | `client.disconnect()` | Close the connection |
56
+ | `client.status` | [`StatusApi`](#statusapi) instance |
57
+ | `client.recorder` | [`RecorderApi`](#recorderapi) instance |
58
+ | `client.jobs` | `JobsApi` instance |
59
+
60
+ ---
61
+
62
+ ## Command List
63
+
64
+ `CommandList` builds a batch of operations sent in a single `execute` call. All methods are chainable and accept an optional `timeOffsetMs` to schedule commands relative to the current frame.
65
+
66
+ ### Sources
67
+
68
+ ```typescript
69
+ const cl = new CommandList();
70
+
71
+ // File source
72
+ cl.sourceAdd('clip1', { file: { url: '/clips/a.mp4' } });
73
+
74
+ // Decklink capture
75
+ cl.sourceAdd('sdi-in', { decklink: { deviceIndex: 0, videoFormat: DecklinkVideoFormat.HD_1080i_50 } });
76
+
77
+ // WebSocket ingest
78
+ cl.sourceAdd('ws-in', { websocket: { channel: 'my-channel' } });
79
+
80
+ // MoQ ingest
81
+ cl.sourceAdd('moq-in', { moq: { url: 'https://relay.example', broadcast: 'feed', key: '' } });
82
+
83
+ cl.sourceRemove('clip1');
84
+ ```
85
+
86
+ ### Compositor
87
+
88
+ ```typescript
89
+ cl.compositorAdd('main', 1920, 1080, false);
90
+
91
+ cl.nodeAdd('bg', { compositorId: 'main', parentAddress: '', nodeType: 'source' });
92
+ cl.nodeAdd('fg', { compositorId: 'main', parentAddress: '', nodeType: 'source' });
93
+
94
+ cl.nodeRemove('main', 'fg');
95
+ cl.compositorRemove('main');
96
+ ```
97
+
98
+ ### Outputs
99
+
100
+ ```typescript
101
+ cl.outputAdd('stream', {
102
+ encoder: { width: 1920, height: 1080, fps: 50, preset: 0, bitrateKbps: 8000 },
103
+ websocket: { channel: 'live-out' },
104
+ });
105
+
106
+ cl.outputUpdate('stream', { /* updated config */ });
107
+ cl.outputRemove('stream');
108
+ ```
109
+
110
+ ### Audio Mixers
111
+
112
+ ```typescript
113
+ import { sesame } from '@stinkycomputing/sesame-api-client';
114
+ const { AUDIO_MIXER_CHANNEL_TYPE_STEREO } = sesame.v1.audio.AudioMixerChannelType;
115
+
116
+ cl.audioMixerAdd('mix1', {
117
+ outputType: AUDIO_MIXER_CHANNEL_TYPE_STEREO,
118
+ order: 0,
119
+ channels: [
120
+ { id: 'ch1', sourceId: 'cam1', channelType: AUDIO_MIXER_CHANNEL_TYPE_STEREO,
121
+ sourceChannels: [0, 1], level: 1.0, pan: 0.0, plugins: [] },
122
+ ],
123
+ });
124
+
125
+ cl.audioMixerChannelAdd('mix1', { id: 'ch2', sourceId: 'cam2', /* ... */ });
126
+ cl.audioMixerChannelRemove('mix1', 'ch2');
127
+ cl.audioMixerRemove('mix1');
128
+ ```
129
+
130
+ ### Properties
131
+
132
+ Property names are type-checked against a generated registry - TypeScript will autocomplete valid names and value shapes.
133
+
134
+ ```typescript
135
+ // Compositor property (typed)
136
+ cl.propertySet({ compositorId: 'main' }, 'cam1-node', 'transform/position', { vecValue: { r: 100, g: 200 } });
137
+
138
+ // Audio mixer property
139
+ cl.propertySet({ audioMixerId: 'mix1' }, 'ch1', 'level', { floatValue: 0.8 });
140
+
141
+ // Source property
142
+ cl.propertySet({ sourceId: 'cam1' }, '', 'decoder/buffer_frames', { intValue: 4 });
143
+ ```
144
+
145
+ ### Transport Commands
146
+
147
+ ```typescript
148
+ cl.transportCommand('cam1', { type: 'play' });
149
+ cl.transportCommand('cam1', { type: 'pause' });
150
+ cl.transportCommand('cam1', { type: 'seek', value: 5_000_000 }); // microseconds
151
+ cl.transportCommand('cam1', { type: 'next_playlist_clip' });
152
+ ```
153
+
154
+ Available types: `play`, `pause`, `stop`, `seek`, `seek_usertime`, `jog`, `live`, `set_recorder`, `load_playlist`, `eject_playlist`, `load_clip`, `set_metadata`, `take`, `pre_roll`, `post_roll`, `next_playlist_clip`, `skip_next_playlist_clip`, `transitions_disabled`, `set_scrubbing`.
155
+
156
+ ### Playlists
157
+
158
+ ```typescript
159
+ cl.playlistLoad('cam1', {
160
+ id: 'my-playlist',
161
+ duration: 0,
162
+ clips: [
163
+ { id: 1, recorderId: 'recorder1', speed: 1, transitionTimeUs: 0,
164
+ startTimeUs: 0, endTimeUs: 10_000_000, audioRouting: [] },
165
+ ],
166
+ });
167
+
168
+ cl.playlistEject('cam1');
169
+ ```
170
+
171
+ ### Animations
172
+
173
+ ```typescript
174
+ import { CommandList, keyframe, EaseKind, sesame } from '@stinkycomputing/sesame-api-client';
175
+ const { HOLD } = sesame.v1.compositor.AnimationChannelEvaluationMode;
176
+
177
+ cl.propertyAnimate(
178
+ { compositorId: 'main' }, 'cam1-node', 'opacity', HOLD, HOLD,
179
+ [
180
+ keyframe(0, { floatValue: 0.0 }),
181
+ keyframe(500_000, { floatValue: 1.0 }, EaseKind.QUADRATIC_INOUT),
182
+ keyframe(1_000_000, { floatValue: 0.5 }, EaseKind.CUBIC_IN, EaseKind.CUBIC_OUT),
183
+ ],
184
+ );
185
+ ```
186
+
187
+ `keyframe(timeUs, value, easingIn?, easingOut?)` - time in microseconds.
188
+
189
+ ### Metadata Bindings
190
+
191
+ Wire output tracks to node capabilities for remote state / control. See [Data Frames](#data-frames) for the other side.
192
+
193
+ ```typescript
194
+ cl.setMetadataBindings([
195
+ // Push AudioMixerStatus from 'mix1' out on the 'audio-state' track of output 'ws-out'
196
+ { sourceId: 'mix1', metadataId: 'audio-state', outputId: 'ws-out', trackName: 'audio-state' },
197
+ // Route incoming 'audio-ctrl' track on source 'ws-in' → mixer 'mix1'
198
+ { sourceId: 'ws-in', trackName: 'audio-ctrl', targetId: 'mix1', metadataId: 'audio-control' },
199
+ ]);
200
+ ```
201
+
202
+ ---
203
+
204
+ ## Data Frames
205
+
206
+ `FRAME_TYPE_DATA` frames carry typed protobuf payloads over output WebSocket/MoQ tracks. Use `parseDataFrame` to decode incoming frames without manually inspecting `data_type`.
207
+
208
+ ### Receiving (output stream → client)
209
+
210
+ ```typescript
211
+ import { WireProtocol, parseDataFrame } from '@stinkycomputing/sesame-api-client';
212
+
213
+ ws.addEventListener('message', async (evt) => {
214
+ const bytes = new Uint8Array(await evt.data.arrayBuffer());
215
+ const frame = WireProtocol.parse(bytes);
216
+ const data = parseDataFrame(frame);
217
+ if (!data) return; // not a DATA frame
218
+
219
+ switch (data.dataType) {
220
+ case 'audio-state':
221
+ // data.status: sesame.v1.audio.AudioMixerStatus
222
+ console.log(data.trackName, data.status.id, data.status.vu);
223
+ break;
224
+ case 'transport-status':
225
+ // data.event: sesame.v1.status.TransportEvent
226
+ console.log(data.event.state, data.event.positionUs);
227
+ break;
228
+ case 'json':
229
+ console.log(data.json); // parsed JS object
230
+ break;
231
+ }
232
+ });
233
+ ```
234
+
235
+ | `dataType` | Payload type | Direction |
236
+ |------------|-------------|-----------|
237
+ | `'json'` | `unknown` (parsed JSON) | server → client |
238
+ | `'transport-status'` | `sesame.v1.status.TransportEvent` | server → client |
239
+ | `'audio-state'` | `sesame.v1.audio.AudioMixerStatus` | server → client |
240
+ | `'transport-control'` | `sesame.v1.sources.TransportControlRequest` | client → server |
241
+ | `'audio-control'` | `sesame.v1.audio.AudioControlRequest` | client → server |
242
+
243
+ ### Sending Remote Control (source stream → server)
244
+
245
+ Send control frames over a WebSocket **source** connection (inbound to the server). The server resolves the target node from the metadata binding - no ID is needed in the payload.
246
+
247
+ ```typescript
248
+ import { buildAudioControlFrame, buildTransportControlFrame, sesame } from '@stinkycomputing/sesame-api-client';
249
+
250
+ // Adjust a mixer channel level and pan
251
+ ws.send(buildAudioControlFrame('audio-ctrl', {
252
+ channels: [
253
+ { channelId: 'ch1', level: 0.8, pan: -0.2 },
254
+ ],
255
+ }));
256
+
257
+ // Adjust a plugin parameter on a channel
258
+ ws.send(buildAudioControlFrame('audio-ctrl', {
259
+ channels: [{
260
+ channelId: 'ch1',
261
+ plugins: [{ pluginId: 'eq1', params: [{ id: 3, floatValue: 200.0 }] }],
262
+ }],
263
+ }));
264
+
265
+ // Queue a transport command on a bound source
266
+ ws.send(buildTransportControlFrame('transport-ctrl', {
267
+ cmdType: sesame.v1.sources.SourceTransportCommandType.SOURCE_TRANSPORT_CMD_PLAY,
268
+ }));
269
+ ```
270
+
271
+ `buildAudioControlFrame(trackName, request)` and `buildTransportControlFrame(trackName, request)` both return a ready-to-send `Uint8Array`.
272
+
273
+ ### Sending Binary Data
274
+
275
+ Send opaque binary payloads on a named metadata track:
276
+
277
+ ```typescript
278
+ import { buildBinaryDataFrame } from '@stinkycomputing/sesame-api-client';
279
+
280
+ const payload = new Uint8Array([0xF0, 0x15, 0x31, /* ... */ 0xF7]);
281
+ ws.send(buildBinaryDataFrame('my-binary-track', payload));
282
+ ```
283
+
284
+ ---
285
+
286
+ ## StatusApi
287
+
288
+ Accessed via `client.status`.
289
+
290
+ ```typescript
291
+ const status = await client.status.getStatus();
292
+ // status: sesame.v1.status.Status - full server state snapshot
293
+
294
+ const devices = await client.status.getIODevices();
295
+ // devices.devices: list of available capture/output hardware
296
+ ```
297
+
298
+ ---
299
+
300
+ ## RecorderApi
301
+
302
+ Accessed via `client.recorder`.
303
+
304
+ ```typescript
305
+ // Add a clip to a recorder
306
+ await client.recorder.addClip('recorder1', {
307
+ id: 1, path: '/recordings/clip1.mp4', durationUs: 30_000_000,
308
+ });
309
+
310
+ // Copy, remove, clear
311
+ await client.recorder.copyClip('recorder1', 1, 2);
312
+ await client.recorder.removeClip('recorder1', 1);
313
+ await client.recorder.removeAllClips('recorder1');
314
+
315
+ // Fetch all clips
316
+ const clips = await client.recorder.getClips('recorder1');
317
+
318
+ // Verify clip files on disk (returns list of error strings)
319
+ const errors = await client.recorder.verifyClips('recorder1');
320
+
321
+ // Set clip metadata
322
+ await client.recorder.setMetadata('recorder1', { /* ITransportMetadata */ });
323
+ ```
324
+
325
+ ---
326
+
327
+ ## Event Subscriptions
328
+
329
+ Subscribe to server-pushed event topics via `addSubscription`:
330
+
331
+ ```typescript
332
+ import { sesame } from '@stinkycomputing/sesame-api-client';
333
+ const { EVENT_TOPIC_TRANSPORT, EVENT_TOPIC_ERROR } = sesame.v1.common.EventTopic;
334
+
335
+ client.addSubscription(EVENT_TOPIC_TRANSPORT, (topic, event) => {
336
+ const transport = event.transportEvent;
337
+ console.log(transport?.state, transport?.positionUs);
338
+ });
339
+
340
+ client.addSubscription(EVENT_TOPIC_ERROR, (topic, event) => {
341
+ console.error(event.errorEvent?.message);
342
+ });
343
+
344
+ // Unsubscribe when done
345
+ client.removeSubscription(EVENT_TOPIC_TRANSPORT, myCallback);
346
+ ```
347
+
348
+ Available topics: `EVENT_TOPIC_TRANSPORT`, `EVENT_TOPIC_ERROR`, `EVENT_TOPIC_CALLBACK`, `EVENT_TOPIC_JOB`, `EVENT_TOPIC_RECORDER`, `EVENT_TOPIC_METADATA`.
349
+
350
+ ---
351
+
352
+ ## Wire Protocol
353
+
354
+ Every WebSocket message is framed as:
355
+
356
+ ```
357
+ [4-byte LE header_size][FrameHeader protobuf][payload bytes]
358
+ ```
359
+
360
+ `WireProtocol.serialize` / `WireProtocol.parse` handle the framing layer:
361
+
362
+ ```typescript
363
+ import { WireProtocol, FrameType, sesame } from '@stinkycomputing/sesame-api-client';
364
+
365
+ // Build a frame manually
366
+ const bytes = WireProtocol.serialize(
367
+ { type: FrameType.FRAME_TYPE_RPC },
368
+ rpcPayload,
369
+ );
370
+
371
+ // Parse an incoming frame
372
+ const { valid, header, payload } = WireProtocol.parse(incoming);
373
+ if (valid && header.type === FrameType.FRAME_TYPE_DATA) {
374
+ // use parseDataFrame() instead - see Data Frames above
375
+ }
376
+ ```
377
+
378
+ Frame types: `FRAME_TYPE_RPC`, `FRAME_TYPE_VIDEO`, `FRAME_TYPE_AUDIO`, `FRAME_TYPE_MUXED`, `FRAME_TYPE_DECODER_DATA`, `FRAME_TYPE_DATA`.
379
+
380
+ ---
381
+
382
+ ## Protobuf Modules
383
+
384
+ All protobuf types are accessible via the `sesame` namespace:
385
+
386
+ ```typescript
387
+ import { sesame } from '@stinkycomputing/sesame-api-client';
388
+ ```
389
+
390
+ | Module | Key contents |
391
+ |--------|-------------|
392
+ | `sesame.v1.wire` | `FrameHeader`, `FrameType`, `DataType`, `MediaCodecData` |
393
+ | `sesame.v1.common` | `Empty`, `Vec4`, `PropValue`, `EventTopic`, `PropertyDomain` |
394
+ | `sesame.v1.sources` | Source configs, `SourceTransportCommand`, `TransportControlRequest` |
395
+ | `sesame.v1.outputs` | Output configs, `EncoderConfig` |
396
+ | `sesame.v1.compositor` | Scene graph, nodes, properties, animations, `KeyFrame` |
397
+ | `sesame.v1.audio` | `AudioMixerConfig`, `AudioMixerStatus`, `AudioControlRequest` |
398
+ | `sesame.v1.recorder` | `RecorderClip`, `PlaylistItem`, `TransitionType` |
399
+ | `sesame.v1.jobs` | Background export/import jobs |
400
+ | `sesame.v1.status` | `Status`, `Event`, `TransportEvent`, `SubscriptionRequest` |
401
+ | `sesame.v1.commands` | `CommandList`, `CommandListItem`, `MetadataBinding` |
402
+ | `sesame.v1.rpc` | `Message`, `Request`, `Response`, `Event` envelope |
403
+
404
+ For the complete field and enum reference see **[Protocol Reference](docs/protocol-reference.md)**.
405
+
406
+ ---
407
+
408
+ ## Bundling
409
+
410
+ All dependencies (`events`, `long`, `protobufjs`) are pure JS - no native modules. Bundles cleanly with esbuild, Vite, webpack, or similar without any special `external` config.
411
+
412
+ ---
413
+
414
+ ## Publishing
415
+
416
+ ```bash
417
+ # Bump version in package.json, then:
418
+ pnpm run publish:api-client # stable
419
+ pnpm run publish:api-client:alpha # pre-release (--tag alpha)
420
+ ```
421
+
422
+ Or from this package directly:
423
+
424
+ ```bash
425
+ npm run build
426
+ npm publish --access public [--tag alpha]
427
+ ```
428
+
429
+ ---
430
+
431
+ ## License
432
+
433
+ MIT