@reactor-team/js-sdk 3.1.0 → 3.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -23,6 +23,16 @@ Full reference and guides live at **[docs.reactor.inc](https://docs.reactor.inc)
23
23
  npm install @reactor-team/js-sdk
24
24
  ```
25
25
 
26
+ No bundler configuration needed: the WebAssembly binding the SDK runs on
27
+ ships inside its own JavaScript module, fetched the first time a `Reactor`
28
+ connects rather than when the SDK is imported. Each release is
29
+ checked against Vite 6, 7 and 8 and Next.js 15 and 16 (webpack and
30
+ Turbopack), in both dev and production builds, except Next.js 15 with
31
+ Turbopack, checked in dev only since its production build is still beta.
32
+
33
+ If your page sets a Content Security Policy, its `script-src` needs
34
+ `'wasm-unsafe-eval'` for the binding to compile.
35
+
26
36
  ## Quickstart
27
37
 
28
38
  ```ts
@@ -30,7 +40,7 @@ import { Reactor } from "@reactor-team/js-sdk";
30
40
 
31
41
  const reactor = new Reactor({
32
42
  modelName: "my-model",
33
- jwt: () => fetchToken(), // or a plain string; omit for an unauthenticated local runtime
43
+ jwt: fetchToken, // your resolver, or a plain string; omit for an unauthenticated local runtime
34
44
  });
35
45
 
36
46
  reactor.on("statusChanged", (status) => {
@@ -78,6 +88,45 @@ const { jwt } = await result.json();
78
88
  See [Authentication](https://docs.reactor.inc/authentication) for the full request shape,
79
89
  including `max_session_duration_seconds` and other constraints.
80
90
 
91
+ ### Refreshing a token mid-session
92
+
93
+ A scoped token reaches only the sessions it created or was bound to. When a session outlives
94
+ its token, a fresh token minted the same way cannot reach it: uploads, reconnects, the session
95
+ delete and clip reads all return `403`. So the SDK tells the resolver what each request is for:
96
+
97
+ ```ts
98
+ async function fetchToken({ sessionId }: JwtRequestContext = {}): Promise<string> {
99
+ // ask your backend for a token, and pass sessionId along when it is set
100
+ }
101
+ ```
102
+
103
+ `sessionId` is set for every request that targets an existing session and unset for session
104
+ creation. Keep returning the token that created the session until shortly before it expires.
105
+ Then mint a replacement bound to that session, which lets it act on the session and create
106
+ nothing else:
107
+
108
+ ```ts
109
+ body: JSON.stringify({
110
+ authorization_details: [
111
+ {
112
+ type: "session",
113
+ resources: { models: { match: ["my-model"] }, sessions: { bind: [sessionId] } },
114
+ },
115
+ ],
116
+ }),
117
+ ```
118
+
119
+ Reactor only binds a session the API key owns, so the question your backend must answer is
120
+ whether *this* browser may hold a token for *that* session. The simplest answer is to create the
121
+ session on the backend and hand the browser `{ jwt, sessionId }` to adopt with
122
+ `connect(jwt, { sessionId })`: the backend then knows the owner. When the browser creates the
123
+ session itself, have it register the session id as soon as the SDK first names it to the
124
+ resolver, which is right after creation, with the token that created it; check that token
125
+ against `GET /sessions/{id}` (Reactor answers only for a token bound to it), and remember the
126
+ session for that browser. Registering only after `connect()` returns is too late when a slow
127
+ model keeps the connect open past the token's expiry. The bundled examples do exactly that in
128
+ `examples/shared/token-server.ts`.
129
+
81
130
  ## Events
82
131
 
83
132
  `on(event, handler)` / `off(event, handler)` / `once(event, handler)`. The ones you'll reach
@@ -105,13 +154,97 @@ matching `schemaReceived`/`capabilitiesReceived` events. See
105
154
  [`getSchema()`](https://docs.reactor.inc/sdk-reference/reactor-class#getschema) and
106
155
  [`Capabilities`](https://docs.reactor.inc/sdk-reference/types#capabilities) for the full shape.
107
156
 
157
+ ## Stepped generation
158
+
159
+ A model that generates one step at a time — rather than streaming continuously — is driven
160
+ with `step()`. Create the client with `generation: "stepped"`; `connect()` fails with a
161
+ `GenerationModeMismatchError` when the runtime does not start the session that way.
162
+
163
+ ```ts
164
+ import { Reactor, StepRefusedError, stepFrameToImageData } from "@reactor-team/js-sdk";
165
+
166
+ const reactor = new Reactor({ modelName: "my-stepped-model", generation: "stepped", jwt });
167
+ await reactor.connect();
168
+
169
+ // Input frames never travel over RTP in a stepped session: claim the input
170
+ // track, push frames through it, and the next step carries them.
171
+ const camera = await reactor.publishTrack("camera");
172
+ await camera.pushFrame(canvasCtx.getImageData(0, 0, 640, 360));
173
+
174
+ try {
175
+ const result = await reactor.step({
176
+ state: { prompt: "a cat" },
177
+ commands: [{ command: "jump", data: { height: 2 } }],
178
+ });
179
+ if (result.error) console.warn(`step ${result.step} failed:`, result.error);
180
+ const frame = result.tracks.find((t) => t.track === "main_video")?.frames[0];
181
+ if (frame) outputCtx.putImageData(stepFrameToImageData(frame), 0, 0);
182
+ } catch (error) {
183
+ if (error instanceof StepRefusedError) console.warn("refused:", error.entry, error.message);
184
+ else throw error;
185
+ }
186
+
187
+ // Every part of a step is optional: this one runs on the model's current state.
188
+ await reactor.step();
189
+ ```
190
+
191
+ - `step({ state, commands })` sends `state`, `commands` and every frame pushed since the
192
+ previous step in one request. The runtime applies them in order — each `state` key as the
193
+ model's `set_<key>` command, then `commands`, then the frames — and then generates. The
194
+ runtime checks the whole request before it applies any of it, so a refused step changes
195
+ nothing; past that check they are separate calls, and a handler that fails does not undo
196
+ the ones before it.
197
+ `sendCommand()` outside a step still works and changes the state the next step uses. A file
198
+ input goes in the way it does for a command: `uploadFile()` it, then put the `FileRef` in a
199
+ command's `data` or a state value.
200
+ - It resolves with `{ step, tracks: [{ track, kind, rate, frames }], elapsed, files, error? }`,
201
+ `step` being the step's number in the session, counted from 1.
202
+ `rate` is the rate a track's frames play at as the runtime reported it — frames per second
203
+ for video, samples per second for audio, `0` when the runtime did not say. `elapsed` is the seconds the step's `generate()` took (or the time the model reported for
204
+ the step); each frame has `dtype`, `shape`, typed-array `data` (`Uint8Array` RGB
205
+ `(height, width, 3)` for video, `Int16Array` planar `(channels, samples)` for audio), and
206
+ optional `metadata` / `captureTimeUs`. `files` lists `{ name, contentType, size }` for each
207
+ file the model kept with the step; the bytes are served over HTTP from the session's step
208
+ results (`GET /sessions/{id}/steps/{step}`), not carried in the result. `error` is set when
209
+ the step ran and the model reported a failure — such a step resolves, it does not reject.
210
+ - `publishTrack(name)` resolves with a handle whose `pushFrame(frame, { metadata?,
211
+ captureTimeUs? })` buffers a frame for the next step (`reactor.pushFrame(name, frame)` does
212
+ the same, for a track published on this connection). The handle stops working once the track
213
+ is unpublished or its connection ends; publish again for a new one. It takes a `VideoFrame`, an `ImageData`, `{ data, width, height, format? }` raw
214
+ `"rgba"`/`"rgb"`/`"bgra"` pixels, or `{ samples, sampleRate, channels? }` interleaved
215
+ `Int16Array` PCM for an audio track. Pushes keep their order, and `step()` waits for every
216
+ push called before it. A refused push rejects, so await it, or `.catch()` it in a frame loop.
217
+ At most one message's worth of frames waits for the next step:
218
+ pushing faster than steps run drops the oldest frames, across tracks. `clearStepInputs()`
219
+ drops whatever is waiting.
220
+ - The `MediaStreamTrack` argument of `publishTrack()` is optional in a stepped session and
221
+ ignored there, with a console warning: no RTP is sent, frames go through `pushFrame()`. In
222
+ a streamed session it is still required, and `publishTrack(name)` without one rejects with
223
+ an `InvalidStateError`.
224
+ - Refusals are typed: `StepRefusedError` (`step_refused`: nothing the step carried was
225
+ applied, and `entry` names the part of the request at fault), `StepNotReadyError`
226
+ (`step_not_ready`: the session cannot take a step yet, and the step did not run),
227
+ `StepCommandFailedError` (`step_command_failed`: a state key or command failed, the calls
228
+ before it stay applied, and the step did not run; `entry` names the call and
229
+ `command_code` the code its handler raised), `StepSessionEndedError`
230
+ (`step_session_ended`: the session ended before the step's result came back),
231
+ `StepResultTooLargeError` (`step_result_too_large`), `UnsupportedFrameEncodingError`, and
232
+ `MessageTooLargeError` when the request is too large to send. The frames pushed for a step
233
+ belong to it: when its request cannot be sent, the step rejects and its frames are dropped
234
+ with it, not kept for the next step.
235
+
108
236
  ## Recording and clips
109
237
 
110
238
  ```ts
111
239
  const clip = await reactor.requestClip(10); // last 10 seconds
112
- await reactor.downloadClipAsFile(clip, "clip.mp4", { jwt: await fetchToken() });
240
+ await reactor.downloadClipAsFile(clip, "clip.mp4", { jwt: reactor.getJwtResolver() });
113
241
  ```
114
242
 
243
+ `jwt` takes a token or a resolver. A resolver is called before every poll of the manifest with
244
+ `{ sessionId: clip.sessionId }`, so a clip that takes a while to become ready still downloads
245
+ after the token that created the session has expired. The same goes for `getJwt` on
246
+ `<ClipPlayer>`, `<ClipDownloadButton>` and `useClipDownload`.
247
+
115
248
  `requestClip()`/`requestRecording()`/`downloadClipAsFile()` live directly on `Reactor`, and the
116
249
  same three are bound on the React store. For a preview or download button instead, drop in
117
250
  `<ClipPlayer>`/`<ClipDownloadButton>`:
@@ -119,8 +252,8 @@ same three are bound on the React store. For a preview or download button instea
119
252
  ```tsx
120
253
  import { ClipPlayer, ClipDownloadButton } from "@reactor-team/js-sdk";
121
254
 
122
- <ClipPlayer clip={clip} getJwt={() => fetchToken()} />
123
- <ClipDownloadButton clip={clip} getJwt={() => fetchToken()} filename="clip.mp4" />
255
+ <ClipPlayer clip={clip} getJwt={fetchToken} />
256
+ <ClipDownloadButton clip={clip} getJwt={fetchToken} filename="clip.mp4" />
124
257
  ```
125
258
 
126
259
  Full API, playback details, and the `useClipDownload` hook: [Recordings](https://docs.reactor.inc/concepts/recordings).
@@ -160,16 +293,13 @@ try {
160
293
  ## React
161
294
 
162
295
  ```tsx
163
- import { useCallback } from "react";
164
296
  import { ReactorProvider, ReactorView, useReactor } from "@reactor-team/js-sdk";
165
297
 
166
298
  function App() {
167
- // Stable across renders — see the note on ReactorProvider below for why
168
- // this matters.
169
- const jwt = useCallback(() => fetchToken(), []);
170
-
299
+ // A module-level fetchToken is stable across renders — see the note on
300
+ // ReactorProvider below for why this matters.
171
301
  return (
172
- <ReactorProvider modelName="my-model" jwtToken={jwt}>
302
+ <ReactorProvider modelName="my-model" jwtToken={fetchToken}>
173
303
  <Status />
174
304
  </ReactorProvider>
175
305
  );
@@ -225,8 +355,9 @@ you want to stay model-agnostic. See [Typed model SDKs](https://docs.reactor.inc
225
355
  ```bash
226
356
  mise run install:js # npm install
227
357
  mise run lint:js # eslint + tsc --noEmit
228
- mise run build:js # tsup -> dist/, then copies reactor-wasm's pkg/ into dist/wasm
358
+ mise run build:js # tsup -> dist/, then copies reactor-wasm's pkg/ into dist/wasm, binary inlined
229
359
  mise run test:js # vitest — pure-JS unit tests against a mocked ReactorClient, no wasm build needed
360
+ mise run test:js:consumer-tests # the packed SDK in real Vite/Next.js apps, in a real browser — see consumer-tests/README.md
230
361
  ```
231
362
 
232
363
  `build:js` fails fast with a clear message if `crates/reactor-wasm/pkg`