realtime-avatar 0.35.0 → 0.37.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +60 -0
- package/dist/{chunk-ARJEAQQT.js → chunk-FFMZERDZ.js} +1 -1
- package/dist/{chunk-WHYDSORD.js → chunk-RBK6EHVH.js} +784 -227
- package/dist/express.d.ts +2 -2
- package/dist/express.js +2 -2
- package/dist/hono.d.ts +2 -2
- package/dist/hono.js +2 -2
- package/dist/index.d.ts +18 -6
- package/dist/index.js +1 -1
- package/dist/nextjs.d.ts +2 -2
- package/dist/nextjs.js +2 -2
- package/dist/server.d.ts +1 -1
- package/dist/server.js +1 -1
- package/dist/tanstack-start.d.ts +2 -2
- package/dist/tanstack-start.js +2 -2
- package/dist/{types-CpnLuiJU.d.ts → types-DVL5VX3N.d.ts} +365 -63
- package/dist/{types-CUvcV0Wq.d.ts → types-RQo4LwZ3.d.ts} +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -204,6 +204,61 @@ Two facts follow from that picture and drive everything else:
|
|
|
204
204
|
|
|
205
205
|
---
|
|
206
206
|
|
|
207
|
+
## Speech without a live call
|
|
208
|
+
|
|
209
|
+
```ts
|
|
210
|
+
const speech = await rta.createSpeech(voiceId, { text: "Hello!", format: "wav" }, { signal });
|
|
211
|
+
return new Response(speech.bytes, { headers: { "content-type": speech.contentType } });
|
|
212
|
+
```
|
|
213
|
+
|
|
214
|
+
Available only from the server entries, `createSpeech` wraps the existing HTTP speech endpoint.
|
|
215
|
+
It returns completed audio, **not streaming audio**, and uses **no LiveKit** or live call.
|
|
216
|
+
It does not listen or provide full-duplex conversation; live calls remain full duplex.
|
|
217
|
+
|
|
218
|
+
The exported `createSpeechRequestSchema` owns `CreateSpeechInput`: text is trimmed and must
|
|
219
|
+
contain 1–2,000 characters; `format` defaults to `mp3` (or choose `wav`); optional `speed` is
|
|
220
|
+
0.5–2 and `language` is a BCP-47 tag. Unknown fields reject. Use a real workspace-accessible
|
|
221
|
+
`vce_…` or `voice_…` ID. `SpeechResult` contains `Uint8Array<ArrayBuffer>` bytes, validated
|
|
222
|
+
`audio/mpeg` or `audio/wav` content type, and nullable `billedBytes`, `billedCredits`, `requestId`.
|
|
223
|
+
Missing metadata stays null; invalid metadata, mismatched media, empty audio or more than
|
|
224
|
+
24 MiB rejects. Container signatures are checked, not the full audio codec.
|
|
225
|
+
|
|
226
|
+
Existing RTA authorization and billing apply: `speech:write`, a paid workspace or purchased
|
|
227
|
+
credits, and an accessible voice. Billing counts UTF-8 bytes of trimmed text (one credit per
|
|
228
|
+
90 bytes, rounded up, minimum one). The endpoint stores neither text nor audio; callers may
|
|
229
|
+
save and replay bytes. Limits are 20 requests per minute and one in progress per workspace.
|
|
230
|
+
|
|
231
|
+
**No automatic retries**, regardless of client `maxRetries`: the endpoint is not idempotent,
|
|
232
|
+
and an ambiguous failure may already be billed. `signal`, `timeoutMs` and `totalTimeoutMs`
|
|
233
|
+
cover the request and binary body; an abort does not guarantee that upstream billing stopped.
|
|
234
|
+
Caller aborts propagate, timeout aborts are `TimeoutError`, and HTTP errors use the existing
|
|
235
|
+
`RealtimeAvatarHttpError` class.
|
|
236
|
+
|
|
237
|
+
### Streaming speech (unreleased)
|
|
238
|
+
|
|
239
|
+
`createSpeechStream(voiceId, { text, speed?, language? }, { signal? })` returns an async generator
|
|
240
|
+
of `SpeechStreamEvent`, starting the paid request on first iteration. It is server-only, uses
|
|
241
|
+
no LiveKit, and leaves `createSpeech` unchanged. Requires platform streaming support; not yet
|
|
242
|
+
published. The root/server exports include canonical generated `createSpeechStreamRequestSchema`,
|
|
243
|
+
`speechStreamEventSchema`, `speechAlignedTextSchema`, `CreateSpeechStreamRequest`,
|
|
244
|
+
`SpeechStreamEvent` and `SpeechAlignedText`. There is no streaming `format` option.
|
|
245
|
+
|
|
246
|
+
Consume incrementally with `for await`. `started` declares signed 16-bit little-endian PCM,
|
|
247
|
+
24 kHz mono. `audio` carries base64 PCM, contiguous zero-based `sequence` and mono-sample
|
|
248
|
+
`sampleOffset`; chunks contain whole samples and at most 16 KiB, with a 180-second total cap.
|
|
249
|
+
`alignment` carries absolute-second text segments; replace earlier segments with the same
|
|
250
|
+
`chunkSeq`, without inferring current-word timing. All events share one UUID `requestId`.
|
|
251
|
+
Exactly one terminal follows: `completed` confirms matching `totalSamples`, `billedCredits`
|
|
252
|
+
and `billedBytes`; `failed` yields a typed canonical API error even after partial audio.
|
|
253
|
+
Read through EOF to reject trailing events. Malformed or truncated streams throw
|
|
254
|
+
`RealtimeAvatarError`, while HTTP refusals retain `RealtimeAvatarHttpError`.
|
|
255
|
+
|
|
256
|
+
The SDK never collects all audio and bounds receive work to 24 MiB total, 256 KiB per event,
|
|
257
|
+
and 64 KiB for HTTP error text. `break`, iterator `return()` and caller abort cancel the reader
|
|
258
|
+
and fetch, including a pending `next()`. The smaller of `timeoutMs` and `totalTimeoutMs`
|
|
259
|
+
covers the entire request/body, including pauses between iterations. **Never automatically
|
|
260
|
+
retry** an ambiguous paid request. Cancellation does not prove that billing stopped.
|
|
261
|
+
|
|
207
262
|
## Custom voice (webhook)
|
|
208
263
|
|
|
209
264
|
Speak a call in your own voice. Each sentence she is about to say is POSTed, signed, to an
|
|
@@ -423,6 +478,11 @@ rta.getRecording(recordingId)
|
|
|
423
478
|
rta.getRecordingAccess(recordingId)
|
|
424
479
|
rta.getConnectionHistory(sessionId) // bounded LiveKit observations; requires usage:read
|
|
425
480
|
|
|
481
|
+
// completed speech audio; server only, no LiveKit, never retried; requires speech:write
|
|
482
|
+
rta.createSpeech(voiceId, { text, format?, speed?, language? }, { signal? })
|
|
483
|
+
// unreleased streaming candidate; generated SSE events, never retried
|
|
484
|
+
rta.createSpeechStream(voiceId, { text, speed?, language? }, { signal? })
|
|
485
|
+
|
|
426
486
|
// pre-rendered video (MP4, no live call); server only, requires renders:write / renders:read
|
|
427
487
|
rta.createRender({ avatarId, voiceId? | voice?, lipGain?, shots }) // 202 queued; one Idempotency-Key across retries
|
|
428
488
|
rta.waitForRender(renderId) // block until ready | failed; RETURNS a failed render, throws only on timeout
|
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import { RealtimeAvatar, ROUTE_TIMEOUT_MS, proxyEndRequestSchema, proxyConnectRequestSchema, isQueued, RealtimeAvatarHttpError, RealtimeAvatarError } from './chunk-
|
|
1
|
+
import { RealtimeAvatar, ROUTE_TIMEOUT_MS, proxyEndRequestSchema, proxyConnectRequestSchema, isQueued, RealtimeAvatarHttpError, RealtimeAvatarError } from './chunk-RBK6EHVH.js';
|
|
2
2
|
|
|
3
3
|
// ../http-client/src/generated/error-semantics.ts
|
|
4
4
|
var ERROR_SEMANTICS = {
|