realtime-avatar 0.36.0 → 0.38.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +36 -6
- package/dist/{chunk-YHNBW3X6.js → chunk-7NZ7U5HN.js} +628 -184
- package/dist/{chunk-EVATXOT2.js → chunk-T4WKWENM.js} +1 -1
- package/dist/express.d.ts +2 -2
- package/dist/express.js +2 -2
- package/dist/hono.d.ts +2 -2
- package/dist/hono.js +2 -2
- package/dist/index.d.ts +17 -8
- package/dist/index.js +1 -1
- package/dist/nextjs.d.ts +2 -2
- package/dist/nextjs.js +2 -2
- package/dist/server.d.ts +1 -1
- package/dist/server.js +1 -1
- package/dist/tanstack-start.d.ts +2 -2
- package/dist/tanstack-start.js +2 -2
- package/dist/{types-aQOs22YE.d.ts → types-CYti2_bv.d.ts} +1 -1
- package/dist/{types-C_0GpIws.d.ts → types-niYtoW5A.d.ts} +300 -12
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -234,6 +234,31 @@ cover the request and binary body; an abort does not guarantee that upstream bil
|
|
|
234
234
|
Caller aborts propagate, timeout aborts are `TimeoutError`, and HTTP errors use the existing
|
|
235
235
|
`RealtimeAvatarHttpError` class.
|
|
236
236
|
|
|
237
|
+
### Streaming speech (unreleased)
|
|
238
|
+
|
|
239
|
+
`createSpeechStream(voiceId, { text, speed?, language? }, { signal? })` returns an async generator
|
|
240
|
+
of `SpeechStreamEvent`, starting the paid request on first iteration. It is server-only, uses
|
|
241
|
+
no LiveKit, and leaves `createSpeech` unchanged. Requires platform streaming support; not yet
|
|
242
|
+
published. The root/server exports include canonical generated `createSpeechStreamRequestSchema`,
|
|
243
|
+
`speechStreamEventSchema`, `speechAlignedTextSchema`, `CreateSpeechStreamRequest`,
|
|
244
|
+
`SpeechStreamEvent` and `SpeechAlignedText`. There is no streaming `format` option.
|
|
245
|
+
|
|
246
|
+
Consume incrementally with `for await`. `started` declares signed 16-bit little-endian PCM,
|
|
247
|
+
24 kHz mono. `audio` carries base64 PCM, contiguous zero-based `sequence` and mono-sample
|
|
248
|
+
`sampleOffset`; chunks contain whole samples and at most 16 KiB, with a 180-second total cap.
|
|
249
|
+
`alignment` carries absolute-second text segments; replace earlier segments with the same
|
|
250
|
+
`chunkSeq`, without inferring current-word timing. All events share one UUID `requestId`.
|
|
251
|
+
Exactly one terminal follows: `completed` confirms matching `totalSamples`, `billedCredits`
|
|
252
|
+
and `billedBytes`; `failed` yields a typed canonical API error even after partial audio.
|
|
253
|
+
Read through EOF to reject trailing events. Malformed or truncated streams throw
|
|
254
|
+
`RealtimeAvatarError`, while HTTP refusals retain `RealtimeAvatarHttpError`.
|
|
255
|
+
|
|
256
|
+
The SDK never collects all audio and bounds receive work to 24 MiB total, 256 KiB per event,
|
|
257
|
+
and 64 KiB for HTTP error text. `break`, iterator `return()` and caller abort cancel the reader
|
|
258
|
+
and fetch, including a pending `next()`. The smaller of `timeoutMs` and `totalTimeoutMs`
|
|
259
|
+
covers the entire request/body, including pauses between iterations. **Never automatically
|
|
260
|
+
retry** an ambiguous paid request. Cancellation does not prove that billing stopped.
|
|
261
|
+
|
|
237
262
|
## Custom voice (webhook)
|
|
238
263
|
|
|
239
264
|
Speak a call in your own voice. Each sentence she is about to say is POSTed, signed, to an
|
|
@@ -455,6 +480,8 @@ rta.getConnectionHistory(sessionId) // bounded LiveKit observations; requires
|
|
|
455
480
|
|
|
456
481
|
// completed speech audio; server only, no LiveKit, never retried; requires speech:write
|
|
457
482
|
rta.createSpeech(voiceId, { text, format?, speed?, language? }, { signal? })
|
|
483
|
+
// unreleased streaming candidate; generated SSE events, never retried
|
|
484
|
+
rta.createSpeechStream(voiceId, { text, speed?, language? }, { signal? })
|
|
458
485
|
|
|
459
486
|
// pre-rendered video (MP4, no live call); server only, requires renders:write / renders:read
|
|
460
487
|
rta.createRender({ avatarId, voiceId? | voice?, lipGain?, shots }) // 202 queued; one Idempotency-Key across retries
|
|
@@ -465,7 +492,7 @@ rta.iterateRenders({ limit? }) // every render, paging handled
|
|
|
465
492
|
rta.getRenderAccess(renderId) // { id, url, expiresAt }; signed for an hour, renew when needed
|
|
466
493
|
|
|
467
494
|
// avatars
|
|
468
|
-
rta.
|
|
495
|
+
rta.createAnimaAvatar({ displayName, voice, settings?, metadata? }) // app-owned rig and rendering, explicit voice, no assets
|
|
469
496
|
rta.createAvatarFromImage({ displayName, imageUrl, motionPrompt?, voice? }) // platform-rendered video
|
|
470
497
|
rta.createAvatarFromVideo({ displayName, videoUrl, voice? }) // DEPRECATED — closed, 422
|
|
471
498
|
rta.listAvatars()
|
|
@@ -517,10 +544,10 @@ const voice: VoiceSpecInput = {
|
|
|
517
544
|
};
|
|
518
545
|
```
|
|
519
546
|
|
|
520
|
-
### Register an app
|
|
547
|
+
### Register an Anima avatar (your app owns the rig and the rendering)
|
|
521
548
|
|
|
522
549
|
```ts
|
|
523
|
-
const avatar = await rta.
|
|
550
|
+
const avatar = await rta.createAnimaAvatar({
|
|
524
551
|
displayName: "Rin",
|
|
525
552
|
voice: { provider: "fish", voice_id: "your-provider-voice-id", model: "s2-pro" },
|
|
526
553
|
metadata: { characterId: "your-character-id" },
|
|
@@ -529,18 +556,21 @@ const avatar = await rta.createLive2dAvatar({
|
|
|
529
556
|
const call = await rta.startCall({ avatarId: avatar.id, mode: "voice" });
|
|
530
557
|
```
|
|
531
558
|
|
|
532
|
-
Requires the platform's
|
|
559
|
+
Requires the platform's `anima` renderer. One `POST /avatars` sends `renderer: "anima"`,
|
|
533
560
|
`sourceKind: "image"` and an explicit voice selector; no assets, automatic casting, clips or
|
|
534
|
-
resting-loop generation are requested. Your app owns the rig. Source fields, motion prompts,
|
|
561
|
+
resting-loop generation are requested. Your app owns the rig and the rendering. Source fields, motion prompts,
|
|
535
562
|
renderer overrides and automatic voice descriptions are refused before network I/O.
|
|
536
563
|
`settings` and `metadata` are optional records and are omitted unless provided.
|
|
537
564
|
|
|
538
|
-
The exported `
|
|
565
|
+
The exported `CreateAnimaAvatarInput` and complete `RegisteredAvatar` response derive from
|
|
539
566
|
canonical generated schemas. The response is parsed once; malformed successful responses throw
|
|
540
567
|
`RealtimeAvatarError` without the rejected body. Existing image/video creation requests are
|
|
541
568
|
unchanged. Existing avatar methods retain `renderer`, defaulting older responses that omit it to
|
|
542
569
|
`"video"`.
|
|
543
570
|
|
|
571
|
+
`renderer: "anima"` replaced `live2d` in 0.38.0 with no alias: the platform refuses `live2d` and
|
|
572
|
+
SDK 0.37.0 and earlier cannot parse an `anima` avatar, so upgrade once the platform serves `anima`.
|
|
573
|
+
|
|
544
574
|
### Validate persisted grants and proxy requests
|
|
545
575
|
|
|
546
576
|
The server root (`realtime-avatar` or `realtime-avatar/server`) exports the existing
|