@npcify/web 0.2.1 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -125,6 +125,9 @@ animate()
125
125
  - **Dialogue & Speech State**:
126
126
  - `character.speaking`, `character.thinking`, `character.currentSpeech`, `character.currentText`, `character.revealedText`, `character.options`, `character.cursor`, `character.speech`.
127
127
  - `await character.createConversation()`, `await character.sendMessage(text)`, `await character.endConversation()`.
128
+ - **Voice Input** ([details](#voice-input-and-input-modes)):
129
+ - `character.bindPushToTalk(options)`, `character.startListening(options)`, `character.stopListening()`, `character.cancelListening()`.
130
+ - `character.listeningState`, `character.onListeningState(callback)`, `character.onTranscript(callback)`, `character.inputModes`.
128
131
  - `await character.dispose()`: Ends conversation, detaches from the scene, removes the audio element, and disposes VRM GPU resources.
129
132
 
130
133
  ## Mode 2: Managed Canvas (`mountNpcify`)
@@ -188,13 +191,76 @@ with any ESM bundler after replacing their configuration placeholders. Keys in b
188
191
  code are visible to visitors. Use narrow scopes; keep keys in your backend when they
189
192
  must remain private. Calls made with a key are billed to its owner.
190
193
 
194
+ ## Voice input and input modes
195
+
196
+ The player can speak a reply when the character owner turned voice input on
197
+ (the STT tab of the character in the dashboard, where the owner also chooses
198
+ the speech recognition provider). The SDK records the microphone and streams
199
+ the audio to the Npcify server. At the release, the server recognises the
200
+ speech and starts the turn. The recognised text is the player's message, as a
201
+ typed reply.
202
+
203
+ ```js
204
+ // Hold the button, or Space, to talk. The release sends the speech.
205
+ const talk = document.querySelector('#talk')
206
+ talk.hidden = !character.inputModes.includes('voice')
207
+ const unbindTalk = character.bindPushToTalk({
208
+ element: talk,
209
+ onResult: ({ text }) => { if (!text) status.textContent = 'No speech was heard.' },
210
+ onShortPress: () => { status.textContent = 'Hold to talk.' },
211
+ onError: (error) => { status.textContent = error.message },
212
+ })
213
+ character.onListeningState((state) => {
214
+ talk.textContent =
215
+ { listening: 'Listening…', recognizing: 'Recognizing…' }[state] ?? 'Hold to talk'
216
+ })
217
+ ```
218
+
219
+ - `bindPushToTalk({ element, key, minHoldMs, listening, enabled, onResult, onShortPress, onError })`:
220
+ press and hold an element, a key (`KeyboardEvent.code`, default `'Space'`;
221
+ `null` binds no key), or both. The key does not start while the focus is in
222
+ a field or on another control, and key repeat is ignored. A press shorter
223
+ than `minHoldMs` (300 ms), or a release before the microphone gave any
224
+ audio, sends nothing and calls `onShortPress`. Escape, window blur and a
225
+ hidden tab cancel. A long press on the element does not open a menu, select
226
+ text or scroll, and a release outside the element ends the press. Returns
227
+ an unbind function.
228
+ - `startListening({ language, send, maxDurationMs, endOnSilenceMs })` starts
229
+ listening from code, for example when the character stops speaking.
230
+ `endOnSilenceMs` stops after that much silence that follows speech, and
231
+ `maxDurationMs` (30 s at most) always stops. `stopListening()` resolves with
232
+ `{ text, language, durationMs, sent, turn, send() }`, or with null when the
233
+ microphone gave no audio yet. `cancelListening()` discards the recording;
234
+ the owner pays only the audio that already went to a streaming provider.
235
+ - `onTranscript(({ text, final }) => …)` gives the final text, and partial
236
+ text when the server's speech model gives it.
237
+ - The first press or `startListening()` asks for the microphone permission.
238
+ An empty result sends nothing.
239
+ - `language` is an optional BCP 47 hint (`'zh'`, `'ja'`, `'en'`). Without it,
240
+ the server detects the language.
241
+ - `inputModes` (an option of `mountNpcify`, `loadCharacter` and `Npcify`) is
242
+ an array of `'options'`, `'text'` and `'voice'`; one mode forces it. Without
243
+ it, every mode that the character permits is available. A baked
244
+ conversation permits only `['options']`, a character without custom
245
+ replies permits only `'options'`, and `'voice'` needs the character's voice
246
+ input (`speechInputEnabled` in the character info). Another configuration
247
+ throws an `InputModeError` when the character loads, and listening on a
248
+ character with voice input off throws one too. With `inputModes` set,
249
+ `sendMessage` with typed text needs `'text'`, a reply option needs
250
+ `'options'` or `'text'`, and listening needs `'voice'`.
251
+ `character.inputModes` gives the modes in use.
252
+ - The character owner pays the recognition at the price of the provider that
253
+ the owner chose, as for the turn's language model.
254
+
191
255
  ## Runtime and lifecycle
192
256
 
193
257
  - `mountNpcify({ canvas, baseUrl, characterId, apiKey, serverRendering })` returns
194
- `{ character, npcify, scene, camera, renderer, controls, dispose }` after loading.
195
- Give the canvas a nonzero CSS size.
258
+ `{ character, npcify, scene, camera, renderer, controls, dispose }` after loading,
259
+ and normalizes the idle animation orientation to face `+Z` by default
260
+ (`normalizeIdleOrientation: true`). Give the canvas a nonzero CSS size.
196
261
  - `loadCharacter({ baseUrl, characterId, apiKey, parent, gazeTarget, shadows })`
197
- loads and returns a `Character` instance for custom Three.js scenes and render loops.
262
+ loads and returns a `Character` instance for custom Three.js scenes and render loops
263
+ (`normalizeIdleOrientation` defaults to `false`).
198
264
  - Call `unlockAudio()` (or `character.audio.unlock()`) during a user gesture before
199
265
  starting playback.
200
266
  - `onOptionsReady` delivers choices after playback finishes. `sendMessage(label)`
package/dist/index.d.ts CHANGED
@@ -39,9 +39,15 @@ interface AvatarPublicInfo {
39
39
  freeFirstNMessages?: number | null;
40
40
  hostedCharacterEnabled?: boolean;
41
41
  listedPublicly?: boolean;
42
+ r18?: boolean;
42
43
  bakedConversationEnabled?: boolean;
43
44
  bakedConversationRevision?: number;
44
45
  allowCustomReply?: boolean;
46
+ /**
47
+ * The owner turned voice input on and chose a provider. Voice also needs
48
+ * `allowCustomReply`. Missing on older servers: off.
49
+ */
50
+ speechInputEnabled?: boolean;
45
51
  autoPlay: boolean;
46
52
  serverRenderingEnabled: boolean;
47
53
  cameraSqt?: Sqt | null;
@@ -280,6 +286,18 @@ interface StagingStreamBillingEvent {
280
286
  }
281
287
  type StagingStreamEvent = StagingStreamStartEvent | StagingStreamTextDeltaEvent | StagingStreamTimelineEvent | StagingStreamPrefixEvent | StagingStreamLiveEvent | StagingStreamSpeechChunkEvent | StagingStreamSpeechCompleteEvent | StagingStreamSpeechErrorEvent | StagingStreamOptionsEvent | StagingStreamErrorEvent | StagingStreamBillingEvent;
282
288
 
289
+ /**
290
+ * Voice input of conversations (docs/voice-input.md): the input modes of the
291
+ * SDKs, and the messages of a speech session between an SDK and the server.
292
+ */
293
+ /** How a player can reply: a reply option, typed text, or speech. */
294
+ declare const INPUT_MODES: readonly ["options", "text", "voice"];
295
+ type InputMode = (typeof INPUT_MODES)[number];
296
+ /** A configuration of input modes that is invalid or that the character does not permit. */
297
+ declare class InputModeError extends Error {
298
+ constructor(message: string);
299
+ }
300
+
283
301
  declare const BAKED_CONVERSATION_VERSION: 1;
284
302
  /** Portable, language-neutral asset. Audio is embedded binary, never a signed URL. */
285
303
  interface BakedConversationAsset {
@@ -319,9 +337,11 @@ declare const decodePerformance: <T = unknown>(input: ArrayBuffer | Uint8Array,
319
337
  interface ConversationTurnOptions {
320
338
  silent?: boolean;
321
339
  parentStagingId?: string | null;
340
+ /** The turn's `Idempotency-Key`; a new UUID when absent. */
341
+ requestId?: string;
322
342
  }
323
343
 
324
- type NpcifyAssetInfo = Pick<AvatarPublicInfo, 'animations' | 'idleAnimationId' | 'bakedConversationEnabled' | 'bakedConversationRevision'>;
344
+ type NpcifyAssetInfo = Pick<AvatarPublicInfo, 'animations' | 'idleAnimationId' | 'bakedConversationEnabled' | 'bakedConversationRevision'> & Partial<Pick<AvatarPublicInfo, 'allowCustomReply' | 'speechInputEnabled'>>;
325
345
 
326
346
  interface LlmThinkingPayload {
327
347
  taskId?: string | null;
@@ -586,6 +606,13 @@ interface AnimationConfig {
586
606
  loop?: boolean;
587
607
  weight?: number;
588
608
  }
609
+ interface AnimationControllerOptions {
610
+ /**
611
+ * Normalize the configured idle animation clip so its average torso heading
612
+ * and neck/head orientation face canonical `+Z`.
613
+ */
614
+ normalizeIdleOrientation?: boolean;
615
+ }
589
616
  interface CharacterAnimationPlayOptions {
590
617
  fadeDuration?: number;
591
618
  loop?: boolean;
@@ -618,14 +645,17 @@ declare class AnimationController implements AnimationControllerLike {
618
645
  prepareAnimations?: (ids: string[]) => Promise<void>;
619
646
  private readonly mixer;
620
647
  private readonly clips;
648
+ private readonly rawClips;
649
+ private readonly normalizedIdleClips;
621
650
  private readonly configs;
651
+ private readonly normalizeIdleOrientation;
622
652
  private previewMixer;
623
653
  private restorePreview;
624
654
  private activeAction;
625
655
  private activeAnimationId;
626
656
  private idleAnimationId;
627
657
  private pendingReturnToIdleAction;
628
- constructor(vrm: VRM, idleAnimationId?: string | null);
658
+ constructor(vrm: VRM, idleAnimationId?: string | null, options?: AnimationControllerOptions);
629
659
  getAnimationMixer(): AnimationMixer;
630
660
  getActiveAction(): AnimationAction | null;
631
661
  getActiveAnimationId(): string | null;
@@ -636,6 +666,7 @@ declare class AnimationController implements AnimationControllerLike {
636
666
  previewAnimation(animation: VRMAnimation): () => void;
637
667
  setIdleAnimation(animationId: string | null): void;
638
668
  clearAnimations(): void;
669
+ private resolveClipForId;
639
670
  playAnimation(animationId: string, fadeDuration?: number, loopOverride?: boolean): void;
640
671
  playIdle(fadeDuration?: number): void;
641
672
  stopAnimation(fadeDuration?: number): void;
@@ -744,6 +775,7 @@ declare const FacialPerformanceLayer: {
744
775
  readonly Viseme: "viseme";
745
776
  readonly Blink: "blink";
746
777
  readonly Gaze: "gaze";
778
+ readonly Overlay: "overlay";
747
779
  };
748
780
  type FacialPerformanceLayerId = (typeof FacialPerformanceLayer)[keyof typeof FacialPerformanceLayer];
749
781
  type FacialMouthMode = 'expression' | 'viseme';
@@ -799,6 +831,9 @@ declare class FacialPerformanceMixer {
799
831
  playCue(channel: FacialChannelName, options?: FacialCueOptions): void;
800
832
  setEmotion(channel: FacialChannelName, weight: number, mouthMode: FacialMouthMode): void;
801
833
  clearEmotion(mouthMode?: FacialMouthMode): void;
834
+ /** Blend temporary emotions without changing timeline targets or mouth ownership. */
835
+ setOverlay(weights: Readonly<Record<string, number>>): void;
836
+ clearOverlay(): void;
802
837
  releaseVisemeMouthMode(): void;
803
838
  /**
804
839
  * Ends the speech source without disturbing any other performance layer.
@@ -948,6 +983,8 @@ declare class StagingPlayer {
948
983
  private readonly audioPlayer;
949
984
  private readonly facialMixer;
950
985
  private readonly animationController;
986
+ private readonly clock;
987
+ private pauseRevision;
951
988
  private playbackController;
952
989
  private generation;
953
990
  private incremental;
@@ -958,6 +995,10 @@ declare class StagingPlayer {
958
995
  private activeExpressionMouthMode;
959
996
  private activeAnimation;
960
997
  constructor(audioPlayer: AudioPlayer, facialMixer: FacialPerformanceMixer, animationController: AnimationControllerLike);
998
+ get paused(): boolean;
999
+ pause(): void;
1000
+ resume(): Promise<void>;
1001
+ private waitUntilResumed;
961
1002
  play(timeline: StagingTimeline, options?: StagingPlaybackOptions): Promise<void>;
962
1003
  updateTimeline(timeline: StagingTimeline, complete?: boolean): void;
963
1004
  private startCursor;
@@ -1103,6 +1144,22 @@ interface NpcifyOptions {
1103
1144
  volume?: number;
1104
1145
  /** Initial speech audio muted flag. */
1105
1146
  muted?: boolean;
1147
+ /**
1148
+ * Normalize the character's idle animation orientation so the torso and head
1149
+ * face canonical `+Z` on average while preserving idle motion.
1150
+ * Defaults to `true` for `Npcify` (Web Chats and Canvas API) and `false` for
1151
+ * `Character` (`loadCharacter` / `createCharacter` Three.js game mode).
1152
+ */
1153
+ normalizeIdleOrientation?: boolean;
1154
+ /**
1155
+ * How the player can reply: `'options'`, `'text'` and `'voice'`, in any
1156
+ * combination; one mode forces it. Without it, every mode that the
1157
+ * character permits. A baked conversation permits only `['options']`; a
1158
+ * character without custom replies permits only `options`; `voice` needs
1159
+ * the character's voice input (`speechInputEnabled`, the STT tab). A mode
1160
+ * that the character does not permit throws an `InputModeError` on load.
1161
+ */
1162
+ inputModes?: InputMode[];
1106
1163
  /** Optional error callback. */
1107
1164
  onError?: (error: Error) => void;
1108
1165
  }
@@ -1169,7 +1226,8 @@ declare class StreamPlaybackController {
1169
1226
  force?: boolean;
1170
1227
  timeline?: StagingTimeline;
1171
1228
  }): Promise<boolean>;
1172
- handleEvent(event: StagingStreamEvent): void;
1229
+ /** Returns the playback result for a timeline event. */
1230
+ handleEvent(event: StagingStreamEvent): Promise<boolean> | undefined;
1173
1231
  cleanup(): void;
1174
1232
  private resetRealtimeSpeech;
1175
1233
  private playTimeline;
@@ -1210,6 +1268,10 @@ declare class CharacterExpressions {
1210
1268
  setEmotion(name: FacialChannelName, weight?: number, mouthMode?: FacialMouthMode): void;
1211
1269
  /** Clear the active emotion expression layer. */
1212
1270
  clearEmotion(mouthMode?: FacialMouthMode): void;
1271
+ /** Set a temporary emotion overlay. Timeline emotions and visemes keep their state. */
1272
+ setOverlay(weights: Readonly<Record<string, number>>): void;
1273
+ /** Ease the temporary overlay back to the current timeline or idle face. */
1274
+ clearOverlay(): void;
1213
1275
  /** Reset all expression layer targets to zero. */
1214
1276
  reset(): void;
1215
1277
  }
@@ -1313,6 +1375,12 @@ declare class CharacterRuntime {
1313
1375
  /** Stop the currently playing skeletal animation. */
1314
1376
  stopAnimation(fadeDuration?: number): void;
1315
1377
  unlockAudio(): Promise<boolean>;
1378
+ /** Freeze the current NPCP timeline, audio, face, and animation. */
1379
+ pausePlayback(): void;
1380
+ /** Continue the same timeline. Call from a gesture if the browser requires it. */
1381
+ resumePlayback(): Promise<void>;
1382
+ /** Cancel current output and pending audio/animation loads. */
1383
+ interruptPlayback(): void;
1316
1384
  getExpressions(): string[];
1317
1385
  getMouthExpressionNames(): string[];
1318
1386
  getExpressionNames(): string[];
@@ -1337,6 +1405,8 @@ declare class CharacterRuntime {
1337
1405
  lookAt(x: number, y: number, z: number): void;
1338
1406
  /** Reset the character's gaze target so the eyes look straight ahead. */
1339
1407
  clearLookAt(): void;
1408
+ /** Briefly look at an object; release restores the prior target and gaze mode. */
1409
+ lookAtTemporarily(target: Object3D): () => void;
1340
1410
  startPerformanceRecording(options?: PerformanceRecordingStartOptions): void;
1341
1411
  stopPerformanceRecording(): PerformanceRecording | null;
1342
1412
  getPerformanceRecording(): PerformanceRecording | null;
@@ -1366,6 +1436,97 @@ declare class CharacterRuntime {
1366
1436
  protected readonly reportError: (error: Error, context?: string) => void;
1367
1437
  }
1368
1438
 
1439
+ declare class MicrophoneError extends Error {
1440
+ readonly code: 'MICROPHONE_PERMISSION_DENIED' | 'MICROPHONE_UNAVAILABLE';
1441
+ constructor(code: 'MICROPHONE_PERMISSION_DENIED' | 'MICROPHONE_UNAVAILABLE', message: string);
1442
+ }
1443
+ /** One frame of 16 kHz mono PCM and its loudness (RMS, 0 to 1). */
1444
+ type CaptureFrame = {
1445
+ pcm: Int16Array;
1446
+ level: number;
1447
+ };
1448
+ /** A source of 16 kHz frames. The SDK uses the microphone; tests use a fake. */
1449
+ interface AudioCapture {
1450
+ start(onFrame: (frame: CaptureFrame) => void): Promise<void>;
1451
+ /** Ends the capture; the last partial frame arrives before this resolves. */
1452
+ stop(): Promise<void>;
1453
+ /** Ends the capture without its last frame. */
1454
+ cancel(): void;
1455
+ }
1456
+
1457
+ type ListeningState = 'idle' | 'starting' | 'listening' | 'recognizing';
1458
+ interface ListeningOptions {
1459
+ /** BCP 47 hint, for example `ja`, `zh` or `en`. Without it the language is detected. */
1460
+ language?: string | null;
1461
+ /** `false`: recognise only; send the text later with `result.send()`. Default `true`. */
1462
+ send?: boolean;
1463
+ /** The recording stops at this time. Default and maximum: 30 seconds. */
1464
+ maxDurationMs?: number;
1465
+ /** For listening without a release: stop after this much silence that follows speech. */
1466
+ endOnSilenceMs?: number | null;
1467
+ parentStagingId?: string | null;
1468
+ silent?: boolean;
1469
+ }
1470
+ interface SpeechInputResult {
1471
+ /** The recognised text; empty when no speech was recognised (then no turn starts). */
1472
+ text: string;
1473
+ /** The detected language, when the recognition vendor gives it. */
1474
+ language: string | null;
1475
+ durationMs: number;
1476
+ /** The server started the turn with this text. */
1477
+ sent: boolean;
1478
+ /** The final payload of the turn, when it was sent. */
1479
+ turn: Promise<StagingResultPayload> | null;
1480
+ /** Sends the text as the player's message, when it was not sent. */
1481
+ send(): Promise<StagingResultPayload>;
1482
+ }
1483
+ interface TranscriptEvent {
1484
+ text: string;
1485
+ /** False for the text so far while the player speaks. */
1486
+ final: boolean;
1487
+ }
1488
+ declare class SpeechInputError extends Error {
1489
+ readonly code: string;
1490
+ constructor(code: string, message?: string);
1491
+ }
1492
+
1493
+ /** What push-to-talk drives: a `Character` or `Npcify` instance. */
1494
+ interface PushToTalkTarget {
1495
+ readonly listeningState: ListeningState;
1496
+ startListening(options?: ListeningOptions): Promise<SpeechInputResult | null>;
1497
+ stopListening(): Promise<SpeechInputResult | null>;
1498
+ cancelListening(): void;
1499
+ }
1500
+ interface PushToTalkOptions {
1501
+ /** A button or any element to press and hold. */
1502
+ element?: HTMLElement | null;
1503
+ /** A `KeyboardEvent.code` to hold. Default `'Space'`; `null` binds no key. */
1504
+ key?: string | null;
1505
+ /** A shorter press sends nothing. Default 300 ms. */
1506
+ minHoldMs?: number;
1507
+ /** The options of each listening, or a function that gives them at each press. */
1508
+ listening?: ListeningOptions | (() => ListeningOptions);
1509
+ /** False while the player cannot reply (for example while options are locked). */
1510
+ enabled?: () => boolean;
1511
+ onResult?: (result: SpeechInputResult) => void;
1512
+ /**
1513
+ * A press sent nothing: it was shorter than `minHoldMs`, or it ended before
1514
+ * the microphone gave any audio (a permission prompt, a slow device). A UI
1515
+ * can say "hold to talk".
1516
+ */
1517
+ onShortPress?: () => void;
1518
+ onError?: (error: unknown) => void;
1519
+ }
1520
+ /**
1521
+ * Press and hold to talk (docs/voice-input.md): an element, a key, or both.
1522
+ * The key does not start while the focus is in a field or on another
1523
+ * control, key repeat is ignored, a short press sends nothing, and window
1524
+ * blur, a hidden tab and Escape cancel. A long press on the element does not
1525
+ * open a menu, select text or scroll. A release anywhere ends the press.
1526
+ * Returns a function that unbinds.
1527
+ */
1528
+ declare const bindPushToTalk: (target: PushToTalkTarget, options?: PushToTalkOptions) => () => void;
1529
+
1369
1530
  declare class Character extends CharacterRuntime {
1370
1531
  private readonly conversationSocket;
1371
1532
  private renderSessionId;
@@ -1376,8 +1537,17 @@ declare class Character extends CharacterRuntime {
1376
1537
  private readonly liveConversation;
1377
1538
  private baked;
1378
1539
  private readonly authoring;
1540
+ protected readonly normalizeIdleOrientation: boolean;
1379
1541
  private readonly bakedAsset?;
1380
1542
  private animationRefreshId;
1543
+ private readonly voice;
1544
+ private readonly configuredInputModes;
1545
+ private resolvedInputModes;
1546
+ /** The character's voice input switch, after the character loads. */
1547
+ private speechInput;
1548
+ private latestOptions;
1549
+ private listeningStateCallback;
1550
+ private transcriptCallback;
1381
1551
  conversationId: string | null;
1382
1552
  constructor(options: NpcifyOptions);
1383
1553
  load(callback?: (info: AssetLoadProgress) => void): Promise<this>;
@@ -1399,18 +1569,57 @@ declare class Character extends CharacterRuntime {
1399
1569
  replayStaging(stagingId: string): Promise<StagingResultPayload>;
1400
1570
  replayStaging(conversationId: string, stagingId: string, options?: PlayStagingOptions): Promise<StagingResultPayload>;
1401
1571
  get isBakedConversation(): boolean;
1572
+ /**
1573
+ * The input modes in use: the configured modes, or every mode that the
1574
+ * character permits. Before `loadAssets`, the configuration or all modes.
1575
+ */
1576
+ get inputModes(): readonly InputMode[];
1577
+ get listeningState(): ListeningState;
1578
+ /**
1579
+ * Opens the microphone and streams the speech to the server. Resolves with
1580
+ * the result when listening ends (`stopListening`, `endOnSilenceMs` or
1581
+ * `maxDurationMs`), or with null when it is cancelled. The first call asks
1582
+ * for the microphone permission.
1583
+ */
1584
+ startListening(options?: ListeningOptions): Promise<SpeechInputResult | null>;
1585
+ /**
1586
+ * Ends the recording; resolves as soon as the server recognised the text.
1587
+ * Before the microphone gave any audio, it cancels and resolves null.
1588
+ */
1589
+ stopListening(): Promise<SpeechInputResult | null>;
1590
+ /** Discards the recording. Nothing is recognised or charged. */
1591
+ cancelListening(): void;
1592
+ /** Press and hold an element or a key (default Space) to talk. Returns an unbind function. */
1593
+ bindPushToTalk(options?: PushToTalkOptions): () => void;
1594
+ onListeningState(callback: (state: ListeningState) => void): () => void;
1595
+ /** Partial text while the player speaks (`final: false`), then the final text. */
1596
+ onTranscript(callback: (event: TranscriptEvent) => void): () => void;
1402
1597
  getConversationWorldline(conversationId?: string | null): Promise<ConversationWorldline>;
1403
1598
  acceptAuthoringResult(payload: StagingResultPayload): Promise<void>;
1404
1599
  playStaging(payload: StagingResultPayload, options?: PlayStagingOptions): Promise<boolean>;
1600
+ /**
1601
+ * Plays one live staging event from another transport, such as an actor
1602
+ * stage. A timeline event returns its playback result.
1603
+ */
1604
+ playStreamEvent(event: StagingStreamEvent): Promise<boolean> | undefined;
1405
1605
  setRenderSessionId(renderSessionId: string | null): void;
1406
1606
  onStagingResult(callback: (payload: StagingResultPayload) => Promise<void> | void): () => void;
1407
1607
  onThinking(callback: (payload: LlmThinkingPayload) => Promise<void> | void): () => void;
1408
1608
  private resetAssets;
1409
1609
  private readonly getAssetInfo;
1410
1610
  private handleStagingResult;
1611
+ /** Throws when the input modes do not include `mode`. */
1612
+ private assertInputMode;
1613
+ /**
1614
+ * A typed reply needs the `text` mode; one of the current reply options
1615
+ * needs `options` or `text`. Without a configuration the server's rules
1616
+ * apply alone, as before.
1617
+ */
1618
+ private assertReplyMode;
1411
1619
  private invokeThinkingCallback;
1412
1620
  }
1413
1621
  declare class Npcify extends Character {
1622
+ constructor(options: NpcifyOptions);
1414
1623
  }
1415
1624
 
1416
1625
  interface LoadCharacterOptions extends CharacterOptions {
@@ -1501,6 +1710,9 @@ interface MountNpcifyOptions {
1501
1710
  apiToken?: string;
1502
1711
  serverRendering?: boolean;
1503
1712
  assetProfile?: 'original' | 'web-optimized-v1';
1713
+ normalizeIdleOrientation?: boolean;
1714
+ /** How the player can reply: `'options'`, `'text'`, `'voice'` (docs/voice-input.md). */
1715
+ inputModes?: InputMode[];
1504
1716
  onError?: (error: Error) => void;
1505
1717
  }
1506
1718
  interface NpcifyView {
@@ -1515,4 +1727,4 @@ interface NpcifyView {
1515
1727
  /** Mount into an existing canvas. Its CSS must give it a nonzero width and height. */
1516
1728
  declare const mountNpcify: (options: MountNpcifyOptions) => Promise<NpcifyView>;
1517
1729
 
1518
- export { AnimationController, type AnimationControllerLike, type AssetLoadProgress, type AudioPerformanceFrameSnapshot, AudioPlayer, BakedConversation, type BakedConversationAsset, Character, CharacterAnimation, type CharacterAnimationPlayOptions, CharacterAudio, type CharacterDisposeOptions, CharacterExpressions, type CharacterLookAtTarget, type CharacterOptions, type CharacterPlaybackCursor, CharacterRuntime, type CharacterRuntimeInitOptions, CharacterSceneRig, type CharacterShadowOptions, type CharacterSpeechState, type CharacterTransformOptions, type ConversationTurnOptions, type FacialChannelName, type FacialCueOptions, type FacialMouthMode, type FacialPerformanceFrameSnapshot, FacialPerformanceLayer, type FacialPerformanceLayerId, FacialPerformanceMixer, type FacialPerformanceModelSnapshot, type LlmThinkingPayload, type LoadCharacterOptions, type MountNpcifyOptions, Npcify, type NpcifyAssetInfo, Character as NpcifyCharacter, type NpcifyOptions, type NpcifyView, PERFORMANCE_FILE_EXTENSION, PERFORMANCE_MEDIA_TYPE, type PerformanceAnalysis, type PerformanceAnalysisOptions, type PerformanceEvent, type PerformanceFrame, type PerformanceIssue, type PerformanceKind, type PerformanceRecording, PerformanceRecordingSchemaVersion, type PerformanceRecordingStartOptions, type PerformanceTimelineSnapshot, type PlayStagingOptions, RemoteAnimationController, type RepeatedVisemeTransitionAnalysis, type StagingOptionsReadyPayload, type StagingPlaybackStart, type StagingSpeechPlaybackEvent, type Vector3Like, type Vector3Tuple, type VisemePerformanceFrameSnapshot, analyzePerformanceRecording, createCharacter, decodePerformance, encodePerformance, loadBakedConversation, loadCharacter, mountNpcify };
1730
+ export { AnimationController, type AnimationControllerLike, type AnimationControllerOptions, type AssetLoadProgress, type AudioCapture, type AudioPerformanceFrameSnapshot, AudioPlayer, BakedConversation, type BakedConversationAsset, type CaptureFrame, Character, CharacterAnimation, type CharacterAnimationPlayOptions, CharacterAudio, type CharacterDisposeOptions, CharacterExpressions, type CharacterLookAtTarget, type CharacterOptions, type CharacterPlaybackCursor, CharacterRuntime, type CharacterRuntimeInitOptions, CharacterSceneRig, type CharacterShadowOptions, type CharacterSpeechState, type CharacterTransformOptions, type ConversationTurnOptions, type FacialChannelName, type FacialCueOptions, type FacialMouthMode, type FacialPerformanceFrameSnapshot, FacialPerformanceLayer, type FacialPerformanceLayerId, FacialPerformanceMixer, type FacialPerformanceModelSnapshot, INPUT_MODES, type InputMode, InputModeError, type ListeningOptions, type ListeningState, type LlmThinkingPayload, type LoadCharacterOptions, MicrophoneError, type MountNpcifyOptions, Npcify, type NpcifyAssetInfo, Character as NpcifyCharacter, type NpcifyOptions, type NpcifyView, PERFORMANCE_FILE_EXTENSION, PERFORMANCE_MEDIA_TYPE, type PerformanceAnalysis, type PerformanceAnalysisOptions, type PerformanceEvent, type PerformanceFrame, type PerformanceIssue, type PerformanceKind, type PerformanceRecording, PerformanceRecordingSchemaVersion, type PerformanceRecordingStartOptions, type PerformanceTimelineSnapshot, type PlayStagingOptions, type PushToTalkOptions, type PushToTalkTarget, RemoteAnimationController, type RepeatedVisemeTransitionAnalysis, SpeechInputError, type SpeechInputResult, type StagingOptionsReadyPayload, type StagingPlaybackStart, type StagingSpeechPlaybackEvent, type TranscriptEvent, type Vector3Like, type Vector3Tuple, type VisemePerformanceFrameSnapshot, analyzePerformanceRecording, bindPushToTalk, createCharacter, decodePerformance, encodePerformance, loadBakedConversation, loadCharacter, mountNpcify };