@npcify/web 0.7.2 → 0.8.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +33 -1
- package/dist/index.d.ts +164 -7
- package/dist/index.js +10466 -9749
- package/package.json +3 -3
package/README.md
CHANGED
|
@@ -122,7 +122,7 @@ animate()
|
|
|
122
122
|
- `character.animation.activeAction`: Active `THREE.AnimationAction | null`.
|
|
123
123
|
- `await character.animation.play(animationId, { fadeDuration, loop })`: Automatically fetches lazy animations if needed and cross-fades.
|
|
124
124
|
- `character.animation.playIdle(fadeDuration)` / `character.animation.stop(fadeDuration)`.
|
|
125
|
-
- Each clip keeps the character's hands out of its own torso, hips and legs: the upper arm turns at the shoulder by the smallest rotation that is necessary (0.7.1). Loose cloth on spring bones (skirts, coat tails) turns the arm by 10 degrees at most, and a clip without contact stays exactly as authored (0.7.2).
|
|
125
|
+
- Each clip keeps the character's hands out of its own torso, hips and legs: the upper arm turns at the shoulder by the smallest rotation that is necessary (0.7.1). Loose cloth on spring bones (skirts, coat tails) turns the arm by 10 degrees at most, and a clip without contact stays exactly as authored (0.7.2). The clip changes only within about 0.3 s around a contact of a hand with the body; turn it off with the option `handClearance: false` (0.7.3; for `mountNpcify`, `new Npcify` and `createCharacter`/`loadCharacter`).
|
|
126
126
|
- **Dialogue & Speech State**:
|
|
127
127
|
- `character.speaking`, `character.thinking`, `character.currentSpeech`, `character.currentText`, `character.revealedText`, `character.options`, `character.cursor`, `character.speech`.
|
|
128
128
|
- `await character.createConversation()`, `await character.sendMessage(text)`, `await character.endConversation()`.
|
|
@@ -130,6 +130,7 @@ animate()
|
|
|
130
130
|
- `character.onConnectionClosed(callback)`: The server closed the conversation connection ([lifecycle](#runtime-and-lifecycle)).
|
|
131
131
|
- **Voice Input** ([details](#voice-input-and-input-modes)):
|
|
132
132
|
- `character.bindPushToTalk(options)`, `character.startListening(options)`, `character.stopListening()`, `character.cancelListening()`.
|
|
133
|
+
- `character.interruptible`, `character.interrupt()`, `character.onInterrupted(callback)`: stop a reply ([details](#interrupting-a-reply)).
|
|
133
134
|
- `character.listeningState`, `character.onListeningState(callback)`, `character.onTranscript(callback)`, `character.inputModes`.
|
|
134
135
|
- `character.notifyUserTyping(active)`, `character.playerActivity`, `character.onPlayerActivity(callback)`: listening reflexes ([details](#voice-input-and-input-modes)).
|
|
135
136
|
- `await character.dispose()`: Ends conversation, detaches from the scene, removes the audio element, and disposes VRM GPU resources.
|
|
@@ -265,6 +266,37 @@ character.onListeningState((state) => {
|
|
|
265
266
|
- The character owner pays the recognition at the price of the provider that
|
|
266
267
|
the owner chose, as for the turn's language model.
|
|
267
268
|
|
|
269
|
+
### Interrupting a reply
|
|
270
|
+
|
|
271
|
+
When the character owner turned on **Players can interrupt replies** (the
|
|
272
|
+
Conversation tab; it needs custom replies and no baked conversation), the
|
|
273
|
+
player can stop a reply while it is generated or plays. The reply field and
|
|
274
|
+
push-to-talk need no lock during the reply:
|
|
275
|
+
|
|
276
|
+
```js
|
|
277
|
+
character.onInterrupted(({ phase, stored }) => {
|
|
278
|
+
// 'thinking' (cancelled, its message joins the next one), 'speaking' or 'played' (kept, cut where it was heard)
|
|
279
|
+
})
|
|
280
|
+
sendButton.disabled = character.thinking && !character.interruptible
|
|
281
|
+
```
|
|
282
|
+
|
|
283
|
+
- `sendMessage(text)` during a reply stops it at once (audio, mouth, face and
|
|
284
|
+
live motion ease to idle within 0.25 s) and sends the new message with the
|
|
285
|
+
heard position. The `sendMessage` of the stopped reply rejects with a
|
|
286
|
+
`NpcifyTurnError` whose `code` is `TURN_INTERRUPTED`
|
|
287
|
+
(`isTurnInterrupted(error)`); it is not an error for the UI.
|
|
288
|
+
- A push-to-talk press during a reply silences it at once. After `minHoldMs`
|
|
289
|
+
the reply stops and the server is told; a shorter press plays the reply
|
|
290
|
+
on. `startListening({ interruptAfterMs })` does the same from code.
|
|
291
|
+
- `interrupt()` stops the running reply without a message (a stop button).
|
|
292
|
+
- `interruptible` is true while a reply runs that the player can interrupt.
|
|
293
|
+
- Too many interrupts before the first sound (6 a minute, 30 an hour) give a
|
|
294
|
+
`NpcifyTurnError` with `code` `INTERRUPT_RATE_LIMITED`; the reply then goes
|
|
295
|
+
on.
|
|
296
|
+
- The owner pays the model tokens up to the stop; a reply stopped before its
|
|
297
|
+
first sound charges the player no message fee
|
|
298
|
+
([pricing](../../docs/ai-pricing.md#interrupted-replies)).
|
|
299
|
+
|
|
268
300
|
## Sight
|
|
269
301
|
|
|
270
302
|
The character can see the player's camera when the character owner turned
|
package/dist/index.d.ts
CHANGED
|
@@ -147,6 +147,12 @@ interface AvatarPublicInfo {
|
|
|
147
147
|
sdkMemoryEnabled?: boolean;
|
|
148
148
|
/** On live turns, the character gets the player's local message time (missing on older servers: off). */
|
|
149
149
|
messageTimeEnabled?: boolean;
|
|
150
|
+
/**
|
|
151
|
+
* Players can interrupt a reply while it plays. On the public info it is the
|
|
152
|
+
* effective value (custom replies on, baked playback off); on the owner's
|
|
153
|
+
* summary it is the owner's switch. Missing on older servers: off.
|
|
154
|
+
*/
|
|
155
|
+
allowInterruption?: boolean;
|
|
150
156
|
cameraSqt?: Sqt | null;
|
|
151
157
|
/** Allowed appearance variants (`default` first) when the character has variants. */
|
|
152
158
|
variants?: AvatarVariantSummary[];
|
|
@@ -178,6 +184,8 @@ interface ConversationWorldlineStaging {
|
|
|
178
184
|
initialVariant?: string;
|
|
179
185
|
activeVariant?: string;
|
|
180
186
|
toolCalls?: StagingToolCall[];
|
|
187
|
+
/** The player interrupted this reply; `text` holds what the player heard. */
|
|
188
|
+
interrupted?: boolean;
|
|
181
189
|
}
|
|
182
190
|
interface ConversationWorldline {
|
|
183
191
|
conversationId: string;
|
|
@@ -451,7 +459,29 @@ interface StagingStreamMotionFramesEvent {
|
|
|
451
459
|
/** `frameCount x (3 + 4 x bones.length)` float32 values, little endian. */
|
|
452
460
|
frames: Uint8Array;
|
|
453
461
|
}
|
|
454
|
-
|
|
462
|
+
/** Where a reply was when the player interrupted it. */
|
|
463
|
+
type StagingInterruptPhase = 'thinking' | 'speaking' | 'played';
|
|
464
|
+
/**
|
|
465
|
+
* The player interrupted a reply (NPCP 1.2 stream event; older receivers
|
|
466
|
+
* ignore it). `stored` is true when the reply stays in the worldline as the
|
|
467
|
+
* parent of the next turn (cut at `heardSeconds` of the staging clock); false
|
|
468
|
+
* when it was cancelled before its first sound.
|
|
469
|
+
*/
|
|
470
|
+
interface StagingStreamInterruptedEvent {
|
|
471
|
+
type: 'interrupted';
|
|
472
|
+
taskId: string;
|
|
473
|
+
stagingId: string;
|
|
474
|
+
phase: StagingInterruptPhase;
|
|
475
|
+
stored: boolean;
|
|
476
|
+
heardSeconds: number;
|
|
477
|
+
/**
|
|
478
|
+
* Only on the client: the SDK stopped the reply but the server did not
|
|
479
|
+
* (it had nothing to stop, refused, or did not answer). The worldline is
|
|
480
|
+
* unchanged; the player can reply.
|
|
481
|
+
*/
|
|
482
|
+
local?: boolean;
|
|
483
|
+
}
|
|
484
|
+
type StagingStreamEvent = StagingStreamStartEvent | StagingStreamTextDeltaEvent | StagingStreamTimelineEvent | StagingStreamPrefixEvent | StagingStreamLiveEvent | StagingStreamSpeechChunkEvent | StagingStreamSpeechCompleteEvent | StagingStreamSpeechErrorEvent | StagingStreamOptionsEvent | StagingStreamErrorEvent | StagingStreamBillingEvent | StagingStreamMotionFramesEvent | StagingStreamInterruptedEvent;
|
|
455
485
|
|
|
456
486
|
/**
|
|
457
487
|
* Voice input of conversations (docs/voice-input.md): the input modes of the
|
|
@@ -507,8 +537,45 @@ interface ConversationTurnOptions {
|
|
|
507
537
|
/** The turn's `Idempotency-Key`; a new UUID when absent. */
|
|
508
538
|
requestId?: string;
|
|
509
539
|
}
|
|
540
|
+
/** What the client played of a reply that the player interrupted. */
|
|
541
|
+
interface InterruptedReplyPosition {
|
|
542
|
+
stagingId: string | null;
|
|
543
|
+
/** Seconds of the staging clock that played; 0 before any sound. */
|
|
544
|
+
heardSeconds: number;
|
|
545
|
+
}
|
|
546
|
+
/**
|
|
547
|
+
* A turn failed with a code of the server that the SDK handles:
|
|
548
|
+
* `TURN_INTERRUPTED` (the player interrupted the reply; not an error for
|
|
549
|
+
* the UI) or `INTERRUPT_RATE_LIMITED` (too many interrupts before the first
|
|
550
|
+
* sound; the running reply goes on).
|
|
551
|
+
*/
|
|
552
|
+
/** The codes of a refused or interrupted turn, or of a refused interrupt. */
|
|
553
|
+
type NpcifyTurnErrorCode = 'TURN_INTERRUPTED' | 'INTERRUPT_RATE_LIMITED' | 'INTERRUPTION_DISABLED' | 'CONVERSATION_ACCESS_DENIED';
|
|
554
|
+
declare class NpcifyTurnError extends Error {
|
|
555
|
+
readonly code: NpcifyTurnErrorCode;
|
|
556
|
+
readonly status: number;
|
|
557
|
+
constructor(code: NpcifyTurnErrorCode, status: number);
|
|
558
|
+
}
|
|
559
|
+
/** The code of a turn error of the SDK (`NpcifyTurnError`, `SpeechInputError`). */
|
|
560
|
+
declare const turnErrorCode: (error: unknown) => string | null;
|
|
561
|
+
/**
|
|
562
|
+
* True when the player interrupted the turn (a typed or a voice turn): not an
|
|
563
|
+
* error for the UI.
|
|
564
|
+
*/
|
|
565
|
+
declare const isTurnInterrupted: (error: unknown) => boolean;
|
|
510
566
|
|
|
511
|
-
|
|
567
|
+
/** What started an interrupt: a sent message or a talk press. */
|
|
568
|
+
type InterruptSource = 'send' | 'talk';
|
|
569
|
+
/** A reply that the player interrupted. */
|
|
570
|
+
interface InterruptedReply extends InterruptedReplyPosition {
|
|
571
|
+
/** The server's task of the reply, when the client knows it. */
|
|
572
|
+
taskId: string | null;
|
|
573
|
+
source: InterruptSource;
|
|
574
|
+
/** Milliseconds from the player's action to silent audio. */
|
|
575
|
+
muteMs: number;
|
|
576
|
+
}
|
|
577
|
+
|
|
578
|
+
type NpcifyAssetInfo = Pick<AvatarPublicInfo, 'animations' | 'idleAnimationId' | 'bakedConversationEnabled' | 'bakedConversationRevision'> & Partial<Pick<AvatarPublicInfo, 'allowCustomReply' | 'allowInterruption' | 'speechInputEnabled' | 'sight' | 'sdkMemoryEnabled' | 'variants'>>;
|
|
512
579
|
interface FetchVrmDownloadUrlOptions {
|
|
513
580
|
apiToken?: string;
|
|
514
581
|
assetProfile?: 'original' | 'web-optimized-v1';
|
|
@@ -577,6 +644,13 @@ declare class ClientTelemetry {
|
|
|
577
644
|
latePhonemeCount: number;
|
|
578
645
|
phonemeLateMs: number;
|
|
579
646
|
}): void;
|
|
647
|
+
/** The time from the player's interrupt to silent audio. */
|
|
648
|
+
reportInterrupt(params: {
|
|
649
|
+
conversationId: string;
|
|
650
|
+
taskId: string;
|
|
651
|
+
stagingId: string;
|
|
652
|
+
muteMs: number;
|
|
653
|
+
}): void;
|
|
580
654
|
}
|
|
581
655
|
|
|
582
656
|
/** Deduplicate animation demand and discard results after model/catalog replacement. */
|
|
@@ -726,7 +800,7 @@ type PerformanceEventPayload = {
|
|
|
726
800
|
state: 'requested' | 'succeeded' | 'blocked';
|
|
727
801
|
} | {
|
|
728
802
|
type: 'staging.stream';
|
|
729
|
-
event: 'start' | 'timeline' | 'options' | 'error';
|
|
803
|
+
event: 'start' | 'timeline' | 'options' | 'error' | 'interrupted';
|
|
730
804
|
stagingId: string;
|
|
731
805
|
taskId: string;
|
|
732
806
|
} | {
|
|
@@ -803,6 +877,8 @@ interface AnimationControllerOptions {
|
|
|
803
877
|
* and neck/head orientation face canonical `+Z`.
|
|
804
878
|
*/
|
|
805
879
|
normalizeIdleOrientation?: boolean;
|
|
880
|
+
/** Keep the hands out of the character's body (default `true`). */
|
|
881
|
+
handClearance?: boolean;
|
|
806
882
|
}
|
|
807
883
|
interface CharacterAnimationPlayOptions {
|
|
808
884
|
fadeDuration?: number;
|
|
@@ -841,6 +917,7 @@ declare class AnimationController implements AnimationControllerLike {
|
|
|
841
917
|
private readonly normalizedIdleClips;
|
|
842
918
|
private readonly configs;
|
|
843
919
|
private readonly normalizeIdleOrientation;
|
|
920
|
+
private readonly handClearance;
|
|
844
921
|
private previewMixer;
|
|
845
922
|
private restorePreview;
|
|
846
923
|
private activeAction;
|
|
@@ -1121,6 +1198,8 @@ type ResumableAudioContext = BaseAudioContext & {
|
|
|
1121
1198
|
};
|
|
1122
1199
|
declare class AudioPlayer {
|
|
1123
1200
|
private readonly visemePlayer;
|
|
1201
|
+
/** The caller paused the audio (`pausePlayback`), maybe before it started. */
|
|
1202
|
+
private pausedByCaller;
|
|
1124
1203
|
private audioElement;
|
|
1125
1204
|
private readonly streams;
|
|
1126
1205
|
private unlocked;
|
|
@@ -1160,6 +1239,13 @@ declare class AudioPlayer {
|
|
|
1160
1239
|
getCurrentTime(): number;
|
|
1161
1240
|
updateVisemes(visemes: ConversationTimelineEntry[]): void;
|
|
1162
1241
|
stop(): void;
|
|
1242
|
+
/** Pauses the playing or starting audio; `resumePlayback` continues it. */
|
|
1243
|
+
pausePlayback(): void;
|
|
1244
|
+
/**
|
|
1245
|
+
* Continues audio paused by `pausePlayback`, also one that had not started.
|
|
1246
|
+
* `started`: the audio played before the pause.
|
|
1247
|
+
*/
|
|
1248
|
+
resumePlayback(started?: boolean): Promise<void>;
|
|
1163
1249
|
reset(): void;
|
|
1164
1250
|
cleanup(): void;
|
|
1165
1251
|
private discardStream;
|
|
@@ -1208,12 +1294,19 @@ declare class StagingPlayer {
|
|
|
1208
1294
|
private readonly appliedToolCallIds;
|
|
1209
1295
|
private activeOptions?;
|
|
1210
1296
|
constructor(audioPlayer: AudioPlayer, facialMixer: FacialPerformanceMixer, animationController: AnimationControllerLike);
|
|
1297
|
+
/** A staging plays (also while it is paused). */
|
|
1298
|
+
get playing(): boolean;
|
|
1211
1299
|
get paused(): boolean;
|
|
1212
1300
|
/**
|
|
1213
1301
|
* Seconds of the staging clock that the tracks show now, or null when no
|
|
1214
1302
|
* staging plays (also while playback waits for the next dialogue block).
|
|
1215
1303
|
*/
|
|
1216
1304
|
get trackTime(): number | null;
|
|
1305
|
+
/**
|
|
1306
|
+
* Seconds of the staging clock that played: the track time, or between
|
|
1307
|
+
* dialogue blocks the time that played so far; 0 before any staging.
|
|
1308
|
+
*/
|
|
1309
|
+
get positionSeconds(): number;
|
|
1217
1310
|
pause(): void;
|
|
1218
1311
|
resume(): Promise<void>;
|
|
1219
1312
|
private waitUntilResumed;
|
|
@@ -1260,6 +1353,8 @@ declare class LiveMotionPlayer {
|
|
|
1260
1353
|
private readonly blendIn;
|
|
1261
1354
|
private readonly blendOut;
|
|
1262
1355
|
private weight;
|
|
1356
|
+
/** A shorter blend-out after an interrupt, until the weight is 0. */
|
|
1357
|
+
private quickBlendOut;
|
|
1263
1358
|
/** The last applied pose, to blend out of it after the frames end. */
|
|
1264
1359
|
private lastPose;
|
|
1265
1360
|
private readonly scratch;
|
|
@@ -1271,6 +1366,11 @@ declare class LiveMotionPlayer {
|
|
|
1271
1366
|
load(stagingId: string, track: StagingMotionTrack, fetchBytes?: (url: string) => Promise<Uint8Array>): Promise<void>;
|
|
1272
1367
|
has(stagingId: string): boolean;
|
|
1273
1368
|
forget(stagingId: string): void;
|
|
1369
|
+
/**
|
|
1370
|
+
* The player interrupted the reply: its frames go, and the pose blends to
|
|
1371
|
+
* the mixer's pose (idle) within `seconds`.
|
|
1372
|
+
*/
|
|
1373
|
+
stop(stagingId: string | null, seconds?: number): void;
|
|
1274
1374
|
/** The current live weight, 0 (idle) to 1 (live frames). */
|
|
1275
1375
|
get liveWeight(): number;
|
|
1276
1376
|
/**
|
|
@@ -1432,6 +1532,13 @@ interface NpcifyOptions {
|
|
|
1432
1532
|
* `Character` (`loadCharacter` / `createCharacter` Three.js game mode).
|
|
1433
1533
|
*/
|
|
1434
1534
|
normalizeIdleOrientation?: boolean;
|
|
1535
|
+
/**
|
|
1536
|
+
* Keep the character's hands out of its own torso, hips and legs in VRMA
|
|
1537
|
+
* clips (the upper arm turns at the shoulder; loose cloth at most 10
|
|
1538
|
+
* degrees; the clip changes only within about 0.3 s around a contact).
|
|
1539
|
+
* Defaults to `true`.
|
|
1540
|
+
*/
|
|
1541
|
+
handClearance?: boolean;
|
|
1435
1542
|
/**
|
|
1436
1543
|
* Playback of live motion frames (characters with real-time motion). The
|
|
1437
1544
|
* default `rootMode: 'anchored'` keeps the character where it stands.
|
|
@@ -1499,6 +1606,11 @@ declare class StreamPlaybackController {
|
|
|
1499
1606
|
private currentCursorValue;
|
|
1500
1607
|
private readonly playedStagingIds;
|
|
1501
1608
|
private readonly states;
|
|
1609
|
+
/** Events of a reply that the player is interrupting, kept until it is decided. */
|
|
1610
|
+
private readonly heldTasks;
|
|
1611
|
+
/** Replies that the player interrupted: their late events are dropped. */
|
|
1612
|
+
private readonly droppedTasks;
|
|
1613
|
+
private interruptedCallback;
|
|
1502
1614
|
constructor(dependencies: StreamPlaybackDependencies);
|
|
1503
1615
|
get speaking(): boolean;
|
|
1504
1616
|
get currentSpeech(): StagingSpeechPlaybackEvent | null;
|
|
@@ -1508,6 +1620,16 @@ declare class StreamPlaybackController {
|
|
|
1508
1620
|
get cursor(): CharacterPlaybackCursor | null;
|
|
1509
1621
|
/** The staging that plays or waits to play, or null. */
|
|
1510
1622
|
get activeStagingId(): string | null;
|
|
1623
|
+
/** The task of the staging that plays or waits to play, or null. */
|
|
1624
|
+
get activeTaskId(): string | null;
|
|
1625
|
+
/** The server stopped a reply that the player interrupted. */
|
|
1626
|
+
onInterrupted(callback: (event: StagingStreamInterruptedEvent) => void): () => void;
|
|
1627
|
+
/** Keeps the later events of a task until `release` or `drop`. */
|
|
1628
|
+
hold(taskId: string): void;
|
|
1629
|
+
/** The interrupt did not happen (a short press, a refused interrupt): play on. */
|
|
1630
|
+
release(taskId: string): void;
|
|
1631
|
+
/** The reply was interrupted: its held and later events are dropped. */
|
|
1632
|
+
drop(taskId: string): void;
|
|
1511
1633
|
/**
|
|
1512
1634
|
* A staging plays or waits to play (its audio or the rest of its stream).
|
|
1513
1635
|
* False after it completed or when autoplay blocked it; the last staging
|
|
@@ -1521,13 +1643,21 @@ declare class StreamPlaybackController {
|
|
|
1521
1643
|
onOptionsReady(callback: (payload: StagingOptionsReadyPayload) => void): () => void;
|
|
1522
1644
|
onTimelineStart(callback: (timeline: StagingTimeline, playback: StagingPlaybackStart) => Promise<void> | void): () => void;
|
|
1523
1645
|
onPlaybackCursor(callback: (cursor: CharacterPlaybackCursor) => void): () => void;
|
|
1524
|
-
|
|
1646
|
+
/** The staging played, or its task was stopped (it never plays later). */
|
|
1647
|
+
hasPlayed(stagingId: string, taskId?: string | null): boolean;
|
|
1525
1648
|
playPayload(payload: StagingResultPayload, options?: {
|
|
1526
1649
|
force?: boolean;
|
|
1527
1650
|
timeline?: StagingTimeline;
|
|
1528
1651
|
}): Promise<boolean>;
|
|
1652
|
+
/**
|
|
1653
|
+
* The SDK stopped a reply that the server did not stop: the app learns it
|
|
1654
|
+
* as an `interrupted` event with `local: true`.
|
|
1655
|
+
*/
|
|
1656
|
+
notifyLocalInterrupt(event: StagingStreamInterruptedEvent): void;
|
|
1529
1657
|
/** Returns the playback result for a timeline event. */
|
|
1530
1658
|
handleEvent(event: StagingStreamEvent): Promise<boolean> | undefined;
|
|
1659
|
+
/** Stops the active staging at once; its audio and buffers go. */
|
|
1660
|
+
stopActive(): void;
|
|
1531
1661
|
cleanup(): void;
|
|
1532
1662
|
private resetRealtimeSpeech;
|
|
1533
1663
|
private playTimeline;
|
|
@@ -1811,6 +1941,11 @@ declare class CharacterRuntime extends CharacterSceneAccess {
|
|
|
1811
1941
|
set gazeTarget(target: Object3D | null);
|
|
1812
1942
|
/** True while the character is actively delivering a speech block or playing voice audio. */
|
|
1813
1943
|
get speaking(): boolean;
|
|
1944
|
+
/**
|
|
1945
|
+
* True while a staging plays or waits for the rest of its stream; false
|
|
1946
|
+
* after it ended or was stopped (an app opens its replies then).
|
|
1947
|
+
*/
|
|
1948
|
+
get playbackActive(): boolean;
|
|
1814
1949
|
/** True while waiting for the next AI turn to begin streaming or playing. */
|
|
1815
1950
|
get thinking(): boolean;
|
|
1816
1951
|
/** Active speech block metadata during playback, or `null` when silent. */
|
|
@@ -2020,6 +2155,14 @@ interface ListeningOptions {
|
|
|
2020
2155
|
endOnSilenceMs?: number | null;
|
|
2021
2156
|
parentStagingId?: string | null;
|
|
2022
2157
|
silent?: boolean;
|
|
2158
|
+
/**
|
|
2159
|
+
* A press during a reply of a character with interruption: the reply is
|
|
2160
|
+
* silenced at once and stopped after this time (push-to-talk passes its
|
|
2161
|
+
* `minHoldMs`); a cancel before it plays the reply on. 0: stop at once.
|
|
2162
|
+
*/
|
|
2163
|
+
interruptAfterMs?: number;
|
|
2164
|
+
/** Internal: the turn interrupts the running reply (`preempt`). */
|
|
2165
|
+
preempt?: boolean;
|
|
2023
2166
|
}
|
|
2024
2167
|
interface SpeechInputResult {
|
|
2025
2168
|
/** The recognised text; empty when no speech was recognised (then no turn starts). */
|
|
@@ -2090,15 +2233,18 @@ declare class Character extends CharacterRuntime {
|
|
|
2090
2233
|
private readonly notifiedStagingIds;
|
|
2091
2234
|
private readonly liveConversation;
|
|
2092
2235
|
private baked;
|
|
2093
|
-
/** The conversation whose live motion was warmed up. */
|
|
2094
|
-
private liveMotionPrepared;
|
|
2095
2236
|
private readonly authoring;
|
|
2096
2237
|
private hostedWebChat;
|
|
2097
2238
|
private memoryIdentifier;
|
|
2098
2239
|
protected readonly normalizeIdleOrientation: boolean;
|
|
2240
|
+
protected readonly handClearance: boolean;
|
|
2099
2241
|
private readonly bakedAsset?;
|
|
2100
2242
|
private animationRefreshId;
|
|
2101
2243
|
private readonly voice;
|
|
2244
|
+
private readonly interruption;
|
|
2245
|
+
/** The character allows interruption (`allowInterruption` of its info). */
|
|
2246
|
+
private allowInterruption;
|
|
2247
|
+
private readonly talk;
|
|
2102
2248
|
private readonly sight;
|
|
2103
2249
|
private readonly inputs;
|
|
2104
2250
|
private listeningStateCallback;
|
|
@@ -2129,6 +2275,15 @@ declare class Character extends CharacterRuntime {
|
|
|
2129
2275
|
dispose(options?: CharacterDisposeOptions): Promise<void>;
|
|
2130
2276
|
sendMessage(message: string, options?: ConversationTurnOptions): Promise<StagingResultPayload>;
|
|
2131
2277
|
sendMessage(conversationId: string, message: string, options?: ConversationTurnOptions): Promise<StagingResultPayload>;
|
|
2278
|
+
/** A reply runs that the player can interrupt (no input lock is needed). */
|
|
2279
|
+
get interruptible(): boolean;
|
|
2280
|
+
/**
|
|
2281
|
+
* Stops the running reply (audio, mouth, face and motion ease to idle
|
|
2282
|
+
* within 0.25 s) and tells the server; null when nothing runs.
|
|
2283
|
+
*/
|
|
2284
|
+
interrupt(): Promise<InterruptedReply | null>;
|
|
2285
|
+
/** The server stopped a reply that the player interrupted. */
|
|
2286
|
+
onInterrupted(callback: (event: StagingStreamInterruptedEvent) => void): () => void;
|
|
2132
2287
|
replayStaging(stagingId: string): Promise<StagingResultPayload>;
|
|
2133
2288
|
replayStaging(conversationId: string, stagingId: string, options?: PlayStagingOptions): Promise<StagingResultPayload>;
|
|
2134
2289
|
get isBakedConversation(): boolean;
|
|
@@ -2295,6 +2450,8 @@ interface MountNpcifyOptions {
|
|
|
2295
2450
|
serverRendering?: boolean;
|
|
2296
2451
|
assetProfile?: 'original' | 'web-optimized-v1';
|
|
2297
2452
|
normalizeIdleOrientation?: boolean;
|
|
2453
|
+
/** Keep the hands out of the character's body in VRMA clips (default `true`). */
|
|
2454
|
+
handClearance?: boolean;
|
|
2298
2455
|
/** How the player can reply: `'options'`, `'text'`, `'voice'` (docs/voice-input.md). */
|
|
2299
2456
|
inputModes?: InputMode[];
|
|
2300
2457
|
/** Player memory identifier for SDK conversations; empty/null uses the character owner's dashboard memory bucket. */
|
|
@@ -2314,4 +2471,4 @@ interface NpcifyView {
|
|
|
2314
2471
|
/** Mount into an existing canvas. Its CSS must give it a nonzero width and height. */
|
|
2315
2472
|
declare const mountNpcify: (options: MountNpcifyOptions) => Promise<NpcifyView>;
|
|
2316
2473
|
|
|
2317
|
-
export { AnimationController, type AnimationControllerLike, type AnimationControllerOptions, type AssetLoadProgress, type AudioCapture, type AudioPerformanceFrameSnapshot, AudioPlayer, type AvatarVariantSummary, BakedConversation, type BakedConversationAsset, type CaptureFrame, Character, CharacterAnimation, type CharacterAnimationPlayOptions, CharacterAudio, type CharacterDisposeOptions, CharacterExpressions, type CharacterLookAtTarget, type CharacterOptions, type CharacterPlaybackCursor, CharacterRuntime, type CharacterRuntimeInitOptions, CharacterSceneRig, type CharacterShadowOptions, type CharacterSpeechState, type CharacterTransformOptions, type CharacterVariantChangeEvent, type ConversationConnectionClosedEvent, ConversationConnectionError, type ConversationConnectionErrorCode, type ConversationJoinFailure, type ConversationTurnOptions, DEFAULT_VARIANT_KEY, type FacialChannelName, type FacialCueOptions, type FacialMouthMode, type FacialPerformanceFrameSnapshot, FacialPerformanceLayer, type FacialPerformanceLayerId, FacialPerformanceMixer, type FacialPerformanceModelSnapshot, type FetchVrmDownloadUrlOptions, INPUT_MODES, type InputMode, InputModeError, type ListeningOptions, type ListeningState, type LiveMotionOptions, LiveMotionPlayer, type LiveMotionRootMode, type LlmThinkingPayload, type LoadCharacterOptions, MicrophoneError, type MountNpcifyOptions, NUDE_VARIANT_KEY, Npcify, type NpcifyAssetInfo, Character as NpcifyCharacter, type NpcifyEventMap, type NpcifyEventName, type NpcifyOptions, type NpcifyView, PERFORMANCE_FILE_EXTENSION, PERFORMANCE_MEDIA_TYPE, type PerformanceAnalysis, type PerformanceAnalysisOptions, type PerformanceEvent, type PerformanceFrame, type PerformanceIssue, type PerformanceKind, type PerformanceRecording, PerformanceRecordingSchemaVersion, type PerformanceRecordingStartOptions, type PerformanceTimelineSnapshot, type PlayStagingOptions, type PlayerActivity, type PushToTalkOptions, type PushToTalkTarget, RemoteAnimationController, type RepeatedVisemeTransitionAnalysis, SightError, type SightFailureCode, type SightObservation, type SightObservationEvent, type SightOptions, type SightPerson, type SightState, SpeechInputError, type SpeechInputResult, type StagingOptionsReadyPayload, type StagingPlaybackStart, type StagingSpeechPlaybackEvent, type StagingSwitchVariantArguments, type StagingToolCall, type StagingToolCallResult, type StagingToolName, type StagingToolResultStatus, type TranscriptEvent, type VariantChangeEvent, type Vector3Like, type Vector3Tuple, type VisemePerformanceFrameSnapshot, analyzePerformanceRecording, bindPushToTalk, createCharacter, decodePerformance, encodePerformance, fetchVariantVrmDownloadUrl, fetchVrmDownloadUrl, loadBakedConversation, loadCharacter, loadVariantVrm, loadVrm, mountNpcify };
|
|
2474
|
+
export { AnimationController, type AnimationControllerLike, type AnimationControllerOptions, type AssetLoadProgress, type AudioCapture, type AudioPerformanceFrameSnapshot, AudioPlayer, type AvatarVariantSummary, BakedConversation, type BakedConversationAsset, type CaptureFrame, Character, CharacterAnimation, type CharacterAnimationPlayOptions, CharacterAudio, type CharacterDisposeOptions, CharacterExpressions, type CharacterLookAtTarget, type CharacterOptions, type CharacterPlaybackCursor, CharacterRuntime, type CharacterRuntimeInitOptions, CharacterSceneRig, type CharacterShadowOptions, type CharacterSpeechState, type CharacterTransformOptions, type CharacterVariantChangeEvent, type ConversationConnectionClosedEvent, ConversationConnectionError, type ConversationConnectionErrorCode, type ConversationJoinFailure, type ConversationTurnOptions, DEFAULT_VARIANT_KEY, type FacialChannelName, type FacialCueOptions, type FacialMouthMode, type FacialPerformanceFrameSnapshot, FacialPerformanceLayer, type FacialPerformanceLayerId, FacialPerformanceMixer, type FacialPerformanceModelSnapshot, type FetchVrmDownloadUrlOptions, INPUT_MODES, type InputMode, InputModeError, type InterruptSource, type InterruptedReply, type InterruptedReplyPosition, type ListeningOptions, type ListeningState, type LiveMotionOptions, LiveMotionPlayer, type LiveMotionRootMode, type LlmThinkingPayload, type LoadCharacterOptions, MicrophoneError, type MountNpcifyOptions, NUDE_VARIANT_KEY, Npcify, type NpcifyAssetInfo, Character as NpcifyCharacter, type NpcifyEventMap, type NpcifyEventName, type NpcifyOptions, NpcifyTurnError, type NpcifyTurnErrorCode, type NpcifyView, PERFORMANCE_FILE_EXTENSION, PERFORMANCE_MEDIA_TYPE, type PerformanceAnalysis, type PerformanceAnalysisOptions, type PerformanceEvent, type PerformanceFrame, type PerformanceIssue, type PerformanceKind, type PerformanceRecording, PerformanceRecordingSchemaVersion, type PerformanceRecordingStartOptions, type PerformanceTimelineSnapshot, type PlayStagingOptions, type PlayerActivity, type PushToTalkOptions, type PushToTalkTarget, RemoteAnimationController, type RepeatedVisemeTransitionAnalysis, SightError, type SightFailureCode, type SightObservation, type SightObservationEvent, type SightOptions, type SightPerson, type SightState, SpeechInputError, type SpeechInputResult, type StagingInterruptPhase, type StagingOptionsReadyPayload, type StagingPlaybackStart, type StagingSpeechPlaybackEvent, type StagingStreamInterruptedEvent, type StagingSwitchVariantArguments, type StagingToolCall, type StagingToolCallResult, type StagingToolName, type StagingToolResultStatus, type TranscriptEvent, type VariantChangeEvent, type Vector3Like, type Vector3Tuple, type VisemePerformanceFrameSnapshot, analyzePerformanceRecording, bindPushToTalk, createCharacter, decodePerformance, encodePerformance, fetchVariantVrmDownloadUrl, fetchVrmDownloadUrl, isTurnInterrupted, loadBakedConversation, loadCharacter, loadVariantVrm, loadVrm, mountNpcify, turnErrorCode };
|