@npcify/web 0.1.7 → 0.2.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,8 +1,9 @@
1
1
  # @npcify/web
2
2
 
3
- Add a Npcify character to any browser application with JavaScript. Includes VRM
4
- rendering, animation, lip sync, live chat, baked conversations, and a canvas
5
- integration. No Vue or React dependency.
3
+ Add a Npcify AI character to any browser application or Three.js 3D game. Includes VRM
4
+ loading, humanoid skeleton and bone access, animation mixing, lip sync, facial
5
+ expressions, Web Audio routing, live chat, baked conversations, and an optional
6
+ managed canvas helper. No Vue or React dependency.
6
7
 
7
8
  ## Install
8
9
 
@@ -13,7 +14,120 @@ npm install @npcify/web
13
14
  Use an ESM-capable bundler, such as Vite. This package includes the required
14
15
  Npcify runtime code; consumers do not need private workspace packages.
15
16
 
16
- ## Plain JavaScript
17
+ ## Mode 1: Three.js Game Integration (`loadCharacter`)
18
+
19
+ Use `loadCharacter` (or `new Character(...)` / `createCharacter(...)`) when
20
+ building a Three.js game or custom 3D scene and you want direct, real-time access
21
+ to the character's `scene`, `mesh`, `skinnedMeshes`, `materials`, humanoid bones,
22
+ `audio` element / Web Audio nodes, facial `expressions`, skeletal `animation`,
23
+ and live `speech` state inside your own render loop.
24
+
25
+ ```js
26
+ import * as THREE from 'three'
27
+ import { loadCharacter } from '@npcify/web'
28
+
29
+ const scene = new THREE.Scene()
30
+ const camera = new THREE.PerspectiveCamera(35, window.innerWidth / window.innerHeight, 0.1, 100)
31
+ camera.position.set(0, 1.4, 2.5)
32
+
33
+ const renderer = new THREE.WebGLRenderer({ antialias: true })
34
+ renderer.setSize(window.innerWidth, window.innerHeight)
35
+ renderer.shadowMap.enabled = true
36
+ document.body.appendChild(renderer.domElement)
37
+
38
+ // Load the character and attach its VRM scene directly to your Three.js scene
39
+ const character = await loadCharacter({
40
+ baseUrl: 'https://api.npcify.ai',
41
+ characterId: 'YOUR_CHARACTER_ID',
42
+ apiKey: 'YOUR_SCOPED_API_KEY',
43
+ parent: scene,
44
+ gazeTarget: camera,
45
+ shadows: true,
46
+ position: [0, 0, 0],
47
+ })
48
+
49
+ // Inspect meshes, materials, and humanoid bones in real time:
50
+ console.log(character.scene) // THREE.Group (vrm.scene)
51
+ console.log(character.mesh) // Primary THREE.SkinnedMesh | THREE.Mesh
52
+ console.log(character.skinnedMeshes) // readonly THREE.SkinnedMesh[]
53
+ console.log(character.materials) // readonly THREE.Material[]
54
+ const headBone = character.getBone('head') // THREE.Object3D | null
55
+
56
+ // Optional: route character voice into Three.js 3D spatial audio (PositionalAudio)
57
+ const listener = new THREE.AudioListener()
58
+ camera.add(listener)
59
+ const positionalAudio = new THREE.PositionalAudio(listener)
60
+ positionalAudio.setNodeSource(character.audio.createMediaElementSource(listener.context))
61
+ headBone?.add(positionalAudio)
62
+
63
+ // Start a conversation inside a user gesture handler:
64
+ startButton.onclick = async () => {
65
+ await character.audio.unlock()
66
+ await character.createConversation()
67
+ }
68
+
69
+ // Advance the character in your game loop and read real-time state:
70
+ const clock = new THREE.Clock()
71
+ function animate() {
72
+ requestAnimationFrame(animate)
73
+ const delta = Math.min(clock.getDelta(), 0.1)
74
+ character.update(delta)
75
+
76
+ // Real-time state on every frame:
77
+ // character.speaking, character.revealedText, character.options
78
+ // character.audio.playing, character.audio.currentTime, character.audio.volume
79
+ // character.expressions.mouthOpenness, character.expressions.weights
80
+ // character.animation.activeAnimationId, character.mixer
81
+
82
+ renderer.render(scene, camera)
83
+ }
84
+ animate()
85
+ ```
86
+
87
+ ### `Character` Real-Time Interface
88
+
89
+ - **3D Scene & Mesh (`Character`)**:
90
+ - `character.scene` / `character.object3d` / `character.root`: Root `THREE.Group` (`vrm.scene`).
91
+ - `character.mesh`: Primary `THREE.SkinnedMesh | THREE.Mesh | null`.
92
+ - `character.meshes`: `readonly THREE.Mesh[]` across the character hierarchy.
93
+ - `character.skinnedMeshes`: `readonly THREE.SkinnedMesh[]`.
94
+ - `character.materials`: Deduplicated `readonly THREE.Material[]`.
95
+ - `character.skeletons`: Deduplicated `readonly THREE.Skeleton[]`.
96
+ - `character.humanoid`: Three-VRM `VRMHumanoid | null`.
97
+ - `character.getBone(name)` / `character.getNormalizedBone(name)`: Look up a humanoid bone `THREE.Object3D` (`'head'`, `'hips'`, `'rightHand'`, etc.).
98
+ - `character.attachTo(parent)` / `character.detach()`: Attach or detach from any `THREE.Object3D`.
99
+ - `character.setShadows({ cast, receive })`: Configure shadow flags across all character meshes.
100
+ - `character.setTransform({ position, rotation, scale })`: Set root transform with `Vector3`/`Euler` or `[x, y, z]` tuples.
101
+ - `character.setGazeTarget(object3d)`: Make template-face eyes follow a `THREE.Camera` or `THREE.Object3D` with their gaze cone and idle eye moves. Other faces do not follow it. `mountNpcify` sets its camera.
102
+ - `character.lookAt(object3d | [x, y, z] | { x, y, z } | x, y, z)`: Look at an object or a world position. A template face keeps its gaze cone, idle eye moves and staging look cues; other faces turn their eyes with three-vrm `vrm.lookAt.target`. Coordinates that are not finite numbers throw a `RangeError`.
103
+ - `character.clearLookAt()`: Remove the gaze target, so that the eyes look ahead.
104
+ - **Audio & Web Audio (`character.audio`)**:
105
+ - `character.audio.element`: Underlying `HTMLAudioElement | null`.
106
+ - `character.audio.playing`, `paused`, `ended`, `currentTime`, `duration`, `unlocked`.
107
+ - `character.audio.volume` (`0..1`), `muted` (`boolean`), `playbackRate` (`number`), `crossOrigin`.
108
+ - `character.audio.activeVisemes`: Active lip-sync viseme timeline.
109
+ - `character.audio.createMediaElementSource(audioContext)`: Create or reuse a `MediaElementAudioSourceNode` for `THREE.PositionalAudio`, `AnalyserNode`, or custom Web Audio graphs.
110
+ - `character.audio.connectAudioNode(destination)`: Connect character voice directly to any Web Audio `AudioNode`.
111
+ - `character.audio.unlock()` / `character.audio.stop()`.
112
+ - **Facial Expressions (`character.expressions`)**:
113
+ - `character.expressions.all`, `emotions`, `visemes`: Available VRM expression channel names.
114
+ - `character.expressions.weights`: Real-time map of applied channel weights (`Record<string, number>`).
115
+ - `character.expressions.mouthOpenness`: Real-time mouth openness in `[0, 1]`.
116
+ - `character.expressions.getWeight(name)` / `setWeight(name, weight)`.
117
+ - `character.expressions.play(name, { weight, durationMs })`.
118
+ - `character.expressions.setEmotion(name, weight)` / `clearEmotion()` / `reset()`.
119
+ - **Skeletal Animation (`character.animation`)**:
120
+ - `character.animation.availableIds`, `idleAnimationId`, `activeAnimationId`.
121
+ - `character.animation.mixer`: Underlying `THREE.AnimationMixer | null`.
122
+ - `character.animation.activeAction`: Active `THREE.AnimationAction | null`.
123
+ - `await character.animation.play(animationId, { fadeDuration, loop })`: Automatically fetches lazy animations if needed and cross-fades.
124
+ - `character.animation.playIdle(fadeDuration)` / `character.animation.stop(fadeDuration)`.
125
+ - **Dialogue & Speech State**:
126
+ - `character.speaking`, `character.thinking`, `character.currentSpeech`, `character.currentText`, `character.revealedText`, `character.options`, `character.cursor`, `character.speech`.
127
+ - `await character.createConversation()`, `await character.sendMessage(text)`, `await character.endConversation()`.
128
+ - `await character.dispose()`: Ends conversation, detaches from the scene, removes the audio element, and disposes VRM GPU resources.
129
+
130
+ ## Mode 2: Managed Canvas (`mountNpcify`)
17
131
 
18
132
  Create a key in your character's **Export → API keys** tab. Choose the characters
19
133
  that integration may access. Copy the key when created; it is shown only once.
@@ -41,15 +155,15 @@ const view = await mountNpcify({
41
155
  serverRendering: false, // Use the setting from Export → Web SDK.
42
156
  onError: (error) => { status.textContent = error.message },
43
157
  })
44
- const { npcify } = view
45
- npcify.onOptionsReady(({ options }) => {
158
+ const { character } = view
159
+ character.onOptionsReady(({ options }) => {
46
160
  replies.replaceChildren()
47
161
  for (const option of options) {
48
162
  const button = document.createElement('button')
49
163
  button.textContent = option.label
50
164
  button.onclick = async () => {
51
165
  replies.replaceChildren()
52
- try { await npcify.sendMessage(option.label) }
166
+ try { await character.sendMessage(option.label) }
53
167
  catch (error) { status.textContent = error.message }
54
168
  }
55
169
  replies.append(button)
@@ -59,8 +173,8 @@ start.disabled = false
59
173
  start.onclick = async () => {
60
174
  start.disabled = true
61
175
  try {
62
- await npcify.unlockAudio()
63
- await npcify.createConversation()
176
+ await character.unlockAudio()
177
+ await character.createConversation()
64
178
  } catch (error) {
65
179
  status.textContent = error.message
66
180
  start.disabled = false
@@ -69,21 +183,26 @@ start.onclick = async () => {
69
183
  window.addEventListener('pagehide', () => { void view.dispose() }, { once: true })
70
184
  ```
71
185
 
72
- Run the included `examples/index.html` and `examples/main.js` with any ESM
73
- bundler after replacing their configuration placeholders. Keys in browser code
74
- are visible to visitors. Use narrow scopes; keep keys in your backend when they
186
+ Run the included `examples/index.html`, `examples/main.js`, or `examples/three-game.js`
187
+ with any ESM bundler after replacing their configuration placeholders. Keys in browser
188
+ code are visible to visitors. Use narrow scopes; keep keys in your backend when they
75
189
  must remain private. Calls made with a key are billed to its owner.
76
190
 
77
191
  ## Runtime and lifecycle
78
192
 
79
193
  - `mountNpcify({ canvas, baseUrl, characterId, apiKey, serverRendering })` returns
80
- `{ npcify, dispose }` after loading. Give the canvas a nonzero CSS size.
81
- - Call `unlockAudio()` during a user gesture before starting playback.
194
+ `{ character, npcify, scene, camera, renderer, controls, dispose }` after loading.
195
+ Give the canvas a nonzero CSS size.
196
+ - `loadCharacter({ baseUrl, characterId, apiKey, parent, gazeTarget, shadows })`
197
+ loads and returns a `Character` instance for custom Three.js scenes and render loops.
198
+ - Call `unlockAudio()` (or `character.audio.unlock()`) during a user gesture before
199
+ starting playback.
82
200
  - `onOptionsReady` delivers choices after playback finishes. `sendMessage(label)`
83
201
  selects a choice. Custom replies depend on the character configuration.
84
202
  - `onStagingResult` delivers full results. `onCharacterReveal` supports subtitles.
203
+ Every `on*` registration method returns an `() => void` unsubscribe function.
85
204
  - `dispose()` stops rendering, releases GPU resources, and disconnects chat.
86
- Call it when navigating away or removing the canvas.
205
+ Call it when navigating away or removing the character.
87
206
  - `serverRendering: true` uses acknowledged WebP frames through an authenticated
88
207
  render session. Protected VRM/VRMA files stay on the server. Remount after a
89
208
  render connection error. This release does not use the hosted site's WebRTC
@@ -95,11 +214,6 @@ must remain private. Calls made with a key are billed to its owner.
95
214
  - ESM only; `require()` is not a supported package entry point. Browser APIs are
96
215
  used when creating a view; do not call `mountNpcify` during server rendering.
97
216
 
98
- For custom rendering, use the exported `Npcify` class (`apiToken` option), call
99
- `loadAssets()`, add `npcify.vrm.scene` to your scene, and call
100
- `npcify.update(deltaSeconds)` from your render loop. Do not update that VRM a
101
- second time. Call `endConversation()` when finished.
102
-
103
217
  ## Maintainer release
104
218
 
105
219
  Bump the package version, merge the validated change into `develop`, then run