@npcify/web 0.1.7 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,8 +1,9 @@
1
1
  # @npcify/web
2
2
 
3
- Add a Npcify character to any browser application with JavaScript. Includes VRM
4
- rendering, animation, lip sync, live chat, baked conversations, and a canvas
5
- integration. No Vue or React dependency.
3
+ Add a Npcify AI character to any browser application or Three.js 3D game. Includes VRM
4
+ loading, humanoid skeleton and bone access, animation mixing, lip sync, facial
5
+ expressions, Web Audio routing, live chat, baked conversations, and an optional
6
+ managed canvas helper. No Vue or React dependency.
6
7
 
7
8
  ## Install
8
9
 
@@ -13,7 +14,118 @@ npm install @npcify/web
13
14
  Use an ESM-capable bundler, such as Vite. This package includes the required
14
15
  Npcify runtime code; consumers do not need private workspace packages.
15
16
 
16
- ## Plain JavaScript
17
+ ## Mode 1: Three.js Game Integration (`loadCharacter`)
18
+
19
+ Use `loadCharacter` (or `new Character(...)` / `createCharacter(...)`) when
20
+ building a Three.js game or custom 3D scene and you want direct, real-time access
21
+ to the character's `scene`, `mesh`, `skinnedMeshes`, `materials`, humanoid bones,
22
+ `audio` element / Web Audio nodes, facial `expressions`, skeletal `animation`,
23
+ and live `speech` state inside your own render loop.
24
+
25
+ ```js
26
+ import * as THREE from 'three'
27
+ import { loadCharacter } from '@npcify/web'
28
+
29
+ const scene = new THREE.Scene()
30
+ const camera = new THREE.PerspectiveCamera(35, window.innerWidth / window.innerHeight, 0.1, 100)
31
+ camera.position.set(0, 1.4, 2.5)
32
+
33
+ const renderer = new THREE.WebGLRenderer({ antialias: true })
34
+ renderer.setSize(window.innerWidth, window.innerHeight)
35
+ renderer.shadowMap.enabled = true
36
+ document.body.appendChild(renderer.domElement)
37
+
38
+ // Load the character and attach its VRM scene directly to your Three.js scene
39
+ const character = await loadCharacter({
40
+ baseUrl: 'https://api.npcify.ai',
41
+ characterId: 'YOUR_CHARACTER_ID',
42
+ apiKey: 'YOUR_SCOPED_API_KEY',
43
+ parent: scene,
44
+ gazeTarget: camera,
45
+ shadows: true,
46
+ position: [0, 0, 0],
47
+ })
48
+
49
+ // Inspect meshes, materials, and humanoid bones in real time:
50
+ console.log(character.scene) // THREE.Group (vrm.scene)
51
+ console.log(character.mesh) // Primary THREE.SkinnedMesh | THREE.Mesh
52
+ console.log(character.skinnedMeshes) // readonly THREE.SkinnedMesh[]
53
+ console.log(character.materials) // readonly THREE.Material[]
54
+ const headBone = character.getBone('head') // THREE.Object3D | null
55
+
56
+ // Optional: route character voice into Three.js 3D spatial audio (PositionalAudio)
57
+ const listener = new THREE.AudioListener()
58
+ camera.add(listener)
59
+ const positionalAudio = new THREE.PositionalAudio(listener)
60
+ positionalAudio.setNodeSource(character.audio.createMediaElementSource(listener.context))
61
+ headBone?.add(positionalAudio)
62
+
63
+ // Start a conversation inside a user gesture handler:
64
+ startButton.onclick = async () => {
65
+ await character.audio.unlock()
66
+ await character.createConversation()
67
+ }
68
+
69
+ // Advance the character in your game loop and read real-time state:
70
+ const clock = new THREE.Clock()
71
+ function animate() {
72
+ requestAnimationFrame(animate)
73
+ const delta = Math.min(clock.getDelta(), 0.1)
74
+ character.update(delta)
75
+
76
+ // Real-time state on every frame:
77
+ // character.speaking, character.revealedText, character.options
78
+ // character.audio.playing, character.audio.currentTime, character.audio.volume
79
+ // character.expressions.mouthOpenness, character.expressions.weights
80
+ // character.animation.activeAnimationId, character.mixer
81
+
82
+ renderer.render(scene, camera)
83
+ }
84
+ animate()
85
+ ```
86
+
87
+ ### `Character` Real-Time Interface
88
+
89
+ - **3D Scene & Mesh (`Character`)**:
90
+ - `character.scene` / `character.object3d` / `character.root`: Root `THREE.Group` (`vrm.scene`).
91
+ - `character.mesh`: Primary `THREE.SkinnedMesh | THREE.Mesh | null`.
92
+ - `character.meshes`: `readonly THREE.Mesh[]` across the character hierarchy.
93
+ - `character.skinnedMeshes`: `readonly THREE.SkinnedMesh[]`.
94
+ - `character.materials`: Deduplicated `readonly THREE.Material[]`.
95
+ - `character.skeletons`: Deduplicated `readonly THREE.Skeleton[]`.
96
+ - `character.humanoid`: Three-VRM `VRMHumanoid | null`.
97
+ - `character.getBone(name)` / `character.getNormalizedBone(name)`: Look up a humanoid bone `THREE.Object3D` (`'head'`, `'hips'`, `'rightHand'`, etc.).
98
+ - `character.attachTo(parent)` / `character.detach()`: Attach or detach from any `THREE.Object3D`.
99
+ - `character.setShadows({ cast, receive })`: Configure shadow flags across all character meshes.
100
+ - `character.setTransform({ position, rotation, scale })`: Set root transform with `Vector3`/`Euler` or `[x, y, z]` tuples.
101
+ - `character.setGazeTarget(object3d)`: Make template-face eyes follow a `THREE.Camera` or `THREE.Object3D`.
102
+ - **Audio & Web Audio (`character.audio`)**:
103
+ - `character.audio.element`: Underlying `HTMLAudioElement | null`.
104
+ - `character.audio.playing`, `paused`, `ended`, `currentTime`, `duration`, `unlocked`.
105
+ - `character.audio.volume` (`0..1`), `muted` (`boolean`), `playbackRate` (`number`), `crossOrigin`.
106
+ - `character.audio.activeVisemes`: Active lip-sync viseme timeline.
107
+ - `character.audio.createMediaElementSource(audioContext)`: Create or reuse a `MediaElementAudioSourceNode` for `THREE.PositionalAudio`, `AnalyserNode`, or custom Web Audio graphs.
108
+ - `character.audio.connectAudioNode(destination)`: Connect character voice directly to any Web Audio `AudioNode`.
109
+ - `character.audio.unlock()` / `character.audio.stop()`.
110
+ - **Facial Expressions (`character.expressions`)**:
111
+ - `character.expressions.all`, `emotions`, `visemes`: Available VRM expression channel names.
112
+ - `character.expressions.weights`: Real-time map of applied channel weights (`Record<string, number>`).
113
+ - `character.expressions.mouthOpenness`: Real-time mouth openness in `[0, 1]`.
114
+ - `character.expressions.getWeight(name)` / `setWeight(name, weight)`.
115
+ - `character.expressions.play(name, { weight, durationMs })`.
116
+ - `character.expressions.setEmotion(name, weight)` / `clearEmotion()` / `reset()`.
117
+ - **Skeletal Animation (`character.animation`)**:
118
+ - `character.animation.availableIds`, `idleAnimationId`, `activeAnimationId`.
119
+ - `character.animation.mixer`: Underlying `THREE.AnimationMixer | null`.
120
+ - `character.animation.activeAction`: Active `THREE.AnimationAction | null`.
121
+ - `await character.animation.play(animationId, { fadeDuration, loop })`: Automatically fetches lazy animations if needed and cross-fades.
122
+ - `character.animation.playIdle(fadeDuration)` / `character.animation.stop(fadeDuration)`.
123
+ - **Dialogue & Speech State**:
124
+ - `character.speaking`, `character.thinking`, `character.currentSpeech`, `character.currentText`, `character.revealedText`, `character.options`, `character.cursor`, `character.speech`.
125
+ - `await character.createConversation()`, `await character.sendMessage(text)`, `await character.endConversation()`.
126
+ - `await character.dispose()`: Ends conversation, detaches from the scene, removes the audio element, and disposes VRM GPU resources.
127
+
128
+ ## Mode 2: Managed Canvas (`mountNpcify`)
17
129
 
18
130
  Create a key in your character's **Export → API keys** tab. Choose the characters
19
131
  that integration may access. Copy the key when created; it is shown only once.
@@ -41,15 +153,15 @@ const view = await mountNpcify({
41
153
  serverRendering: false, // Use the setting from Export → Web SDK.
42
154
  onError: (error) => { status.textContent = error.message },
43
155
  })
44
- const { npcify } = view
45
- npcify.onOptionsReady(({ options }) => {
156
+ const { character } = view
157
+ character.onOptionsReady(({ options }) => {
46
158
  replies.replaceChildren()
47
159
  for (const option of options) {
48
160
  const button = document.createElement('button')
49
161
  button.textContent = option.label
50
162
  button.onclick = async () => {
51
163
  replies.replaceChildren()
52
- try { await npcify.sendMessage(option.label) }
164
+ try { await character.sendMessage(option.label) }
53
165
  catch (error) { status.textContent = error.message }
54
166
  }
55
167
  replies.append(button)
@@ -59,8 +171,8 @@ start.disabled = false
59
171
  start.onclick = async () => {
60
172
  start.disabled = true
61
173
  try {
62
- await npcify.unlockAudio()
63
- await npcify.createConversation()
174
+ await character.unlockAudio()
175
+ await character.createConversation()
64
176
  } catch (error) {
65
177
  status.textContent = error.message
66
178
  start.disabled = false
@@ -69,21 +181,26 @@ start.onclick = async () => {
69
181
  window.addEventListener('pagehide', () => { void view.dispose() }, { once: true })
70
182
  ```
71
183
 
72
- Run the included `examples/index.html` and `examples/main.js` with any ESM
73
- bundler after replacing their configuration placeholders. Keys in browser code
74
- are visible to visitors. Use narrow scopes; keep keys in your backend when they
184
+ Run the included `examples/index.html`, `examples/main.js`, or `examples/three-game.js`
185
+ with any ESM bundler after replacing their configuration placeholders. Keys in browser
186
+ code are visible to visitors. Use narrow scopes; keep keys in your backend when they
75
187
  must remain private. Calls made with a key are billed to its owner.
76
188
 
77
189
  ## Runtime and lifecycle
78
190
 
79
191
  - `mountNpcify({ canvas, baseUrl, characterId, apiKey, serverRendering })` returns
80
- `{ npcify, dispose }` after loading. Give the canvas a nonzero CSS size.
81
- - Call `unlockAudio()` during a user gesture before starting playback.
192
+ `{ character, npcify, scene, camera, renderer, controls, dispose }` after loading.
193
+ Give the canvas a nonzero CSS size.
194
+ - `loadCharacter({ baseUrl, characterId, apiKey, parent, gazeTarget, shadows })`
195
+ loads and returns a `Character` instance for custom Three.js scenes and render loops.
196
+ - Call `unlockAudio()` (or `character.audio.unlock()`) during a user gesture before
197
+ starting playback.
82
198
  - `onOptionsReady` delivers choices after playback finishes. `sendMessage(label)`
83
199
  selects a choice. Custom replies depend on the character configuration.
84
200
  - `onStagingResult` delivers full results. `onCharacterReveal` supports subtitles.
201
+ Every `on*` registration method returns an `() => void` unsubscribe function.
85
202
  - `dispose()` stops rendering, releases GPU resources, and disconnects chat.
86
- Call it when navigating away or removing the canvas.
203
+ Call it when navigating away or removing the character.
87
204
  - `serverRendering: true` uses acknowledged WebP frames through an authenticated
88
205
  render session. Protected VRM/VRMA files stay on the server. Remount after a
89
206
  render connection error. This release does not use the hosted site's WebRTC
@@ -95,11 +212,6 @@ must remain private. Calls made with a key are billed to its owner.
95
212
  - ESM only; `require()` is not a supported package entry point. Browser APIs are
96
213
  used when creating a view; do not call `mountNpcify` during server rendering.
97
214
 
98
- For custom rendering, use the exported `Npcify` class (`apiToken` option), call
99
- `loadAssets()`, add `npcify.vrm.scene` to your scene, and call
100
- `npcify.update(deltaSeconds)` from your render loop. Do not update that VRM a
101
- second time. Call `endConversation()` when finished.
102
-
103
215
  ## Maintainer release
104
216
 
105
217
  Bump the package version, merge the validated change into `develop`, then run