@vizij/runtime-react 0.2.0 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +420 -121
- package/dist/index.cjs +1692 -650
- package/dist/index.d.cts +63 -8
- package/dist/index.d.ts +63 -8
- package/dist/index.js +1701 -667
- package/package.json +23 -21
- package/LICENSE +0 -201
package/README.md
CHANGED
|
@@ -1,182 +1,481 @@
|
|
|
1
1
|
# @vizij/runtime-react
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
`@vizij/runtime-react` is the bundle-first React runtime for Vizij faces. It loads a GLB or prebuilt world, extracts embedded Vizij metadata, composes the rig/pose/program graphs into the behavior of an Arora device ([`@vizij/runtime`](https://www.npmjs.com/package/@vizij/runtime) — an Arora runtime compiled to WebAssembly), mirrors resolved values into the renderer store, and exposes a React-friendly control surface for apps.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
The package is intentionally aimed at app authors. If your app wants to render a Vizij face and drive it through authored rig inputs, this is the layer to build on.
|
|
6
|
+
|
|
7
|
+
> Status: experimental. Public API is still moving with the runtime/export pipeline.
|
|
8
|
+
|
|
9
|
+
## What It Handles
|
|
10
|
+
|
|
11
|
+
- load a face from a GLB URL, a `Blob`, or an already loaded world
|
|
12
|
+
- extract the embedded `VIZIJ_bundle` payload when present
|
|
13
|
+
- merge explicit `rig`, `pose`, `animations`, and `programs` with discovered bundle content
|
|
14
|
+
- compose the registered graphs into the device's behavior and drive its step loop
|
|
15
|
+
- expose runtime status, controls, diagnostics, and update hooks through React context
|
|
16
|
+
- render the resolved face with `VizijRuntimeFace`
|
|
6
17
|
|
|
7
18
|
## Installation
|
|
8
19
|
|
|
9
20
|
```bash
|
|
10
|
-
pnpm add @vizij/runtime-react
|
|
21
|
+
pnpm add @vizij/runtime-react react react-dom
|
|
22
|
+
```
|
|
23
|
+
|
|
24
|
+
If your app also imports lower-level renderer or engine APIs directly, install those packages too:
|
|
25
|
+
|
|
26
|
+
```bash
|
|
27
|
+
pnpm add @vizij/render @vizij/runtime
|
|
11
28
|
```
|
|
12
29
|
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
30
|
+
`@vizij/runtime-react`, `@vizij/render`, and `@vizij/runtime` should stay on the same workspace/release line.
|
|
31
|
+
|
|
32
|
+
### Bundler Notes
|
|
33
|
+
|
|
34
|
+
The runtime depends on Vizij wasm packages transitively. Your bundler needs to emit `.wasm` assets and allow async wasm loading.
|
|
35
|
+
|
|
36
|
+
Example `next.config.js`:
|
|
37
|
+
|
|
38
|
+
```js
|
|
39
|
+
module.exports = {
|
|
40
|
+
webpack: (config) => {
|
|
41
|
+
config.experiments = {
|
|
42
|
+
...(config.experiments ?? {}),
|
|
43
|
+
asyncWebAssembly: true,
|
|
44
|
+
};
|
|
45
|
+
config.module.rules.push({
|
|
46
|
+
test: /\.wasm$/,
|
|
47
|
+
type: "asset/resource",
|
|
48
|
+
});
|
|
49
|
+
return config;
|
|
50
|
+
},
|
|
51
|
+
};
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
If you override wasm URLs manually, pass plain string URLs to the underlying init helpers. Avoid wrappers that turn them into `RelativeURL` objects.
|
|
55
|
+
|
|
56
|
+
## Quick Start
|
|
37
57
|
|
|
38
58
|
```tsx
|
|
39
59
|
import {
|
|
40
60
|
VizijRuntimeFace,
|
|
41
61
|
VizijRuntimeProvider,
|
|
42
62
|
useVizijRuntime,
|
|
63
|
+
type VizijAssetBundle,
|
|
43
64
|
} from "@vizij/runtime-react";
|
|
44
65
|
|
|
45
|
-
|
|
66
|
+
const faceAssetUrl = new URL("./face.glb", import.meta.url).href;
|
|
46
67
|
|
|
47
|
-
const assetBundle = {
|
|
48
|
-
namespace: "demo",
|
|
49
|
-
glb: {
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
68
|
+
const assetBundle: VizijAssetBundle = {
|
|
69
|
+
namespace: "demo-face",
|
|
70
|
+
glb: {
|
|
71
|
+
kind: "url",
|
|
72
|
+
src: faceAssetUrl,
|
|
73
|
+
aggressiveImport: true,
|
|
74
|
+
},
|
|
75
|
+
pose: {
|
|
76
|
+
stageNeutralFilter: (_id, path) => !path.includes("/color/"),
|
|
54
77
|
},
|
|
55
78
|
};
|
|
56
79
|
|
|
57
80
|
export function App() {
|
|
58
81
|
return (
|
|
59
82
|
<VizijRuntimeProvider assetBundle={assetBundle} autostart>
|
|
60
|
-
<
|
|
61
|
-
<VizijRuntimeFace />
|
|
83
|
+
<RuntimeStage />
|
|
62
84
|
</VizijRuntimeProvider>
|
|
63
85
|
);
|
|
64
86
|
}
|
|
65
87
|
|
|
66
|
-
function
|
|
67
|
-
const { loading, error,
|
|
68
|
-
|
|
69
|
-
if (
|
|
70
|
-
return <p>Runtime
|
|
88
|
+
function RuntimeStage() {
|
|
89
|
+
const { loading, ready, error, stagePoseNeutral } = useVizijRuntime();
|
|
90
|
+
|
|
91
|
+
if (loading) return <p>Loading face…</p>;
|
|
92
|
+
if (error) return <p>Runtime failed: {error.message}</p>;
|
|
93
|
+
if (!ready) return <p>Preparing runtime…</p>;
|
|
94
|
+
|
|
95
|
+
return (
|
|
96
|
+
<div>
|
|
97
|
+
<button onClick={() => stagePoseNeutral(true)}>Reset pose</button>
|
|
98
|
+
<VizijRuntimeFace className="face-canvas" showSafeArea={false} />
|
|
99
|
+
</div>
|
|
100
|
+
);
|
|
71
101
|
}
|
|
72
102
|
```
|
|
73
103
|
|
|
74
|
-
The provider
|
|
104
|
+
The provider resolves the face bundle, boots its Arora device with the composed graphs as the device's behavior, and publishes the merged runtime state through `useVizijRuntime()`.
|
|
105
|
+
|
|
106
|
+
## Core Concepts
|
|
107
|
+
|
|
108
|
+
### Bundle-first runtime
|
|
109
|
+
|
|
110
|
+
The main contract is `VizijAssetBundle`. In the default workflow you hand the runtime one GLB and let it discover as much as possible from the embedded `VIZIJ_bundle`.
|
|
111
|
+
|
|
112
|
+
Explicit overrides still work. If you provide `rig`, `pose`, `animations`, or `programs`, the runtime merges them with embedded bundle data and deduplicates animations/programs by id.
|
|
113
|
+
|
|
114
|
+
### One device per provider
|
|
115
|
+
|
|
116
|
+
Each `VizijRuntimeProvider` owns one Arora device. The device runs the
|
|
117
|
+
composed graph as its behavior on a shared key/value store: graph `input`
|
|
118
|
+
nodes read store paths each tick, graph outputs write back, and the provider
|
|
119
|
+
mirrors changed values into the renderer store after every step. Providers
|
|
120
|
+
are fully isolated from each other — multiple faces mean multiple devices,
|
|
121
|
+
and namespacing keeps their store keys apart.
|
|
122
|
+
|
|
123
|
+
`driveRuntime={false}` mounts a runtime that does not step its device
|
|
124
|
+
from its own loop; use it for surfaces that are stepped manually (see
|
|
125
|
+
[Manual stepping](#manual-stepping)) or at a background cadence.
|
|
126
|
+
|
|
127
|
+
### Where the composed graph comes from
|
|
128
|
+
|
|
129
|
+
The device runs **one** graph, composed from several sources
|
|
130
|
+
(`composeGraphSpecs`; the live list is `graphSourcesRef` in
|
|
131
|
+
`VizijRuntimeProvider`). Each source has a distinct provenance:
|
|
132
|
+
|
|
133
|
+
- **Rig graph** — shipped in the loaded GLB/asset bundle (`VIZIJ_bundle`'s
|
|
134
|
+
rig, or the explicit `rig` override). It maps rig input paths to the
|
|
135
|
+
face's morph/bone/material writes: this is the face itself.
|
|
136
|
+
- **Pose-driver graph** — the bundle's `pose` graph (or the
|
|
137
|
+
`pose-driver`/`pose` graph discovered in the bundle). It turns high-level
|
|
138
|
+
pose controls into rig-input writes, which the rig source reads back
|
|
139
|
+
through the shared store paths on the next tick.
|
|
140
|
+
- **Program graphs** — one source per _playing_ program: procedural graphs
|
|
141
|
+
from the bundle's `programs` started via the transport, and (in
|
|
142
|
+
`vizij-authoring`) the motiongraph editor's graph, published as a program
|
|
143
|
+
so it evaluates on the device like everything else.
|
|
144
|
+
- **Animations** — a single source, composed whenever any clip is playing.
|
|
145
|
+
It is an `ExternalFunction` node that steps the **animation module**
|
|
146
|
+
([`@vizij/animation-module`](https://www.npmjs.com/package/@vizij/animation-module))
|
|
147
|
+
every device tick off the golden `arora/dt`. Clips register into the module
|
|
148
|
+
as data (through its call surface), the module samples them **inside the
|
|
149
|
+
device**, and its per-track outputs are routed on to the rig-input store
|
|
150
|
+
paths (VIZ-61 Stage B — the JS clip pipeline no longer samples clips).
|
|
151
|
+
|
|
152
|
+
Sources are namespaced by id (`source::node`) so nodes can't collide;
|
|
153
|
+
**store paths are deliberately shared** — that is the cross-source contract.
|
|
154
|
+
|
|
155
|
+
**How a change reaches the device.** When the composition changes — a program
|
|
156
|
+
starts or stops, the rig or a source is swapped — the runtime diffs the old and
|
|
157
|
+
new composed specs and **patches the running graph in place** through the
|
|
158
|
+
behavior interpreter's `apply(GraphDiff)`: only the nodes and edges the change
|
|
159
|
+
touched are re-installed (VIZ-79). A structurally empty diff is a no-op; if a
|
|
160
|
+
patch ever fails the runtime falls back to a whole-spec `loadGraph`. The Vizij
|
|
161
|
+
node graph is a full behavior interpreter — it accepts both an incremental
|
|
162
|
+
`apply(GraphDiff)` and a whole `load` (see
|
|
163
|
+
[`vizij-arora-behavior/docs/node-graph.md`](https://github.com/vizij-ai/vizij-rs/blob/main/crates/interop/vizij-arora-behavior/docs/node-graph.md)).
|
|
164
|
+
The device, its store, and its loaded module set are untouched across the
|
|
165
|
+
change; the device is rebuilt only when the **module set** itself changes.
|
|
166
|
+
Stateful nodes keep their state too — springs, dampers, and the graph clock
|
|
167
|
+
carry over, and a node the edit didn't touch keeps its integration state, so a
|
|
168
|
+
program starting or stopping (or an unrelated edit) does not reset it.
|
|
169
|
+
|
|
170
|
+
### Animations and the device
|
|
171
|
+
|
|
172
|
+
Playback lives inside the device, end to end. Clips load into the animation
|
|
173
|
+
module with their **final store keys resolved at load** (the rig routing the
|
|
174
|
+
JS pipeline used to apply per tick), the composed animations source steps
|
|
175
|
+
the module off the golden `arora/dt`, and a path-less `output` node applies
|
|
176
|
+
the sampled batch onto those keys — no JS touches the per-tick path.
|
|
177
|
+
|
|
178
|
+
Transport rides the module (0.2.0): `play`/`pause` resume and hold the
|
|
179
|
+
player, `stop` resets its playhead (the next tick emits the clip's t=0
|
|
180
|
+
pose; `stopAnimation({ clearOutputs: false })` holds the pose instead),
|
|
181
|
+
`seekAnimation`, `setAnimationLoop(false)` (one-shot), and
|
|
182
|
+
`playAnimation({ speed, weight })` are real player commands.
|
|
183
|
+
`getAnimationState()` reads the module's `player_states` feedback — a live
|
|
184
|
+
playhead, duration, playing, speed — and `playAnimation()`'s promise
|
|
185
|
+
resolves when a non-looping clip reaches its end.
|
|
186
|
+
|
|
187
|
+
Remaining fidelity gap: authored `cubic` keyframes carry no explicit
|
|
188
|
+
handles in the stored form, so they sample the engine's default ease
|
|
189
|
+
(`linear`/`step` timing rides through as explicit bezier handles).
|
|
190
|
+
|
|
191
|
+
### Asset reloads vs graph re-registration
|
|
192
|
+
|
|
193
|
+
When you swap the `assetBundle` prop, the runtime decides whether it needs to reload assets or only re-register controllers. That behavior is controlled by `updateTier` and is also exposed as `resolveRuntimeUpdatePlan()`.
|
|
194
|
+
|
|
195
|
+
This matters for tooling apps like `vizij-authoring`, where graphs/animations can change without replacing the face asset itself.
|
|
196
|
+
|
|
197
|
+
## `VizijAssetBundle`
|
|
198
|
+
|
|
199
|
+
```ts
|
|
200
|
+
type VizijAssetBundle = {
|
|
201
|
+
namespace?: string;
|
|
202
|
+
faceId?: string;
|
|
203
|
+
glb: VizijGlbAsset;
|
|
204
|
+
rig?: VizijGraphAsset;
|
|
205
|
+
pose?: {
|
|
206
|
+
graph?: VizijGraphAsset;
|
|
207
|
+
config?: PoseRigConfig;
|
|
208
|
+
stageNeutralFilter?: (id: string, path: string) => boolean;
|
|
209
|
+
};
|
|
210
|
+
animations?: VizijAnimationAsset[];
|
|
211
|
+
programs?: VizijProgramAsset[];
|
|
212
|
+
initialInputs?: Record<string, ValueJSON>;
|
|
213
|
+
metadata?: Record<string, unknown>;
|
|
214
|
+
bundle?: VizijBundleExtension | null;
|
|
215
|
+
};
|
|
216
|
+
```
|
|
217
|
+
|
|
218
|
+
### `glb`
|
|
219
|
+
|
|
220
|
+
Required. One of:
|
|
75
221
|
|
|
76
|
-
|
|
222
|
+
- `{ kind: "url", src, aggressiveImport?, rootBounds? }`
|
|
223
|
+
- `{ kind: "blob", blob, aggressiveImport?, rootBounds? }`
|
|
224
|
+
- `{ kind: "world", world, animatables, bundle? }`
|
|
77
225
|
|
|
78
|
-
|
|
226
|
+
Use `kind: "world"` when your app already loaded the scene and wants runtime-react to wire only the engine/runtime layer.
|
|
79
227
|
|
|
80
|
-
|
|
228
|
+
### `rig`
|
|
81
229
|
|
|
82
|
-
`
|
|
230
|
+
Optional `VizijGraphAsset` for the main rig graph. When omitted, the runtime looks for a compatible graph in the embedded bundle.
|
|
83
231
|
|
|
84
|
-
|
|
85
|
-
- `rig`: optional. When omitted, the runtime looks for a `VIZIJ_bundle` extension inside the GLB and registers the first rig graph it finds.
|
|
86
|
-
- `pose`: optional. Provide your own pose graph/config or just a `stageNeutralFilter`; bundled pose data is pulled from the GLB when present.
|
|
87
|
-
- `animations`: optional array. Explicit entries merge with animations discovered in the GLB bundle (deduped by id). Use `playAnimation` to trigger them.
|
|
88
|
-
- `programs`: optional array. Explicit entries merge with bundled `motiongraph` programs (deduped by id). Use `playProgram` / `pauseProgram` / `stopProgram` to control them.
|
|
89
|
-
- `initialInputs`: optional map of ValueJSON that seeds inputs before autostart.
|
|
90
|
-
- `metadata`: free-form dictionary you can read back through `useVizijRuntime().assetBundle.metadata`.
|
|
91
|
-
- `bundle`: optional. Supply pre-parsed bundle metadata when you manage world loading yourself. When loading from GLB/Blob the runtime fills this with the extracted `VizijBundleExtension`.
|
|
232
|
+
### `pose`
|
|
92
233
|
|
|
93
|
-
|
|
234
|
+
Optional pose graph/config surface:
|
|
235
|
+
|
|
236
|
+
- `graph`: pose-driver graph override
|
|
237
|
+
- `config`: `PoseRigConfig` used by pose-aware UIs and pose-blending configuration
|
|
238
|
+
- `stageNeutralFilter`: lets you skip specific neutral writes, for example baked color channels
|
|
239
|
+
|
|
240
|
+
`PoseRigConfig.poseGroups` defines how subsets of poses are grouped for local blend behavior and wider composition. Group labels such as `viseme` or `emotion` are not runtime path segments. Runtime-facing pose writes still use canonical per-pose paths.
|
|
241
|
+
|
|
242
|
+
### `animations`
|
|
243
|
+
|
|
244
|
+
Optional authored clips. These merge with embedded bundle animations and extracted GLTF animation clips.
|
|
245
|
+
|
|
246
|
+
### `programs`
|
|
247
|
+
|
|
248
|
+
Optional procedural programs. These merge with embedded bundle `motiongraph` entries.
|
|
249
|
+
|
|
250
|
+
### `initialInputs`
|
|
251
|
+
|
|
252
|
+
Optional `ValueJSON` map staged before autostart/manual stepping.
|
|
253
|
+
|
|
254
|
+
### `metadata`
|
|
255
|
+
|
|
256
|
+
Arbitrary app metadata. The runtime keeps it attached to the resolved `assetBundle`.
|
|
257
|
+
|
|
258
|
+
### `bundle`
|
|
259
|
+
|
|
260
|
+
Optional pre-parsed `VizijBundleExtension`. Useful when you already decoded bundle metadata yourself.
|
|
94
261
|
|
|
95
262
|
## Provider Props
|
|
96
263
|
|
|
97
|
-
|
|
98
|
-
- `namespace` / `faceId`: override bundle values to reuse the same assets under multiple namespaces or faces.
|
|
99
|
-
- `autoCreate` + `createOptions`: forwarded to `OrchestratorProvider`. Leave `autoCreate` enabled unless you need manual control over WASM creation.
|
|
100
|
-
- `autostart`: when true, the orchestrator starts ticking as soon as assets register.
|
|
101
|
-
- `driveOrchestrator`: defaults to `true`. In shared orchestrator setups, set this to `false` on non-driver runtimes so only one loop calls `stepRuntime` while others continue to stage inputs/animations.
|
|
102
|
-
- `mergeStrategy`: orchestrator merge strategy, defaults to additive for outputs/intermediate values.
|
|
103
|
-
- `orchestratorScope`: control orchestrator reuse. `"auto"` (default) reuses a parent `OrchestratorProvider` when present, `"shared"` requires one, and `"isolated"` always spins up a dedicated instance.
|
|
104
|
-
- `onRegisterControllers(ids)`: observe graph and animation controller ids that were registered.
|
|
105
|
-
- `onStatusChange(status)`: gets every status update (loading events, errors, controller updates).
|
|
106
|
-
|
|
107
|
-
Inputs are staged in a per-frame buffer and flushed together before each `step()`, minimizing wasm boundary crossings even when multiple drivers write in the same tick.
|
|
108
|
-
|
|
109
|
-
The provider exposes context via `useVizijRuntime()` once `loading` flips to `false`.
|
|
110
|
-
|
|
111
|
-
## Hooks and Helpers
|
|
112
|
-
|
|
113
|
-
- `useVizijRuntime()`: returns the full runtime context (status, setters, animation helpers, and the original asset bundle). Common properties:
|
|
114
|
-
- `loading`, `ready`, `error`, `errors`: lifecycle state.
|
|
115
|
-
- `namespace`, `faceId`, `rootId`: resolved IDs the renderer/orchestrator share.
|
|
116
|
-
- `outputPaths`: namespaced output signal paths detected in the registered graphs (matches orchestrator write keys).
|
|
117
|
-
- `controllers`: `{ graphs, anims }` that were installed.
|
|
118
|
-
- `setInput(path, value)`, `setValue(id, namespace, value)`: talk directly to the orchestrator or renderer store.
|
|
119
|
-
- `stagePoseNeutral(force)`: restore the neutral pose captured at export.
|
|
120
|
-
- `animateValue(path, target, options)`, `cancelAnimation(path)`: tween rig values with built-in easing.
|
|
121
|
-
- `playAnimation(id, options)`, `stopAnimation(id)`: drive bundle animations that were provided in `animations`.
|
|
122
|
-
- `playProgram(id)`, `pauseProgram(id)`, `stopProgram(id)`: drive bundled procedural programs discovered from `motiongraph` bundle entries or supplied via `programs`.
|
|
123
|
-
- `step(dt)` / `advanceAnimations(dt)`: manually tick the orchestrator if you run outside `autostart`.
|
|
124
|
-
- `useRigInput(path)`: returns `[value, setter]` for a single rig input. The setter writes through the orchestrator while the value mirrors the renderer store.
|
|
125
|
-
- `useVizijOutputs(paths)`: subscribes to renderer output paths (`RawValue` map) for UI or logging.
|
|
126
|
-
- `registerInputDriver(id, factory)`: attach custom drivers (speech-to-anim, sensors). The factory receives `setInput` and `setRendererValue` helpers and must return `{ start, stop, dispose }`.
|
|
264
|
+
`VizijRuntimeProviderProps`:
|
|
127
265
|
|
|
128
|
-
|
|
266
|
+
- `assetBundle`: required runtime bundle
|
|
267
|
+
- `namespace`, `faceId`: override resolved ids without mutating the incoming bundle
|
|
268
|
+
- `updateTier`: `"auto"` (default), `"assets"`, or `"graphs"`
|
|
269
|
+
- `autoCreate`: load the engine wasm and boot the device automatically on mount
|
|
270
|
+
- `autostart`: start the runtime loop automatically after registration
|
|
271
|
+
- `driveRuntime`: whether this runtime instance should call `step()` during its loop
|
|
272
|
+
- `mergeStrategy`: forwarded to graph registration
|
|
273
|
+
- `transformOutputWrite(write)`: intercept or drop output writes before they hit the renderer store
|
|
274
|
+
- `onRegisterControllers(ids)`: receive registered graph/animation ids
|
|
275
|
+
- `onStatusChange(status)`: subscribe to runtime status changes
|
|
129
276
|
|
|
130
|
-
|
|
131
|
-
- Compose your own UI with the renderer primitives exported from `@vizij/render`. The runtime only handles wiring and state.
|
|
277
|
+
### Important runtime flags
|
|
132
278
|
|
|
133
|
-
|
|
279
|
+
- `autostart` controls whether the device begins stepping automatically once ready.
|
|
280
|
+
- `driveRuntime={false}` is useful for faces stepped manually or at a background cadence.
|
|
281
|
+
- `transformOutputWrite` is the hook to remap or suppress specific runtime outputs before they update renderer state.
|
|
134
282
|
|
|
135
|
-
|
|
283
|
+
## Runtime Context API
|
|
136
284
|
|
|
137
|
-
|
|
138
|
-
- `registration`: orchestrator graph registration problems.
|
|
139
|
-
- `driver`: input driver lifecycle exceptions.
|
|
140
|
-
- `animation`: failures while sampling clip tracks.
|
|
285
|
+
Use `useVizijRuntime()` inside the provider tree.
|
|
141
286
|
|
|
142
|
-
|
|
287
|
+
### Status and identity
|
|
143
288
|
|
|
144
|
-
|
|
289
|
+
- `loading`, `ready`
|
|
290
|
+
- `error`, `errors`
|
|
291
|
+
- `namespace`, `faceId`, `rootId`
|
|
292
|
+
- `controllers.graphs`, `controllers.anims`
|
|
293
|
+
- `outputPaths`
|
|
294
|
+
- `stepHz`
|
|
295
|
+
- `assetBundle`
|
|
296
|
+
- `inputConstraints`
|
|
145
297
|
|
|
146
|
-
|
|
298
|
+
`inputConstraints` is built from graph metadata and is the right source for slider defaults/ranges in tooling UIs.
|
|
147
299
|
|
|
148
|
-
|
|
300
|
+
### Input and renderer writes
|
|
149
301
|
|
|
150
|
-
|
|
302
|
+
- `setInput(path, value, shape?)`
|
|
303
|
+
- `getValueSnapshot(path)` — the current engine-store value of a path (reads your own writes)
|
|
304
|
+
- `setValue(id, namespace, value)`
|
|
305
|
+
- `stagePoseNeutral(force?)`
|
|
151
306
|
|
|
152
|
-
|
|
307
|
+
### Runtime graph updates
|
|
153
308
|
|
|
154
|
-
|
|
309
|
+
- `setGraphBundle(bundle, options?)`
|
|
155
310
|
|
|
156
|
-
|
|
157
|
-
2. Drop the GLB into a `VizijAssetBundle` and let the runtime extract rig/pose/animation/program data automatically.
|
|
158
|
-
3. Optionally override or extend bundle contents by setting `rig`, `pose`, `animations`, or `programs` manually (useful for tooling builds or custom staging).
|
|
159
|
-
4. Host GLB URLs or include them via bundler asset imports (`new URL("./face.glb", import.meta.url).href`).
|
|
311
|
+
`setGraphBundle()` lets you swap `rig`, `pose`, `animations`, and `programs` at runtime. This is the API that tooling apps use when the face asset stays the same but the authored runtime bundle changes.
|
|
160
312
|
|
|
161
|
-
|
|
313
|
+
### Value animation helpers
|
|
162
314
|
|
|
163
|
-
|
|
315
|
+
- `animateValue(path, target, options?)`
|
|
316
|
+
- `cancelAnimation(path)`
|
|
317
|
+
- `setAnimationActive(active)`
|
|
318
|
+
- `isAnimationActive()`
|
|
164
319
|
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
320
|
+
`animateValue()` is the simple way to tween a single rig input path with built-in easing.
|
|
321
|
+
|
|
322
|
+
### Clip transport
|
|
323
|
+
|
|
324
|
+
- `playAnimation(id, options?)`
|
|
325
|
+
- `pauseAnimation(id)`
|
|
326
|
+
- `seekAnimation(id, timeSeconds)`
|
|
327
|
+
- `setAnimationLoop(id, enabled)`
|
|
328
|
+
- `getAnimationState(id)`
|
|
329
|
+
- `stopAnimation(id, options?)`
|
|
330
|
+
|
|
331
|
+
### Program transport
|
|
332
|
+
|
|
333
|
+
- `playProgram(id)`
|
|
334
|
+
- `pauseProgram(id)`
|
|
335
|
+
- `stopProgram(id, options?)`
|
|
336
|
+
- `getProgramState(id)`
|
|
337
|
+
|
|
338
|
+
### Manual stepping
|
|
339
|
+
|
|
340
|
+
- `step(dt, opts?)`
|
|
341
|
+
- `advanceAnimations(dt)`
|
|
342
|
+
|
|
343
|
+
Use manual stepping when you do not want the provider to own the runtime loop or when hidden/shared faces need low-frequency background stepping.
|
|
344
|
+
|
|
345
|
+
### Driver registration
|
|
346
|
+
|
|
347
|
+
- `registerInputDriver(id, factory)`
|
|
348
|
+
|
|
349
|
+
Custom input drivers receive:
|
|
350
|
+
|
|
351
|
+
- `setInput(path, value, shape?)`
|
|
352
|
+
- `setRendererValue(id, namespace, valueOrUpdater)`
|
|
353
|
+
- `namespace`
|
|
354
|
+
- `faceId`
|
|
355
|
+
|
|
356
|
+
Return `{ start, stop, dispose }`.
|
|
357
|
+
|
|
358
|
+
## Hooks
|
|
359
|
+
|
|
360
|
+
### `useVizijRuntime()`
|
|
361
|
+
|
|
362
|
+
Throws when used outside the provider.
|
|
363
|
+
|
|
364
|
+
### `useOptionalVizijRuntime()`
|
|
365
|
+
|
|
366
|
+
Returns `null` when no provider is present. This is useful for shared components that can operate with or without a runtime.
|
|
367
|
+
|
|
368
|
+
### `useRigInput(path)`
|
|
369
|
+
|
|
370
|
+
Returns `[value, setValue]` for a single runtime input path. The setter writes into the device's store, while the value mirrors the renderer store.
|
|
371
|
+
|
|
372
|
+
### `useVizijOutputs(paths)`
|
|
373
|
+
|
|
374
|
+
Subscribes to renderer output values for the current namespace and returns a path-to-value map.
|
|
375
|
+
|
|
376
|
+
## Components
|
|
377
|
+
|
|
378
|
+
### `VizijRuntimeFace`
|
|
379
|
+
|
|
380
|
+
`VizijRuntimeFace` renders the resolved face using the current runtime namespace and root id.
|
|
381
|
+
|
|
382
|
+
It accepts normal `Vizij` renderer props except `rootId` and `namespace`, which are owned by the runtime. It also supports `namespaceOverride` when you need to inspect another namespace while keeping the current runtime context.
|
|
383
|
+
|
|
384
|
+
## Exported Utilities
|
|
385
|
+
|
|
386
|
+
### Pose path helpers
|
|
387
|
+
|
|
388
|
+
- `buildRigInputPath(faceId, path)`
|
|
389
|
+
- `buildPoseWeightInputPathSegment(poseId)`
|
|
390
|
+
- `buildPoseWeightRelativePath(poseId)`
|
|
391
|
+
- `buildPoseWeightPathMap(poses, faceId)`
|
|
392
|
+
|
|
393
|
+
These are the canonical helpers for pose-weight paths. The current runtime/export contract is:
|
|
394
|
+
|
|
395
|
+
```text
|
|
396
|
+
rig/{faceId}/poses/{poseId}.weight
|
|
171
397
|
```
|
|
172
398
|
|
|
173
|
-
|
|
399
|
+
That contract does not change when the pose belongs to a group labeled `viseme` or `emotion`. Group membership exists to control blend behavior, not to build a different input path family.
|
|
400
|
+
|
|
401
|
+
### Pose semantics helpers
|
|
402
|
+
|
|
403
|
+
- `normalizePoseSemanticKey()`
|
|
404
|
+
- `getPoseSemanticKey()`
|
|
405
|
+
- `resolvePoseMembership()`
|
|
406
|
+
- `resolvePoseSemantics()`
|
|
407
|
+
- `filterPosesBySemanticKind()`
|
|
408
|
+
- `buildSemanticPoseWeightPathMap()`
|
|
409
|
+
- constants such as `VISEME_POSE_KEYS`, `EMOTION_POSE_KEYS`, and `EXPRESSIVE_EMOTION_POSE_KEYS`
|
|
410
|
+
|
|
411
|
+
These helpers are convenience utilities for example apps that need to order or match current poses without hard-coding face-specific names. Pose groups themselves still exist to control blending/composition, and these helpers do not introduce paths such as `rig/{faceId}/visemes/...`.
|
|
412
|
+
|
|
413
|
+
### Automatic input-path detection and pose-control bridging
|
|
414
|
+
|
|
415
|
+
Runtime-react also auto-detects the actual rig input paths it needs from the registered rig graph:
|
|
174
416
|
|
|
175
|
-
|
|
417
|
+
1. `collectInputPathMap()` scans `input` nodes and records aliases for authored channel ids.
|
|
418
|
+
2. When both direct rig inputs and `pose/control` inputs exist, the direct rig input path is preferred for bare channel ids.
|
|
419
|
+
3. Compiled pose graphs still emit internal pose-control outputs on:
|
|
420
|
+
|
|
421
|
+
```text
|
|
422
|
+
rig/{faceId}/pose/control/{inputId}
|
|
423
|
+
```
|
|
424
|
+
|
|
425
|
+
4. The provider bridges those internal outputs back onto the detected rig input path when possible and falls back to the native `pose/control` path only when necessary.
|
|
426
|
+
|
|
427
|
+
This is why dependent apps should prefer runtime-react helpers and resolved metadata over hard-coded face-specific input paths.
|
|
428
|
+
|
|
429
|
+
### Face control helpers
|
|
430
|
+
|
|
431
|
+
- `resolveFaceControls(assetBundle, runtimeFaceId?, inputConstraints?)`
|
|
432
|
+
- `mapNormalizedControlValue(control, value)`
|
|
433
|
+
- `mapUnitControlValue(control, value)`
|
|
434
|
+
|
|
435
|
+
Use these when you want to build gaze/blink/eyelid controls from runtime metadata rather than hard-coded paths.
|
|
436
|
+
|
|
437
|
+
### Update policy helpers
|
|
438
|
+
|
|
439
|
+
- `resolveRuntimeUpdatePlan(previous, next, tier)`
|
|
440
|
+
|
|
441
|
+
This is the same policy used internally by the provider to decide between:
|
|
442
|
+
|
|
443
|
+
- reloading assets
|
|
444
|
+
- only re-registering graphs/animations/programs
|
|
445
|
+
- doing nothing
|
|
446
|
+
|
|
447
|
+
## Common Patterns
|
|
448
|
+
|
|
449
|
+
### Multiple faces in one app
|
|
450
|
+
|
|
451
|
+
Mount one `VizijRuntimeProvider` per face; each owns its device. Give hidden or non-driver faces `driveRuntime={false}` and step them at a background cadence (see `vizij-showcase`'s `HiddenStepController`).
|
|
452
|
+
|
|
453
|
+
### Bundle-first player
|
|
454
|
+
|
|
455
|
+
See [`apps/demo-vizij-player`](../../../apps/demo-vizij-player/README.md) for the reference “one bundled GLB in, runtime UI out” flow.
|
|
456
|
+
|
|
457
|
+
### Fullscreen face tutorials
|
|
458
|
+
|
|
459
|
+
See:
|
|
460
|
+
|
|
461
|
+
- [`apps/tutorial-fullscreen-face/tutorial.md`](../../../apps/tutorial-fullscreen-face/tutorial.md)
|
|
462
|
+
- [`apps/tutorial-agent-face/tutorial.md`](../../../apps/tutorial-agent-face/tutorial.md)
|
|
463
|
+
|
|
464
|
+
### Runtime-truthful authoring
|
|
465
|
+
|
|
466
|
+
See [`apps/vizij-authoring/README.md`](../../../apps/vizij-authoring/README.md) for the `setGraphBundle()` and `transformOutputWrite()` tooling workflow.
|
|
467
|
+
|
|
468
|
+
## Development
|
|
469
|
+
|
|
470
|
+
```bash
|
|
471
|
+
pnpm --filter "@vizij/runtime-react" build
|
|
472
|
+
pnpm --filter "@vizij/runtime-react" test
|
|
473
|
+
pnpm --filter "@vizij/runtime-react" typecheck
|
|
474
|
+
pnpm --filter "@vizij/runtime-react" lint
|
|
475
|
+
pnpm --filter "@vizij/runtime-react" dev
|
|
476
|
+
```
|
|
176
477
|
|
|
177
|
-
When
|
|
478
|
+
When you change runtime behavior, validate at least one bundle-first app and one shared-runtime app. In this repo the fastest pair is usually:
|
|
178
479
|
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
3. Build, test, and pack the runtime filter (`pnpm --filter "@vizij/runtime-react" build`, `pnpm --filter "@vizij/runtime-react" test`, `pnpm --filter "@vizij/runtime-react" exec npm pack --dry-run`).
|
|
182
|
-
4. Push a tag named `npm-runtime-react-vX.Y.Z`. The shared GitHub Action handles `npm publish`.
|
|
480
|
+
- `demo-vizij-player`
|
|
481
|
+
- `vizij-showcase` or `vizij-authoring`
|