@hypersoniclabs/helix-mcp 0.2.5 → 0.2.12
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +81 -11
- package/dist/continuumCanary.d.ts +17 -0
- package/dist/continuumCanary.js +17 -0
- package/dist/continuumCanary.js.map +1 -0
- package/dist/server.d.ts +14 -1
- package/dist/server.js +4058 -162
- package/dist/server.js.map +1 -1
- package/dist/tsconfig.build.tsbuildinfo +1 -1
- package/dist/vehicleTools.d.ts +86 -0
- package/dist/vehicleTools.js +229 -0
- package/dist/vehicleTools.js.map +1 -0
- package/docs/avatar-face.md +115 -0
- package/docs/bridge.md +98 -0
- package/docs/bring-your-world.md +117 -0
- package/docs/catalog.md +69 -1
- package/docs/character-animation.md +442 -0
- package/docs/character-attachments.md +166 -0
- package/docs/character-world.md +785 -130
- package/docs/continuum.md +153 -0
- package/docs/items.md +73 -0
- package/docs/lighting-world.md +667 -0
- package/docs/locomotion-clip-spec.md +294 -0
- package/docs/manifest.md +31 -6
- package/docs/multiplayer-logic.md +460 -17
- package/docs/multiplayer-templates/chrono-orchard.md +36 -22
- package/docs/multiplayer-templates/collect-a-thon.md +28 -38
- package/docs/multiplayer-templates/collections.md +24 -24
- package/docs/multiplayer-templates/hangout.md +124 -111
- package/docs/multiplayer-templates/npc-wave.md +310 -0
- package/docs/multiplayer-templates/obby.md +13 -16
- package/docs/multiplayer-templates/persistent-progress.md +218 -0
- package/docs/multiplayer-templates/physics-bumper.md +27 -9
- package/docs/multiplayer-templates/physics-football.md +22 -8
- package/docs/multiplayer-templates/relic-bearers.md +12 -15
- package/docs/multiplayer-templates/server-motion.md +16 -19
- package/docs/multiplayer-templates/shooter-range.md +275 -0
- package/docs/multiplayer-templates/team-control.md +28 -15
- package/docs/multiplayer-templates/turn-arena.md +31 -22
- package/docs/multiplayer-templates/voice-radio.md +166 -0
- package/docs/multiplayer-templates/wave-survival.md +7 -8
- package/docs/multiplayer-templates/world-shop.md +240 -0
- package/docs/multiplayer-world.md +222 -129
- package/docs/npc-world.md +623 -0
- package/docs/publishing.md +108 -28
- package/docs/purchases.md +223 -0
- package/docs/scene-performance.md +64 -0
- package/docs/screenshots.md +140 -0
- package/docs/sdk.md +324 -5
- package/docs/shooter-worlds.md +537 -0
- package/docs/terrain.md +173 -0
- package/docs/upgrades.md +324 -0
- package/docs/vehicles.md +727 -0
- package/docs/world-inspect.md +156 -0
- package/docs/world-look.md +241 -0
- package/docs/world-recipe.md +65 -6
- package/package.json +15 -4
- package/skills/README.md +91 -0
- package/skills/helix-assets/SKILL.md +491 -0
- package/skills/helix-assets/references/asset-sources.md +143 -0
- package/skills/helix-assets/references/vault-api.md +105 -0
- package/skills/helix-avatar-qa/SKILL.md +85 -0
- package/skills/helix-avatars/SKILL.md +206 -0
- package/skills/helix-avatars/references/contract.md +166 -0
- package/skills/helix-avatars/references/dynamics.md +367 -0
- package/skills/helix-avatars/references/face.md +50 -0
- package/skills/helix-avatars/references/publish.md +76 -0
- package/skills/helix-avatars/references/qa.md +251 -0
- package/skills/helix-avatars/references/rigging.md +88 -0
- package/skills/helix-avatars/references/source-generated.md +190 -0
- package/skills/helix-avatars/references/source-model.md +90 -0
- package/skills/helix-avatars/references/source-rigid.md +90 -0
- package/skills/helix-avatars/references/source-vrm.md +61 -0
- package/skills/helix-gauntlet/SKILL.md +128 -0
- package/skills/helix-multiplayer/SKILL.md +150 -0
- package/skills/helix-multiplayer/references/dsl-capability-map.md +107 -0
- package/skills/helix-vehicles/SKILL.md +218 -0
- package/skills/helix-vehicles/references/addons.md +212 -0
- package/skills/helix-vehicles/references/appearance.md +339 -0
- package/skills/helix-vehicles/references/audio-import.md +138 -0
- package/skills/helix-vehicles/references/audio.md +580 -0
- package/skills/helix-vehicles/references/cabin.md +225 -0
- package/skills/helix-vehicles/references/host-manifest.md +174 -0
- package/skills/helix-vehicles/references/physics-from-specs.md +463 -0
- package/skills/helix-vehicles/references/publish.md +214 -0
- package/skills/helix-vehicles/references/qa.md +177 -0
- package/skills/helix-vehicles/references/reference-package.json +3481 -0
- package/skills/helix-vehicles/references/reference-package.md +69 -0
- package/skills/helix-vehicles/references/source-beamng.md +167 -0
- package/skills/helix-vehicles/references/source-concept.md +38 -0
- package/skills/helix-vehicles/references/source-model.md +100 -0
- package/skills/helix-vehicles/references/source-scratch.md +60 -0
- package/skills/helix-world-build/SKILL.md +376 -0
- package/skills/helix-world-build/references/config-gates.md +104 -0
- package/skills/helix-world-director/SKILL.md +210 -0
- package/skills/helix-world-qa/SKILL.md +371 -0
- package/skills/helix-world-qa/references/perf-budgets.md +240 -0
- package/skills/helix-world-qa/references/perf-handle.md +125 -0
- package/skills/helix-world-qa/references/visual-scorecard.md +85 -0
package/docs/catalog.md
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# Platform catalog —
|
|
1
|
+
# Platform catalog — packages and Vault assets
|
|
2
2
|
|
|
3
3
|
HELIX Instant publishes reusable **packages** that worlds consume, distinct from worlds themselves:
|
|
4
4
|
|
|
@@ -21,6 +21,74 @@ HELIX Instant publishes reusable **packages** that worlds consume, distinct from
|
|
|
21
21
|
3. The response also carries `codeBaseUrl` (the downloadable code) and `assetBaseUrl` (the CDN the
|
|
22
22
|
code streams animations/textures from). Assets are never embedded in a consumer's bundle.
|
|
23
23
|
|
|
24
|
+
## Vault assets
|
|
25
|
+
|
|
26
|
+
Vault is the durable resource catalog for things a world composes: `prop`, `character`, `animation`,
|
|
27
|
+
`audio`, `texture`, `material`, `environment`, `terrain`, `vfx`, `decal`, `sky`,
|
|
28
|
+
`gaussian_splat`, `scene`, `interactive_item` (an executable definition: subtype `emote` for a global
|
|
29
|
+
emote, or a weapon/other interaction — see `read_doc({ name: "shooter-worlds" })`), and
|
|
30
|
+
`unreal_package`. Raw reusable emote motion remains kind `animation`; the executable, ownable wrapper
|
|
31
|
+
is `interactive_item` with subtype `emote`. A URL is a resolved rendition; the UUID
|
|
32
|
+
returned as `assetId` is the handle a world or agent keeps.
|
|
33
|
+
|
|
34
|
+
Use this ladder:
|
|
35
|
+
|
|
36
|
+
1. `list_materials` for the small platform-default surface palette, then `use_material` for one
|
|
37
|
+
material's map URLs. Both take an optional `resolution` (e.g. `"1k"`, `"2k"`; `1024`/`2048` are
|
|
38
|
+
accepted aliases) to pick which texture resolution you get; omit it for the material's default.
|
|
39
|
+
The output names the resolved resolution and every one the material offers. A resolution the
|
|
40
|
+
material does not carry is an error listing the valid values — it is never silently swapped for
|
|
41
|
+
a different one, so a `2k` request that returns `1k` maps cannot happen. Only photographed or
|
|
42
|
+
scanned materials are offered: procedural glass and water are not, the listing names them under
|
|
43
|
+
`unlisted`, and `use_material` on one is an error saying why. Apply a material with the visual
|
|
44
|
+
system (`read_doc({ name: "world-look" })`), using the `assetBaseUrl` in `use_material`'s output.
|
|
45
|
+
2. `search_assets` with typed filters. For a browser world, normally include `engine: "web"`.
|
|
46
|
+
Add `origin: "generated"` when the trust boundary matters, and `official: true` to narrow to the
|
|
47
|
+
platform-vouched tier first (widen only when it has no fit). Each result carries separate
|
|
48
|
+
`selectionExplanation` components for lexical matches, hard filters, measured performance,
|
|
49
|
+
observed reuse, and unavailable signals. Do not collapse those into an invented score.
|
|
50
|
+
3. `get_asset` for the complete typed metadata and resolved artifact; `list_asset_versions` for
|
|
51
|
+
immutable history; `track_asset` for observed world use.
|
|
52
|
+
4. `install_asset` to download the served version and related artifacts into the world. The CLI
|
|
53
|
+
verifies SHA-256 and byte count before writing `public/helix.assets.json`. For materials,
|
|
54
|
+
`materialRenditions` defaults to `runtime` (lean KTX2 maps); choose `source` for editable PNG
|
|
55
|
+
maps or `all` only when both families are required. Non-material installs are unchanged.
|
|
56
|
+
5. **USE the installed artifact according to its kind — install alone renders nothing.** Props and
|
|
57
|
+
characters need the world's KTX2-aware GLB loader before `scene.add`. Animations are reusable
|
|
58
|
+
clips: bind them to a compatible character with `THREE.AnimationMixer`; never `scene.add` an
|
|
59
|
+
animation. An `interactive_item` is a descriptor, not geometry. A subtype `emote` wrapper that is
|
|
60
|
+
published as a global owned emote is inventory-driven and should not be manually installed into a
|
|
61
|
+
world. There is currently no public `loadVaultAsset`/`listVaultAssets` helper; do not import one.
|
|
62
|
+
|
|
63
|
+
For emotes, use two explicit discovery queries:
|
|
64
|
+
|
|
65
|
+
- reusable motion for a generic world: `search_assets({ kinds: ["animation"], skeleton: "humanoid" })`
|
|
66
|
+
- executable/ownable global emotes: `search_assets({ kinds: ["interactive_item"], subtype: "emote" })`
|
|
67
|
+
|
|
68
|
+
The wrapper metadata says `joinMode` (`partner`, `sync`, or `none`), playback, participants, roles,
|
|
69
|
+
phases, authored alignment, portability, and ownership requirements. A collectible changes who may
|
|
70
|
+
equip/initiate it; a nearby participant may still join without ownership.
|
|
71
|
+
6. Generate only when search found nothing appropriate. `start_asset_generation` returns a job id;
|
|
72
|
+
record it and call `poll_asset_generation` instead of starting another job after a disconnect.
|
|
73
|
+
|
|
74
|
+
Generation routes remain explicit:
|
|
75
|
+
|
|
76
|
+
- Prop and character use main Dreamer because their final artifact passes the universal-item mesh
|
|
77
|
+
lane.
|
|
78
|
+
- Image, material, audio, and Gaussian splat use the shared asset broker. Materials are validated
|
|
79
|
+
PBR bundles (1K default, explicit 2K), and splats declare `object` or `environment`.
|
|
80
|
+
- Animation generation is unavailable and fails before a chargeable job is created. Search for or
|
|
81
|
+
upload a skeleton-gated animation instead.
|
|
82
|
+
|
|
83
|
+
For text-to-speech, call `list_voices` before `generate_audio`. Choose a returned voice id and one
|
|
84
|
+
of its supported language codes; follow `pageInfo.nextPageToken` for more results. HELIX exposes
|
|
85
|
+
only safe selection metadata and previews, never vendor credentials, cloning, or voice
|
|
86
|
+
administration.
|
|
87
|
+
|
|
88
|
+
A successful default-on job reports `vaultAssetId` and `vaultAutoPublish: "published"`. An explicit
|
|
89
|
+
creator preference can return `"disabled"`. Provider/deployment absence is an error, never a
|
|
90
|
+
placeholder success.
|
|
91
|
+
|
|
24
92
|
## What you cannot do here
|
|
25
93
|
|
|
26
94
|
Publishing systems/abilities/asset-packs is **internal/first-party only** — there is no publish tool
|
|
@@ -0,0 +1,442 @@
|
|
|
1
|
+
# Character animation — author gestures as JSON (the pose DSL)
|
|
2
|
+
|
|
3
|
+
Read this only when a world needs CUSTOM character animation. Most worlds never do: the humanoid-character
|
|
4
|
+
system already ships locomotion, jumps and the standard emote set. Reach for the pose DSL when you want a
|
|
5
|
+
gesture the platform does not provide — a wave, a salute, a world-specific ritual — without exporting a GLB.
|
|
6
|
+
|
|
7
|
+
Worlds define and play their own character gestures as **JSON keyframe poses** — no GLB export, no
|
|
8
|
+
platform publish. Registration validates everything at once; playback replicates by default in
|
|
9
|
+
multiplayer (see the multiplayer-world doc — oneshots relay, loops sync so late joiners see them).
|
|
10
|
+
|
|
11
|
+
**Prerequisite — the world's INSTALLED humanoid-character must be ≥ 0.2.12** (where the gesture DSL was
|
|
12
|
+
added). On a world you did not scaffold this session, run `check_for_updates({ projectDir })` FIRST: an
|
|
13
|
+
older install shows as in-range-stale, and the fix is just `install_world_packages` + rebuild — any
|
|
14
|
+
existing `^0.2` pin already allows 0.2.12, so no helix.json edit is needed. For gestures to REPLICATE
|
|
15
|
+
(not just play locally) the same check also flags a stale project SDK — update it and rebuild too.
|
|
16
|
+
Compile-time `ik` keys and runtime IK (both below) need **≥ 0.2.15**: on an older install an `ik` field is
|
|
17
|
+
silently ignored when the key also has `bones`, so the tools refuse it with a version message — same fix,
|
|
18
|
+
`install_world_packages` + rebuild. **After any install that moves the system version, RESTART the dev
|
|
19
|
+
server and this MCP session** — a running process keeps serving the previously loaded module, which reads
|
|
20
|
+
as "ik keys don't evaluate" long after the upgrade. Prefer **≥ 0.2.18** for `ik` authoring — each older
|
|
21
|
+
tier has a known, fixed solver defect that upgrading beats hand-tuning around: 0.2.15 rolls the forearm
|
|
22
|
+
on chest-front goals (no humerus roll); 0.2.16–0.2.17 mangle DEEP elbow folds (close-to-shoulder / sip
|
|
23
|
+
range: `[±160, −80s, ∓160]` eulers + clamp warnings + an off-target hand) and silently ignore `item`
|
|
24
|
+
intents. The tools gate `item` on the installed system's capability.
|
|
25
|
+
|
|
26
|
+
## SCOPE — what this tool is and is not for (read before you start)
|
|
27
|
+
|
|
28
|
+
This is for **short, simple, upper-body-led motion**. It is NOT an animation authoring suite, and trying to
|
|
29
|
+
build complex full-body choreography with it wastes a lot of your effort for a poor result.
|
|
30
|
+
|
|
31
|
+
**Good fits — these land quickly, often first try:**
|
|
32
|
+
|
|
33
|
+
- Player actions: wave, salute, point, sip, knock, throw a lever, offer an item.
|
|
34
|
+
- Tool and weapon actions: a mining swing, a one-handed melee strike, a chop, a hammer blow.
|
|
35
|
+
- Stances and held poses: hurt/clutching, carrying, cold/shivering, slumped, alert.
|
|
36
|
+
- One-shot accents: nod, shrug, flinch, headshake, recoil, a wince.
|
|
37
|
+
- Additive overlays that ride locomotion: a limp, a hunch, a drunken sway, a breathing idle.
|
|
38
|
+
|
|
39
|
+
**Bad fits — do NOT attempt these here:**
|
|
40
|
+
|
|
41
|
+
- **Dances.** Real choreography needs weight transfer, footwork and full-body timing; it is normally
|
|
42
|
+
motion-captured. A simple 4-beat arm routine is reachable, anything more is not.
|
|
43
|
+
- **Fight choreography and acrobatics** — combos, spins, ninja jumps, rolls, flips, sweeps.
|
|
44
|
+
- **Anything where the feet must step, plant, pivot, or leave the ground.**
|
|
45
|
+
- **Anything needing the character to travel through space.**
|
|
46
|
+
|
|
47
|
+
**Why, concretely — these are hard limits, not tuning problems:** a gesture is *bone rotations only*,
|
|
48
|
+
layered over the locomotion system. There is **no root motion** (the character cannot move), **no foot
|
|
49
|
+
planting or leg IK** (feet slide or float if you rotate the legs), and **no weight shift**, because the
|
|
50
|
+
locomotion graph owns the lower body. (The arm `ik` keys below solve HAND placement — they change none of
|
|
51
|
+
this.) Add that joint-space posing has no notion of balance, and a convincing dance or fight move is
|
|
52
|
+
simply outside what the format represents.
|
|
53
|
+
|
|
54
|
+
**The signal you have gone too far:** you are keying legs *and* arms *and* torso to hit the same beat, you
|
|
55
|
+
are past ~6 keys of full-body coordination, or you are on your third visual iteration and it still reads
|
|
56
|
+
wrong. Stop and pick a different approach.
|
|
57
|
+
|
|
58
|
+
**What to use instead:** the platform's shipped locomotion and emote clips, or an authored/motion-captured
|
|
59
|
+
GLB clip. Text-to-animation generation is planned as a separate API for exactly this gap — when a world
|
|
60
|
+
needs a real dance or fight move, that is the route, not this DSL.
|
|
61
|
+
|
|
62
|
+
## Put each clip in its OWN .json file — never inline them in world code
|
|
63
|
+
|
|
64
|
+
**This is not a style preference.** A single clip runs to 100–300 lines of bone values; three of them inlined
|
|
65
|
+
in `main.ts` triples the file and buries the world's actual logic in coordinate soup. Worse for you: every
|
|
66
|
+
later edit to that file forces you to re-read poses you are not touching, so a one-line gameplay change
|
|
67
|
+
costs you the whole gesture library in context. Keep them out of the way:
|
|
68
|
+
|
|
69
|
+
```
|
|
70
|
+
src/
|
|
71
|
+
main.ts ← stays about the world
|
|
72
|
+
gestures/
|
|
73
|
+
wave.json
|
|
74
|
+
hurt-belly.json
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
```js
|
|
78
|
+
import wave from './gestures/wave.json';
|
|
79
|
+
import hurtBelly from './gestures/hurt-belly.json';
|
|
80
|
+
|
|
81
|
+
for (const clip of [wave, hurtBelly]) mp.registerAnimation(clip);
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
Vite inlines the JSON at build, so this costs nothing at runtime. When you iterate on a pose, you edit ONE
|
|
85
|
+
small file and re-read only that, and `inspect_gesture` measures that file directly — no build required.
|
|
86
|
+
The `multiplayer-hangout` template does exactly this.
|
|
87
|
+
|
|
88
|
+
The worked one-shot, `gestures/wave.json`. Solved by MEASUREMENT, not by eye: `upperarm x=-60` puts the
|
|
89
|
+
elbow's swing plane frontal, so the elbow hinge alone sweeps the hand sideways with no vertical drift;
|
|
90
|
+
shoulder+clavicle add a smaller same-direction swing and LEAD by 50 ms via their own key times, so the chain
|
|
91
|
+
overlaps instead of snapping together. Fingers key BOTH knuckle rows (`*_01` + `*_02`) — unkeyed fingers
|
|
92
|
+
keep the rig's rest curl and read as a half-fist:
|
|
93
|
+
|
|
94
|
+
```json
|
|
95
|
+
{
|
|
96
|
+
"name": "wave", "duration": 1.6, "play": "oneshot", "compose": "override", "mask": "rightArm",
|
|
97
|
+
"keys": [
|
|
98
|
+
{ "t": 0, "ease": "ease", "bones": { "clavicle_r": [0,0,0], "upperarm_r": [0,0,0], "lowerarm_r": [0,0,0], "hand_r": [0,0,0],
|
|
99
|
+
"index_01_r": [0,0,0], "index_02_r": [0,0,0], "middle_01_r": [0,0,0], "middle_02_r": [0,0,0], "ring_01_r": [0,0,0],
|
|
100
|
+
"ring_02_r": [0,0,0], "pinky_01_r": [0,0,0], "pinky_02_r": [0,0,0], "thumb_01_r": [0,0,0], "thumb_02_r": [0,0,0] } },
|
|
101
|
+
{ "t": 0.3, "ease": "ease", "bones": { "clavicle_r": [0,0,15.5], "upperarm_r": [-60,0,28] } },
|
|
102
|
+
{ "t": 0.35, "ease": "ease", "bones": { "lowerarm_r": [0,-88,0], "hand_r": [-10,0,0],
|
|
103
|
+
"index_01_r": [0,-12,0], "index_02_r": [0,-5,0], "middle_01_r": [0,-12,0], "middle_02_r": [0,-5,0], "ring_01_r": [0,-12,0],
|
|
104
|
+
"ring_02_r": [0,-5,0], "pinky_01_r": [0,-12,0], "pinky_02_r": [0,-5,0], "thumb_01_r": [0,-25,0], "thumb_02_r": [0,-5,0] } },
|
|
105
|
+
{ "t": 0.6, "ease": "ease", "bones": { "clavicle_r": [0,0,12.5], "upperarm_r": [-60,0,22] } },
|
|
106
|
+
{ "t": 0.65, "ease": "ease", "bones": { "lowerarm_r": [0,-52,0] } },
|
|
107
|
+
{ "t": 0.9, "ease": "ease", "bones": { "clavicle_r": [0,0,15.5], "upperarm_r": [-60,0,28] } },
|
|
108
|
+
{ "t": 0.95, "ease": "ease", "bones": { "lowerarm_r": [0,-88,0] } },
|
|
109
|
+
{ "t": 1.2, "ease": "ease", "bones": { "clavicle_r": [0,0,12.5], "upperarm_r": [-60,0,22] } },
|
|
110
|
+
{ "t": 1.25, "ease": "ease", "bones": { "lowerarm_r": [0,-52,0], "hand_r": [-10,0,0],
|
|
111
|
+
"index_01_r": [0,-12,0], "index_02_r": [0,-5,0], "middle_01_r": [0,-12,0], "middle_02_r": [0,-5,0], "ring_01_r": [0,-12,0],
|
|
112
|
+
"ring_02_r": [0,-5,0], "pinky_01_r": [0,-12,0], "pinky_02_r": [0,-5,0], "thumb_01_r": [0,-25,0], "thumb_02_r": [0,-5,0] } },
|
|
113
|
+
{ "t": 1.6, "ease": "ease", "bones": { "clavicle_r": [0,0,0], "upperarm_r": [0,0,0], "lowerarm_r": [0,0,0], "hand_r": [0,0,0],
|
|
114
|
+
"index_01_r": [0,0,0], "index_02_r": [0,0,0], "middle_01_r": [0,0,0], "middle_02_r": [0,0,0], "ring_01_r": [0,0,0],
|
|
115
|
+
"ring_02_r": [0,0,0], "pinky_01_r": [0,0,0], "pinky_02_r": [0,0,0], "thumb_01_r": [0,0,0], "thumb_02_r": [0,0,0] } }
|
|
116
|
+
]
|
|
117
|
+
}
|
|
118
|
+
```
|
|
119
|
+
|
|
120
|
+
```js
|
|
121
|
+
import wave from './gestures/wave.json';
|
|
122
|
+
mp.registerAnimation(wave); // → { warnings } ; throws with EVERY issue listed at once
|
|
123
|
+
mp.playAnimation('wave'); // local player, replicated by default
|
|
124
|
+
mp.playAnimation('sway', { loop: true }); // synced loop — late joiners see it
|
|
125
|
+
mp.stopAnimation(); // fade out + clear the replicated loop
|
|
126
|
+
```
|
|
127
|
+
|
|
128
|
+
The rules that make gestures work:
|
|
129
|
+
|
|
130
|
+
- **Every value is REST-RELATIVE**: `[x°, y°, z°]` Euler degrees, composed twist-first (internally 'ZYX' —
|
|
131
|
+
X applies first, so the swing carries the twist), and `[0,0,0]` = the character's rest
|
|
132
|
+
pose (a relaxed A-pose, arms ~55° down) for EVERY bone in BOTH modes. `compose: "override"` replaces
|
|
133
|
+
the animation on keyed bones (mask required — it is the conflict unit: a new override fades out any
|
|
134
|
+
overlapping one); `compose: "additive"` stacks deltas on top of whatever plays (nod over walk).
|
|
135
|
+
Unkeyed mask bones keep the underlying animation. Twist bones cannot be keyed.
|
|
136
|
+
- Masks: `upperBody`, `lowerBody`, `leftArm`, `rightArm`, `spine`, `armsHead` — also usable as `overrideMask`
|
|
137
|
+
to mix compose modes in one clip (below). Caps: ≤ 24 keys, ≤ 10 s,
|
|
138
|
+
names `[A-Za-z0-9][A-Za-z0-9_\-.:]*` ≤ 64. `ease` per key: `linear` | `ease` | `hold` (toward the
|
|
139
|
+
next key); loops wrap last→first continuously.
|
|
140
|
+
- **Trigger from input/interaction handlers** (they run only on the acting player), never from
|
|
141
|
+
replicated-state observers (they run on every client — everyone would gesture).
|
|
142
|
+
|
|
143
|
+
**The axis reference — read it with `gesture_axis_table`, do not guess.** Every bone group has a verified
|
|
144
|
+
human-terms meaning per Euler axis plus an advisory natural range; outside that range the compiler WARNS
|
|
145
|
+
(still compiles) because that is where poses start looking broken. **A warning means "look at this", not
|
|
146
|
+
"wrong"** — the ranges were validated by grid-searching a real gesture against measured criteria: the
|
|
147
|
+
unconstrained best `upperarm_r x` was −70, and clamping to the natural −60 cost 1.7% of pose quality while
|
|
148
|
+
the range still correctly rejected the far tail. Stay inside them unless measurement says otherwise.
|
|
149
|
+
|
|
150
|
+
The table is NOT reproduced here on purpose — it lives in one place, the character system itself, so what
|
|
151
|
+
you read is what YOUR pinned version actually lints against. Two ways to get it, no shell needed:
|
|
152
|
+
**`gesture_axis_table({ world })`** for the whole table (call it BEFORE picking rotation values — guessing
|
|
153
|
+
costs a revise cycle per bone), and `inspect_gesture` prints the rows for the bones your clip keys alongside
|
|
154
|
+
every warning, which is when you need them again.
|
|
155
|
+
|
|
156
|
+
The shape worth memorising: **X twists along the bone; −Y is the "bend" family (elbow, knee, bow, nod);
|
|
157
|
+
+Z is the lateral plane (raise, split, tilt, shrug)**. Same values on `_l`/`_r` bones = mirrored motion.
|
|
158
|
+
And the rule that overrides all of it: **these meanings hold FROM REST ONLY.** Rotations chain, so once a
|
|
159
|
+
parent has rotated, a child's "raises the arm" axis points wherever the parent carried it — that is what
|
|
160
|
+
`inspect_gesture` with `axes: true` measures, and why a value that "should" work sometimes does not.
|
|
161
|
+
|
|
162
|
+
**Worked MIXED example — `hurt-belly`** (loop 1.2 s, `compose: 'additive'` + `overrideMask: 'rightArm'`):
|
|
163
|
+
the bow rides the locomotion while the arm ignores its swing. Fist clutched across the abdomen, head
|
|
164
|
+
down, spine bowed, slow wince. `spine_02/_03 [0,−12,0]`, `neck_01 [0,−10,0]`, `head [0,−25,0]`,
|
|
165
|
+
`clavicle_r [0,−8,0]`, `upperarm_r [60,−10,−20]`, `lowerarm_r [0,−60,0]`, fingers fisted (`*_01 y 75..78`,
|
|
166
|
+
`*_02 y 82..88`, `thumb_01 [0,40,0]`); deepen everything ~3–5° at t=0.6 and let the loop wrap back.
|
|
167
|
+
Measured: forearm near-horizontal across the body, hand ~4 cm clear of the belly, fingertips closed 50–60%,
|
|
168
|
+
face 59° below horizontal. Composes over `walk` — legs keep striding, the torso bow rides the walk, the fist stays put.
|
|
169
|
+
|
|
170
|
+
Practical recipes: arm overhead ≈ `upperarm z 145`; hanging at the side ≈ `z −35`; hand-to-mouth (mug hold)
|
|
171
|
+
≈ `clavicle [0,−10,0]` + `upperarm [−15,−35,5]` + `lowerarm [0,−108,0]` + `hand [0,0,40]` + curled fingers;
|
|
172
|
+
sit ≈ `thigh [0,90,0]` + `calf [0,−90,0]`; bow ≈ distribute across the spine (`spine_02/_03 [0,−20,0]` each —
|
|
173
|
+
never one bone at −40) + `head [0,−50,0]`.
|
|
174
|
+
|
|
175
|
+
**Timing and structure — the rules that bite on ACTIONS (tool swings, strikes, gestures):**
|
|
176
|
+
|
|
177
|
+
- **A one-shot and a loop of the same action are DIFFERENT clips.** A one-shot must start AND end at rest,
|
|
178
|
+
or it snaps when it finishes. A loop must contain NO rest pose, or the character returns to neutral
|
|
179
|
+
between repetitions and the action stutters. Do not play a one-shot on loop and expect a repeating
|
|
180
|
+
action: author the loop separately, keeping the same poses and timings and dropping the rest bookends —
|
|
181
|
+
the wrap segment (last key back to the first) carries the recovery.
|
|
182
|
+
- **Punch comes from key SPACING, not from `hold`.** `hold` means "do not interpolate to the next key", so
|
|
183
|
+
across a large movement it TELEPORTS. Give the strike a short segment (0.12–0.16s) and the recovery a
|
|
184
|
+
longer one (0.3s+); keep `ease` throughout. Reserve `hold` for small accents and for freezing a grip
|
|
185
|
+
through a dwell.
|
|
186
|
+
- **A swing needs the TORSO, not just the arm.** Coil the spine one way in the wind-up and uncoil it
|
|
187
|
+
through the strike (`spine_02/_03` x, ±15–20), with the other arm counter-swinging. An arm swinging alone
|
|
188
|
+
reads as a flail.
|
|
189
|
+
- **Two-handed grips: author BOTH hands as `ik` goals on the implied handle, then CHECK THE SPACING.**
|
|
190
|
+
Give each hand a position goal a fixed offset apart along the handle instead of placing two independent
|
|
191
|
+
joint-space arms and hoping they meet. The verification still stands: measure hand-to-hand distance at
|
|
192
|
+
EVERY key; it must stay roughly constant or the handle visibly stretches. The shipped pickaxe swing holds
|
|
193
|
+
20.6 / 21.5 / 17.8 cm across wind-up, strike and follow-through.
|
|
194
|
+
|
|
195
|
+
**Composition is where authoring fails — respect these rules:**
|
|
196
|
+
|
|
197
|
+
- **ROTATIONS CHAIN — the table is only valid near rest.** The biggest trap, and the reason blind authoring
|
|
198
|
+
fails. Every axis meaning was probed with the parents at bind pose; once a parent rotates it carries the
|
|
199
|
+
child's frame with it, so "raises the arm out/up" or "waves the fingers inward" mean *different world
|
|
200
|
+
motions* at a posed shoulder. A value that was right at rest is wrong the moment anything above it moves.
|
|
201
|
+
**Never author a large multi-joint pose from the table alone — measure the posed chain** (tooling below).
|
|
202
|
+
Measured examples, all counter-intuitive:
|
|
203
|
+
- Rest is an **A-pose, arms down and slightly forward**, so `upperarm_r z ≈ 100` ("raise out/up") carries
|
|
204
|
+
the arm up *and backward* and parks the hand behind the head — invisible head-on, obvious from the side.
|
|
205
|
+
- At a ~95° raise the forearm **cannot be made to point up by twist at all** (swept the full 360°: best
|
|
206
|
+
vertical component 0.03). Elevation must come from the upper arm, not from folding the elbow.
|
|
207
|
+
- `hand_r` rotations **do not move the wrist position**, only its orientation — probe a fingertip. To
|
|
208
|
+
MOVE the wrist, use an `ik` key (below): a position goal is exactly what that is for.
|
|
209
|
+
- The table is exact only near rest and per-axis. Values COMPOSE (twist first, then Y, then Z): a raised
|
|
210
|
+
arm's **elbow-fold plane and palm** end up wherever the composition puts them — tune the upperarm X twist
|
|
211
|
+
*at the raised pose* until the elbow pit and palm face the intended way (not predictable from the table).
|
|
212
|
+
- **Move the CHAIN, not one joint** — a single-joint sweep reads robotic. Give neighbouring joints a smaller
|
|
213
|
+
same-direction contribution (verify the sign; it is often counter-intuitive) and their **own key times,
|
|
214
|
+
30–60 ms ahead** of the driving joint so the motion overlaps through the chain. Amplitudes shrink as you
|
|
215
|
+
move away from the driver. Worked example: the wave is **elbow-driven** (a real wave pivots at the elbow),
|
|
216
|
+
with the shoulder and clavicle adding ±3/±1.5 and leading by 50 ms.
|
|
217
|
+
- **Open hands must key the fingers.** Unkeyed fingers keep the rig's rest curl and read as a half-fist;
|
|
218
|
+
extend with negative Y (`*_01 y −12`, `*_02 y −5`, `thumb_01 y −25`).
|
|
219
|
+
- **Shoulder girdle participates**: raises past horizontal add `clavicle z +10..15`; forward reaches add
|
|
220
|
+
`clavicle y −10..15` — without it the shoulder looks dislocated.
|
|
221
|
+
- **Sparse tracks**: a bone interpolates between its OWN keys — to hold it still while others move, repeat
|
|
222
|
+
its value at the end of the hold (see the wave's fingers, repeated at t=1.25), or it sags.
|
|
223
|
+
- Head/spine/finger/leg gestures land first-try; **arm poses with position/orientation intent take 2–3
|
|
224
|
+
visual rounds in joint space — author those with `ik` keys instead (next block).** Either way, verify
|
|
225
|
+
close-up from the FRONT and the SIDE (forward-lean hides frontally), checking the palm and elbow-pit
|
|
226
|
+
directions specifically, and play it in motion before shipping.
|
|
227
|
+
|
|
228
|
+
**Position-intent arm poses: author the GOAL with an `ik` key, not a joint-space search (≥ 0.2.15).**
|
|
229
|
+
When a key's intent is "hand HERE, palm facing THAT" — touch the face, plant a hand on a rail, meet a
|
|
230
|
+
handle — state it directly and let registration solve the rotations:
|
|
231
|
+
|
|
232
|
+
```js
|
|
233
|
+
{ "t": 0.9, "ease": "hold", "ik": { "hand_r": { "position": [-0.05, 1.55, 0.14], "palm": "back", "elbow": "down" } } }
|
|
234
|
+
```
|
|
235
|
+
|
|
236
|
+
- `position` is CHARACTER-space meters (+x = the character's left, y up from the feet, +z forward). `palm`
|
|
237
|
+
hints the facing (`inward|outward|up|down|forward|back`); `elbow` the fold (`down|out|up` — omit for the
|
|
238
|
+
adaptive default, which wings cross-body reaches naturally). Both hands can carry goals in one key.
|
|
239
|
+
- **Holding a prop? Orient the ITEM, not the palm (`item`, ≥ 0.2.18).** The grip socket is the item frame
|
|
240
|
+
(+y = the item's up, +z = where it points): `item: { up: 'up' }` carries a mug level,
|
|
241
|
+
`item: { up: [0, 0.7, -0.7] }` tips it toward the mouth for a sip, `item: { point: 'forward' }` aims a
|
|
242
|
+
beam/tip. One axis per goal (direction word or character-space vector); mutually exclusive with `palm`
|
|
243
|
+
(both drive the wrist). The solve puts natural forearm pronation (≤30°) into the roll and the wrist
|
|
244
|
+
takes the rest, guard-capped. Verify with the `up=`/`itemUp` readouts — palm hints are TOUCH semantics
|
|
245
|
+
and cannot express "keep the cup level"; probing them for that is wasted turns.
|
|
246
|
+
- **When ik fights you, drop to joint space — that is the design, not a workaround.** ik states simple
|
|
247
|
+
intent (hand here, palm there, item level); an exact articulation you already know is a `bones` euler
|
|
248
|
+
key, and an ik `position` WITHOUT palm/item leaves `hand_*` free to euler-key in the same key. Mixing
|
|
249
|
+
the two in one key is the intended authoring style.
|
|
250
|
+
- **Reachable envelope (canonical rig):** each shoulder joint sits ≈1.44 m up and 0.19 m off center; the
|
|
251
|
+
arm chain is 0.278 m (upper) + 0.273 m (forearm) = **0.55 m shoulder→wrist**. A goal farther than that
|
|
252
|
+
from the shoulder cannot land — the solve clamps and `inspect_gesture` warns with the cm shortfall.
|
|
253
|
+
Useful heights: hips 0.96, chest 1.35, shoulders ≈1.44, chin ≈1.55.
|
|
254
|
+
- **`position` places the WRIST (the `hand_*` bone origin), not the fingertips.** The hand extends ≈15–19
|
|
255
|
+
cm beyond it (curl-dependent; palm center ≈7 cm). To touch a surface with the fingers, author the goal
|
|
256
|
+
≈10–15 cm short of the contact point — a goal ON the surface buries the hand in it.
|
|
257
|
+
- Solved ANALYTICALLY at registration into ordinary joint keys — deterministic, so every client compiles
|
|
258
|
+
the identical clip; playback and the wire never know ik existed.
|
|
259
|
+
- Rules: an ik-solved bone must be override (compose `override`, or inside `overrideMask`), must sit in
|
|
260
|
+
the mask, and a bone cannot be euler-keyed AND ik-solved in the same key. Author the key's OTHER bones
|
|
261
|
+
(torso lean, fingers) first — the solve runs against them, so a spine bend is compensated for free.
|
|
262
|
+
- The numeric loop is closed for you: `inspect_gesture` prints an **ik goals** section — the target, the
|
|
263
|
+
resolved joint eulers, and where the hand actually LANDS with its cm error — plus a `hands:` landmark
|
|
264
|
+
line per frame; out-of-reach targets and joint-limit clamps warn with `keys[N].ik` labels. Expect
|
|
265
|
+
first-try landings.
|
|
266
|
+
|
|
267
|
+
**Measure first, look second — and never trust one camera angle.** Screenshot-guessing an arm pose costs
|
|
268
|
+
~8 rounds even with vision; measuring got a full-body additive pose right in zero. `inspect_gesture` runs the real compiler and player against the real rig and
|
|
269
|
+
reports where the bones actually landed. Three modes, one per question you cannot see the answer to:
|
|
270
|
+
|
|
271
|
+
```
|
|
272
|
+
inspect_gesture({ clip, world, t: [0.35], bones: ["hand_r"] }) → where the bones ARE (+ palm/finger facing)
|
|
273
|
+
inspect_gesture({ clip, world, motion: true, → what the clip DOES over time
|
|
274
|
+
motionWindow: [0.35, 1.25] })
|
|
275
|
+
inspect_gesture({ clip, world, axes: true, probe: "hand_r" }) → what each keyed axis does AT this pose
|
|
276
|
+
```
|
|
277
|
+
|
|
278
|
+
- **`t`** — per bone: world position, direction down the bone, landmark English ("0.08m above head, 0.28m
|
|
279
|
+
to the character-right, 0.14m in front, points up"), plus **palm/finger facing** for hands.
|
|
280
|
+
- **`motion`** — path extent split lateral / vertical / depth, dominant axis, purity, reversals. **A static
|
|
281
|
+
frame cannot show this.** It is what catches a gesture moving along the wrong axis: the broken wave read
|
|
282
|
+
*"0.14m front-to-back"*, the fixed one *"0.21m side-to-side"*. Always pass `motionWindow`, or the
|
|
283
|
+
rest→pose→rest travel swamps the gesture itself.
|
|
284
|
+
- **`axes`** — for each keyed axis, the world motion a `+10°` edit produces **at the posed chain**: the
|
|
285
|
+
question the table cannot answer. Use it to find which axis does what you want BEFORE guessing values.
|
|
286
|
+
|
|
287
|
+
The inspect tools deliberately contain **no goal-seeking search** — they report measurements and never
|
|
288
|
+
judge. The sanctioned solver is the `ik` key above: analytic, at registration, for hand-position goals.
|
|
289
|
+
Hand-roll a search over the returned numbers only if a gesture needs something beyond that.
|
|
290
|
+
|
|
291
|
+
**The measure-first recipe** (this authored a full-body additive pose in ZERO visual rounds). Do not write a
|
|
292
|
+
clip, call it done, and hand it to a human to judge — that costs them a round trip per mistake and they
|
|
293
|
+
cannot tell you WHY it is wrong.
|
|
294
|
+
|
|
295
|
+
**The order is a rule, not a preference: measure until the numbers are right, THEN look, THEN hand it over.**
|
|
296
|
+
Steps 0–6 are `inspect_gesture` and cost milliseconds; step 7 is `capture_gesture` and costs a browser launch.
|
|
297
|
+
Looking first wastes the expensive call on a clip whose problems the numbers would have named for free.
|
|
298
|
+
|
|
299
|
+
0. **Compile it before anything else.** `inspect_gesture({ clip, world })` returns compile errors and
|
|
300
|
+
natural-range warnings, each quoting the axis meaning. A clip that warns on five bones is not ready to
|
|
301
|
+
look at. This step needs no `t`, no `motion` — just the clip.
|
|
302
|
+
1. **Probe the landmarks** — inspect any trivial clip with `bones: ["pelvis","spine_01","spine_02","head"]` to learn
|
|
303
|
+
where the body actually is on this rig. Do not assume.
|
|
304
|
+
2. **Measure the mesh** if clearance matters — bones say nothing about the body's SURFACE.
|
|
305
|
+
3. **Pose the TORSO FIRST, then solve the arm against it.** The load-bearing step: arms hang off the spine,
|
|
306
|
+
so an arm solved against an unbent torso is wrong the moment the spine bends. Same chaining rule, one
|
|
307
|
+
level up.
|
|
308
|
+
4. **Target relative to a landmark** (hand relative to `spine_02`), not absolute world space, so the target
|
|
309
|
+
survives the torso moving.
|
|
310
|
+
5. **Verify with `t` and `motion`.** Two traps: leaf bones like `head` have NO child, so `dir` is
|
|
311
|
+
`null` — take the world quaternion instead; and never measure the axis the rotation turns ABOUT (a nod
|
|
312
|
+
is about the head's local Y, so comparing local Y reads 0° and looks like the key did nothing).
|
|
313
|
+
6. **When a value that SHOULD work does not, measure the chain.** `axes: true` reports what a +10° edit to
|
|
314
|
+
each keyed axis does at the posed chain — the one question the table cannot answer, and usually the
|
|
315
|
+
answer: at a ~95° shoulder raise the forearm cannot be made to point up by twist at all, so elevation
|
|
316
|
+
has to come from the upper arm.
|
|
317
|
+
7. **Then LOOK at it yourself** — `capture_gesture({ clip, world })` renders the clip as a contact sheet
|
|
318
|
+
(front and one side, five times across it) and returns it as an image. Judge the silhouette and the limb
|
|
319
|
+
relationships. Read both rows: a limb displaced sideways overlaps the body in the SIDE view purely by
|
|
320
|
+
projection, so the front row shows lateral separation and the side row shows depth — that pair is what
|
|
321
|
+
catches an arm raised behind the head, which reads as a perfect wave from the front alone.
|
|
322
|
+
|
|
323
|
+
**`at` captures ONE moment instead of a sequence** — the two views then sit side by side at ~1.8x the
|
|
324
|
+
size. Reach for it when a human asks you to fix one specific thing: the whole image goes to that instant
|
|
325
|
+
instead of a fifth of it, which is the difference between guessing at a hand's orientation and reading it.
|
|
326
|
+
|
|
327
|
+
**You almost never need `views`,** because the default is chosen per mode and each is right for its job.
|
|
328
|
+
A sequence gets front + whichever side the clip actually keys (derived from its bones and mask, so a
|
|
329
|
+
left-handed gesture is never rendered from the flank its own torso hides) — the ORTHOGONAL pair, which is
|
|
330
|
+
what makes "moves side-to-side, not front-to-back" readable. A single moment gets front + `three-quarter`,
|
|
331
|
+
40° off the front on the clip's active side, because that lifts a limb CLEAR of the torso instead of
|
|
332
|
+
hiding it behind: a pure side view of a laterally displaced arm overlaps the head by projection and cannot
|
|
333
|
+
tell you in-front from behind. Override for a reach behind the body (`back`). Never pair front with back,
|
|
334
|
+
or left with right — each looks down ONE axis, so neither view shows depth; a `three-quarter` never has
|
|
335
|
+
that problem because it mixes both.
|
|
336
|
+
**What the render can and cannot answer.** Trust it for silhouette, which limb is where, joint bend
|
|
337
|
+
direction and gross orientation — the "does this read as a person doing X" judgement, which is the whole
|
|
338
|
+
reason to look. Do NOT read distances or which body part moved off it. Both are measurably unreliable:
|
|
339
|
+
in testing, an elbow sitting 0.20 m FORWARD of the torso was read as "tucked against the ribs", and a
|
|
340
|
+
wind-up where the TORSO had leaned 0.19 m back was read as "the arms are behind the head" — the gestalt
|
|
341
|
+
was right, the attribution wrong. Depth *magnitude* is the weak axis, and the `three-quarter` does not fix
|
|
342
|
+
it; it fixes in-front-versus-behind *ordering*. And a natural-range violation is invisible in a render
|
|
343
|
+
entirely — the `pickaxe-swing` sheet looks fine while breaking the shoulder's swing limit by 10°. So when
|
|
344
|
+
the image raises a question, go back to `inspect_gesture` for the answer, never look harder at the image.
|
|
345
|
+
|
|
346
|
+
8. **Only then hand it to a human**, saying what you measured. If 0–7 were done, their first look confirms
|
|
347
|
+
rather than corrects.
|
|
348
|
+
|
|
349
|
+
`capture_gesture` costs a headless browser launch — seconds, against milliseconds for `inspect_gesture`. So
|
|
350
|
+
it is the CLOSING check, once per clip, not the inner loop.
|
|
351
|
+
|
|
352
|
+
**Additive cannot guarantee CONTACT.** The base clip keeps moving the limb under your delta, so a pose that
|
|
353
|
+
touches the body at rest drifts off while walking. Floating poses tolerate it ("fist held across the belly"
|
|
354
|
+
survives the walk swing); "palm resting ON the stomach" needs `overrideMask` on that limb.
|
|
355
|
+
|
|
356
|
+
**Mixing both modes in ONE clip — `overrideMask`.** For a gesture that must ride the locomotion AND hold a
|
|
357
|
+
limb still:
|
|
358
|
+
|
|
359
|
+
```js
|
|
360
|
+
{ compose: 'additive', overrideMask: 'rightArm', keys: [{ t: 0, bones: {
|
|
361
|
+
spine_02: [0,-12,0], head: [0,-25,0], // additive — bow rides the walk
|
|
362
|
+
upperarm_r: [60,-10,-20], lowerarm_r: [0,-60,0], // override — arm ignores the walk's swing
|
|
363
|
+
} }] }
|
|
364
|
+
```
|
|
365
|
+
|
|
366
|
+
Keyed bones inside `overrideMask` override; the rest stay additive. Only valid with `compose: 'additive'`.
|
|
367
|
+
|
|
368
|
+
**Never hand-split this into two clips.** It looks right in single-player and is BROKEN in multiplayer: the
|
|
369
|
+
wire carries one loop slot per player, so two simultaneous looping clips replicate only whichever started
|
|
370
|
+
second — remote players and late joiners see half the gesture. Oneshots are unaffected.
|
|
371
|
+
|
|
372
|
+
**`weight` is not a substitute** and works oppositely per mode: on override it blends base→pose (a real
|
|
373
|
+
strength knob); on additive it scales the delta toward nothing, so lowering it gives MORE base motion.
|
|
374
|
+
|
|
375
|
+
**Then verify in motion**, from the FRONT and the SIDE, checking the palm and elbow-pit directions — a good
|
|
376
|
+
scrub frame can still swing wrong.
|
|
377
|
+
|
|
378
|
+
**Runtime IK is a separate thing from clips (≥ 0.2.15, replicates).** For LIVE world targets — press a wall
|
|
379
|
+
button, hold a door handle, glance at an object — do not author a clip at all: the character reaches with
|
|
380
|
+
its arm chain at runtime, and other players see it (the target point replicates; every client runs the
|
|
381
|
+
same solver locally).
|
|
382
|
+
|
|
383
|
+
```js
|
|
384
|
+
mp.local.ik.reach('right', buttonMesh, { press: true }); // one-off: touch, hold ~0.12 s, withdraw
|
|
385
|
+
const grab = mp.local.ik.reach('right', handleMesh, { engage: {} }); // holds while near + facing; auto re-grabs
|
|
386
|
+
mp.local.ik.lookAt(handleMesh); // interaction gaze; lookAt(null) clears
|
|
387
|
+
grab.release(); // when the interaction is over for good
|
|
388
|
+
```
|
|
389
|
+
|
|
390
|
+
`engage` takes `{ maxDistanceM, maxAngleDeg, from: 'shoulder'|'character' }` (default distance: arm reach
|
|
391
|
+
+ 0.25 m); `press` and `engage` are mutually exclusive. The facing arc (≥ 0.2.17) is the **reaching arm's
|
|
392
|
+
workspace, not a forward cone**: per arm the window runs from ~15° across the body midline to ~135° on
|
|
393
|
+
that arm's own side (`maxAngleDeg`, default 75, is the half-width around a center leaned 60° toward the
|
|
394
|
+
arm). Facing the target directly always engages, the same-side flank engages even somewhat behind the
|
|
395
|
+
shoulder line, cross-body and the deep back never do — so pick the hand on the target's side. The arc is
|
|
396
|
+
measured from the **character model's forward, never the camera** — in third person the camera orbits
|
|
397
|
+
independently, so a camera-based facing check is wrong; gate custom prompts with the character forward too
|
|
398
|
+
(`new THREE.Vector3(0,0,1).applyQuaternion(mp.local.model.quaternion)`, flattened horizontally — the
|
|
399
|
+
character-world doc has the full snippet).
|
|
400
|
+
|
|
401
|
+
**`orientMatch: true` adopts the target's world frame as the HAND-BONE frame, verbatim** (85° wrist-guard
|
|
402
|
+
cap) — nothing about it is palm-aware. The canonical hand-bone frame, measured: fingers run along local
|
|
403
|
+
**+x** (left hand; −x right) and the relaxed palm normal is ≈ local `[0.1, 0.5, -0.85]` (left; fully
|
|
404
|
+
negated right) — not a clean axis, so do NOT derive palm-on-surface rotations by hand. The working
|
|
405
|
+
pattern: place a helper `Object3D` at the contact point, calibrate its rotation by eye once, and reach
|
|
406
|
+
for the helper with `orientMatch: true`. Off (the default), the solve leaves the wrist to the
|
|
407
|
+
animation/grip — correct for engage-grabs of free props. Runtime reach orientation has no numeric
|
|
408
|
+
readback yet (`inspect_gesture` covers clips only) — verify orientMatch poses visually.
|
|
409
|
+
|
|
410
|
+
A press can gate ITSELF (≥ 0.2.16): `press: { when: {} }` takes the same envelope shape and defaults as
|
|
411
|
+
`engage`, evaluated once at the moment of the request — out of range or facing away, **nothing fires**
|
|
412
|
+
(no arm motion, no reach-through-the-body) and the returned handle reports `fired: false`:
|
|
413
|
+
|
|
414
|
+
```js
|
|
415
|
+
const h = mp.local.ik.reach('right', pressPoint, { press: { holdS: 0.1, when: {} } });
|
|
416
|
+
if (!h.fired) hud.flash('too far away');
|
|
417
|
+
``` One reach slot per hand — a new reach
|
|
418
|
+
replaces the old — and an active reach overrides whatever a clip does to that arm (IK solves after
|
|
419
|
+
gestures). Drive ONLY `mp.local` from interaction handlers; remote players' arms are driven by replication.
|
|
420
|
+
|
|
421
|
+
**Rig reach numbers — place interactables with these, do not measure in-world.** On the canonical rig the
|
|
422
|
+
arm chain is **0.55 m shoulder→wrist** (0.278 + 0.273), shoulders ≈1.44 m up and ±0.19 m off center, so
|
|
423
|
+
the default engage envelope is **≈0.80 m from the shoulder** (the runtime measures each avatar's own
|
|
424
|
+
chain, so non-standard avatars self-adjust). Practical placement: put grab/press targets **0.4–0.7 m**
|
|
425
|
+
from where the character will stand, at 0.9–1.3 m height. Beyond the envelope an `engage` stays armed but
|
|
426
|
+
never grabs, and a `press` reaches full extension then gives up after `timeoutS`. Separate wire caps
|
|
427
|
+
(replication validity, not reach): a reach target must be ≤ **3 m** from the player and a look target
|
|
428
|
+
≤ **50 m**, or that frame's upload is skipped.
|
|
429
|
+
|
|
430
|
+
**The reach target is the WRIST, not the fingertips.** The solver drives the `hand_*` bone origin onto
|
|
431
|
+
the target; fingers extend ≈15–19 cm beyond it (palm ≈7 cm). To **press** a button with the fingers, do
|
|
432
|
+
not target the button itself — target a point ≈**10–15 cm in front of its face** (offset along the
|
|
433
|
+
surface normal), or the wrist lands on the button and the hand punches through:
|
|
434
|
+
|
|
435
|
+
```js
|
|
436
|
+
const pressPoint = button.position.clone().addScaledVector(buttonFaceNormal, 0.12);
|
|
437
|
+
mp.local.ik.reach('right', pressPoint, { press: true });
|
|
438
|
+
```
|
|
439
|
+
|
|
440
|
+
Targeting the object directly IS right for `engage` grabs of free-standing props (handle, orb, lever) —
|
|
441
|
+
the object should end up in the palm, and the grip finger poses do the wrapping.
|
|
442
|
+
|