@kubohiroya/turbowarp-realtime-motion-capture 0.3.0 → 0.5.0

package/README.ja.md

@@ -14,13 +14,13 @@ multiview pose estimationを使うcamera／fusion application向けの複合Turb
 - WebGPU MoveNet MultiPoseで最大6人を追跡し、PoseFrame2D v1／v2 JSONを返します。
 - このpackageが所有するmultiview-pose JSON contractを検証します。
 - chessboardとOpenCV.jsで共有cameraをcalibrationします。
-- 外部PoseFrame3D v1を宣言的なA-Frame avatar rigへretargetします。
+- 外部PoseFrame3D v1をA-FrameのVRMアバターへretargetします。
 - camera timingを計測し、同期した2D poseを3Dへ統合して、サイリウムで演者を識別します。
 
 ## 要件と安全性
 
 - 信頼できるunsandboxed custom extensionを利用できるTurboWarp
-- QR／時刻用WebRTC 0.3.0、video用Camera Source 0.5.0、avatar用A-Frame 0.3.0
+- QR／時刻用WebRTC 0.3.0、video用Camera Source 0.5.0、avatar用A-Frame 0.5.0
 - 姿勢推定用TensorFlow.js WebGPUとcalibration用WebAssembly
 
 全機能は起動時固定・既定OFFです。MoveNetにCPU／WASM／WebGL fallbackはなく、初回model loadは
@@ -37,7 +37,7 @@ custom extensionとして読み込みます。通常の順序はCamera Source、
 version固定CDN URL:
 
 ```text
-https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-realtime-motion-capture@0.2.0/dist/turbowarp-realtime-motion-capture.js
+https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-realtime-motion-capture@0.5.0/dist/turbowarp-realtime-motion-capture.js
 ```
 
 npm packageが公開するのはbrowser向けstandalone bundle、schema、文書です。Composition APIは

package/README.md

@@ -14,13 +14,13 @@ Composite TurboWarp blocks for camera and fusion applications using multiview po
 - Runs WebGPU MoveNet MultiPose for up to six people and reports PoseFrame2D v1/v2 JSON.
 - Validates the owned multiview-pose JSON contracts.
 - Calibrates shared cameras with a chessboard and OpenCV.js.
-- Retargets external PoseFrame3D v1 data to declarative A-Frame avatar rigs.
+- Retargets external PoseFrame3D v1 data to VRM avatars in A-Frame.
 - Measures camera timing, fuses synchronized 2D poses into 3D, and identifies performers by glow sticks.
 
 ## Requirements and safety
 
 - TurboWarp with trusted unsandboxed custom extensions enabled.
-- WebRTC 0.3.0 for QR/time, Camera Source 0.5.0 for video, and A-Frame 0.3.0 for avatars.
+- WebRTC 0.3.0 for QR/time, Camera Source 0.5.0 for video, and A-Frame 0.5.0 for avatars.
 - TensorFlow.js WebGPU support for pose and WebAssembly support for calibration.
 
 All features are startup-fixed and OFF by default. MoveNet has no CPU, WASM, or WebGL fallback and
@@ -35,7 +35,7 @@ extension. The usual order is Camera Source, WebRTC, A-Frame, then Realtime Moti
 not used may be omitted. A version-pinned package URL is:
 
 ```text
-https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-realtime-motion-capture@0.3.0/dist/turbowarp-realtime-motion-capture.js
+https://cdn.jsdelivr.net/npm/@kubohiroya/turbowarp-realtime-motion-capture@0.5.0/dist/turbowarp-realtime-motion-capture.js
 ```
 
 The npm package distributes a standalone browser bundle, schemas, and documentation. It does not

package/dist/extension-manifest.json

@@ -31,6 +31,16 @@
       "blockType": "REPORTER",
       "arguments": []
     },
+    {
+      "opcode": "avatarExpressionNames",
+      "blockType": "REPORTER",
+      "arguments": [
+        {
+          "id": "PERSON_ID",
+          "type": "STRING"
+        }
+      ]
+    },
     {
       "opcode": "avatarRetargetError",
       "blockType": "REPORTER",
@@ -525,7 +535,7 @@
           "type": "STRING"
         },
         {
-          "id": "TEMPLATE_JSON",
+          "id": "VRM_URL",
           "type": "STRING"
         }
       ]
@@ -535,6 +545,24 @@
       "blockType": "COMMAND",
       "arguments": []
     },
+    {
+      "opcode": "setAvatarExpression",
+      "blockType": "COMMAND",
+      "arguments": [
+        {
+          "id": "NAME",
+          "type": "STRING"
+        },
+        {
+          "id": "PERSON_ID",
+          "type": "STRING"
+        },
+        {
+          "id": "WEIGHT",
+          "type": "NUMBER"
+        }
+      ]
+    },
     {
       "opcode": "setFrameSyncDisplayRefresh",
       "blockType": "COMMAND",

package/dist/turbowarp-realtime-motion-capture.js

@@ -541,20 +541,20 @@
   			"opcode": "registerAvatarAsset",
   			"feature": "avatarRetargetV1",
   			"blockType": "COMMAND",
-  			"text": "register avatar asset [ASSET_ID] template JSON [TEMPLATE_JSON] rig JSON [RIG_JSON]",
-  			"description": "Registers an A-Frame 0.3.0 template and its Kalidokit rig-output selector mapping.",
+  			"text": "register avatar asset [ASSET_ID] VRM [VRM_URL] rig JSON [RIG_JSON]",
+  			"description": "Registers a VRM model and its root placement for avatars driven through TurboWarp-A-Frame capability v2.",
   			"arguments": {
   				"ASSET_ID": {
   					"type": "STRING",
   					"defaultValue": "actor"
   				},
-  				"TEMPLATE_JSON": {
+  				"VRM_URL": {
   					"type": "STRING",
-  					"defaultValue": "{\"type\":\"group\",\"children\":[]}"
+  					"defaultValue": "avatar.vrm"
   				},
   				"RIG_JSON": {
   					"type": "STRING",
-  					"defaultValue": "{\"bones\":[{\"selector\":\"#{avatar}-left-arm\",\"rig\":\"LeftUpperArm\"}]}"
+  					"defaultValue": "{\"rootScale\":1,\"rootOffset\":[0,0,0]}"
   				}
   			}
   		},
@@ -563,7 +563,7 @@
   			"feature": "avatarRetargetV1",
   			"blockType": "COMMAND",
   			"text": "bind person [PERSON_ID] to avatar [INSTANCE_ID] asset [ASSET_ID] under [PARENT] confidence [CONFIDENCE]",
-  			"description": "Creates an avatar instance and binds one PoseFrame3D person ID to it.",
+  			"description": "Creates an avatar instance, loads its VRM, and binds one PoseFrame3D person ID to it. The block waits until the VRM is ready.",
   			"arguments": {
   				"PERSON_ID": {
   					"type": "STRING",
@@ -615,6 +615,38 @@
   				}
   			}
   		},
+  		{
+  			"opcode": "setAvatarExpression",
+  			"feature": "avatarRetargetV1",
+  			"blockType": "COMMAND",
+  			"text": "set avatar expression [NAME] to [WEIGHT] for person [PERSON_ID]",
+  			"description": "Sets a VRM expression, weight clamped to 0 through 1, on the avatar bound to a PoseFrame3D person ID. An avatar whose VRM is still loading is skipped.",
+  			"arguments": {
+  				"NAME": {
+  					"type": "STRING",
+  					"defaultValue": "happy"
+  				},
+  				"WEIGHT": {
+  					"type": "NUMBER",
+  					"defaultValue": 1
+  				},
+  				"PERSON_ID": {
+  					"type": "STRING",
+  					"defaultValue": "performer-1"
+  				}
+  			}
+  		},
+  		{
+  			"opcode": "avatarExpressionNames",
+  			"feature": "avatarRetargetV1",
+  			"blockType": "REPORTER",
+  			"text": "avatar expressions for person [PERSON_ID]",
+  			"description": "Returns the VRM expression names of the avatar bound to a person as a JSON array, or an empty array while it loads.",
+  			"arguments": { "PERSON_ID": {
+  				"type": "STRING",
+  				"defaultValue": "performer-1"
+  			} }
+  		},
   		{
   			"opcode": "resetAvatarRetarget",
   			"feature": "avatarRetargetV1",
@@ -78248,19 +78280,35 @@
   	"loadTemplate",
   	"createFromTemplate",
   	"setPosition",
-  	"setRotation",
   	"emitEvent",
   	"deleteSelector",
   	"countSelector",
+  	"loadVrm",
+  	"setVrmBoneRotation",
+  	"setVrmExpression",
+  	"vrmExpressionNames",
   	"requireVersion"
   ];
-  function requireAFramePublicBlocks(runtime) {
+  /**
+  * Returns TurboWarp-A-Frame capability v2, the only version avatar retargeting accepts.
+  * A build that cannot provide v2 is refused rather than driven through an older contract.
+  */
+  function requireAFrameCapability(runtime) {
   	const candidate = runtime[AFRAME_CAPABILITY_KEY];
-  	if (typeof candidate !== "object" || candidate === null) throw new Error("TurboWarp-A-Frame capability v1 must be loaded before avatar retargeting.");
-  	for (const method of METHODS) if (typeof Reflect.get(candidate, method) !== "function") throw new Error(`TurboWarp-A-Frame capability v1 is missing ${method}().`);
-  	const capability = candidate;
-  	if (capability.version !== 1) throw new Error(`TurboWarp-A-Frame capability v1 is required; found version ${String(capability.version)}.`);
-  	return capability.requireVersion(1);
+  	if (typeof candidate !== "object" || candidate === null) throw new Error("TurboWarp-A-Frame capability v2 must be loaded before avatar retargeting.");
+  	const requireVersion = Reflect.get(candidate, "requireVersion");
+  	if (typeof requireVersion !== "function") throw new Error("TurboWarp-A-Frame capability v2 is missing requireVersion().");
+  	let capability;
+  	try {
+  		capability = requireVersion.call(candidate, 2);
+  	} catch (error) {
+  		throw new Error(`TurboWarp-A-Frame capability v2 is required: ${error instanceof Error ? error.message : String(error)}`);
+  	}
+  	if (typeof capability !== "object" || capability === null) throw new Error("TurboWarp-A-Frame capability v2 is required.");
+  	for (const method of METHODS) if (typeof Reflect.get(capability, method) !== "function") throw new Error(`TurboWarp-A-Frame capability v2 is missing ${method}(); TurboWarp-A-Frame 0.5.0 or later provides it.`);
+  	const version = Reflect.get(capability, "version");
+  	if (version !== 2) throw new Error(`TurboWarp-A-Frame capability v2 is required; found version ${String(version)}.`);
+  	return capability;
   }
   //#endregion
   //#region node_modules/.pnpm/kalidokit@1.1.5/node_modules/kalidokit/dist/utils/helpers.js
@@ -79815,18 +79863,42 @@
   //#region src/avatar/controller.ts
   var identifiers = /^[A-Za-z0-9._-]{1,64}$/u;
   var eventNames = /^[A-Za-z0-9._:-]{1,80}$/u;
-  var KALIDOKIT_RIG_KEY_SET = new Set(KALIDOKIT_RIG_KEYS);
+  /** Every avatar instance is an empty node that the VRM is loaded onto. */
+  var HOLDER_TEMPLATE = JSON.stringify({ type: "empty" });
+  /**
+  * Kalidokit solves a mirrored selfie view: its `Right*` outputs come from the performer's
+  * left landmarks. A stage avatar moves the performer's own side, so each output drives the
+  * opposite VRM humanoid bone.
+  */
+  var VRM_BONES = {
+  	RightUpperArm: "leftUpperArm",
+  	RightLowerArm: "leftLowerArm",
+  	LeftUpperArm: "rightUpperArm",
+  	LeftLowerArm: "rightLowerArm",
+  	RightHand: "leftHand",
+  	LeftHand: "rightHand",
+  	RightUpperLeg: "leftUpperLeg",
+  	RightLowerLeg: "leftLowerLeg",
+  	LeftUpperLeg: "rightUpperLeg",
+  	LeftLowerLeg: "rightLowerLeg",
+  	Spine: "spine",
+  	Hips: "hips"
+  };
+  /**
+  * The joints each Kalidokit output is computed from. Its `Right*` outputs read the
+  * performer's left landmarks (BlazePose 11, 13, 15, 23, 25, 27), and `Left*` the right.
+  */
   var RIG_REQUIRED_JOINTS = {
-  	RightUpperArm: ["right_shoulder", "right_elbow"],
-  	RightLowerArm: ["right_elbow", "right_wrist"],
-  	LeftUpperArm: ["left_shoulder", "left_elbow"],
-  	LeftLowerArm: ["left_elbow", "left_wrist"],
-  	RightHand: ["right_wrist"],
-  	LeftHand: ["left_wrist"],
-  	RightUpperLeg: ["right_hip", "right_knee"],
-  	RightLowerLeg: ["right_knee", "right_ankle"],
-  	LeftUpperLeg: ["left_hip", "left_knee"],
-  	LeftLowerLeg: ["left_knee", "left_ankle"],
+  	RightUpperArm: ["left_shoulder", "left_elbow"],
+  	RightLowerArm: ["left_elbow", "left_wrist"],
+  	LeftUpperArm: ["right_shoulder", "right_elbow"],
+  	LeftLowerArm: ["right_elbow", "right_wrist"],
+  	RightHand: ["left_wrist"],
+  	LeftHand: ["right_wrist"],
+  	RightUpperLeg: ["left_hip", "left_knee"],
+  	RightLowerLeg: ["left_knee", "left_ankle"],
+  	LeftUpperLeg: ["right_hip", "right_knee"],
+  	LeftLowerLeg: ["right_knee", "right_ankle"],
   	Spine: [
   		"left_shoulder",
   		"right_shoulder",
@@ -79845,21 +79917,23 @@
   		this.runtime = runtime;
   		this.solver = solver;
   	}
-  	registerAsset(assetIdValue, templateJson, rigJson) {
+  	registerAsset(assetIdValue, vrmUrlValue, rigJson) {
   		const assetId = identifier(assetIdValue, "avatar asset ID");
-  		parseJsonObject(templateJson, "Avatar template JSON");
+  		const vrmUrl = nonEmpty(vrmUrlValue, "VRM URL");
+  		if (vrmUrl.length > 2048) throw new Error("VRM URL exceeds 2048 characters.");
   		const rig = parseRigMapping(rigJson);
-  		const aframe = requireAFramePublicBlocks(this.runtime);
+  		const aframe = requireAFrameCapability(this.runtime);
   		const templateId = `twmp-avatar-${assetId}`;
-  		aframe.loadTemplate(templateId, templateJson);
+  		aframe.loadTemplate(templateId, HOLDER_TEMPLATE);
   		this.assets.set(assetId, {
   			id: assetId,
   			templateId,
+  			vrmUrl,
   			rig
   		});
   		this.succeed("configured");
   	}
-  	bind(personIdValue, instanceIdValue, assetIdValue, parentSelectorValue, confidenceValue) {
+  	async bind(personIdValue, instanceIdValue, assetIdValue, parentSelectorValue, confidenceValue) {
   		const personId = identifier(personIdValue, "person ID");
   		const instanceId = identifier(instanceIdValue, "avatar instance ID");
   		const assetId = identifier(assetIdValue, "avatar asset ID");
@@ -79871,25 +79945,41 @@
   		const existingInstance = [...this.bindings.values()].find((binding) => binding.instanceId === instanceId);
   		if (existingInstance && existingInstance.personId !== personId) throw new Error(`Avatar instance is already bound: ${instanceId}`);
   		if (!existingPerson && this.bindings.size >= 6) throw new Error("Avatar retargeting supports at most six people.");
-  		const aframe = requireAFramePublicBlocks(this.runtime);
+  		const aframe = requireAFrameCapability(this.runtime);
   		if (instanceId !== existingPerson?.instanceId && aframe.countSelector(`#${instanceId}`) > 0) throw new Error(`A-Frame node already exists: ${instanceId}`);
   		if (existingPerson) this.removeBinding(aframe, existingPerson);
   		if (aframe.countSelector(`#${instanceId}`) > 0) throw new Error(`A-Frame node already exists: ${instanceId}`);
   		aframe.createFromTemplate(asset.templateId, instanceId, parentSelector);
-  		this.bindings.set(personId, {
+  		const binding = {
   			personId,
   			instanceId,
   			assetId,
   			confidence,
-  			recognized: false
-  		});
+  			recognized: false,
+  			loaded: false
+  		};
+  		this.bindings.set(personId, binding);
+  		this.lastState = "loading";
+  		try {
+  			await aframe.loadVrm(asset.vrmUrl, `#${instanceId}`);
+  		} catch (error) {
+  			if (this.bindings.get(personId) === binding) {
+  				this.bindings.delete(personId);
+  				aframe.deleteSelector(`#${instanceId}`);
+  				this.lastState = "error";
+  				this.lastError = `${personId}: ${message(error)}`;
+  			}
+  			throw error;
+  		}
+  		if (this.bindings.get(personId) !== binding) return;
+  		binding.loaded = true;
   		this.succeed("bound");
   	}
   	unbind(personIdValue) {
   		const personId = identifier(personIdValue, "person ID");
   		const binding = this.bindings.get(personId);
   		if (!binding) return;
-  		this.removeBinding(requireAFramePublicBlocks(this.runtime), binding);
+  		this.removeBinding(requireAFrameCapability(this.runtime), binding);
   		this.succeed("configured");
   	}
   	apply(frameJson, pose2dJson) {
@@ -79901,7 +79991,7 @@
   		try {
   			frame = parseFrame(frameJson);
   			pose2d = parsePoseFrame2D(pose2dJson);
-  			aframe = requireAFramePublicBlocks(this.runtime);
+  			aframe = requireAFrameCapability(this.runtime);
   		} catch (error) {
   			this.lastState = "error";
   			this.lastError = message(error);
@@ -79910,23 +80000,26 @@
   		const people = new Map(frame.persons.map((person) => [person.personId, person]));
   		const screenPeople = new Map(pose2d.persons.map((person) => [person.trackingId, person]));
   		const errors = [];
-  		for (const binding of [...this.bindings.values()]) try {
-  			if (aframe.countSelector(`#${binding.instanceId}`) !== 1) {
-  				this.bindings.delete(binding.personId);
-  				errors.push(`${binding.personId}: avatar instance is missing after scene reset`);
-  				continue;
-  			}
-  			const person = people.get(binding.personId);
-  			const screenPerson = screenPeople.get(binding.personId);
-  			if (!person || !screenPerson || person.score < binding.confidence || screenPerson.score < binding.confidence) {
-  				this.setRecognized(aframe, binding, false, frame.timestampUs);
-  				continue;
+  		for (const binding of [...this.bindings.values()]) {
+  			if (!binding.loaded) continue;
+  			try {
+  				if (aframe.countSelector(`#${binding.instanceId}`) !== 1) {
+  					this.bindings.delete(binding.personId);
+  					errors.push(`${binding.personId}: avatar instance is missing after scene reset`);
+  					continue;
+  				}
+  				const person = people.get(binding.personId);
+  				const screenPerson = screenPeople.get(binding.personId);
+  				if (!person || !screenPerson || person.score < binding.confidence || screenPerson.score < binding.confidence) {
+  					this.setRecognized(aframe, binding, false, frame.timestampUs);
+  					continue;
+  				}
+  				this.applyPerson(aframe, binding, person, screenPerson, pose2d);
+  				this.setRecognized(aframe, binding, true, frame.timestampUs);
+  				this.lastUpdated += 1;
+  			} catch (error) {
+  				errors.push(`${binding.personId}: ${message(error)}`);
   			}
-  			this.applyPerson(aframe, binding, person, screenPerson, pose2d);
-  			this.setRecognized(aframe, binding, true, frame.timestampUs);
-  			this.lastUpdated += 1;
-  		} catch (error) {
-  			errors.push(`${binding.personId}: ${message(error)}`);
   		}
   		if (errors.length > 0) {
   			this.lastState = "partial";
@@ -79936,7 +80029,7 @@
   	reset() {
   		const candidate = this.runtime.turbowarpAFrameCapability;
   		if (typeof candidate === "object" && candidate !== null) try {
-  			const aframe = requireAFramePublicBlocks(this.runtime);
+  			const aframe = requireAFrameCapability(this.runtime);
   			for (const binding of [...this.bindings.values()]) this.removeBinding(aframe, binding);
   		} catch {
   			this.bindings.clear();
@@ -79946,6 +80039,21 @@
   		this.lastUpdated = 0;
   		this.succeed("idle");
   	}
+  	/**
+  	* Sets an expression on the VRM bound to a person. Connecting Performance DSL effects to
+  	* expressions belongs to the application; this only reaches the right avatar. A binding
+  	* whose VRM is still loading is skipped, as frames are.
+  	*/
+  	setExpression(personIdValue, name, weight) {
+  		const binding = this.requireBinding(personIdValue);
+  		if (!binding.loaded) return;
+  		requireAFrameCapability(this.runtime).setVrmExpression(`#${binding.instanceId}`, nonEmpty(name, "expression name"), weight);
+  	}
+  	expressionNames(personIdValue) {
+  		const binding = this.requireBinding(personIdValue);
+  		if (!binding.loaded) return [];
+  		return requireAFrameCapability(this.runtime).vrmExpressionNames(`#${binding.instanceId}`);
+  	}
   	bindingCount() {
   		return this.bindings.size;
   	}
@@ -79973,15 +80081,12 @@
   			const [offsetX, offsetY, offsetZ] = asset.rig.rootOffset;
   			aframe.setPosition(`#${binding.instanceId}`, root.x * asset.rig.rootScale + offsetX, root.y * asset.rig.rootScale + offsetY, root.z * asset.rig.rootScale + offsetZ);
   		}
-  		for (const bone of asset.rig.bones) this.applyBone(aframe, binding, bone, rig, worldKeypoints, screenKeypoints);
+  		for (const key of KALIDOKIT_RIG_KEYS) this.applyBone(aframe, binding, key, rig, worldKeypoints, screenKeypoints);
   	}
-  	applyBone(aframe, binding, bone, rig, worldKeypoints, screenKeypoints) {
-  		if (!requiredJoints(bone.rig).every((id) => (worldKeypoints.get(id)?.score ?? 0) >= binding.confidence && (screenKeypoints.get(id)?.score ?? 0) >= binding.confidence)) return;
-  		const selector = expandSelector(bone.selector, binding.instanceId);
-  		if (aframe.countSelector(selector) !== 1) throw new Error(`Rig selector must match exactly one node: ${selector}`);
-  		const rotation = rotationForRig(rig, bone.rig);
-  		const [offsetX, offsetY, offsetZ] = bone.offsetDegrees;
-  		aframe.setRotation(selector, radiansToDegrees(rotation.x) + offsetX, radiansToDegrees(rotation.y) + offsetY, radiansToDegrees(rotation.z) + offsetZ);
+  	applyBone(aframe, binding, key, rig, worldKeypoints, screenKeypoints) {
+  		if (!requiredJoints(key).every((id) => (worldKeypoints.get(id)?.score ?? 0) >= binding.confidence && (screenKeypoints.get(id)?.score ?? 0) >= binding.confidence)) return;
+  		const [x, y, z] = vrmBoneDegrees(rotationForRig(rig, key));
+  		aframe.setVrmBoneRotation(`#${binding.instanceId}`, VRM_BONES[key], x, y, z);
   	}
   	setRecognized(aframe, binding, recognized, timestampUs) {
   		if (binding.recognized === recognized) return;
@@ -79999,6 +80104,12 @@
   		aframe.deleteSelector(`#${binding.instanceId}`);
   		this.bindings.delete(binding.personId);
   	}
+  	requireBinding(personIdValue) {
+  		const personId = identifier(personIdValue, "person ID");
+  		const binding = this.bindings.get(personId);
+  		if (!binding) throw new Error(`No avatar is bound to person: ${personId}`);
+  		return binding;
+  	}
   	succeed(state) {
   		this.lastState = state;
   		this.lastError = "";
@@ -80022,48 +80133,24 @@
   }
   function parseRigMapping(source) {
   	const value = parseJsonObject(source, "Avatar rig mapping JSON");
+  	if ("bones" in value) throw new Error("Rig mapping no longer takes bones: VRM humanoid bones are driven directly.");
   	rejectUnknownKeys(value, /* @__PURE__ */ new Set([
   		"rootScale",
   		"rootOffset",
   		"recognitionStartEvent",
-  		"recognitionEndEvent",
-  		"bones"
+  		"recognitionEndEvent"
   	]), "rig mapping");
   	const rootScale = finite(value.rootScale ?? 1, "rootScale");
   	if (rootScale <= 0) throw new Error("rootScale must be greater than zero.");
-  	const rootOffset = vector3(value.rootOffset ?? [
-  		0,
-  		0,
-  		0
-  	], "rootOffset");
-  	const recognitionStartEvent = eventName(value.recognitionStartEvent ?? "twmp-recognition-start", "recognitionStartEvent");
-  	const recognitionEndEvent = eventName(value.recognitionEndEvent ?? "twmp-recognition-end", "recognitionEndEvent");
-  	if (!Array.isArray(value.bones) || value.bones.length === 0 || value.bones.length > 32) throw new Error("Rig mapping bones must contain between 1 and 32 entries.");
   	return {
   		rootScale,
-  		rootOffset,
-  		recognitionStartEvent,
-  		recognitionEndEvent,
-  		bones: value.bones.map((entry, index) => parseBone(entry, index))
-  	};
-  }
-  function parseBone(value, index) {
-  	const bone = record(value, `bones[${index}]`);
-  	rejectUnknownKeys(bone, /* @__PURE__ */ new Set([
-  		"selector",
-  		"rig",
-  		"offsetDegrees"
-  	]), `bones[${index}]`);
-  	const selector = nonEmpty(bone.selector, `bones[${index}].selector`);
-  	if (!selector.includes("{avatar}")) throw new Error(`bones[${index}].selector must contain {avatar}.`);
-  	return {
-  		selector,
-  		rig: rigKey(bone.rig, `bones[${index}].rig`),
-  		offsetDegrees: vector3(bone.offsetDegrees ?? [
+  		rootOffset: vector3(value.rootOffset ?? [
   			0,
   			0,
   			0
-  		], `bones[${index}].offsetDegrees`)
+  		], "rootOffset"),
+  		recognitionStartEvent: eventName(value.recognitionStartEvent ?? "twmp-recognition-start", "recognitionStartEvent"),
+  		recognitionEndEvent: eventName(value.recognitionEndEvent ?? "twmp-recognition-end", "recognitionEndEvent")
   	};
   }
   function parseJsonObject(source, label) {
@@ -80083,10 +80170,6 @@
   	const unknown = Object.keys(value).find((key) => !allowed.has(key));
   	if (unknown) throw new Error(`${label} contains unknown field: ${unknown}`);
   }
-  function rigKey(value, label) {
-  	if (typeof value !== "string" || !KALIDOKIT_RIG_KEY_SET.has(value)) throw new Error(`${label} must be a supported Kalidokit pose rig key.`);
-  	return value;
-  }
   function vector3(value, label) {
   	if (!Array.isArray(value) || value.length !== 3) throw new Error(`${label} must contain exactly three numbers.`);
   	return [
@@ -80117,8 +80200,20 @@
   	if (!eventNames.test(normalized)) throw new Error(`Invalid ${label}.`);
   	return normalized;
   }
-  function expandSelector(selector, instanceId) {
-  	return selector.replaceAll("{avatar}", instanceId);
+  /**
+  * Converts a Kalidokit rotation in radians to Euler degrees on a VRM normalized bone.
+  *
+  * Kalidokit's rotations are for VRM 0.x bone axes, which face -Z; normalized bones use the
+  * VRM 1.0 axes, a half turn about Y away, which negates x and z. Driving the opposite side
+  * (see VRM_BONES) also mirrors the rotation across the body's midline, which negates y and z.
+  * Together the signs become (-x, -y, z).
+  */
+  function vrmBoneDegrees(rotation) {
+  	return [
+  		-radiansToDegrees(rotation.x),
+  		-radiansToDegrees(rotation.y),
+  		radiansToDegrees(rotation.z)
+  	];
   }
   function radiansToDegrees(value) {
   	return value * 180 / Math.PI;
@@ -82954,11 +83049,11 @@
   	}
   	registerAvatarAsset(args) {
   		this.requireAvatarEnabled();
-  		this.avatar.registerAsset(Scratch.Cast.toString(args.ASSET_ID), Scratch.Cast.toString(args.TEMPLATE_JSON), Scratch.Cast.toString(args.RIG_JSON));
+  		this.avatar.registerAsset(Scratch.Cast.toString(args.ASSET_ID), Scratch.Cast.toString(args.VRM_URL), Scratch.Cast.toString(args.RIG_JSON));
   	}
   	bindAvatarPerson(args) {
   		this.requireAvatarEnabled();
-  		this.avatar.bind(Scratch.Cast.toString(args.PERSON_ID), Scratch.Cast.toString(args.INSTANCE_ID), Scratch.Cast.toString(args.ASSET_ID), Scratch.Cast.toString(args.PARENT), Scratch.Cast.toNumber(args.CONFIDENCE));
+  		return this.avatar.bind(Scratch.Cast.toString(args.PERSON_ID), Scratch.Cast.toString(args.INSTANCE_ID), Scratch.Cast.toString(args.ASSET_ID), Scratch.Cast.toString(args.PARENT), Scratch.Cast.toNumber(args.CONFIDENCE));
   	}
   	unbindAvatarPerson(args) {
   		this.requireAvatarEnabled();
@@ -82968,6 +83063,14 @@
   		this.requireAvatarEnabled();
   		this.avatar.apply(Scratch.Cast.toString(args.POSE3D_JSON), Scratch.Cast.toString(args.POSE2D_JSON));
   	}
+  	setAvatarExpression(args) {
+  		this.requireAvatarEnabled();
+  		this.avatar.setExpression(Scratch.Cast.toString(args.PERSON_ID), Scratch.Cast.toString(args.NAME), Scratch.Cast.toNumber(args.WEIGHT));
+  	}
+  	avatarExpressionNames(args) {
+  		this.requireAvatarEnabled();
+  		return JSON.stringify(this.avatar.expressionNames(Scratch.Cast.toString(args.PERSON_ID)));
+  	}
   	resetAvatarRetarget() {
   		this.avatar.reset();
   	}

package/docs/architecture.ja.md

@@ -134,32 +134,47 @@ detectorのdisposeで解放します。
 ## PoseFrame3D avatar retarget
 
 `avatarRetargetV1`は独立した起動時固定・既定OFF flagです。runtime key
-`turbowarpAFrameCapability`へ`requireVersion(1)`を呼び、TurboWarp-A-Frame capabilityの公開同期
-scene操作7種だけを利用します。A-Frame DOM、Three.js `object3D`、GLTF内部boneへはアクセス
-しません。capability v1は`@kubohiroya/turbowarp-aframe@0.3.0`で公開済みです。
-
-asset登録では宣言的template JSONをA-Frameへ送り、検証済みrig mappingを保持します。各boneは
-対応するKalidokit pose rig出力、`{avatar}`を含むselector、任意Euler offset degreeで定義します。
-適用時は対応するexact-v1 PoseFrame3DとPoseFrame2Dの両方を要求します。PoseFrame3Dの`personId`と
-PoseFrame2Dの`trackingId`が一致するpersonだけを結合しますが、これは時刻alignmentではありません。
+`turbowarpAFrameCapability`へ`requireVersion(2)`を呼び、それ以外のversionは受け付けません。
+capability v2を持たないA-Frame（0.3.0など）は拒否します。そのcapabilityのscene操作とVRM操作だけを
+利用し、A-Frame DOM、Three.js object、GLTF内部へはアクセスしません。capability v2は
+`@kubohiroya/turbowarp-aframe@0.4.0`で公開され、adapterが同じく要求する表情の操作は0.5.0で
+加わったため、peerは0.5.0に固定しています。
+
+asset登録ではVRMのURLと検証済みのroot配置を保持します。personをbindすると、holder templateから
+空のnodeを作ってVRMを読み込み、準備ができるまで待ちます。読み込みに失敗したらnodeとbindingを
+取り除きます。適用時は対応するexact-v1 PoseFrame3DとPoseFrame2Dの両方を要求します。
+PoseFrame3Dの`personId`とPoseFrame2Dの`trackingId`が一致するpersonだけを結合しますが、これは
+時刻alignmentではありません。
 
 adapterは両方のCOCO-17 recordを、exact pinした`kalidokit@1.1.5`が要求する33 positionへ
 決定論的に変換します。screen座標にはPoseFrame2Dの`frameWidth`／`frameHeight`を使い、world座標は
 外部serviceの値を維持します。不足するBlazePose face／hand／foot pointは低visibilityで中点補間
 または複製します。`runtime: "tfjs"`、`enableLegs: true`のKalidokit `Pose.solve`だけをrotation
-solverとし、radian出力をA-Frame degreeへ変換します。hips結果にroot scale／offsetを適用し、
+solverとし、hips結果にroot scale／offsetを適用し、
 自前rotation fallbackは持ちません。joint／personがbinding threshold未満なら該当transformだけを
 skipし、直前値を維持します。
 
+Kalidokitは、VRM 0.xのボーンの軸で、鏡に映した自撮りの視点を解きます。`Right*`の出力は演者の
+左のlandmarkから来るため、各出力は反対側のVRMヒューマノイドのボーンを回し、演者自身の側を
+動かします。回転は度で正規化されたボーンへ書きます。正規化されたボーンはVRM 0.xからYまわりに
+半回転したVRM 1.0の軸を使うためxとzの符号が反転し、反対側を動かすことで正中面について鏡映する
+ためyとzの符号が反転します。合わせて符号は(-x, -y, z)になります。これはKalidokit 1.1.5と
+VRMに合成の姿勢を入れて確かめました。腕を30°下ろす、片腕を水平に保つ、膝を上げる、前腕を前へ
+伸ばす、のいずれも演者自身の手足が同じ向きに動きました。Kalidokitの腕の出力は、水平より上げた
+腕と下げた腕をほとんど区別せず、体を傾けてもspineの傾きを返さなかったため、spineとhipsの符号は
+同じ導出によるもので、まだ観測していません。
+
 最大6 person IDを一意なtemplate instanceへbindします。recognition遷移は設定可能なA-Frame
-eventで通知し、application側がPerformance DSLのstart／end effectへ接続できます。1人の
+eventで通知し、application側がPerformance DSLのstart／end effectへ接続できます。effectを
+アバター自身に表せるよう、adapterはperson IDにbindしたアバターへVRMの表情を設定します。どの
+表情をいつ使うかはapplicationが決めます。1人の
 capability失敗は`partial`診断へ集約し、他avatarを継続します。rebind、明示reset、project
 lifecycle reset、disposeでは可能ならend eventを送り、生成instanceと一時状態をcleanupします。
 
 PoseFrame3Dは別実装の3D serviceから届くexact v1境界dataです。`timestampUs`は不透明値として
 recognition event dataへcopyするだけです。frame alignment、履歴保持／query、triangulation、
 3D solveは行いません。Kalidokitは上流でdeprecatedでありnative BlazePose landmarkを想定するため、
-このCOCO-17拡張は明示的な精度制約です。release前に対象GLTF rigを実browserで検証します。
+このCOCO-17拡張は明示的な精度制約です。release前に実際の記録でretargetを検証します。
 
 ## フレーム同期パターンの縦切り