three-mediapipe-rig 0.0.2 → 0.0.4

package/README.md

@@ -1,16 +1,38 @@
 
 ![cover](cover.jpg)
 
-# three-mediapipe-rig
+# Control a skeleton with the webcam
 
-Integrate [Google MediaPipe](https://ai.google.dev/edge/mediapipe/solutions/guide)'s **webcam motion tracking** with [Three.js](https://threejs.org/) skeletal rigs. Load a GLTF/GLB character, bind it, and drive its body, hands, and face from a webcam or video — in just a few lines of code.
+Integrate [Google MediaPipe](https://ai.google.dev/edge/mediapipe/solutions/guide)'s **webcam motion tracking** with [Three.js](https://threejs.org/) skeletal rigs. 
 
-Use your webcam ( or a video ) to drive a skeleton.
+The motion from the webcam will be applied to a skeleton. Angle based so it works with any size skeleton. 
 
 This will run 3 models: face, body, hands. So expect a FPS drop.
 
 LIVE EXAMPLE: https://bandinopla.github.io/three-mediapipe-rig/
 
+---
+
+## Table of Contents
+
+- [Features](#features)
+- [Installation](#installation)
+- [Quick Start](#quick-start)
+  - [Skeleton](#skeleton)
+  - [Facial Animation](#facial-animation)
+- [API](#api)
+  - [`setupTracker(config?)`](#setuptrackerconfig)
+  - [Tracker API](#tracker-api-return-value-of-setuptracker)
+- [Bone Naming](#bone-naming)
+  - [Default bone names](#default-bone-names)
+  - [Custom bone map example](#custom-bone-map-example)
+- [Multiple Characters](#multiple-characters)
+- [Debugging with Video](#debugging-with-video)
+- [Recording](#recording)
+- [License](#license)
+
+---
+
 ## Features
 
 - **Full-body pose tracking** — shoulders, arms, hips, legs, and head
@@ -19,6 +41,7 @@ LIVE EXAMPLE: https://bandinopla.github.io/three-mediapipe-rig/
 - **Automatic bone binding** — maps MediaPipe landmarks to your rig's skeleton using a configurable bone-name map
 - **Webcam & video input** — use a live webcam feed or a pre-recorded video for motion capture
 - **Debug tools** — preview the video/image feed overlaid with landmark visualizations
+- **Recording** — record the motion capture to a file ( .glb )
 
 ## Installation
 ```
@@ -29,41 +52,32 @@ npm install three-mediapipe-rig
 
 ## Quick Start
 
-```ts
-// 1. Create your renderer
-const renderer = new THREE.WebGPURenderer({ antialias: true });
-renderer.setSize(window.innerWidth, window.innerHeight);
-document.body.appendChild(renderer.domElement);
+```ts 
 
-// 2. Initialize the tracker (loads MediaPipe models)
+// 1. Initialize the tracker (loads MediaPipe models)
 await setupTracker({ ...config... })
  
 const rig = scene.getObjectByName("rig")!;
 
-// 3. Bind the rig to the tracker
+// 2. Bind the rig to the tracker
 const binding = tracker.bind(rig);
  
 
-// 4. Start the webcam ( must be initialized by a user triggered event like a click )
-tracker.start();
-
-// 5. Update in your render loop
-const clock = new THREE.Timer();
-renderer.setAnimationLoop((time: number) => {
-  const delta = clock.update(time).getDelta();
+// 3. Start the webcam ( must be initialized by a user triggered event like a click )
+button.addEventListener('click', () => {
+    tracker.start(); //<-- will ask webcam access!
+})
 
-  // 6. update the skeleton...
-  binding?.update(delta);
 
-  renderer.render(scene, camera);
-});
+// 4. Update the rig in your render loop ( this keels the skeleton in-sync)
+binding?.update(delta);
 ```
 
 ### Skeleton
-You can use the skeleton provided in `rig.blend` or use your own and provide a bone name mapping so we know where the bones are in the second argument for the `.bind` method. But pay attention to the bone role of the provided skeleton, as it is the one expected by this module.
+You can use the skeleton provided in `rig.blend` or use your own and provide a bone name mapping so we know where the bones are in the second argument for the `.bind` method. But pay attention to the [**bone roll**](https://docs.blender.org/manual/en/latest/animation/armatures/bones/editing/bone_roll.html) of the provided skeleton, as it is the one expected by this module.
 
 ### Facial Animation
-Media Pipe provides blend shape keys for the face ( estimated from the webcam ). The face it is expected to be a separated mesh, just the face, with blend shape keys named as the ones provided by Media Pipe. See [Blend Shape Keys reference](/face-blendshapekeys.md) You don't have to have all of them, if they are not found, they will be ignored.
+Media Pipe provides blend shape keys for the face ( estimated from the webcam ). The face it is expected to be a separated mesh with a name that starts with "face", with blend shape keys named as the ones provided by Media Pipe. See [Blend Shape Keys reference](/face-blendshapekeys.md) You don't have to have all of them, if they are not found, they will be ignored.
 
 
 ## API
@@ -251,6 +265,24 @@ const tracker = await setupTracker({
 });
 ```
 
+## Recording
+You can record the motion of a rig by doing this. Notice the track names of the clip will use the names of the bones from the currently recorded rig.
+
+```ts
+// Start recording
+yourRigBindHandler.startRecording()
+
+// after a while...
+const result = yourRigBindHandler.stopRecording();
+
+// to save the recording to a file....
+result.saveToFile();
+
+// or if you just want the AnimationClip...
+result.clip
+```
+But this will save a .glb containing the rig (mesh, textures, etc...) You would have to strip the mesh data after export or just accept the full GLB and strip it with a post-process tool like [gltf-transform](https://gltf-transform.dev/). The saved .glb will contain the animation clip, the rig and the textures. Still havent figured out a way to export a minimal file like just the animation... 
+
 --- 
 
 ## License