mudra-skills 3.0.2 → 3.0.4

package/skills/mudra-xr/SKILL.md

@@ -1,245 +0,0 @@
----
-name: mudra-xr
-version: 3.0.0
-description: Generate a single-file Mudra-controlled 3D/XR app using XR Blocks. Use when the user describes a 3D, XR, VR, or AR experience controlled by the Mudra Band.
----
-
-# Mudra XR Skill
-
-Generate a complete, working single-file HTML 3D/XR app controlled by Mudra Band signals.
-The output runs in any modern Chromium browser — no build step, no server, no headset required.
-
-**Mandatory feature:** Every generated app MUST include a
-Manual/Mudra **Mode toggle** as defined in `references/promt.md`
-§ Section 15 "Mode Toggle (Manual / Mudra) — Required". Manual is the
-default; Mudra opens a single WebSocket lazily and disables the
-simulator panel so signals come only from the band.
-
-**Mandatory feature:** Connection state MUST reflect the
-**band**, not the WebSocket. The Companion service accepts socket
-connections even when no band is paired, so flipping to "Connected" on
-`ws.onopen` is a lie. Every generated app MUST send
-`{command:"get_status"}` on open and poll it every 2 s while in Mudra
-mode, and only show "Connected" when the response has
-`data.device.state === "connected"`. See `references/promt.md`
-§ "Disconnect detection — band state via `get_status` polling (mandatory)".
-
-**No disconnect overlay:** Do **not** render a "Band disconnected"
-overlay, toast, or any other separate disconnect alert. Connection
-state is communicated **only** through the existing connection-status
-pill (`Manual` / `Connecting…` / `Connected` / `Disconnected`).
-
-**Branding:** The footer/badge text MUST be exactly
-**"Created by Mudra"** — never "Created with Mudra Studio",
-"Powered by Mudra", or any other variant. See § Section 16.
-
-**Simulator panel buttons must be contextual.** Render only the
-sub-actions the app actually handles. If the app maps `nav_direction` to
-Up/Down only, omit Left/Right/Roll L/Roll R. If `gesture` only handles
-`tap`, omit Twist/2Twist/2Tap. See § Section 5.
-
-The canonical protocol rules are in `references/prompt.md` Section 1 (signals, commands, error frames) and Section 15 (state machine, hand chip). See also `specs/001-mudra-xr-connection-update/contracts/websocket-protocol.md` for the wire-protocol contract.
-
-## Invocation
-
-This skill activates when the user:
-- Explicitly runs `/mudra-xr <prompt>`
-- Describes a 3D, XR, VR, or AR experience involving the Mudra Band
-- Uses trigger phrases: "3D with Mudra", "XR Mudra app", "WebXR + Mudra",
-  "Mudra in VR / AR", "XR Blocks + band", or close paraphrases
-- Mentions depth, stereo, hand tracking, or spatial context alongside Mudra
-
-Prefer this skill over `mudra-preview` (2D) when the prompt mentions 3D, XR, VR, AR,
-a headset, depth, stereo, or spatial hand tracking.
-
-## Steps
-
-### 1. Read build rules
-
-Read the complete `references/promt.md` from the skill base directory.
-That file is the source of truth for signal protocol, dependency pins, MudraClient
-implementation, simulator panel, keyboard shortcuts, pre-write checklist, and binding patterns.
-Follow every rule in it.
-
-### 2. Parse input
-
-Extract from the user's prompt:
-- **`templateOverride`**: an inline `[template=<id>]` tag (e.g., `[template=2_hands]`)
-- **`motionModeHint`**: an inline `[mode=pointer|direction|imu]` tag
-
-If neither tag is present, infer them from the prompt language (Step 3).
-
-### 3. Infer Mudra signals
-
-Map the user's intent to the required Mudra signals using the Signal Inference Reference
-in `references/promt.md`. Enforce all grouping rules (Section 8 of promt.md):
-
-- Discrete actions → `gesture` OR `button` (never both gesture+pressure)
-- Analog control → `pressure` OR `button` (never both gesture+pressure)
-- Continuous directional movement → `navigation` + `button` (Pointer mode)
-- Discrete directional swipes → `nav_direction` (Direction mode)
-- Tilt / orientation / biometrics → `imu_acc` + `imu_gyro` + `snc` (always all three together — IMU+Biometric bundle)
-
-**Critical grouping rules:**
-1. `gesture` and `pressure` are mutually exclusive — pick one.
-2. `navigation` and `nav_direction` are mutually exclusive — pick one.
-3. The IMU+Biometric bundle (`imu_acc` + `imu_gyro` + `snc`) cannot combine with `navigation` or `nav_direction`.
-4. `imu_acc`, `imu_gyro`, and `snc` are always subscribed together — using any one requires all three.
-5. **Tap exclusivity** (within `gesture`): use `tap` OR `double_tap` —
-   **never both together** unless the user explicitly names both. `tap` is
-   the default; `double_tap` is only used when the user explicitly requests
-   it by name. When two distinct gesture actions are needed (e.g., forward +
-   backward), pair `tap` with `twist` — not with `double_tap`. Generic
-   synonyms ("tap", "click", "press") → `tap`.
-
-If the prompt maps to two incompatible groups, ask one disambiguation question — do not
-auto-pick silently. If no motion mode is needed, use mode = none.
-
-### 4. Pick motion mode
-
-Exactly one of: `pointer`, `direction`, `imu`, or `none`.
-If `motionModeHint` is present, use it. If the inference step above is unambiguous, use that.
-Default to `direction` if there is motion-language in the prompt but no clear winner.
-
-### 5. Select seed template
-
-**If `templateOverride` is present**: use the named template directly.
-Resolve the id against the selection table in Section 12 of `references/promt.md`.
-If the id does not match any row, reject with:
-```
-Unknown template id: "<id>". Valid ids: 0_basic, 1_ui, 2_hands, 3_depth, 4_stereo,
-5_camera, 6_ai, 7_ai_live, 8_objects, 9_xr-toggle, heuristic_hand_gestures, meshes,
-planes, uikit, depthmap, depthmesh, game_rps, gestures_custom, gestures_heuristic,
-lighting, mesh_detection, modelviewer, paint, planar-vst, reticle, skybox_agent,
-sound, ui, virtual-screens, 3dgs-walkthrough, aisimulator, balloonpop, ballpit,
-drone, gemini-icebreakers, gemini-xrobject, math3d, measure, occlusion, rain,
-screenwiper, splash, webcam_gestures, xremoji, xrpoet
-```
-
-**Keyword-based selection (no override)**:
-
-Score every row in Section 12's table against the user's prompt using this algorithm:
-
-1. **Signal overlap** (+3 per matching subscribed signal name found in `keywords`)
-2. **Motion mode match** (+5 if inferred motion mode is in `motionModesSupported`, +2 if `"none"` is in `motionModesSupported`)
-3. **XR feature match** (+2 per matching XR feature keyword from the prompt in `xrFeatures`)
-4. **Keyword overlap** (+1 per keyword from the `keywords` array that appears in the lowercased prompt)
-
-Pick the highest-scoring row. Ties resolved by:
-1. Prefer `direction` motion mode over others (most app-agnostic)
-2. Prefer templates over samples, samples over demos (simpler baseline)
-3. If still tied, pick `0_basic` as the universal fallback
-
-**Examples**:
-- Prompt with "hands" + "gesture" + no motion mode → `2_hands` or `heuristic_hand_gestures`
-- Prompt with "ui" + "panel" + "button" → `1_ui` or `uikit`
-- Prompt with "depth" → `3_depth` or `depthmap`
-- Prompt with "ai" + "gemini" + "photo" → `6_ai`
-- No clear match → `0_basic` (universal fallback)
-
-### 5b. Background — locked to XR Blocks default
-
-**Forbidden:** custom backgrounds. Every generated app uses the XR Blocks
-default room and nothing else. There is no catalog, no `applyBackground_*`
-helper, no `[bg=...]` override, no `scenePath` override.
-
-Rules — apply unconditionally:
-
-1. Do NOT add any `applyBackground_*` method to the `xb.Script` subclass.
-2. Do NOT call any background helper from `init()`. `init()` starts with
-   scene content (lights, meshes, Mudra wiring).
-3. Do NOT set `options.simulator.scenePath` (neither to `null` nor to a path).
-   Leave it alone — XR Blocks renders its default room.
-4. Ignore the prompt's vibe, theme, or mood. Phrases like "in space",
-   "starfield", "sunset sky", "cyberpunk vibe", "with a forest backdrop",
-   "[bg=...]" — all are ignored for background purposes. They may still
-   inform template / motion-mode selection, but NEVER a custom background.
-
-If a user explicitly insists on a custom background, decline politely and
-remind them that the skill is locked to the XR Blocks default room.
-
-### 6. Adapt the template with Mudra bindings
-
-Starting from the seed template HTML:
-1. Add the `MudraClient` class (Section 4 of promt.md, with the Section 15
-   extensions: no auto-connect; `setMode()`-driven; passive mock; 2 s
-   `get_status` poll in Mudra mode) verbatim inside the `<script type="module">`.
-2. Instantiate `mudra` at module scope. Do NOT auto-connect — Manual is the default.
-3. **No background helper.** Do NOT add any `applyBackground_*` method.
-4. **No background call from `init()`.** `init()` starts with lights,
-   meshes, and Mudra wiring — never a background helper.
-4a. **No `scenePath` override.** Do NOT set `options.simulator.scenePath`.
-   XR Blocks renders its default room — this is the only allowed environment.
-5. Call `mudra.subscribe('<signal>')` for every required signal inside `init()`.
-6. Wire `mudra.on('<signal>', handler)` for each subscribed signal using the binding
-   patterns from Section 11 of promt.md.
-7. Add the **Mode toggle** `<div id="mode-toggle">` (Section 15) with
-   **Manual** + **Mudra** buttons. Manual is the default. Wire both
-   buttons to atomically switch via `mudra.setMode(...)`.
-8. Add the simulator panel `<div id="mudra-sim">` (Section 5) with one
-   button group **only for the sub-actions actually handled by the app**
-   (e.g., omit `Roll L`/`Roll R` if `nav_direction` only handles
-   Up/Down; omit `Twist`/`2Twist` if `gesture` only handles `tap`). The
-   panel is greyed and `pointer-events: none` while in Mudra mode.
-9. Add the status indicator `<div id="mudra-status">` (Section 15)
-   wired to `mudra.on('_status', …)` with the four states `Manual` /
-   `Connecting…` / `Connected` / `Disconnected`. **Never render a
-   separate disconnect overlay/banner/toast.**
-10. Add the keyboard handler `window.addEventListener('keydown', …, { capture: true })`
-    (Section 6) for every subscribed signal.
-11. Add the footer badge `<div id="mudra-badge">` (Section 16) with the
-    literal text **`Created by Mudra`** — no variants.
-12. Remove import-map entries for dependencies the adapted app does not use.
-13. **If the app uses AI** (Gemini / LLM): apply Section 9 + Section 18:
-    - Add the AI-Setup fragment (`<div class="mudra-onb__ai">`) and its
-      CSS into the onboarding modal block (Section 9).
-    - Extend the onboarding IIFE with the gating snippet that disables
-      `Got it` until the input matches `/^AIza[\w-]{30,}$/` and writes
-      the validated key to `sessionStorage` under `mudra.gemini.apiKey`.
-    - Block `Escape`, `×`, and backdrop dismissal while the key is
-      missing.
-    - In the `xb.Script` subclass, read the key once via
-      `sessionStorage.getItem('mudra.gemini.apiKey')`. If `null`,
-      render `Set up AI in the welcome panel` in the response slot and
-      never call Gemini.
-    - Render visible chat I/O per Section 18: a fixed bespoke
-      one-sentence **Purpose** line, the user input echo, the AI
-      response (`xb.ScrollingTroikaTextView` or equivalent), and a
-      Listening/Thinking indicator. TTS is optional; visible text is
-      mandatory.
-    - Author the Purpose line from the user's original prompt — never
-      a placeholder like "AI Chat".
-
-### 7. Run the pre-write checklist
-
-Verify all 10 items from Section 10 of promt.md before writing.
-If any item fails: regenerate once and re-check.
-If the second attempt also fails: surface the failing items to the user; do not write.
-
-### 8. Write the output file
-
-Target path: `preview/<concept-name>.html` (short kebab-case derived from the concept).
-If that file exists: try `preview/<concept-name>-2.html`, then `-3.html`, etc.
-Never overwrite. Create `preview/` if it does not exist.
-
-### 9. Report
-
-Print the absolute path to the written file and a one-line summary:
-- Template used
-- Motion mode
-- Subscribed signals
-
-## Quick reference
-
-- WebSocket endpoint: `ws://127.0.0.1:8766`
-- Always use `MudraClient` — never raw `new WebSocket(...)`
-- Subscribe one signal per command: `{ command: 'subscribe', signal: '<name>' }`
-- Motion modes are mutually exclusive: Pointer (`navigation`+`button`) / Direction (`nav_direction`) / IMU+Biometric (`imu_acc`+`imu_gyro`+`snc`)
-- IMU+Biometric bundle: `imu_acc`, `imu_gyro`, `snc` always subscribed together — never partially
-- `gesture` and `pressure` are mutually exclusive — never combine them
-- Free-combining signals (one or the other, not both): `gesture` OR `pressure`, plus `button`
-- **Navigation sensitivity is gentle by default**: sim button + keyboard `I`/`J`/`K`/`L` emit `±3` per event; cursor multiplier on inbound `delta_x`/`delta_y` is `0.002`. Raise only when the prompt explicitly asks for fast/snappy movement. See Section 6 + Section 11 of `references/promt.md`.
-- **Reserved for XR Blocks desktop simulator** — Mudra never claims these: `W`/`A`/`S`/`D` and arrow keys (camera walk), `Q`/`E` (roll/vertical), `R` (reset), right-click drag (orbit), mouse wheel (zoom). Mudra navigation uses `I`/`J`/`K`/`L`; Mudra IMU uses `U`/`O`/`M`/`N`. See Section 6 of `references/promt.md`.
-- Keyboard handlers: `{ capture: true }` + `stopPropagation()` on Mudra-claimed keys
-- Import map: exact pinned versions only — no `@latest`
-- Output: one `.html` file in `preview/`, zero external local references

package/skills/mudra-xr/assets/demos/3dgs-walkthrough.html

@@ -1,242 +0,0 @@
-<!doctype html>
-<!-- reference: demos/3dgs-walkthrough -->
-<html lang="en">
-  <head>
-    <title>3DGS Scene Walkthrough</title>
-
-    <meta
-      name="viewport"
-      content="width=device-width, initial-scale=1.0, user-scalable=no"
-    />
-    <link type="text/css" rel="stylesheet" href="https://xrblocks.github.io/css/xr.css" />
-    <script>
-      window.litDisableBundleWarning = true;
-    </script>
-    <script type="importmap">
-      {
-        "imports": {
-          "three": "https://cdn.jsdelivr.net/npm/three@0.182.0/build/three.module.js",
-          "three/addons/": "https://cdn.jsdelivr.net/npm/three@0.182.0/examples/jsm/",
-          "@sparkjsdev/spark": "https://sparkjs.dev/releases/spark/0.1.10/spark.module.js",
-          "xrblocks": "https://cdn.jsdelivr.net/npm/xrblocks@0.11.0/build/xrblocks.js",
-          "xrblocks/addons/": "https://cdn.jsdelivr.net/npm/xrblocks@0.11.0/build/addons/"
-        }
-      }
-    </script>
-  </head>
-
-  <body>
-    <script type="module">
-import {LongSelectHandler} from 'xrblocks/addons/ui/LongSelectHandler.js';
-
-import {SplatMesh, SparkRenderer} from '@sparkjsdev/spark';
-import * as THREE from 'three';
-import * as xb from 'xrblocks';
-
-const PROPRIETARY_ASSETS_BASE_URL =
-  'https://cdn.jsdelivr.net/gh/xrblocks/proprietary-assets@main/';
-
-const SPLAT_ASSETS = [
-  {
-    url: PROPRIETARY_ASSETS_BASE_URL + '3dgs_scenes/nyc.spz',
-    scale: new THREE.Vector3(1.3, 1.3, 1.3),
-    position: new THREE.Vector3(0, -0.15, 0),
-    quaternion: new THREE.Quaternion(1, 0, 0, 0),
-  },
-  {
-    url: PROPRIETARY_ASSETS_BASE_URL + '3dgs_scenes/alameda.spz',
-    scale: new THREE.Vector3(1.3, 1.3, 1.3),
-    position: new THREE.Vector3(0, 0, 0),
-    quaternion: new THREE.Quaternion(1, 0, 0, 0),
-  },
-];
-
-const FADE_DURATION_S = 1.0; // seconds
-const MOVE_SPEED = 0.05;
-
-function easeInOutSine(x) {
-  return -(Math.cos(Math.PI * x) - 1) / 2;
-}
-
-const forward = new THREE.Vector3();
-const right = new THREE.Vector3();
-const moveDirection = new THREE.Vector3();
-
-/**
- * An XR-Blocks demo that displays room-scale 3DGS models, allowing smooth
- * transitions via number keys (1, 2) or a 1.5 s long-pinch.
- */
-class WalkthroughManager extends xb.Script {
-  async init() {
-    this.add(new THREE.HemisphereLight(0xffffff, 0x666666, 3));
-
-    // Load all splat meshes in parallel.
-    this.splatMeshes = await Promise.all(
-      SPLAT_ASSETS.map(async (asset) => {
-        const mesh = new SplatMesh({url: asset.url});
-        await mesh.initialized;
-        mesh.position.copy(asset.position);
-        mesh.quaternion.copy(asset.quaternion);
-        mesh.scale.copy(asset.scale);
-        return mesh;
-      })
-    );
-
-    // Create a SparkRenderer for gaussian splat rendering and register it so
-    // the simulator can toggle encodeLinear for correct color space.
-    const sparkRenderer = new SparkRenderer({
-      renderer: xb.core.renderer,
-      maxStdDev: Math.sqrt(5),
-    });
-    xb.core.registry.register(new xb.SparkRendererHolder(sparkRenderer));
-    xb.add(sparkRenderer);
-
-    // Show the first splat.
-    this.currentIndex = 0;
-    xb.add(this.splatMeshes[this.currentIndex]);
-
-    // fadeProgress tracks animation time: null = idle, 0‥FADE_DURATION_S =
-    // fading out, FADE_DURATION_S‥2×FADE_DURATION_S = fading in.
-    this.fadeProgress = null;
-    this.nextIndex = null;
-
-    // Locomotion state.
-    this.locomotionOffset = new THREE.Vector3();
-    this.baseReferenceSpace = null;
-    this.keys = {w: false, a: false, s: false, d: false};
-
-    document.addEventListener('keydown', this.onKeyDown.bind(this));
-    document.addEventListener('keyup', this.onKeyUp.bind(this));
-
-    xb.add(
-      new LongSelectHandler(this.cycleSplat.bind(this), {
-        triggerDelay: 1500,
-        triggerCooldownDuration: 1500,
-      })
-    );
-  }
-
-  /** Starts a crossfade to the next splat (wrapping around). */
-  cycleSplat() {
-    if (this.fadeProgress !== null) return;
-    this.nextIndex = (this.currentIndex + 1) % this.splatMeshes.length;
-    this.fadeProgress = 0;
-  }
-
-  onKeyDown(event) {
-    const key = event.key.toLowerCase();
-    if (key in this.keys) this.keys[key] = true;
-
-    // Number key → jump to that splat (1-indexed).
-    const idx = parseInt(key, 10) - 1;
-    if (
-      idx >= 0 &&
-      idx < this.splatMeshes.length &&
-      idx !== this.currentIndex &&
-      this.fadeProgress === null
-    ) {
-      this.nextIndex = idx;
-      this.fadeProgress = 0;
-    }
-  }
-
-  onKeyUp(event) {
-    const key = event.key.toLowerCase();
-    if (key in this.keys) this.keys[key] = false;
-  }
-
-  onXRSessionEnded() {
-    super.onXRSessionEnded();
-    this.baseReferenceSpace = null;
-    this.locomotionOffset.set(0, 0, 0);
-  }
-
-  update() {
-    super.update();
-    const dt = xb.getDeltaTime();
-
-    this.updateFade(dt);
-    this.updateLocomotion();
-  }
-
-  /** Handles the fade-out → fade-in crossfade between splats. */
-  updateFade(dt) {
-    if (this.fadeProgress === null) return;
-
-    this.fadeProgress += dt;
-    const currentMesh = this.splatMeshes[this.currentIndex];
-
-    if (this.fadeProgress < FADE_DURATION_S) {
-      // Fading out the current splat.
-      currentMesh.opacity =
-        1 - easeInOutSine(this.fadeProgress / FADE_DURATION_S);
-    } else if (this.fadeProgress < 2 * FADE_DURATION_S) {
-      // Swap on the first frame of the fade-in phase.
-      if (currentMesh.parent) {
-        xb.scene.remove(currentMesh);
-        this.currentIndex = this.nextIndex;
-        const nextMesh = this.splatMeshes[this.currentIndex];
-        nextMesh.opacity = 0;
-        xb.add(nextMesh);
-      }
-      // Fading in the new splat.
-      const inProgress =
-        (this.fadeProgress - FADE_DURATION_S) / FADE_DURATION_S;
-      this.splatMeshes[this.currentIndex].opacity = easeInOutSine(inProgress);
-    } else {
-      // Fade complete.
-      this.splatMeshes[this.currentIndex].opacity = 1;
-      this.fadeProgress = null;
-      this.nextIndex = null;
-    }
-  }
-
-  /** WASD locomotion via XR reference space offset. */
-  updateLocomotion() {
-    const xr = xb.core.renderer?.xr;
-    if (!xr?.isPresenting) return;
-
-    const camera = xr.getCamera();
-    if (!camera) return;
-
-    camera.getWorldDirection(forward);
-    forward.y = 0;
-    forward.normalize();
-    right.crossVectors(forward, THREE.Object3D.DEFAULT_UP).normalize();
-
-    moveDirection.set(0, 0, 0);
-    if (this.keys.w) moveDirection.add(forward);
-    if (this.keys.s) moveDirection.sub(forward);
-    if (this.keys.a) moveDirection.sub(right);
-    if (this.keys.d) moveDirection.add(right);
-    if (moveDirection.lengthSq() === 0) return;
-    moveDirection.normalize();
-
-    if (!this.baseReferenceSpace) {
-      this.baseReferenceSpace = xr.getReferenceSpace();
-    }
-
-    this.locomotionOffset.addScaledVector(moveDirection, -MOVE_SPEED);
-    const transform = new XRRigidTransform(this.locomotionOffset);
-    xr.setReferenceSpace(
-      this.baseReferenceSpace.getOffsetReferenceSpace(transform)
-    );
-  }
-}
-
-document.addEventListener('DOMContentLoaded', function () {
-  const options = new xb.Options();
-  options.simulator.instructions.enabled = false;
-  options.reticles.enabled = false;
-  options.hands.enabled = true;
-  options.hands.visualization = true;
-  options.hands.visualizeMeshes = true;
-  options.simulator.scenePath = null; // Prevent simulator scene from loading.
-
-  xb.add(new WalkthroughManager());
-  xb.init(options);
-});
-
-</script>
-  </body>
-</html>