mudra-skills 1.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (149) hide show
  1. package/.claude-plugin/plugin.json +19 -0
  2. package/CLAUDE.md +35 -0
  3. package/bin/install.js +25 -0
  4. package/package.json +28 -0
  5. package/skills/mudra-master/SKILL.md +202 -0
  6. package/skills/mudra-master/mudra-preview/SKILL.md +151 -0
  7. package/skills/mudra-master/mudra-preview/assets/ar-menu.html +382 -0
  8. package/skills/mudra-master/mudra-preview/assets/document-scroller.html +468 -0
  9. package/skills/mudra-master/mudra-preview/assets/drum-machine.html +788 -0
  10. package/skills/mudra-master/mudra-preview/assets/emg-visualizer.html +390 -0
  11. package/skills/mudra-master/mudra-preview/assets/generative-art.html +408 -0
  12. package/skills/mudra-master/mudra-preview/assets/gesture-assistant.html +342 -0
  13. package/skills/mudra-master/mudra-preview/assets/gesture-speech.html +444 -0
  14. package/skills/mudra-master/mudra-preview/assets/hands-free-desktop.html +376 -0
  15. package/skills/mudra-master/mudra-preview/assets/model-rotator.html +365 -0
  16. package/skills/mudra-master/mudra-preview/assets/mudra-duel.html +3200 -0
  17. package/skills/mudra-master/mudra-preview/assets/mudra-monitor.html +1451 -0
  18. package/skills/mudra-master/mudra-preview/assets/mudra-ultimate-template.html +891 -0
  19. package/skills/mudra-master/mudra-preview/assets/music-sequencer.html +455 -0
  20. package/skills/mudra-master/mudra-preview/assets/neural-pong.html +380 -0
  21. package/skills/mudra-master/mudra-preview/assets/neural-snake.html +364 -0
  22. package/skills/mudra-master/mudra-preview/assets/presentation-controller.html +345 -0
  23. package/skills/mudra-master/mudra-preview/assets/pressure-painter.html +285 -0
  24. package/skills/mudra-master/mudra-preview/assets/runner.html +1812 -0
  25. package/skills/mudra-master/mudra-preview/assets/smart-home.html +461 -0
  26. package/skills/mudra-master/mudra-preview/assets/space-invaders.html +1154 -0
  27. package/skills/mudra-master/mudra-preview/assets/waterful-ring-toss.html +1624 -0
  28. package/skills/mudra-master/mudra-preview/references/agent_protocol.json +646 -0
  29. package/skills/mudra-master/mudra-preview/references/promt.md +16829 -0
  30. package/skills/mudra-master/mudra-xr/SKILL.md +245 -0
  31. package/skills/mudra-master/mudra-xr/assets/demos/3dgs-walkthrough.html +242 -0
  32. package/skills/mudra-master/mudra-xr/assets/demos/aisimulator.html +792 -0
  33. package/skills/mudra-master/mudra-xr/assets/demos/balloonpop.html +422 -0
  34. package/skills/mudra-master/mudra-xr/assets/demos/ballpit.html +284 -0
  35. package/skills/mudra-master/mudra-xr/assets/demos/drone.html +41 -0
  36. package/skills/mudra-master/mudra-xr/assets/demos/gemini-icebreakers.html +338 -0
  37. package/skills/mudra-master/mudra-xr/assets/demos/gemini-xrobject.html +302 -0
  38. package/skills/mudra-master/mudra-xr/assets/demos/math3d.html +188 -0
  39. package/skills/mudra-master/mudra-xr/assets/demos/measure.html +247 -0
  40. package/skills/mudra-master/mudra-xr/assets/demos/occlusion.html +251 -0
  41. package/skills/mudra-master/mudra-xr/assets/demos/rain.html +498 -0
  42. package/skills/mudra-master/mudra-xr/assets/demos/screenwiper.html +432 -0
  43. package/skills/mudra-master/mudra-xr/assets/demos/splash.html +519 -0
  44. package/skills/mudra-master/mudra-xr/assets/demos/webcam_gestures.html +253 -0
  45. package/skills/mudra-master/mudra-xr/assets/demos/xremoji.html +833 -0
  46. package/skills/mudra-master/mudra-xr/assets/demos/xrpoet.html +151 -0
  47. package/skills/mudra-master/mudra-xr/assets/samples/depthmap.html +266 -0
  48. package/skills/mudra-master/mudra-xr/assets/samples/depthmesh.html +100 -0
  49. package/skills/mudra-master/mudra-xr/assets/samples/game_rps.html +1099 -0
  50. package/skills/mudra-master/mudra-xr/assets/samples/gestures_custom.html +467 -0
  51. package/skills/mudra-master/mudra-xr/assets/samples/gestures_heuristic.html +256 -0
  52. package/skills/mudra-master/mudra-xr/assets/samples/lighting.html +163 -0
  53. package/skills/mudra-master/mudra-xr/assets/samples/mesh_detection.html +46 -0
  54. package/skills/mudra-master/mudra-xr/assets/samples/modelviewer.html +183 -0
  55. package/skills/mudra-master/mudra-xr/assets/samples/paint.html +133 -0
  56. package/skills/mudra-master/mudra-xr/assets/samples/planar-vst.html +45 -0
  57. package/skills/mudra-master/mudra-xr/assets/samples/reticle.html +95 -0
  58. package/skills/mudra-master/mudra-xr/assets/samples/skybox_agent.html +384 -0
  59. package/skills/mudra-master/mudra-xr/assets/samples/sound.html +390 -0
  60. package/skills/mudra-master/mudra-xr/assets/samples/ui.html +496 -0
  61. package/skills/mudra-master/mudra-xr/assets/samples/virtual-screens.html +808 -0
  62. package/skills/mudra-master/mudra-xr/assets/templates/0_basic.html +64 -0
  63. package/skills/mudra-master/mudra-xr/assets/templates/1_ui.html +107 -0
  64. package/skills/mudra-master/mudra-xr/assets/templates/2_hands.html +183 -0
  65. package/skills/mudra-master/mudra-xr/assets/templates/3_depth.html +89 -0
  66. package/skills/mudra-master/mudra-xr/assets/templates/4_stereo.html +84 -0
  67. package/skills/mudra-master/mudra-xr/assets/templates/5_camera.html +117 -0
  68. package/skills/mudra-master/mudra-xr/assets/templates/6_ai.html +154 -0
  69. package/skills/mudra-master/mudra-xr/assets/templates/7_ai_live.html +253 -0
  70. package/skills/mudra-master/mudra-xr/assets/templates/8_objects.html +95 -0
  71. package/skills/mudra-master/mudra-xr/assets/templates/9_xr-toggle.html +90 -0
  72. package/skills/mudra-master/mudra-xr/assets/templates/heuristic_hand_gestures.html +102 -0
  73. package/skills/mudra-master/mudra-xr/assets/templates/meshes.html +47 -0
  74. package/skills/mudra-master/mudra-xr/assets/templates/planes.html +47 -0
  75. package/skills/mudra-master/mudra-xr/assets/templates/uikit.html +187 -0
  76. package/skills/mudra-master/mudra-xr/references/agent_protocol.json +878 -0
  77. package/skills/mudra-master/mudra-xr/references/promt.md +1516 -0
  78. package/skills/mudra-preview/SKILL.md +151 -0
  79. package/skills/mudra-preview/assets/ar-menu.html +382 -0
  80. package/skills/mudra-preview/assets/document-scroller.html +468 -0
  81. package/skills/mudra-preview/assets/drum-machine.html +788 -0
  82. package/skills/mudra-preview/assets/emg-visualizer.html +390 -0
  83. package/skills/mudra-preview/assets/generative-art.html +408 -0
  84. package/skills/mudra-preview/assets/gesture-assistant.html +342 -0
  85. package/skills/mudra-preview/assets/gesture-speech.html +444 -0
  86. package/skills/mudra-preview/assets/hands-free-desktop.html +376 -0
  87. package/skills/mudra-preview/assets/model-rotator.html +365 -0
  88. package/skills/mudra-preview/assets/mudra-duel.html +3200 -0
  89. package/skills/mudra-preview/assets/mudra-monitor.html +1451 -0
  90. package/skills/mudra-preview/assets/mudra-ultimate-template.html +891 -0
  91. package/skills/mudra-preview/assets/music-sequencer.html +455 -0
  92. package/skills/mudra-preview/assets/neural-pong.html +380 -0
  93. package/skills/mudra-preview/assets/neural-snake.html +364 -0
  94. package/skills/mudra-preview/assets/presentation-controller.html +345 -0
  95. package/skills/mudra-preview/assets/pressure-painter.html +285 -0
  96. package/skills/mudra-preview/assets/runner.html +1812 -0
  97. package/skills/mudra-preview/assets/smart-home.html +461 -0
  98. package/skills/mudra-preview/assets/space-invaders.html +1154 -0
  99. package/skills/mudra-preview/assets/waterful-ring-toss.html +1624 -0
  100. package/skills/mudra-preview/references/agent_protocol.json +646 -0
  101. package/skills/mudra-preview/references/promt.md +16829 -0
  102. package/skills/mudra-xr/SKILL.md +245 -0
  103. package/skills/mudra-xr/assets/demos/3dgs-walkthrough.html +242 -0
  104. package/skills/mudra-xr/assets/demos/aisimulator.html +792 -0
  105. package/skills/mudra-xr/assets/demos/balloonpop.html +422 -0
  106. package/skills/mudra-xr/assets/demos/ballpit.html +284 -0
  107. package/skills/mudra-xr/assets/demos/drone.html +41 -0
  108. package/skills/mudra-xr/assets/demos/gemini-icebreakers.html +338 -0
  109. package/skills/mudra-xr/assets/demos/gemini-xrobject.html +302 -0
  110. package/skills/mudra-xr/assets/demos/math3d.html +188 -0
  111. package/skills/mudra-xr/assets/demos/measure.html +247 -0
  112. package/skills/mudra-xr/assets/demos/occlusion.html +251 -0
  113. package/skills/mudra-xr/assets/demos/rain.html +498 -0
  114. package/skills/mudra-xr/assets/demos/screenwiper.html +432 -0
  115. package/skills/mudra-xr/assets/demos/splash.html +519 -0
  116. package/skills/mudra-xr/assets/demos/webcam_gestures.html +253 -0
  117. package/skills/mudra-xr/assets/demos/xremoji.html +833 -0
  118. package/skills/mudra-xr/assets/demos/xrpoet.html +151 -0
  119. package/skills/mudra-xr/assets/samples/depthmap.html +266 -0
  120. package/skills/mudra-xr/assets/samples/depthmesh.html +100 -0
  121. package/skills/mudra-xr/assets/samples/game_rps.html +1099 -0
  122. package/skills/mudra-xr/assets/samples/gestures_custom.html +467 -0
  123. package/skills/mudra-xr/assets/samples/gestures_heuristic.html +256 -0
  124. package/skills/mudra-xr/assets/samples/lighting.html +163 -0
  125. package/skills/mudra-xr/assets/samples/mesh_detection.html +46 -0
  126. package/skills/mudra-xr/assets/samples/modelviewer.html +183 -0
  127. package/skills/mudra-xr/assets/samples/paint.html +133 -0
  128. package/skills/mudra-xr/assets/samples/planar-vst.html +45 -0
  129. package/skills/mudra-xr/assets/samples/reticle.html +95 -0
  130. package/skills/mudra-xr/assets/samples/skybox_agent.html +384 -0
  131. package/skills/mudra-xr/assets/samples/sound.html +390 -0
  132. package/skills/mudra-xr/assets/samples/ui.html +496 -0
  133. package/skills/mudra-xr/assets/samples/virtual-screens.html +808 -0
  134. package/skills/mudra-xr/assets/templates/0_basic.html +64 -0
  135. package/skills/mudra-xr/assets/templates/1_ui.html +107 -0
  136. package/skills/mudra-xr/assets/templates/2_hands.html +183 -0
  137. package/skills/mudra-xr/assets/templates/3_depth.html +89 -0
  138. package/skills/mudra-xr/assets/templates/4_stereo.html +84 -0
  139. package/skills/mudra-xr/assets/templates/5_camera.html +117 -0
  140. package/skills/mudra-xr/assets/templates/6_ai.html +154 -0
  141. package/skills/mudra-xr/assets/templates/7_ai_live.html +253 -0
  142. package/skills/mudra-xr/assets/templates/8_objects.html +95 -0
  143. package/skills/mudra-xr/assets/templates/9_xr-toggle.html +90 -0
  144. package/skills/mudra-xr/assets/templates/heuristic_hand_gestures.html +102 -0
  145. package/skills/mudra-xr/assets/templates/meshes.html +47 -0
  146. package/skills/mudra-xr/assets/templates/planes.html +47 -0
  147. package/skills/mudra-xr/assets/templates/uikit.html +187 -0
  148. package/skills/mudra-xr/references/agent_protocol.json +878 -0
  149. package/skills/mudra-xr/references/promt.md +1516 -0
@@ -0,0 +1,1516 @@
1
+ # Mudra XR Skill — Build Rules
2
+
3
+ This document is the canonical reference for the `mudra-xr` Claude Code skill.
4
+ Read it in full before generating any app. Every rule below is non-negotiable
5
+ unless explicitly marked as configurable.
6
+
7
+ ---
8
+
9
+ ## Section 1 — Mudra Protocol
10
+
11
+ ### WebSocket endpoint
12
+
13
+ ```
14
+ ws://127.0.0.1:8766
15
+ ```
16
+
17
+ Always construct the connection through `MudraClient` (Section 4).
18
+ Never use raw `new WebSocket(...)`.
19
+
20
+ ### Nine canonical signals
21
+
22
+ | Signal | Category | Description |
23
+ |--------|----------|-------------|
24
+ | `gesture` | Discrete | Hand gesture events (tap, double_tap, twist, double_twist) |
25
+ | `button` | Discrete | Button hold / release |
26
+ | `pressure` | Analog | Finger pressure 0–100, normalized 0–1 |
27
+ | `navigation` | Motion (Pointer) | Continuous delta_x / delta_y cursor movement |
28
+ | `nav_direction` | Motion (Direction) | Discrete directional swipes: None, Right, Left, Up, Down, Roll Left, Roll Right |
29
+ | `imu_acc` | Motion (IMU) | Accelerometer values [x, y, z] m/s², frequency 1125 Hz |
30
+ | `imu_gyro` | Motion (IMU) | Gyroscope values [x, y, z] deg/s, frequency 1125 Hz |
31
+ | `snc` | Biometric | 3 de-interleaved channel arrays [[ch1], [ch2], [ch3]] |
32
+ | `battery` | Status | Battery level 0–100, charging boolean |
33
+
34
+ ### Subscription handshake
35
+
36
+ Send one command per signal — never use plural `signals`, arrays, or batch commands:
37
+
38
+ ```js
39
+ // CORRECT
40
+ ws.send(JSON.stringify({ command: 'subscribe', signal: 'gesture' }));
41
+ ws.send(JSON.stringify({ command: 'subscribe', signal: 'pressure' }));
42
+
43
+ // WRONG — never do this
44
+ ws.send(JSON.stringify({ command: 'subscribe', signals: ['gesture', 'pressure'] }));
45
+ ws.send(JSON.stringify({ command: 'subscribe', signal: ['gesture', 'pressure'] }));
46
+ ```
47
+
48
+ ### Full command surface
49
+
50
+ `subscribe`, `unsubscribe`, `get_subscriptions`, `enable`, `disable`,
51
+ `get_status`, `get_docs`, `trigger_gesture`
52
+
53
+ ### Inbound message payload shapes
54
+
55
+ ```js
56
+ // gesture
57
+ { type: 'gesture', data: { type: 'tap'|'double_tap'|'twist'|'double_twist', confidence: 0–1, timestamp }, timestamp }
58
+
59
+ // button
60
+ { type: 'button', data: { state: 'pressed'|'released', timestamp }, timestamp }
61
+
62
+ // pressure
63
+ { type: 'pressure', data: { value: 0–100, normalized: 0–1, timestamp }, timestamp }
64
+
65
+ // navigation
66
+ { type: 'navigation', data: { delta_x: number, delta_y: number, timestamp }, timestamp }
67
+
68
+ // nav_direction
69
+ { type: 'nav_direction', data: { direction: 'Right'|'Left'|'Up'|'Down'|'Roll Left'|'Roll Right'|'None', timestamp }, timestamp }
70
+
71
+ // imu_acc
72
+ { type: 'imu_acc', data: { values: [x, y, z], frequency: 1125, timestamp }, timestamp }
73
+
74
+ // imu_gyro
75
+ { type: 'imu_gyro', data: { values: [x, y, z], frequency: 1125, timestamp }, timestamp }
76
+
77
+ // snc — extend rolling buffers (500 samples/channel) with all samples per callback
78
+ { type: 'snc', data: { values: [[ch1_samples], [ch2_samples], [ch3_samples]], timestamp }, timestamp }
79
+
80
+ // battery
81
+ { type: 'battery', data: { level: 0–100, charging: boolean, timestamp }, timestamp }
82
+
83
+ // connection_status
84
+ { type: 'connection_status', data: { status: 'connected'|'disconnected', message: string }, timestamp }
85
+ ```
86
+
87
+ ---
88
+
89
+ ## Section 2 — Canonical Dependency Pins
90
+
91
+ Use the exact versions below. Never use `@latest` or version ranges.
92
+ Import map `<script type="importmap">` must contain **only** the dependencies
93
+ the app actually uses — no unused entries.
94
+
95
+ ```json
96
+ {
97
+ "imports": {
98
+ "three": "https://cdn.jsdelivr.net/npm/three@0.182.0/build/three.module.js",
99
+ "three/addons/": "https://cdn.jsdelivr.net/npm/three@0.182.0/examples/jsm/",
100
+ "troika-three-text": "https://cdn.jsdelivr.net/gh/protectwise/troika@028b81cf308f0f22e5aa8e78196be56ec1997af5/packages/troika-three-text/src/index.js",
101
+ "troika-three-utils": "https://cdn.jsdelivr.net/gh/protectwise/troika@v0.52.4/packages/troika-three-utils/src/index.js",
102
+ "troika-worker-utils": "https://cdn.jsdelivr.net/gh/protectwise/troika@v0.52.4/packages/troika-worker-utils/src/index.js",
103
+ "bidi-js": "https://esm.sh/bidi-js@%5E1.0.2?target=es2022",
104
+ "webgl-sdf-generator": "https://esm.sh/webgl-sdf-generator@1.1.1/es2022/webgl-sdf-generator.mjs",
105
+ "lit": "https://cdn.jsdelivr.net/gh/lit/dist@3/core/lit-core.min.js",
106
+ "lit/": "https://esm.run/lit@3/",
107
+ "xrblocks": "https://cdn.jsdelivr.net/npm/xrblocks@0.11.0/build/xrblocks.js",
108
+ "xrblocks/addons/": "https://cdn.jsdelivr.net/npm/xrblocks@0.11.0/build/addons/"
109
+ }
110
+ }
111
+ ```
112
+
113
+ XR Blocks stylesheet (always `<link>` in `<head>`):
114
+
115
+ ```html
116
+ <link type="text/css" rel="stylesheet" href="https://xrblocks.github.io/css/xr.css" />
117
+ ```
118
+
119
+ Lit bundle warning suppression (place before import map):
120
+
121
+ ```html
122
+ <script>window.litDisableBundleWarning = true;</script>
123
+ ```
124
+
125
+ ---
126
+
127
+ ## Section 3 — XR Blocks Lifecycle Composition
128
+
129
+ ### Module-scope MudraClient
130
+
131
+ Instantiate `MudraClient` exactly once at module scope — not inside `init()`.
132
+ This ensures the WebSocket open-attempt starts immediately on page load so the
133
+ 1500 ms timeout (Section 4) begins counting before the XR scene initializes.
134
+
135
+ ```js
136
+ // Module scope — outside any class
137
+ const mudra = new MudraClient('ws://127.0.0.1:8766');
138
+
139
+ class MainScript extends xb.Script {
140
+ init() {
141
+ // Wire mudra handlers here after scene objects are created
142
+ mudra.on('gesture', (data) => { ... });
143
+ mudra.on('pressure', (data) => { ... });
144
+ }
145
+ update() { /* called every frame by xb */ }
146
+ }
147
+ ```
148
+
149
+ ### xb.Script class structure
150
+
151
+ Every generated app's top-level logic lives inside a class extending `xb.Script`.
152
+ The class name may be anything (`MainScript`, `CubeApp`, etc.) but must extend `xb.Script`.
153
+
154
+ ```js
155
+ class MainScript extends xb.Script {
156
+ /** Called once after XR Blocks and the WebGL context are ready. */
157
+ init() {
158
+ // Add lights, meshes, Mudra bindings here.
159
+ this.add(new THREE.HemisphereLight(0xffffff, 0x666666, 3));
160
+ // Place objects at xb.user.objectDistance in front of viewer:
161
+ this.mesh.position.set(0, xb.user.height - 0.5, -xb.user.objectDistance);
162
+ }
163
+
164
+ /** Called every animation frame. Keep cheap — no allocations. */
165
+ update() { }
166
+
167
+ /** Fires when a pinch/controller-trigger starts in XR. */
168
+ onSelectStart(event) { }
169
+
170
+ /** Fires when a pinch/controller-trigger ends in XR. */
171
+ onSelectEnd(event) { }
172
+
173
+ /** Fires every frame while a pinch is held in XR. */
174
+ onSelecting(event) { }
175
+ }
176
+ ```
177
+
178
+ ### Entry point
179
+
180
+ ```js
181
+ document.addEventListener('DOMContentLoaded', function () {
182
+ xb.add(new MainScript());
183
+ xb.init(new xb.Options());
184
+ });
185
+ ```
186
+
187
+ ### Key spatial constants
188
+
189
+ - `xb.user.height` — floor-relative eye height (approx. 1.6 m)
190
+ - `xb.user.objectDistance` — recommended arm-length distance for objects (approx. 0.8 m)
191
+ - Y-up coordinate system; Z is toward the viewer (negative Z = in front of you)
192
+
193
+ ---
194
+
195
+ ## Section 4 — Mock WebSocket Fallback (MudraClient)
196
+
197
+ ### Policy
198
+
199
+ - Start the WebSocket open attempt immediately on page load.
200
+ - If the WebSocket does not open within **1500 ms**, activate the mock automatically.
201
+ - If the WebSocket closes mid-session (band disconnect), flip to mock without a page reload.
202
+ - The mock fires exactly the same message format as the real device.
203
+ - App code must not branch on `_useMock` — it must receive the same messages either way.
204
+
205
+ ### Connection-status state machine
206
+
207
+ ```
208
+ [page load]
209
+ │
210
+ ▼
211
+ connecting ──── ws opens within 1500 ms ──→ connected
212
+ │
213
+ └── timeout or error ──────────────────→ simulated
214
+ │
215
+ connected ──── ws.onclose fires ───────────────→ disconnected-simulated
216
+ simulated ──── ws.onclose fires ───────────────→ (already simulated, no change)
217
+ ```
218
+
219
+ ### Required MudraClient implementation
220
+
221
+ Include this class verbatim in every generated app:
222
+
223
+ ```js
224
+ class MudraClient {
225
+ constructor(url) {
226
+ this._handlers = {};
227
+ this._subscriptions = new Set();
228
+ this._timers = [];
229
+ this._status = 'connecting';
230
+ this._notifyStatus('connecting');
231
+
232
+ const timeout = setTimeout(() => this._startMock(), 1500);
233
+
234
+ try {
235
+ this._ws = new WebSocket(url);
236
+ this._ws.onopen = () => {
237
+ clearTimeout(timeout);
238
+ this._status = 'connected';
239
+ this._notifyStatus('connected');
240
+ this._subscriptions.forEach(sig =>
241
+ this._ws.send(JSON.stringify({ command: 'subscribe', signal: sig }))
242
+ );
243
+ };
244
+ this._ws.onmessage = (e) => {
245
+ const msg = JSON.parse(e.data);
246
+ if (this._handlers[msg.type]) this._handlers[msg.type](msg.data);
247
+ };
248
+ this._ws.onclose = () => {
249
+ clearTimeout(timeout);
250
+ if (this._status === 'connected') {
251
+ this._status = 'disconnected-simulated';
252
+ this._notifyStatus('disconnected-simulated');
253
+ this._startMock();
254
+ }
255
+ };
256
+ this._ws.onerror = () => {
257
+ clearTimeout(timeout);
258
+ this._startMock();
259
+ };
260
+ } catch (_) {
261
+ clearTimeout(timeout);
262
+ this._startMock();
263
+ }
264
+ }
265
+
266
+ /** Register a handler for a signal type. Call before subscribe(). */
267
+ on(signal, fn) {
268
+ this._handlers[signal] = fn;
269
+ }
270
+
271
+ /** Subscribe to a signal. Safe to call before the WebSocket opens. */
272
+ subscribe(signal) {
273
+ this._subscriptions.add(signal);
274
+ if (this._ws && this._ws.readyState === WebSocket.OPEN) {
275
+ this._ws.send(JSON.stringify({ command: 'subscribe', signal }));
276
+ }
277
+ }
278
+
279
+ /** Send an arbitrary command to the band service. */
280
+ send(cmd) {
281
+ if (this._ws && this._ws.readyState === WebSocket.OPEN) {
282
+ this._ws.send(JSON.stringify(cmd));
283
+ } else if (cmd.command === 'trigger_gesture') {
284
+ this._dispatchMockGesture(cmd.data.type);
285
+ }
286
+ }
287
+
288
+ /** Current connection status string. */
289
+ get status() { return this._status; }
290
+
291
+ _notifyStatus(s) {
292
+ if (this._handlers['_status']) this._handlers['_status'](s);
293
+ }
294
+
295
+ _emit(payload) {
296
+ if (this._handlers[payload.type]) this._handlers[payload.type](payload.data);
297
+ }
298
+
299
+ _dispatchMockGesture(type) {
300
+ this._emit({ type: 'gesture', data: { type, confidence: 0.99, timestamp: Date.now() } });
301
+ }
302
+
303
+ _startMock() {
304
+ if (this._status === 'simulated' || this._status === 'disconnected-simulated') return;
305
+ const wasPreviouslyConnected = this._status === 'connected';
306
+ this._status = wasPreviouslyConnected ? 'disconnected-simulated' : 'simulated';
307
+ this._notifyStatus(this._status);
308
+ // Passive mock: no auto-firing. Signals fire ONLY from sim-panel clicks,
309
+ // keyboard shortcuts, or real WebSocket messages. This keeps the scene
310
+ // stable and makes the sim panel the single, explicit source of motion.
311
+ }
312
+
313
+ destroy() {
314
+ this._timers.forEach(t => clearInterval(t));
315
+ if (this._ws) this._ws.close();
316
+ }
317
+ }
318
+ ```
319
+
320
+ ---
321
+
322
+ ## Section 5 — Simulator Panel
323
+
324
+ ### Purpose
325
+
326
+ The simulator panel lets a user exercise every subscribed Mudra signal without
327
+ a band or XR headset. It is a **2D DOM overlay** (not spatial XR UI) and must
328
+ always be visible in flat-screen mode. It disappears automatically when the
329
+ browser transitions to an immersive WebXR session (native WebXR DOM suppression).
330
+
331
+ ### DOM structure
332
+
333
+ ```html
334
+ <div id="mudra-sim" style="
335
+ position: fixed; bottom: 0; left: 0; right: 0;
336
+ background: rgba(0,0,0,0.75); backdrop-filter: blur(4px);
337
+ padding: 8px 12px; display: flex; flex-wrap: wrap; gap: 8px;
338
+ z-index: 9999; font-family: system-ui, sans-serif;">
339
+ <!-- One button group per subscribed signal -->
340
+ </div>
341
+ ```
342
+
343
+ ### Button groups per signal (render ONLY the sub-actions the app actually uses)
344
+
345
+ The catalog below is the **maximal** set per signal. The simulator panel
346
+ MUST render **only** the sub-actions the generated app handles — never
347
+ extras. Examples:
348
+
349
+ - App maps `gesture.tap` only → render `Tap`. Omit `2Tap`, `Twist`, `2Twist`.
350
+ - App maps `nav_direction` to Up/Down only → render `↑`, `↓`. Omit
351
+ `←`, `→`, `Roll L`, `Roll R`.
352
+ - App uses `imu_acc` for X-axis tilt only → render `Tilt X+`, `Tilt X−`.
353
+ Omit `Tilt Y+`, `Tilt Y−`.
354
+
355
+ When in doubt, walk every `mudra.on('<signal>', …)` handler and emit a
356
+ button only for sub-actions referenced inside it. Unused buttons are a
357
+ checklist failure (see Section 10, item 6).
358
+
359
+ | Signal | Maximal buttons (subset based on app handlers) |
360
+ |--------|---------|
361
+ | `gesture` | `Tap`, `2Tap`, `Twist`, `2Twist` |
362
+ | `nav_direction` | `↑`, `↓`, `←`, `→`, `Roll L`, `Roll R` |
363
+ | `navigation` | `↑`, `↓`, `←`, `→` (each emits one delta event of ±3 — gentle/low-sensitivity default; raise per-app only if the prompt explicitly asks for fast/snappy movement) |
364
+ | `pressure` | Slider `0–100` (label shows current value) |
365
+ | `button` | `Press`, `Release` |
366
+ | `imu_acc` | `Tilt X+`, `Tilt X−`, `Tilt Y+`, `Tilt Y−` (5-frame burst at ±2 m/s²) |
367
+ | `imu_gyro` | `Rot X+`, `Rot X−`, `Rot Y+`, `Rot Y−` (5-frame burst at ±10 deg/s) |
368
+ | `snc` | `Spike` (burst of elevated samples on all 3 channels) |
369
+
370
+ ### Button firing rules
371
+
372
+ - Every button fires via the **same code path** as a real Mudra signal.
373
+ Call the same handler that `mudra.on(signal, handler)` would invoke.
374
+ - For `gesture` buttons: also call `mudra.send({ command: 'trigger_gesture', data: { type } })`
375
+ so the round-trip works when a real band is connected.
376
+ - Never duplicate logic between the simulator path and the real-signal path.
377
+
378
+ ```js
379
+ // Example: gesture sim button wires to the same handler as real signals
380
+ function simGesture(type) {
381
+ mudra.send({ command: 'trigger_gesture', data: { type } });
382
+ handleGesture({ type, confidence: 1.0, timestamp: Date.now() });
383
+ }
384
+
385
+ // Example: pressure slider
386
+ pressureSlider.addEventListener('input', () => {
387
+ const norm = pressureSlider.value / 100;
388
+ handlePressure({ value: +pressureSlider.value, normalized: norm, timestamp: Date.now() });
389
+ });
390
+ ```
391
+
392
+ ### Visibility
393
+
394
+ - `#mudra-sim` is always visible in flat-screen mode.
395
+ - Do NOT conditionally hide it when the band is connected (both sources are valid per FR-014).
396
+ - The 2D DOM disappears automatically in immersive WebXR — no JS required.
397
+
398
+ ---
399
+
400
+ ## Section 6 — Keyboard Shortcuts + XR Blocks Precedence
401
+
402
+ ### Canonical keyboard map
403
+
404
+ | Key | Signal / Action |
405
+ |-----|----------------|
406
+ | `Space` | `gesture` → tap |
407
+ | `Shift` | `button` → press (keydown) / release (keyup) |
408
+ | `[` | `pressure` decrease (−10, min 0) |
409
+ | `]` | `pressure` increase (+10, max 100) |
410
+ | `i` | `nav_direction` → Up OR `navigation` delta_y +3 |
411
+ | `k` | `nav_direction` → Down OR `navigation` delta_y −3 |
412
+ | `j` | `nav_direction` → Left OR `navigation` delta_x −3 |
413
+ | `l` | `nav_direction` → Right OR `navigation` delta_x +3 |
414
+
415
+ **Navigation sensitivity default**: keyboard `I`/`J`/`K`/`L` and sim panel
416
+ buttons emit deltas of **±3** per event (gentle / low-sensitivity
417
+ baseline). This keeps cursor/pan motion calm and predictable. Only raise
418
+ the magnitude when the prompt explicitly calls for fast/snappy movement
419
+ (racing, arcade-twitch, etc.).
420
+ | `u` | `imu_acc` tilt X+ burst |
421
+ | `o` | `imu_acc` tilt X− burst |
422
+ | `m` | `imu_gyro` rot Y+ burst |
423
+ | `n` | `imu_gyro` rot Y− burst |
424
+
425
+ For `imu` bursts: fire 5 synthetic frames at ±[2, 0, 9.81] m/s² (acc) or ±[10, 0, 0.5] deg/s (gyro).
426
+
427
+ ### Reserved keys & mouse — owned by XR Blocks desktop simulator
428
+
429
+ The XR Blocks desktop simulator (active whenever the app runs in a flat
430
+ browser without a WebXR session) owns the keys and mouse gestures that
431
+ let the user navigate the 3D scene. Mudra MUST NOT claim or
432
+ `stopPropagation()` any of these — they are how the user orbits, walks,
433
+ and zooms while previewing the app:
434
+
435
+ | Input | XR Blocks role |
436
+ |-------|----------------|
437
+ | `W` `A` `S` `D` | Walk camera forward / left / back / right |
438
+ | `ArrowUp` `ArrowDown` `ArrowLeft` `ArrowRight` | Camera nav (alt to WASD) |
439
+ | `Q` `E` | Camera roll / vertical |
440
+ | `R` | Reset camera pose |
441
+ | Right-click drag | Orbit / look around |
442
+ | Mouse wheel | Zoom in / out |
443
+
444
+ **Why Mudra uses `I`/`J`/`K`/`L` and `U`/`O`/`M`/`N`**: these keys are
445
+ explicitly off the XR Blocks reserved set, so the desktop simulator
446
+ keeps full camera control while the band-driven keyboard shortcuts
447
+ remain available for testing without a band. Never reassign a Mudra
448
+ shortcut onto a reserved key, even when the app does not subscribe to a
449
+ navigation signal.
450
+
451
+ ### Attachment rule (critical)
452
+
453
+ Mudra keyboard handlers MUST attach with `capture: true` and call
454
+ `event.stopPropagation()` on every key the app subscribes to. This prevents
455
+ XR Blocks' desktop-simulator bubble-phase listeners from double-firing.
456
+
457
+ ```js
458
+ window.addEventListener('keydown', (e) => {
459
+ switch (e.code) {
460
+ case 'Space':
461
+ e.stopPropagation();
462
+ simGesture('tap');
463
+ break;
464
+ case 'BracketLeft':
465
+ e.stopPropagation();
466
+ adjustPressure(-10);
467
+ break;
468
+ case 'BracketRight':
469
+ e.stopPropagation();
470
+ adjustPressure(+10);
471
+ break;
472
+ case 'KeyI':
473
+ e.stopPropagation();
474
+ handleNavDirection({ direction: 'Up', timestamp: Date.now() });
475
+ break;
476
+ // ... etc for subscribed keys only — using I/J/K/L for nav and U/O/M/N for IMU
477
+ }
478
+ }, { capture: true });
479
+ ```
480
+
481
+ Only intercept keys for signals the app actually subscribes to. Do NOT
482
+ `stopPropagation` on keys that no Mudra signal handles — XR Blocks needs
483
+ those, especially the reserved set above (WASD, arrows, Q/E/R, mouse).
484
+
485
+ ---
486
+
487
+ ## Section 7 — Connection-Status Indicator
488
+
489
+ ### Required DOM element
490
+
491
+ Every generated app must include exactly one visible element that reflects
492
+ the current `MudraClient` status:
493
+
494
+ ```html
495
+ <div id="mudra-status" style="
496
+ position: fixed; top: 8px; right: 12px;
497
+ padding: 4px 10px; border-radius: 999px;
498
+ font-size: 0.8rem; font-family: system-ui, sans-serif;
499
+ background: rgba(0,0,0,0.6); color: #fff;
500
+ z-index: 9999;">Connecting…</div>
501
+ ```
502
+
503
+ ### Text states
504
+
505
+ | MudraClient status | textContent |
506
+ |-------------------|-------------|
507
+ | `connecting` | `Connecting…` |
508
+ | `connected` | `Connected` |
509
+ | `simulated` | `Simulated` |
510
+ | `disconnected-simulated` | `Disconnected — simulated` |
511
+
512
+ ### Wiring
513
+
514
+ ```js
515
+ mudra.on('_status', (s) => {
516
+ const chip = document.getElementById('mudra-status');
517
+ const labels = {
518
+ 'connecting': 'Connecting…',
519
+ 'connected': 'Connected',
520
+ 'simulated': 'Simulated',
521
+ 'disconnected-simulated': 'Disconnected — simulated',
522
+ };
523
+ chip.textContent = labels[s] ?? s;
524
+ });
525
+ ```
526
+
527
+ ### Visibility
528
+
529
+ - Visible in flat-screen mode at all times.
530
+ - Disappears automatically in immersive XR (native DOM suppression).
531
+ - Place in the top-right corner by default; adapt if the template uses that space.
532
+
533
+ ---
534
+
535
+ ## Section 8 — Signal Grouping Rules (Non-Negotiable)
536
+
537
+ ### Signal group classification
538
+
539
+ | Group | Signals | Combine freely with |
540
+ |-------|---------|---------------------|
541
+ | **Pointer** | `navigation`, `button` | `gesture` OR `button` (not `pressure` if `gesture` is used) |
542
+ | **Direction** | `nav_direction` | `gesture` OR `pressure` OR `button` (but not `gesture`+`pressure` together) |
543
+ | **IMU+Biometric** | `imu_acc`, `imu_gyro`, `snc` | `gesture` OR `pressure` OR `button` (but not `gesture`+`pressure` together) |
544
+ | *(none)* | — | `gesture` OR `pressure` OR `button` (but not `gesture`+`pressure` together) |
545
+
546
+ ### Bundling rule — IMU+Biometric (CRITICAL)
547
+
548
+ `imu_acc`, `imu_gyro`, and `snc` are an **inseparable bundle**. If the user's prompt
549
+ implies any one of them, subscribe to **all three**. Never subscribe to only one or
550
+ two of them.
551
+
552
+ ```js
553
+ // CORRECT — all three always together
554
+ mudra.subscribe('imu_acc');
555
+ mudra.subscribe('imu_gyro');
556
+ mudra.subscribe('snc');
557
+
558
+ // WRONG — partial subscriptions
559
+ mudra.subscribe('snc'); // missing imu_acc and imu_gyro
560
+ mudra.subscribe('imu_acc'); // missing imu_gyro and snc
561
+ ```
562
+
563
+ ### XOR rules (all non-negotiable)
564
+
565
+ 1. **Gesture ⊕ Pressure** — an app may use `gesture` OR `pressure`, never both.
566
+ 2. **Navigation ⊕ Nav_direction** — an app may use `navigation` OR `nav_direction`, never both.
567
+ 3. **Pointer/Direction ⊕ IMU+Biometric** — `navigation` and `nav_direction` cannot be combined with the IMU+Biometric bundle (`imu_acc`/`imu_gyro`/`snc`).
568
+
569
+ ### Illegal combinations
570
+
571
+ ```
572
+ // REJECT these signal sets
573
+ gesture + pressure
574
+ navigation + nav_direction
575
+ navigation + imu_acc
576
+ navigation + imu_gyro
577
+ navigation + snc
578
+ nav_direction + imu_acc
579
+ nav_direction + imu_gyro
580
+ nav_direction + snc
581
+ button + nav_direction // button belongs to Pointer mode only
582
+ ```
583
+
584
+ ### Valid signal sets (examples)
585
+
586
+ ```
587
+ gesture + button
588
+ pressure + button
589
+ navigation + button
590
+ navigation + button + gesture
591
+ nav_direction
592
+ nav_direction + pressure + button
593
+ imu_acc + imu_gyro + snc
594
+ imu_acc + imu_gyro + snc + gesture
595
+ imu_acc + imu_gyro + snc + button
596
+ imu_acc + imu_gyro + snc + pressure + button
597
+ ```
598
+
599
+ ### Inference priority for ties
600
+
601
+ When the user prompt maps to multiple modes equally, ask one disambiguation
602
+ question — do not silently pick one.
603
+
604
+ Default to `direction` mode (`nav_direction`) when there is no navigation
605
+ language in the prompt at all and a motion mode is required by the template.
606
+
607
+ ---
608
+
609
+ ## Section 9 — AI API Key Handling (onboarding-gated, mandatory for AI apps)
610
+
611
+ ### Lifecycle: `sessionStorage` keyed by `mudra.gemini.apiKey`
612
+
613
+ For any generated app that calls a Gemini / LLM endpoint, the API key MUST
614
+ be entered through the **onboarding modal** before the user can use the
615
+ app. The key is stored in `sessionStorage` under the literal key
616
+ `mudra.gemini.apiKey` — it persists across reloads in the same tab and
617
+ clears when the tab closes. **Do NOT use `localStorage`. Do NOT use a
618
+ `prompt()` popup. Do NOT prompt on first AI call.**
619
+
620
+ ```js
621
+ // Read at app start
622
+ const apiKey = sessionStorage.getItem('mudra.gemini.apiKey');
623
+
624
+ // Write only from the onboarding "AI Setup" step
625
+ sessionStorage.setItem('mudra.gemini.apiKey', enteredKey);
626
+ ```
627
+
628
+ ### Required onboarding "AI Setup" fragment
629
+
630
+ For AI apps only, the onboarding modal MUST include the following
631
+ fragment inside `<section class="mudra-onb__body">`, placed AFTER the
632
+ actions table and BEFORE `</section>`:
633
+
634
+ ```html
635
+ <div class="mudra-onb__ai" data-uses-ai>
636
+ <h3 class="mudra-onb__ai-title">AI Setup</h3>
637
+ <p class="mudra-onb__ai-lede">
638
+ This app uses Google Gemini. Paste your API key to continue —
639
+ it's stored only in this browser tab (<code>sessionStorage</code>)
640
+ and is never sent anywhere except Google's API.
641
+ <a href="https://aistudio.google.com/" target="_blank" rel="noopener">Get a key →</a>
642
+ </p>
643
+ <input
644
+ id="mudra-onb-ai-key"
645
+ class="mudra-onb__ai-input"
646
+ type="password"
647
+ autocomplete="off"
648
+ spellcheck="false"
649
+ placeholder="Paste Gemini API key (starts with AIza…)"
650
+ aria-label="Gemini API key"
651
+ />
652
+ <p class="mudra-onb__ai-hint" data-role="hint"></p>
653
+ </div>
654
+ ```
655
+
656
+ And the matching CSS (added to the existing `<style>`):
657
+
658
+ ```css
659
+ .mudra-onb__ai { margin-top: 14px; padding-top: 12px; border-top: 1px solid #eee; }
660
+ .mudra-onb__ai-title { margin: 0 0 6px; font-size: 0.95rem; font-weight: 700; color: #111; }
661
+ .mudra-onb__ai-lede { margin: 0 0 10px; color: #555; font-size: 0.85rem; }
662
+ .mudra-onb__ai-lede a { color: #0d9488; }
663
+ .mudra-onb__ai-input {
664
+ width: 100%; box-sizing: border-box;
665
+ padding: 9px 12px; border: 1px solid #d0d0d0; border-radius: 8px;
666
+ font: 13px/1.4 ui-monospace, SFMono-Regular, Menlo, monospace;
667
+ background: #f8f9fb; color: #111;
668
+ }
669
+ .mudra-onb__ai-input:focus { outline: 2px solid #14b8a6; border-color: #14b8a6; background: #fff; }
670
+ .mudra-onb__ai-hint { margin: 6px 2px 0; font-size: 0.75rem; color: #b91c1c; min-height: 1em; }
671
+ .mudra-onb__continue:disabled { opacity: 0.45; cursor: not-allowed; }
672
+ ```
673
+
674
+ ### Behaviour rules (AI apps only)
675
+
676
+ 1. **`Got it` button starts disabled.** The IIFE that wires the modal
677
+ reads `dialog.querySelector('.mudra-onb__ai')` — if it exists, the
678
+ `.mudra-onb__continue` button is disabled until
679
+ `.mudra-onb__ai-input` is non-empty AND matches `/^AIza[\w-]{30,}$/`
680
+ (Google API-key prefix sanity check).
681
+ 2. **On click of `Got it`**, write the trimmed value to
682
+ `sessionStorage.setItem('mudra.gemini.apiKey', value)`, then close
683
+ the modal.
684
+ 3. **At every page load**, the IIFE checks `sessionStorage` first:
685
+ - If `mudra.gemini.apiKey` is present and matches the prefix regex,
686
+ the AI-Setup fragment is hidden (the user already provided a key
687
+ this session) and `Got it` is enabled immediately.
688
+ - If absent or malformed, the AI-Setup fragment is shown and the
689
+ modal CANNOT be dismissed by `Escape`, the `×` close button, or
690
+ the reopen `?` button without entering a valid key. `dialog.close()`
691
+ paths called from those handlers are no-ops while the key is
692
+ missing.
693
+ 4. **The `?` reopen button** for AI apps re-runs the gating logic. If
694
+ the user clears `sessionStorage` mid-session and reopens, the
695
+ AI-Setup fragment renders again.
696
+ 5. **Reading the key in app code:** the `xb.Script` subclass reads
697
+ `sessionStorage.getItem('mudra.gemini.apiKey')` in `init()`. If the
698
+ key is `null`, the AI portion of the app stays inert (no calls to
699
+ Gemini) and the visible chat panel (Section 18) renders a `Set up
700
+ AI in the welcome panel` placeholder.
701
+ 6. **Never bake a key into the HTML source.** Pre-write regex scan
702
+ (`/AIza[A-Za-z0-9_-]{30,}|sk-[A-Za-z0-9_-]{32,}/` excluding the
703
+ literal placeholder string `AIza…`) must return zero matches.
704
+ 7. **Never auto-read from URL params, `localStorage`, or `prompt()`.**
705
+ 8. **Non-AI apps** ignore this section entirely. The AI-Setup fragment
706
+ is omitted; the modal works as defined in Section 17 unchanged.
707
+
708
+ ### Canonical Gemini model — `gemini-2.5-flash` only
709
+
710
+ For any generated app that calls Gemini via the **REST `generateContent`
711
+ endpoint**, the model ID MUST be exactly `gemini-2.5-flash`. No other
712
+ model IDs are permitted for REST text/chat/vision generation. This is a
713
+ hard pin — preview aliases get retired by Google and the app then 404s.
714
+
715
+ ```js
716
+ // CORRECT — the only permitted REST model for text/chat/vision
717
+ const url = `https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent?key=${encodeURIComponent(apiKey)}`;
718
+
719
+ // FORBIDDEN — preview aliases, dated aliases, retired families
720
+ gemini-1.5-flash, gemini-1.5-flash-latest, gemini-1.5-pro
721
+ gemini-2.5-flash-preview-09-2025, gemini-2.5-flash-preview-04-2025
722
+ gemini-flash-latest, gemini-pro, gemini-2.5-flash-002 (any -NNN suffix)
723
+ ```
724
+
725
+ Pre-write regex scan — every generated HTML file MUST satisfy:
726
+
727
+ - `/generativelanguage\.googleapis\.com\/v1beta\/models\/([a-z0-9-]+):generateContent/`
728
+ → captured model ID MUST equal `gemini-2.5-flash`. Any other capture
729
+ fails the pre-write checklist.
730
+
731
+ Out-of-scope use cases (allowed exceptions, REST-only rule does NOT apply):
732
+
733
+ - **Live API** (WebSocket / xrblocks `xb.core.ai.startLiveSession()`):
734
+ the Live endpoint uses its own model set (`gemini-2.0-flash-live-001`,
735
+ `gemini-2.5-flash-native-audio-preview-12-2025`). Apps that genuinely
736
+ need streaming audio/video may use those, but every text-chat app
737
+ must use the REST pin above.
738
+ - **Image generation** via `:generateContent`: `gemini-2.5-flash-image`
739
+ is permitted only when the app's purpose is image output. Default to
740
+ the text pin otherwise.
741
+
742
+ If an app needs a different model, raise it to the user before writing —
743
+ do not silently swap in a preview alias.
744
+
745
+ ---
746
+
747
+ ## Section 10 — Pre-Write Checklist + Collision Handling
748
+
749
+ Before calling `Write` to emit a generated app, verify all items:
750
+
751
+ | # | Check | Pass condition |
752
+ |---|-------|----------------|
753
+ | 1 | Single file | Exactly one `<html>` document; all CSS in `<style>`; all JS in `<script>` or `<script type="module">` |
754
+ | 2 | Import map | One `<script type="importmap">` block; contents match canonical pins (Section 2) exactly; no unused entries |
755
+ | 3 | xb.Script entry | Top-level logic inside `class <Name> extends xb.Script`; `xb.add(new <Name>())` + `xb.init(new xb.Options())` on `DOMContentLoaded` |
756
+ | 4 | MudraClient | One `MudraClient` instance at module scope; URL = `ws://127.0.0.1:8766`; does NOT auto-connect; `setMode()` drives connect/disconnect |
757
+ | 5 | Subscribe commands | Every used signal has exactly one `mudra.subscribe('<signal>')` call; none outside the signal set |
758
+ | 6 | Simulator panel | `<div id="mudra-sim">` present; ONLY buttons for sub-actions actually handled by the app (no extras like Roll L/R or Twist if unused); buttons fire via handler, not inline `onclick` |
759
+ | 7 | Keyboard bindings | `window.addEventListener('keydown', …, { capture: true })` present; `event.stopPropagation()` on every Mudra-claimed key |
760
+ | 8 | Status indicator | `<div id="mudra-status">` present; text states are `Manual` / `Connecting…` / `Connected` / `Disconnected` (Section 15); no `simulated` strings |
761
+ | 9 | AI key gating | If `usesAI`: the AI-Setup fragment (Section 9) is present inside `.mudra-onb__body`; key is read from `sessionStorage.getItem('mudra.gemini.apiKey')` only; ZERO `prompt(` calls for the key; ZERO `localStorage` references; ZERO baked keys (regex scan) |
762
+ | 9a | Visible AI chat I/O | If `usesAI`: the scene renders BOTH the latest user input AND the AI response as visible text (xb.ScrollingTroikaTextView, troika `Text`, or xb.SpatialPanel rows). The visible "Purpose" line states what the app does in one sentence. TTS may exist but is never the only output (Section 18) |
763
+ | 10 | Background lockdown | ZERO `applyBackground_*` methods in the class; ZERO calls to a background helper from `init()`; no `options.simulator.scenePath` line anywhere. Generated apps use the XR Blocks default room only (Section 14) |
764
+ | 11 | Mode toggle | `<div id="mode-toggle">` with **Manual** + **Mudra** buttons; Manual is the default on load; toggle remains clickable when disconnected; flipping atomically opens/closes the socket per Section 15 |
765
+ | 12 | Band-state polling | In Mudra mode the app sends `{command:"get_status"}` on `ws.onopen` and every 2000 ms thereafter; pill flips to `Connected` ONLY when `data.device.state === "connected"` |
766
+ | 13 | No disconnect overlay | No banner / toast / modal / inline alert ever rendered for disconnect — pill is the only indicator |
767
+ | 14 | Footer | Exactly one `<div id="mudra-badge">` containing the literal text `Created by Mudra` (no variants) |
768
+ | 15 | Mock is passive | `MudraClient._startMock()` (or equivalent) starts NO intervals — synthetic signals come only from sim-panel clicks and keyboard shortcuts |
769
+ | 16 | Gemini model pin | If the app calls `generativelanguage.googleapis.com/v1beta/models/<id>:generateContent`, the captured `<id>` MUST equal `gemini-2.5-flash`. No preview / dated / latest aliases. Live-API and image-gen exceptions per Section 9 |
770
+
771
+ ### Retry policy
772
+
773
+ If any check fails:
774
+ 1. Regenerate once and re-run the full checklist.
775
+ 2. If the second attempt also fails, surface the specific failing check(s)
776
+ to the user and do **not** write the file.
777
+
778
+ ### File collision / auto-suffix rule
779
+
780
+ - Target filename: `preview/<name>.html`
781
+ - If that file already exists, try `preview/<name>-2.html`, then `-3.html`, etc.
782
+ - Never overwrite an existing file.
783
+ - Always report the final written path to the user.
784
+
785
+ ---
786
+
787
+ ## Section 11 — Signal → XR Binding Patterns
788
+
789
+ Use these as code snippets when adapting a template.
790
+ Copy and adapt — do not invent new patterns from scratch.
791
+
792
+ ### gesture.tap → onSelectStart forwarding
793
+
794
+ ```js
795
+ // In init():
796
+ mudra.on('gesture', (data) => {
797
+ if (data.type === 'tap') this.onSelectStart({ source: 'mudra' });
798
+ if (data.type === 'double_tap') this.onSelectEnd({ source: 'mudra' });
799
+ });
800
+ mudra.subscribe('gesture');
801
+ ```
802
+
803
+ ### pressure → scale / color mapping
804
+
805
+ ```js
806
+ // In init():
807
+ mudra.on('pressure', (data) => {
808
+ const s = 0.5 + data.normalized * 1.5; // scale 0.5 … 2.0
809
+ this.mesh.scale.setScalar(s);
810
+ const hue = data.normalized * 0.8; // hue 0 (red) … 0.8 (blue)
811
+ this.mesh.material.color.setHSL(hue, 0.9, 0.5);
812
+ });
813
+ mudra.subscribe('pressure');
814
+ ```
815
+
816
+ ### nav_direction → spatial menu step
817
+
818
+ ```js
819
+ // In init():
820
+ mudra.on('nav_direction', (data) => {
821
+ switch (data.direction) {
822
+ case 'Up': this.menuIndex = Math.max(0, this.menuIndex - 1); break;
823
+ case 'Down': this.menuIndex = Math.min(this.items.length - 1, this.menuIndex + 1); break;
824
+ case 'Right': this.selectItem(this.menuIndex); break;
825
+ case 'Left': this.goBack(); break;
826
+ }
827
+ this.updateMenuHighlight();
828
+ });
829
+ mudra.subscribe('nav_direction');
830
+ ```
831
+
832
+ ### imu_acc + imu_gyro + snc bundle → subscribe all three together
833
+
834
+ `imu_acc`, `imu_gyro`, and `snc` are always subscribed together. Register handlers
835
+ for each signal you actually use in the app, but always send all three subscribe commands.
836
+
837
+ ```js
838
+ // In init():
839
+ mudra.on('imu_acc', (data) => {
840
+ const [ax, ay] = data.values;
841
+ this.mesh.rotation.x = THREE.MathUtils.clamp(ax * 0.1, -Math.PI / 4, Math.PI / 4);
842
+ this.mesh.rotation.z = THREE.MathUtils.clamp(ay * 0.1, -Math.PI / 4, Math.PI / 4);
843
+ });
844
+ mudra.on('imu_gyro', (data) => {
845
+ const [gx] = data.values;
846
+ this.mesh.rotation.y += gx * 0.001;
847
+ });
848
+ mudra.on('snc', (data) => {
849
+ const ch1 = data.values[0];
850
+ const latest = ch1[ch1.length - 1]; // most recent sample
851
+ const norm = Math.min(1, Math.abs(latest) / 500); // normalize
852
+ this.overlay.material.opacity = norm * 0.7;
853
+ });
854
+ // ALWAYS subscribe all three — they form an inseparable bundle
855
+ mudra.subscribe('imu_acc');
856
+ mudra.subscribe('imu_gyro');
857
+ mudra.subscribe('snc');
858
+ ```
859
+
860
+ ### navigation → continuous cursor / pan
861
+
862
+ **Default sensitivity multiplier: `0.002`** (gentle / low-sensitivity).
863
+ Use this baseline for all navigation/cursor/pan bindings. Only raise it
864
+ when the prompt explicitly asks for fast/snappy movement.
865
+
866
+ ```js
867
+ // In init():
868
+ const NAV_SENSITIVITY = 0.002; // gentle default — slow, predictable
869
+ mudra.on('navigation', (data) => {
870
+ this.cursorX = THREE.MathUtils.clamp(this.cursorX + data.delta_x * NAV_SENSITIVITY, -1, 1);
871
+ this.cursorY = THREE.MathUtils.clamp(this.cursorY - data.delta_y * NAV_SENSITIVITY, -1, 1);
872
+ this.cursor.position.set(this.cursorX, this.cursorY, -xb.user.objectDistance);
873
+ });
874
+ mudra.subscribe('navigation');
875
+ ```
876
+
877
+ ---
878
+
879
+ ## Section 12 — Template Selection Table
880
+
881
+ One row per asset in `assets/`. The skill's keyword-based selector scores each row
882
+ against the user's prompt (signal names, motion keywords, XR feature words).
883
+
884
+ | id | path | keywords | motionModesSupported | xrFeatures |
885
+ |----|------|----------|---------------------|------------|
886
+ | `0_basic` | `assets/templates/0_basic.html` | `["basic","cylinder","pinch","color","simple","starter"]` | `["none","pointer"]` | `["input"]` |
887
+ | `1_ui` | `assets/templates/1_ui.html` | `["ui","spatial","panel","text","sdf","font","button","draggable","troika"]` | `["pointer"]` | `["input","ui"]` |
888
+ | `2_hands` | `assets/templates/2_hands.html` | `["hands","hand","pinch","gesture","joints","finger","hand-tracking"]` | `["pointer"]` | `["hands","input"]` |
889
+ | `3_depth` | `assets/templates/3_depth.html` | `["depth","mesh","depth-sensing","plane","environment","occlusion"]` | `["none"]` | `["depth-sensing","mesh-detection"]` |
890
+ | `4_stereo` | `assets/templates/4_stereo.html` | `["stereo","video","passthrough","camera","background","environment","feed"]` | `["none"]` | `["camera","passthrough"]` |
891
+ | `5_camera` | `assets/templates/5_camera.html` | `["camera","video","passthrough","texture","scene","background"]` | `["none","pointer"]` | `["camera","input"]` |
892
+ | `6_ai` | `assets/templates/6_ai.html` | `["ai","gemini","query","vision","photo","capture","llm","multimodal"]` | `["pointer"]` | `["input","camera"]` |
893
+ | `7_ai_live` | `assets/templates/7_ai_live.html` | `["ai","gemini","live","speech","transcription","voice","microphone","audio","real-time"]` | `["pointer"]` | `["input"]` |
894
+ | `8_objects` | `assets/templates/8_objects.html` | `["objects","detection","model","3d","place","anchor","ar","environment"]` | `["pointer"]` | `["mesh-detection","input"]` |
895
+ | `9_xr-toggle` | `assets/templates/9_xr-toggle.html` | `["toggle","xr","enter","exit","session","button","transition"]` | `["none","pointer"]` | `["input"]` |
896
+ | `heuristic_hand_gestures` | `assets/templates/heuristic_hand_gestures.html` | `["gesture","heuristic","hand","recognition","custom","pattern","hand-tracking"]` | `["none"]` | `["hands"]` |
897
+ | `meshes` | `assets/templates/meshes.html` | `["mesh","environment","scan","plane","floor","wall","surface"]` | `["none"]` | `["mesh-detection"]` |
898
+ | `planes` | `assets/templates/planes.html` | `["plane","floor","wall","surface","anchor","environment","horizontal","vertical"]` | `["none"]` | `["plane-detection"]` |
899
+ | `uikit` | `assets/templates/uikit.html` | `["ui","kit","component","widget","button","icon","material","text","panel","menu"]` | `["pointer"]` | `["input","ui"]` |
900
+ | `depthmap` | `assets/samples/depthmap.html` | `["depth","map","visualization","color","gradient","environment","scan"]` | `["none"]` | `["depth-sensing"]` |
901
+ | `depthmesh` | `assets/samples/depthmesh.html` | `["depth","mesh","wireframe","environment","scan","geometry"]` | `["none"]` | `["depth-sensing","mesh-detection"]` |
902
+ | `game_rps` | `assets/samples/game_rps.html` | `["game","rps","rock","paper","scissors","gesture","compete","fun","hand"]` | `["none"]` | `["hands"]` |
903
+ | `gestures_custom` | `assets/samples/gestures_custom.html` | `["gesture","custom","recognize","train","pose","hand-tracking","hands"]` | `["none"]` | `["hands"]` |
904
+ | `gestures_heuristic` | `assets/samples/gestures_heuristic.html` | `["gesture","heuristic","recognize","hand","pose","pinch","open","fist"]` | `["none"]` | `["hands"]` |
905
+ | `lighting` | `assets/samples/lighting.html` | `["lighting","light","shadow","scene","animals","3d","models","environment"]` | `["pointer"]` | `["input"]` |
906
+ | `mesh_detection` | `assets/samples/mesh_detection.html` | `["mesh","detection","environment","scan","ar","surface"]` | `["none"]` | `["mesh-detection"]` |
907
+ | `modelviewer` | `assets/samples/modelviewer.html` | `["model","viewer","3d","gltf","glb","object","rotate","inspect","load"]` | `["pointer"]` | `["input"]` |
908
+ | `paint` | `assets/samples/paint.html` | `["paint","draw","brush","stroke","canvas","art","color","gesture"]` | `["pointer"]` | `["input","hands"]` |
909
+ | `planar-vst` | `assets/samples/planar-vst.html` | `["plane","vst","passthrough","video","portal","ar","surface"]` | `["none"]` | `["plane-detection","camera"]` |
910
+ | `reticle` | `assets/samples/reticle.html` | `["reticle","cursor","pointer","aim","gaze","target","floor","placement"]` | `["none","pointer"]` | `["plane-detection","input"]` |
911
+ | `skybox_agent` | `assets/samples/skybox_agent.html` | `["skybox","sky","background","ai","gemini","generate","environment","image"]` | `["pointer"]` | `["input"]` |
912
+ | `sound` | `assets/samples/sound.html` | `["sound","audio","music","spatial","3d-audio","positional","play"]` | `["pointer"]` | `["input"]` |
913
+ | `ui` | `assets/samples/ui.html` | `["ui","panel","button","menu","list","text","interface","spatial"]` | `["pointer"]` | `["input","ui"]` |
914
+ | `virtual-screens` | `assets/samples/virtual-screens.html` | `["screen","virtual","window","share","stream","desktop","browser","display"]` | `["pointer"]` | `["input"]` |
915
+ | `3dgs-walkthrough` | `assets/demos/3dgs-walkthrough.html` | `["gaussian","splat","3dgs","scene","walkthrough","room","photo","realistic"]` | `["imu","direction"]` | `["input"]` |
916
+ | `aisimulator` | `assets/demos/aisimulator.html` | `["ai","simulate","gemini","agent","roleplay","character","conversation","npc"]` | `["pointer"]` | `["input"]` |
917
+ | `balloonpop` | `assets/demos/balloonpop.html` | `["balloon","pop","game","gesture","fun","pinch","particle","explosion"]` | `["none"]` | `["hands","input"]` |
918
+ | `ballpit` | `assets/demos/ballpit.html` | `["ball","physics","pit","throw","gravity","interact","fun","ammo"]` | `["pointer"]` | `["input","hands"]` |
919
+ | `drone` | `assets/demos/drone.html` | `["drone","fly","navigate","control","imu","tilt","direction","pilot"]` | `["imu","direction"]` | `["input"]` |
920
+ | `gemini-icebreakers` | `assets/demos/gemini-icebreakers.html` | `["gemini","icebreaker","question","conversation","ai","social","fun"]` | `["pointer"]` | `["input"]` |
921
+ | `gemini-xrobject` | `assets/demos/gemini-xrobject.html` | `["gemini","object","recognize","camera","ar","label","identify","vision"]` | `["pointer"]` | `["input","camera"]` |
922
+ | `math3d` | `assets/demos/math3d.html` | `["math","3d","formula","graph","equation","plot","visualization","education"]` | `["pointer"]` | `["input","ui"]` |
923
+ | `measure` | `assets/demos/measure.html` | `["measure","ruler","tape","distance","ar","depth","spatial","length"]` | `["none"]` | `["depth-sensing","mesh-detection"]` |
924
+ | `occlusion` | `assets/demos/occlusion.html` | `["occlusion","depth","animal","model","shadow","ar","realistic","environment"]` | `["pointer"]` | `["input","depth-sensing"]` |
925
+ | `rain` | `assets/demos/rain.html` | `["rain","particle","weather","atmosphere","shader","visual","instanced"]` | `["none","pointer"]` | `["input"]` |
926
+ | `screenwiper` | `assets/demos/screenwiper.html` | `["wiper","screen","wipe","clear","gesture","brush","clean","effect"]` | `["pointer"]` | `["input","hands"]` |
927
+ | `splash` | `assets/demos/splash.html` | `["splash","paint","decal","physics","shoot","color","ball","impact"]` | `["pointer"]` | `["input"]` |
928
+ | `webcam_gestures` | `assets/demos/webcam_gestures.html` | `["webcam","mediapipe","gesture","hand","camera","tracking","no-headset","flat"]` | `["none"]` | `[]` |
929
+ | `xremoji` | `assets/demos/xremoji.html` | `["emoji","expression","face","gesture","fun","balloon","tensorflow","social"]` | `["none"]` | `["hands"]` |
930
+ | `xrpoet` | `assets/demos/xrpoet.html` | `["poem","poetry","ai","gemini","generate","creative","camera","writing"]` | `["pointer"]` | `["input","camera"]` |
931
+
932
+ ---
933
+
934
+ ## Section 13 — Template Inference + Override
935
+
936
+ *(Populated after Phase 5 / T032.)*
937
+
938
+ ---
939
+
940
+ ## Section 14 — Background Lockdown (XR Blocks default only)
941
+
942
+ **Custom backgrounds are forbidden.** Every generated XR app uses the XR
943
+ Blocks default room and nothing else. There is no catalog, no helper, no
944
+ override, no `[bg=...]` tag, no `scenePath` override.
945
+
946
+ ### Hard rules — apply unconditionally
947
+
948
+ 1. The `xb.Script` subclass MUST NOT contain any `applyBackground_*` method.
949
+ 2. `init()` MUST NOT call any background helper. It starts with lights,
950
+ meshes, and Mudra wiring.
951
+ 3. The entry point MUST NOT set `options.simulator.scenePath` — not to
952
+ `null`, not to a path. Leave it alone so XR Blocks renders its default
953
+ room.
954
+ 4. No `THREE.SphereGeometry` dome, no custom skybox `Mesh`, no
955
+ `THREE.Points` starfield, no `THREE.GridHelper` floor, no equirectangular
956
+ `TextureLoader().load(...)` for background purposes. (Per-scene
957
+ geometry that the app actually needs is fine — the ban is on standalone
958
+ environment domes/floors/skies.)
959
+ 5. Prompt cues like "in space", "starfield", "sunset sky", "cyberpunk
960
+ vibe", "with a forest backdrop", or even literal `[bg=<id>]` tags MUST
961
+ be IGNORED for background purposes. They may still inform template /
962
+ motion-mode selection.
963
+ 6. If the user explicitly insists on a custom background, decline and
964
+ remind them that this skill is locked to the XR Blocks default room.
965
+
966
+ ### Pre-write regex (Section 10 check #10)
967
+
968
+ The generated source MUST satisfy ALL of the following:
969
+
970
+ - `/applyBackground_/` → zero matches.
971
+ - `/options\.simulator\.scenePath/` → zero matches.
972
+
973
+ If either pattern matches, the file fails the pre-write checklist and is
974
+ not written.
975
+
976
+ ---
977
+ ## Section 15 — Mode Toggle (Manual / Mudra) — Required
978
+
979
+ **This section supersedes Section 4's auto-fallback "simulated" status and
980
+ Section 7's `simulated` / `disconnected-simulated` states for all new
981
+ apps.** Every XR app generated by this skill MUST implement the Mode
982
+ Toggle exactly as specified here. Canonical protocol:
983
+ `references/agent_protocol.json` (v2.0).
984
+
985
+ ### Summary
986
+
987
+ The user picks how the app is driven via a visible **Mode** control,
988
+ rendered as a 2D DOM overlay (it disappears automatically in immersive
989
+ WebXR sessions, like the simulator panel and status pill):
990
+
991
+ - **Manual** (default on load): the simulator panel is fully interactive.
992
+ Sim-panel actions inject synthetic signal messages into the same handler
993
+ pipeline that real WebSocket messages would flow through. **No WebSocket
994
+ connection is opened in Manual mode.** The XR scene reacts only to
995
+ signal handlers — never to direct DOM clicks bypassing the signal path.
996
+ - **Mudra**: the app opens one WebSocket to `ws://127.0.0.1:8766` and
997
+ subscribes to its signals one-at-a-time. The simulator panel is
998
+ visually disabled (greyed-out, `pointer-events: none`,
999
+ `aria-disabled="true"`) and emits no synthetic signals. The
1000
+ connection-status pill reflects band-pairing state via `get_status`
1001
+ polling — **no separate overlay, toast, or banner is rendered.**
1002
+
1003
+ **The Mode toggle MUST remain fully clickable and keyboard-focusable at
1004
+ all times** — including while the band is disconnected. Manual is the
1005
+ default on first load.
1006
+
1007
+ ### State machine
1008
+
1009
+ ```text
1010
+ type Mode = "manual" | "mudra" // default "manual"
1011
+ type ConnectionState =
1012
+ | "idle" // No socket open. Always the case in Manual.
1013
+ | "connecting" // Socket opening, OR socket open but band-pairing not yet confirmed via get_status.
1014
+ | "connected" // Socket open AND last get_status response had data.device.state === "connected".
1015
+ | "disconnected" // Socket closed/errored, OR socket open but data.device.state !== "connected".
1016
+ ```
1017
+
1018
+ Lazy-WS lifecycle (mandatory):
1019
+
1020
+ | Transition | Action |
1021
+ |------------|--------|
1022
+ | page load | `mode = "manual"`, `connectionState = "idle"`, NO socket |
1023
+ | Manual → Mudra | open new socket, set `connectionState = "connecting"` |
1024
+ | Mudra → Manual | close socket, cancel any in-flight reconnect timer, set `connectionState = "idle"` |
1025
+ | WS `open` (in Mudra) | keep `connectionState = "connecting"`, send all `subscribe` commands, send `{command:"get_status"}`, start status-poll timer |
1026
+ | inbound `status` with `data.device.state === "connected"` (in Mudra) | `connectionState = "connected"` |
1027
+ | inbound `status` with `data.device.state !== "connected"` (in Mudra) | `connectionState = "disconnected"`, **keep socket open** — do NOT closeSocket, do NOT schedule WS reconnect; status-poll will surface the band coming back |
1028
+ | inbound `connection_status: connected` (in Mudra) | request a fresh `get_status`; do not flip the pill on this alone |
1029
+ | inbound `connection_status: disconnected` (in Mudra) | `connectionState = "disconnected"` |
1030
+ | WS error / WS close (in Mudra) | `connectionState = "disconnected"`, stop status-poll, schedule socket reconnect |
1031
+ | reconnect tick (in Mudra & socket dead) | open new socket → `connecting` |
1032
+
1033
+ **Single-socket guarantee:** never have two `WebSocket` instances open at
1034
+ once. Use a connection token to neutralise rapid-toggle races (see the
1035
+ `MudraClient` extensions below).
1036
+
1037
+ ### MudraClient changes vs. Section 4
1038
+
1039
+ The `MudraClient` from Section 4 must be extended (or replaced) so it:
1040
+
1041
+ 1. Does NOT auto-connect in its constructor. Connection is driven by
1042
+ `setMode("mudra")` only.
1043
+ 2. Exposes `setMode(mode)` to flip between `"manual"` and `"mudra"`.
1044
+ 3. In Manual mode, the `_startMock()` interval generators (if any) are
1045
+ NOT started. Mock signals must be **passive**: emitted only when the
1046
+ sim panel is clicked or a keyboard shortcut fires. (Memory:
1047
+ `MudraClient mock must be passive` — strip auto-firing intervals from
1048
+ `_startMock()`; sim panel clicks and keys are the only signal source.)
1049
+ 4. On entering Mudra mode, opens one WebSocket, sends `subscribe` for
1050
+ every signal in its subscription list, sends `{command:"get_status"}`
1051
+ immediately, and starts a 2 s `get_status` poll while the socket is
1052
+ `OPEN`.
1053
+ 5. Emits `_status` events with the new four-state vocabulary:
1054
+ `"idle" | "connecting" | "connected" | "disconnected"`. The legacy
1055
+ `"simulated"` / `"disconnected-simulated"` strings are removed.
1056
+
1057
+ ### Disconnect detection — band state via `get_status` polling (mandatory)
1058
+
1059
+ **The WebSocket handshake to `127.0.0.1:8766` only proves the Companion
1060
+ service is up. It does NOT prove the user's Mudra Band is paired.** The
1061
+ Companion accepts socket connections even when no band is bonded — so
1062
+ flipping the pill to "Connected" on `ws.onopen` is wrong. The pill MUST
1063
+ reflect the band itself, not the socket.
1064
+
1065
+ The source of truth is the `status` response to `{command:"get_status"}`:
1066
+
1067
+ ```json
1068
+ > {"command":"get_status"}
1069
+ < {"type":"status","data":{"device":{"state":"connected", ... }, ...}, "timestamp": ...}
1070
+ < {"type":"status","data":{"device":{"state":"disconnected", ...}, ...}, "timestamp": ...}
1071
+ ```
1072
+
1073
+ Rules:
1074
+
1075
+ 1. On `ws.onopen` (in Mudra mode): stay in `connecting`; send all
1076
+ `subscribe` commands; send `{command:"get_status"}`; start a
1077
+ **status-poll timer** that sends `{command:"get_status"}` every
1078
+ **2000 ms** while `mode === "mudra"` and the socket is `OPEN`.
1079
+ 2. On inbound `{type:"status"}` (in Mudra mode):
1080
+ - `data?.device?.state === "connected"` → `setState("connected")`.
1081
+ - Else → `setState("disconnected")`. Keep the socket open. Do NOT
1082
+ `closeSocket()`. The next poll tick picks the band up after pairing.
1083
+ 3. On inbound `{type:"connection_status"}`: hint only. On `disconnected`,
1084
+ flip the pill. On `connected`, send a fresh `{command:"get_status"}`
1085
+ and let the `status` handler do the actual transition.
1086
+ 4. On WS `error` / `close` (in Mudra mode): `setState("disconnected")`,
1087
+ stop the status-poll timer, `scheduleReconnect()`.
1088
+ 5. On Manual mode: stop the status-poll timer in `closeSocket()`.
1089
+
1090
+ Do not poll faster than 1 s; do not poll slower than 5 s. 2 s is
1091
+ mandated. The pill MAY sit on `Connecting…` for up to one poll cycle
1092
+ (~2 s) after entering Mudra mode while the first `status` round-trips —
1093
+ that is correct behaviour.
1094
+
1095
+ ### Reconnect backoff
1096
+
1097
+ While `mode === "mudra" && connectionState === "disconnected"` AND the
1098
+ socket itself is dead (not just the band):
1099
+
1100
+ ```js
1101
+ const RECONNECT_DELAYS_MS = [1000, 2000, 5000, 5000, 5000]; // capped at 5s
1102
+ ```
1103
+
1104
+ Reset the index on every successful `connected` transition.
1105
+
1106
+ ### Status pill text states (replaces Section 7)
1107
+
1108
+ | connectionState | textContent | colour hint |
1109
+ |-----------------|-------------|-------------|
1110
+ | `idle` (Manual) | `Manual` | neutral |
1111
+ | `connecting` | `Connecting…` | amber |
1112
+ | `connected` | `Connected` | green |
1113
+ | `disconnected` | `Disconnected` | red |
1114
+
1115
+ The pill is the **only** disconnect indicator. No banner, toast, or
1116
+ modal. The simulator panel is greyed (reduced opacity,
1117
+ `pointer-events: none`) when in Mudra + `disconnected`, but the pill is
1118
+ still the only textual disconnect cue.
1119
+
1120
+ ### Mode-toggle DOM sketch
1121
+
1122
+ ```html
1123
+ <div id="mode-toggle" role="tablist" aria-label="Mode">
1124
+ <button id="mode-manual" role="tab" aria-selected="true">Manual</button>
1125
+ <button id="mode-mudra" role="tab" aria-selected="false">Mudra</button>
1126
+ </div>
1127
+ <div id="mudra-status" class="conn-manual">Manual</div>
1128
+ ```
1129
+
1130
+ On every mode change, atomically: cancel any reconnect timer, stop the
1131
+ status-poll, close any open socket, reset `connToken`, then either
1132
+ (Manual) leave `connectionState = "idle"` OR (Mudra) call `openSocket()`.
1133
+
1134
+ ---
1135
+
1136
+ ## Section 16 — Footer Branding — "Created by Mudra"
1137
+
1138
+ Every generated app MUST render a small footer/badge with the **exact**
1139
+ text **`Created by Mudra`**. Never "Created with Mudra Studio", never
1140
+ "Powered by Mudra", never any other variant. The badge is a 2D DOM
1141
+ overlay (disappears in immersive WebXR like the rest of the chrome).
1142
+
1143
+ ```html
1144
+ <div id="mudra-badge" style="
1145
+ position: fixed; bottom: 8px; right: 12px;
1146
+ padding: 4px 10px; border-radius: 999px;
1147
+ font-size: 0.75rem; font-family: system-ui, sans-serif;
1148
+ background: rgba(0,0,0,0.5); color: #fff; opacity: 0.85;
1149
+ z-index: 9999; pointer-events: none;">Created by Mudra</div>
1150
+ ```
1151
+
1152
+ Position is flexible if it would overlap the simulator panel — keep the
1153
+ text identical.
1154
+
1155
+ ---
1156
+
1157
+ ## Section 17 — Onboarding Modal (mandatory, STRICT) — feature 008-strict-onboarding-templates
1158
+
1159
+ > **Supersedes feature 005's loose modal rules.** The locked layout is
1160
+ > **Template 3 — split-card**. The DOM, CSS, and JS below MUST be emitted
1161
+ > **verbatim**. Only the per-app content slots may vary.
1162
+
1163
+ Every generated XR app MUST ship a first-run onboarding modal that greets
1164
+ the user and lists every action the app supports, with paired **Mudra** and
1165
+ **Manual** controls per action. The modal closes via `×` (skip), the
1166
+ **Continue** button, or `Escape`. It re-opens via a small floating `?` icon
1167
+ **only outside immersive XR** — in-XR re-onboarding is out of scope for v1
1168
+ (per feature 005 clarification Q3).
1169
+
1170
+ The binding contract is `specs/008-strict-onboarding-templates/contracts/onboarding-block.md`. The blocks below are the verbatim copies emitted into every app.
1171
+
1172
+ ### XR-specific behavior (read carefully — unchanged from feature 005)
1173
+
1174
+ The onboarding modal is a **2D HTML overlay** shown before immersive entry.
1175
+
1176
+ - **MANDATORY: Disable the XR Blocks default Welcome overlay.** When the
1177
+ Simulator addon is imported (`import 'xrblocks/addons/simulator/SimulatorAddons.js';`),
1178
+ XR Blocks injects its own "Welcome to XR Blocks!" intro modal. This
1179
+ overlay competes visually with the Mudra onboarding modal and must be
1180
+ suppressed. Immediately after constructing the options object, set:
1181
+
1182
+ ```js
1183
+ const options = new xb.Options();
1184
+ options.simulator.instructions.enabled = false; // suppress XR Blocks default Welcome overlay; the Mudra onboarding modal replaces it
1185
+ // ...rest of options config
1186
+ xb.init(options);
1187
+ ```
1188
+
1189
+ - **Do NOT** mirror the Mudra modal as a 3D panel inside the XR scene. In-XR re-onboarding is explicitly out of scope for v1.
1190
+ - The inline `<script>` already listens for `xrsession-start` / `vr-session-start` / `ar-session-start` events and hides both the modal and the `?` icon during immersive sessions. Do NOT edit that wiring.
1191
+ - The modal layers above the XR canvas in 2D mode (`z-index: 100`). The Mudra badge from Section 16 and the simulator panel sit beneath it, by design.
1192
+
1193
+ ### Required palette addition
1194
+
1195
+ Every generated app's `:root` MUST add one variable beyond the canonical palette:
1196
+
1197
+ ```css
1198
+ --on-primary: #0c0d10; /* dark text on the primary-blue Continue button */
1199
+ ```
1200
+
1201
+ ### Locked DOM (paste at end of `<body>`)
1202
+
1203
+ ```html
1204
+ <!-- === BEGIN onboarding-block === (Template 3 — Split Card, feature 008) -->
1205
+ <!-- IMPORTANT: do NOT add the `open` attribute. In 3D apps this matters
1206
+ particularly — XR Blocks' canvas is appended to <body> AFTER this
1207
+ dialog, and a non-modal dialog (HTML `open`) does NOT enter the
1208
+ browser's top layer. The dialog looks visible but the canvas can
1209
+ intercept clicks on Continue / ×. The IIFE below calls showModal()
1210
+ on load, which elevates the dialog above XR Blocks' canvas
1211
+ regardless of z-index. -->
1212
+ <dialog id="mudra-onboarding" data-mudra-onboarding data-app-name="{APP_NAME}">
1213
+ <div class="ob-card">
1214
+ <button class="ob-x" aria-label="Skip onboarding" data-ob-close>×</button>
1215
+ <div class="ob-left">
1216
+ <div class="ob-brand-block">
1217
+ <span class="ob-brand-mark">Mudra Studio</span>
1218
+ <h2 class="ob-brand-name">{APP_NAME_HEAD} <em>{APP_NAME_TAIL}</em></h2>
1219
+ <p class="ob-tagline">{APP_TAGLINE}</p>
1220
+ </div>
1221
+ <span class="ob-brand-footer">Created by Mudra</span>
1222
+ </div>
1223
+ <div class="ob-right">
1224
+ <h3 class="ob-section-title">How to use this app</h3>
1225
+ <div class="ob-chip-grid" id="ob-rows"></div>
1226
+ <div class="ob-continue-row"><button class="ob-continue" data-ob-close>Continue</button></div>
1227
+ </div>
1228
+ </div>
1229
+ </dialog>
1230
+ <button id="mudra-onboarding-help" class="ob-help-btn" aria-label="Reopen onboarding" hidden>?</button>
1231
+ <!-- === END onboarding-block === -->
1232
+ ```
1233
+
1234
+ ### Locked CSS (paste inside the existing `<style>` block in `<head>` — add one if the template does not have one)
1235
+
1236
+ ```css
1237
+ /* === BEGIN onboarding-block === (Template 3 — Split Card, feature 008) */
1238
+ #mudra-onboarding{position:fixed;inset:0;border:0;padding:0;background:transparent;width:100%;height:100%;max-width:none;max-height:none;display:grid;place-items:center;z-index:100;color:var(--text);}
1239
+ #mudra-onboarding::backdrop{background:rgba(0,0,0,0.55);backdrop-filter:blur(6px);}
1240
+ #mudra-onboarding[hidden],#mudra-onboarding:not([open]){display:none;}
1241
+ .ob-card{position:relative;background:var(--card);backdrop-filter:blur(10px);border:1px solid rgba(255,255,255,0.08);border-radius:18px;width:min(720px,94vw);max-height:88vh;overflow:auto;display:grid;grid-template-columns:1fr 1fr;gap:0;box-shadow:0 20px 60px rgba(0,0,0,0.5);font-family:'Poppins',system-ui,sans-serif;}
1242
+ .ob-x{position:absolute;top:14px;right:14px;width:32px;height:32px;border-radius:50%;appearance:none;border:0;background:rgba(255,255,255,0.08);color:var(--text);font-size:18px;cursor:pointer;z-index:2;}
1243
+ .ob-x:hover{background:rgba(255,255,255,0.16);}
1244
+ .ob-left{padding:36px 28px;background:linear-gradient(135deg,rgba(108,140,255,0.16),rgba(185,124,255,0.12));border-right:1px solid rgba(255,255,255,0.06);display:flex;flex-direction:column;justify-content:space-between;border-radius:18px 0 0 18px;}
1245
+ .ob-brand-block{display:flex;flex-direction:column;gap:14px;}
1246
+ .ob-brand-mark{display:flex;align-items:center;gap:10px;font-size:13px;letter-spacing:0.16em;text-transform:uppercase;color:var(--text-secondary);}
1247
+ .ob-brand-mark::before{content:"";display:inline-block;width:24px;height:2px;background:var(--primary);}
1248
+ .ob-brand-name{font-size:34px;font-weight:700;line-height:1.05;margin:0;}
1249
+ .ob-brand-name em{font-style:normal;color:var(--accent);}
1250
+ .ob-tagline{color:var(--text-secondary);font-size:15px;margin:6px 0 0;line-height:1.5;}
1251
+ .ob-brand-footer{font-size:11px;letter-spacing:0.18em;text-transform:uppercase;color:var(--text-secondary);}
1252
+ .ob-right{padding:32px 28px 24px;display:flex;flex-direction:column;}
1253
+ .ob-section-title{font-size:12px;letter-spacing:0.1em;text-transform:uppercase;color:var(--text-secondary);margin:0 0 14px;}
1254
+ .ob-chip-grid{display:grid;grid-template-columns:1fr;gap:8px;flex:1;}
1255
+ .ob-chip{display:grid;grid-template-columns:1fr auto auto;gap:10px;align-items:center;background:rgba(255,255,255,0.04);border:1px solid rgba(255,255,255,0.05);border-radius:10px;padding:10px 12px;font-size:14px;}
1256
+ .ob-chip .nm{font-weight:500;}
1257
+ .ob-chip .mu{font-size:12px;padding:3px 9px;border-radius:999px;background:rgba(108,140,255,0.2);color:var(--primary);font-weight:600;}
1258
+ .ob-chip .mn{font-size:12px;color:var(--text-secondary);}
1259
+ .ob-continue-row{display:flex;justify-content:flex-end;margin-top:18px;}
1260
+ .ob-continue{appearance:none;border:0;background:var(--primary);color:var(--on-primary);font:inherit;font-weight:600;padding:10px 22px;border-radius:10px;cursor:pointer;}
1261
+ .ob-continue:hover{filter:brightness(1.1);}
1262
+ .ob-help-btn{position:fixed;bottom:16px;right:16px;appearance:none;border:0;width:36px;height:36px;border-radius:50%;background:var(--card);color:var(--text);font-size:18px;cursor:pointer;backdrop-filter:blur(10px);border:1px solid rgba(255,255,255,0.12);z-index:50;}
1263
+ @media (max-width:640px){
1264
+ .ob-card{grid-template-columns:1fr;}
1265
+ .ob-left{border-right:0;border-bottom:1px solid rgba(255,255,255,0.06);border-radius:18px 18px 0 0;padding:24px;}
1266
+ .ob-brand-name{font-size:24px;}
1267
+ .ob-right{padding:20px;}
1268
+ .ob-continue{width:100%;}
1269
+ .ob-continue-row{justify-content:stretch;}
1270
+ }
1271
+ /* === END onboarding-block === */
1272
+ ```
1273
+
1274
+ ### Locked JS (paste inside an inline `<script>` — NOT `type="module"` — at end of `<body>`)
1275
+
1276
+ ```js
1277
+ // === BEGIN onboarding-block === (Template 3 — Split Card, feature 008)
1278
+ window.MUDRA_ONBOARDING_ACTIONS = [
1279
+ // Filled by the skill from the app's subscribed signals. See "App-aware filter — strict" below.
1280
+ ];
1281
+
1282
+ (function () {
1283
+ const root = document.getElementById('mudra-onboarding');
1284
+ const help = document.getElementById('mudra-onboarding-help');
1285
+ const grid = document.getElementById('ob-rows');
1286
+
1287
+ grid.innerHTML = window.MUDRA_ONBOARDING_ACTIONS.map(r => `
1288
+ <div class="ob-chip">
1289
+ <span class="nm">${r.action}</span>
1290
+ <span class="mu">${r.mudra}</span>
1291
+ <span class="mn">${r.manual}</span>
1292
+ </div>`).join('');
1293
+
1294
+ function isInImmersiveXR() {
1295
+ return !!(window.xb && window.xb.session && window.xb.session.isImmersive);
1296
+ }
1297
+ function openOb() { if (!root.open) root.showModal(); help.hidden = true; }
1298
+ function closeOb() { if (root.open) root.close(); help.hidden = false; }
1299
+
1300
+ // Close wiring — BOTH `.ob-x` and `.ob-continue` carry [data-ob-close].
1301
+ // This single querySelectorAll attaches the same `closeOb` to each;
1302
+ // do not add separate listeners or split the behavior.
1303
+ root.querySelectorAll('[data-ob-close]').forEach(b => b.addEventListener('click', closeOb));
1304
+
1305
+ // Re-open wiring — floating help-pill (mouse/touch) and the `?` key.
1306
+ // `?` is gated on isInImmersiveXR() so it never re-opens inside VR/AR.
1307
+ help.addEventListener('click', openOb);
1308
+ document.addEventListener('keydown', (e) => {
1309
+ if (e.key === 'Escape' && root.open) closeOb();
1310
+ if (e.key === '?' && !root.open && !isInImmersiveXR()) openOb();
1311
+ });
1312
+
1313
+ // Hide modal + help during immersive XR sessions (preserves feature 005's contract).
1314
+ ['xrsession-start','vr-session-start','ar-session-start'].forEach(ev =>
1315
+ window.addEventListener(ev, () => { if (root.open) root.close(); help.hidden = true; })
1316
+ );
1317
+
1318
+ // Open on load — UNCONDITIONAL showModal(). The dialog enters the
1319
+ // browser's top layer so XR Blocks' canvas (appended after this
1320
+ // script) can never intercept clicks on Continue / ×. Do not gate on
1321
+ // `root.open`; do not use `root.show()`.
1322
+ root.showModal();
1323
+ })();
1324
+ // === END onboarding-block ===
1325
+ ```
1326
+
1327
+ ### Close behavior — what each control does (verbatim explanation for the model)
1328
+
1329
+ | Control | When it fires | Effect |
1330
+ |---|---|---|
1331
+ | **Continue** (`.ob-continue`) | User clicks / activates the primary CTA in the right column | `root.close()`. Help-pill `?` becomes visible bottom-right. User starts using the app (or enters XR via the XR Blocks button). |
1332
+ | **×** (`.ob-x`) | User clicks / activates the circular X in the top-right of the card | Same as Continue — `root.close()` + show `?` pill. The two paths are intentionally equivalent. X is the visual "skip" affordance. (Exception: AI-Setup extension blocks both Continue and × until a valid API key is entered — see "AI-app extension" below.) |
1333
+ | **Escape** | User presses Esc while modal is open | Native `<dialog>` close + our `keydown` handler unhides the `?` pill. (AI extension also intercepts this via the `cancel` event when no valid key is set.) |
1334
+ | **`?` key** | User presses `?` while modal is closed AND NOT inside immersive XR | `root.showModal()` reopens. Inside VR/AR the key is a no-op so the user is never yanked out of the scene. |
1335
+ | **Help-pill** (`#mudra-onboarding-help`) | User clicks the floating `?` button bottom-right (visible only after the modal has been closed once, and hidden during immersive XR) | `root.showModal()` reopens. |
1336
+ | **`xrsession-start` / `vr-session-start` / `ar-session-start`** | User enters VR/AR via the XR Blocks Enter button | Force-close the modal; hide the help-pill so it does not appear in-world. |
1337
+ | **Page load** | Every fresh render | `root.showModal()` runs unconditionally inside the IIFE. Help-pill starts hidden. |
1338
+
1339
+ ### How "close" is wired (REQUIRED — do not refactor)
1340
+
1341
+ 1. Both close buttons carry the `data-ob-close` attribute. The single line `root.querySelectorAll('[data-ob-close]').forEach(b => b.addEventListener('click', closeOb))` is the entire mouse/touch close wiring. The AI-Setup extension below adds its own `capture:true` listener on top of these to gate close when no API key is present — it does not replace the underlying wiring.
1342
+ 2. Keyboard close is handled by the `document`-level `keydown` listener so Escape fires even when focus is outside the dialog.
1343
+ 3. Opening on load uses `root.showModal()` (modal mode, top layer). Never `root.show()` (non-modal) and never the `open` attribute in HTML. In 3D apps this is REQUIRED — non-modal dialogs do not enter the top layer and XR Blocks' canvas can intercept clicks behind the modal.
1344
+
1345
+ ### Anti-patterns — will fail review
1346
+
1347
+ - ❌ `<dialog ... open>` in HTML. Open via `showModal()` only.
1348
+ - ❌ Calling `root.show()` instead of `root.showModal()`.
1349
+ - ❌ Omitting `options.simulator.instructions.enabled = false;` — XR Blocks' Welcome overlay then competes with the Mudra modal.
1350
+ - ❌ Adding click-outside-to-close, time-out auto-close, or any extra close path.
1351
+ - ❌ Hiding `#mudra-onboarding-help` permanently after first close — it must reappear (and remain hidden ONLY during immersive XR).
1352
+ - ❌ Wiring `closeOb` to a button that lacks the `data-ob-close` attribute.
1353
+ - ❌ Differentiating × from Continue behaviorally (e.g., "X means skip, Continue means save"). They are the same close path.
1354
+
1355
+ ### Per-app content slots — the ONLY things you may vary
1356
+
1357
+ | Slot | Source | Notes |
1358
+ |---|---|---|
1359
+ | `{APP_NAME}` (`data-app-name`) | Derived from generated HTML filename | Override only to fix acronym capitalization (e.g., `data-app-name="AR Menu"` for `ar-menu.html`). |
1360
+ | `{APP_NAME_HEAD}` / `{APP_NAME_TAIL}` | App name split into leading word + trailing word(s). Trailing word gets the `<em>` accent. | Single-word names: HEAD is whole word, TAIL is empty (emit `<em></em>`). |
1361
+ | `{APP_TAGLINE}` | One-line description. MUST end with a period. MUST NOT exceed 90 characters. | Example: "Stack blocks with a flick of the wrist." |
1362
+ | `MUDRA_ONBOARDING_ACTIONS` | Per `actions-array.md`. | See filter rule below. |
1363
+
1364
+ ### `MUDRA_ONBOARDING_ACTIONS` shape (renamed from feature-005 `ACTIONS`)
1365
+
1366
+ ```js
1367
+ window.MUDRA_ONBOARDING_ACTIONS = [
1368
+ { action: "Toggle XR session", mudra: "Thumb tap", manual: "Enter", mode: "gesture" },
1369
+ { action: "Look around", mudra: "Lift wrist", manual: "Right-click + drag", mode: "imu_acc" },
1370
+ { action: "Move", mudra: "Swipe", manual: "W A S D", mode: "nav_direction" }
1371
+ ];
1372
+ ```
1373
+
1374
+ Each row has four required fields:
1375
+
1376
+ - **`action`** — the behavior in plain English (NOT the control name).
1377
+ - **`mudra`** — the Mudra control prose (`"Tap"`, `"Twist"`, `"Press 70%"`, `"Tilt left"`).
1378
+ - **`manual`** — keyboard / mouse fallback. Use `"—"` (em dash) if there is no Manual equivalent (common in XR for camera / look controls).
1379
+ - **`mode`** — one of: `gesture` | `button` | `pressure` | `navigation` | `nav_direction` | `imu_acc` | `imu_gyro` | `snc`.
1380
+
1381
+ ### App-aware filter — STRICT (feature 008, FR-010)
1382
+
1383
+ Before emitting `MUDRA_ONBOARDING_ACTIONS`, the skill MUST filter:
1384
+
1385
+ 1. **Build the subscribed set** — every canonical signal this specific app subscribes to.
1386
+ 2. **Drop every row** whose `mode` is not in the subscribed set. **Forbidden** to emit a row for an unsubscribed signal — that is a generation-time bug, not a runtime filter.
1387
+ 3. **Verify exactly one motion mode** (`navigation` | `nav_direction` | `imu_acc`/`imu_gyro` family) across all rows (Constitution Principle III).
1388
+ 4. **Verify Manual ↔ Mudra parity** — every row has both a Mudra and a Manual cell; `"—"` is the only acceptable Manual placeholder.
1389
+ 5. **No collisions** — no two rows share the same `manual` value or the same effective `mudra` value.
1390
+
1391
+ ### Anti-patterns (will fail review)
1392
+
1393
+ - ❌ Omitting `options.simulator.instructions.enabled = false;` — the XR Blocks default Welcome overlay then duplicates / overrides the Mudra onboarding modal.
1394
+ - ❌ Mirroring the modal as a 3D panel inside the XR scene (out of scope v1).
1395
+ - ❌ Editing the `xrsession-start` / `xrsession-end` hide-show wiring.
1396
+ - ❌ Emitting a row whose `mode` is not in the app's subscribed set.
1397
+ - ❌ Mixing two motion modes in the same array.
1398
+ - ❌ A row with `manual: null` or `manual: ""` — use `"—"`.
1399
+ - ❌ Two rows with the same `manual` value — keyboard collision.
1400
+ - ❌ Branding strings other than literal `Created by Mudra`. ("Mudra Studio" stays only as the `.ob-brand-mark` line above the app name.)
1401
+ - ❌ CTA labels other than `Continue`.
1402
+
1403
+ Binding contracts: `specs/008-strict-onboarding-templates/contracts/onboarding-block.md` and `specs/008-strict-onboarding-templates/contracts/actions-array.md`.
1404
+
1405
+ ### AI-app extension (mandatory when `usesAI`)
1406
+
1407
+ The AI-Setup fragment is rendered as an **extra row inside `.ob-chip-grid`** — not as a separate section. The fragment goes at the **top** of the chip grid (above all action chips) so the user sees it before any controls.
1408
+
1409
+ ```html
1410
+ <!-- Inject AS THE FIRST CHILD of #ob-rows when usesAI is true -->
1411
+ <div class="ob-chip ob-chip-ai" id="ob-chip-ai" data-role="ai-setup">
1412
+ <span class="nm">AI key</span>
1413
+ <input class="mudra-onb__ai-input" type="password" placeholder="AIza…" aria-label="Gemini API key" autocomplete="off" />
1414
+ <span class="mn" data-role="hint" aria-live="polite"></span>
1415
+ </div>
1416
+ ```
1417
+
1418
+ ```css
1419
+ /* Append to the locked onboarding-block CSS for AI-using apps only */
1420
+ .ob-chip-ai{grid-template-columns:auto 1fr auto;}
1421
+ .ob-chip-ai .mudra-onb__ai-input{appearance:none;background:rgba(255,255,255,0.04);color:var(--text);border:1px solid rgba(255,255,255,0.12);border-radius:8px;padding:6px 10px;font:inherit;font-size:13px;}
1422
+ .ob-chip-ai .mudra-onb__ai-input:focus{outline:2px solid var(--primary);outline-offset:0;}
1423
+ .ob-chip-ai .mn[data-role="hint"]{color:var(--warning);}
1424
+ ```
1425
+
1426
+ The IIFE that wires the modal MUST be extended with this gating block, inserted just before the final unconditional `root.showModal();`:
1427
+
1428
+ ```js
1429
+ // AI-Setup gating — only runs when the fragment is present (usesAI === true)
1430
+ const aiFragment = document.getElementById('ob-chip-ai');
1431
+ const continueBtn = root.querySelector('.ob-continue');
1432
+ const closeBtn = root.querySelector('.ob-x');
1433
+ const KEY_NAME = "mudra.gemini.apiKey";
1434
+ const KEY_REGEX = /^AIza[\w-]{30,}$/;
1435
+
1436
+ const hasValidStoredKey = () => {
1437
+ const k = sessionStorage.getItem(KEY_NAME);
1438
+ return typeof k === "string" && KEY_REGEX.test(k.trim());
1439
+ };
1440
+
1441
+ if (aiFragment) {
1442
+ const input = aiFragment.querySelector(".mudra-onb__ai-input");
1443
+ const hint = aiFragment.querySelector('[data-role="hint"]');
1444
+
1445
+ const refreshGate = () => {
1446
+ if (hasValidStoredKey()) {
1447
+ aiFragment.hidden = true;
1448
+ continueBtn.disabled = false;
1449
+ return;
1450
+ }
1451
+ aiFragment.hidden = false;
1452
+ const v = input.value.trim();
1453
+ const ok = KEY_REGEX.test(v);
1454
+ continueBtn.disabled = !ok;
1455
+ hint.textContent = (!v || ok) ? "" : "Key should start with \"AIza\" and be ~39 chars.";
1456
+ };
1457
+
1458
+ input.addEventListener("input", refreshGate);
1459
+ refreshGate();
1460
+
1461
+ continueBtn.addEventListener("click", () => {
1462
+ const v = input.value.trim();
1463
+ if (KEY_REGEX.test(v)) sessionStorage.setItem(KEY_NAME, v);
1464
+ }, { capture: true });
1465
+
1466
+ const blockClose = (e) => {
1467
+ if (hasValidStoredKey()) return;
1468
+ e.preventDefault();
1469
+ e.stopPropagation();
1470
+ input.focus();
1471
+ };
1472
+ closeBtn.addEventListener("click", blockClose, { capture: true });
1473
+ root.addEventListener("cancel", blockClose); // Escape
1474
+ }
1475
+ ```
1476
+
1477
+ When the AI fragment is **absent** (non-AI app), the existing wiring runs untouched — `Continue` enables immediately, `Escape` / `×` dismiss as normal.
1478
+
1479
+ > **Migration note (feature 008, 2026-05-14):** the legacy `ACTIONS` variable is renamed to `MUDRA_ONBOARDING_ACTIONS`. The legacy `mudra-onb__*` class names from feature 005 are replaced by `ob-*` (the locked block's class names). The `Got it` CTA label is renamed to `Continue`. The AI-Setup fragment moves from `.mudra-onb__body` (separate section) into `.ob-chip-grid` (first chip).
1480
+
1481
+ ---
1482
+
1483
+ ## Section 18 — Visible AI Chat I/O (mandatory when `usesAI`)
1484
+
1485
+ Every AI-using app MUST render the conversation as on-screen text in the
1486
+ 3D scene — TTS / speech synthesis is optional, never a substitute.
1487
+
1488
+ ### Required visible elements
1489
+
1490
+ | Element | What it shows | How to render |
1491
+ |---------|---------------|---------------|
1492
+ | **Purpose line** | One short sentence stating what the app does ("Ask anything — I'll answer.", "Tell me the date.", etc.). Visible at all times. | Troika `Text` or a top row in an `xb.SpatialPanel`. |
1493
+ | **User input echo** | The latest user message (transcript from speech recognition OR typed text). Updates as soon as input is captured. | A bordered/highlighted row in the panel, e.g. prefixed `💬 You: …`. |
1494
+ | **AI response** | The most recent AI reply in full readable text. Scrollable / wrapping. | `xb.ScrollingTroikaTextView` or a tall row in the panel, prefixed `🤖 AI: …`. |
1495
+ | **Listening / Thinking indicator** | Distinguishes idle / listening / thinking states. | Avatar pulse + a single-word status line ("Listening…", "Thinking…", "Tap to talk"). |
1496
+
1497
+ ### Rules
1498
+
1499
+ 1. **Both sides of every exchange must be visible.** Voice-only output
1500
+ is a checklist failure. The chat panel must accumulate at least the
1501
+ last user turn AND the last AI turn at the same time.
1502
+ 2. **The Purpose line is fixed** for the lifetime of the app. Author it
1503
+ from the user's prompt — e.g. `"Create 3d AI I can ask the date"` →
1504
+ Purpose `"Voice assistant — ask anything, tap to talk."`. Never use
1505
+ a placeholder like `"AI Chat"` alone.
1506
+ 3. **Show typed input when speech recognition is unavailable.** If
1507
+ `webkitSpeechRecognition` / `SpeechRecognition` are missing, render
1508
+ an `<input type="text">` inside an XR Blocks panel OR a 2D overlay
1509
+ below the simulator panel. Both echo and reply still render in the
1510
+ 3D scene.
1511
+ 4. **When no API key is set yet**, the chat panel renders the literal
1512
+ text `Set up AI in the welcome panel` in the response slot — do not
1513
+ attempt any API call.
1514
+ 5. **TTS** (`speechSynthesis`) is allowed but optional. If present, it
1515
+ speaks the AI response in addition to displaying it. Never as a
1516
+ replacement.