mudra-skills 1.2.0 → 3.0.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -17,7 +17,7 @@ ws://127.0.0.1:8766
17
17
  Always construct the connection through `MudraClient` (Section 4).
18
18
  Never use raw `new WebSocket(...)`.
19
19
 
20
- ### Nine canonical signals
20
+ ### Eight canonical signals
21
21
 
22
22
  | Signal | Category | Description |
23
23
  |--------|----------|-------------|
@@ -29,7 +29,7 @@ Never use raw `new WebSocket(...)`.
29
29
  | `imu_acc` | Motion (IMU) | Accelerometer values [x, y, z] m/s², frequency 1125 Hz |
30
30
  | `imu_gyro` | Motion (IMU) | Gyroscope values [x, y, z] deg/s, frequency 1125 Hz |
31
31
  | `snc` | Biometric | 3 de-interleaved channel arrays [[ch1], [ch2], [ch3]] |
32
- | `battery` | Status | Battery level 0–100, charging boolean |
32
+
33
33
 
34
34
  ### Subscription handshake
35
35
 
@@ -47,14 +47,15 @@ ws.send(JSON.stringify({ command: 'subscribe', signal: ['gesture', 'pressure'] }
47
47
 
48
48
  ### Full command surface
49
49
 
50
- `subscribe`, `unsubscribe`, `get_subscriptions`, `enable`, `disable`,
51
- `get_status`, `get_docs`, `trigger_gesture`
50
+ `subscribe`, `unsubscribe`, `get_subscriptions`, `get_status`, `trigger_gesture`
51
+
52
+ **Removed**: `enable`, `disable`, `get_docs` — the new Dart server rejects them with `unknown_command`.
52
53
 
53
54
  ### Inbound message payload shapes
54
55
 
55
56
  ```js
56
- // gesture
57
- { type: 'gesture', data: { type: 'tap'|'double_tap'|'twist'|'double_twist', confidence: 0–1, timestamp }, timestamp }
57
+ // gesture — no confidence field
58
+ { type: 'gesture', data: { type: 'tap'|'double_tap'|'twist'|'double_twist', timestamp }, timestamp }
58
59
 
59
60
  // button
60
61
  { type: 'button', data: { state: 'pressed'|'released', timestamp }, timestamp }
@@ -69,21 +70,40 @@ ws.send(JSON.stringify({ command: 'subscribe', signal: ['gesture', 'pressure'] }
69
70
  { type: 'nav_direction', data: { direction: 'Right'|'Left'|'Up'|'Down'|'Roll Left'|'Roll Right'|'None', timestamp }, timestamp }
70
71
 
71
72
  // imu_acc
72
- { type: 'imu_acc', data: { values: [x, y, z], frequency: 1125, timestamp }, timestamp }
73
+ { type: 'imu_acc', data: { values: [x, y, z], frequency: number, frequency_std: number, timestamp }, timestamp }
73
74
 
74
75
  // imu_gyro
75
- { type: 'imu_gyro', data: { values: [x, y, z], frequency: 1125, timestamp }, timestamp }
76
+ { type: 'imu_gyro', data: { values: [x, y, z], frequency: number, frequency_std: number, timestamp }, timestamp }
76
77
 
77
78
  // snc — extend rolling buffers (500 samples/channel) with all samples per callback
78
- { type: 'snc', data: { values: [[ch1_samples], [ch2_samples], [ch3_samples]], timestamp }, timestamp }
79
+ { type: 'snc', data: { values: [[ch1_samples], [ch2_samples], [ch3_samples]], frequency: number, frequency_std: number, timestamp }, timestamp }
79
80
 
80
- // battery
81
- { type: 'battery', data: { level: 0–100, charging: boolean, timestamp }, timestamp }
81
+ // status — response to get_status command
82
+ { type: 'status', data: { device: { name, address, battery, charging, firmware, serial_number, hand, state, firmware_config: { target, active } }, subscriptions: { snc, imu_acc, imu_gyro, pressure, gesture, navigation, nav_direction, button } }, timestamp }
82
83
 
83
- // connection_status
84
- { type: 'connection_status', data: { status: 'connected'|'disconnected', message: string }, timestamp }
84
+ // subscription_status — response to subscribe/unsubscribe
85
+ { type: 'subscription_status', data: { signal: string, subscribed: boolean }, timestamp }
85
86
  ```
86
87
 
88
+ **Removed**: `confidence` field from gesture (not emitted by Dart server).
89
+
90
+ **Legacy / not emitted by Dart server**: `connection_status` frames were sent by the old Python server on connect. The new Dart server sends **no unsolicited frames** on open — do NOT gate any UI on receiving `connection_status`.
91
+
92
+ ### Error frames
93
+
94
+ The server emits `{type:"error", data:{error:"<code>", message:"..."}, timestamp}` for protocol violations. Codes generated apps may receive:
95
+
96
+ | `data.error` code | Meaning | App must do |
97
+ |---|---|---|
98
+ | `invalid_json` | Malformed JSON sent | Log; never expected from generated apps |
99
+ | `missing_command` | No `command` field | Log; never expected |
100
+ | `unknown_command` | Command not in the supported set | Log; never expected after spec 001 |
101
+ | `missing_signal` | `subscribe`/`unsubscribe` without `signal` | Log; never expected |
102
+ | `invalid_signal` | Unrecognised signal name | Log; never expected after spec 001 |
103
+ | `client_already_connected` | Another tab/process owns the single client slot | **Show terminal "close the other tab" message; do NOT retry** |
104
+
105
+ For any unexpected error code: log and remain in current state. Do not crash the scene.
106
+
87
107
  ---
88
108
 
89
109
  ## Section 2 — Canonical Dependency Pins
@@ -129,18 +149,19 @@ Lit bundle warning suppression (place before import map):
129
149
  ### Module-scope MudraClient
130
150
 
131
151
  Instantiate `MudraClient` exactly once at module scope — not inside `init()`.
132
- This ensures the WebSocket open-attempt starts immediately on page load so the
133
- 1500 ms timeout (Section 4) begins counting before the XR scene initializes.
152
+ The constructor does NOT open a WebSocket; connection is driven by `mudra.setMode("mudra")` only (Section 15). Module-scope instantiation ensures the `MudraClient` object is ready before the XR scene initializes.
134
153
 
135
154
  ```js
136
- // Module scope — outside any class
137
- const mudra = new MudraClient('ws://127.0.0.1:8766');
155
+ // Module scope — outside any class (no URL arg; URL is baked into MudraClient per Section 4)
156
+ const mudra = new MudraClient();
138
157
 
139
158
  class MainScript extends xb.Script {
140
159
  init() {
141
- // Wire mudra handlers here after scene objects are created
142
- mudra.on('gesture', (data) => { ... });
143
- mudra.on('pressure', (data) => { ... });
160
+ // Register signal handlers and subscribe here, AFTER scene objects are created
161
+ mudra.on('gesture', (data) => { /* handle tap / twist / etc. */ });
162
+ mudra.on('pressure', (data) => { /* handle finger pressure */ });
163
+ mudra.subscribe('gesture');
164
+ mudra.subscribe('pressure');
144
165
  }
145
166
  update() { /* called every frame by xb */ }
146
167
  }
@@ -192,29 +213,26 @@ document.addEventListener('DOMContentLoaded', function () {
192
213
 
193
214
  ---
194
215
 
195
- ## Section 4 — Mock WebSocket Fallback (MudraClient)
216
+ ## Section 4 — MudraClient (WebSocket + Mode Toggle glue)
196
217
 
197
218
  ### Policy
198
219
 
199
- - Start the WebSocket open attempt immediately on page load.
200
- - If the WebSocket does not open within **1500 ms**, activate the mock automatically.
201
- - If the WebSocket closes mid-session (band disconnect), flip to mock without a page reload.
202
- - The mock fires exactly the same message format as the real device.
203
- - App code must not branch on `_useMock` — it must receive the same messages either way.
220
+ - The constructor does **NOT** open a WebSocket. Connection is driven by `setMode("mudra")` only.
221
+ - In Manual mode the sim panel and keyboard shortcuts inject synthetic signals through `_emit()` — the exact same handler path as real WebSocket messages. No auto-firing timers. No mock auto-activation.
222
+ - In Mudra mode a real WebSocket opens to `ws://127.0.0.1:8766`. On successful open: replay the subscription record + send `get_status` + start the 2 s status-poll.
223
+ - If the socket closes for a non-conflict reason: schedule reconnect with backoff `[1 s, 2 s, 5 s, 10 s]` (capped). The `_suppressReconnect` flag prevents retry on `client_already_connected`.
224
+ - If `client_already_connected` error arrives: transition to `already-in-use`; set `_suppressReconnect = true`; do not retry.
225
+ - The 6-state `ConnectionState` vocabulary and status-pill wiring live in **Section 15** (canonical). This section documents only the `MudraClient` class itself.
204
226
 
205
227
  ### Connection-status state machine
206
228
 
229
+ See Section 15 for the full 6-state transition diagram. Vocabulary:
230
+
231
+ ```text
232
+ idle | connecting | connected | ws-only | reconnecting | already-in-use
207
233
  ```
208
- [page load]
209
- │
210
- ▼
211
- connecting ──── ws opens within 1500 ms ──→ connected
212
- │
213
- └── timeout or error ──────────────────→ simulated
214
- │
215
- connected ──── ws.onclose fires ───────────────→ disconnected-simulated
216
- simulated ──── ws.onclose fires ───────────────→ (already simulated, no change)
217
- ```
234
+
235
+ `simulated` and `disconnected-simulated` are **removed**. No auto-mock activation.
218
236
 
219
237
  ### Required MudraClient implementation
220
238
 
@@ -222,53 +240,26 @@ Include this class verbatim in every generated app:
222
240
 
223
241
  ```js
224
242
  class MudraClient {
225
- constructor(url) {
226
- this._handlers = {};
227
- this._subscriptions = new Set();
228
- this._timers = [];
229
- this._status = 'connecting';
230
- this._notifyStatus('connecting');
243
+ static _WS_URL = 'ws://127.0.0.1:8766';
244
+ static _BACKOFF = [1000, 2000, 5000, 10000]; // ms, last value is the cap
231
245
 
232
- const timeout = setTimeout(() => this._startMock(), 1500);
233
-
234
- try {
235
- this._ws = new WebSocket(url);
236
- this._ws.onopen = () => {
237
- clearTimeout(timeout);
238
- this._status = 'connected';
239
- this._notifyStatus('connected');
240
- this._subscriptions.forEach(sig =>
241
- this._ws.send(JSON.stringify({ command: 'subscribe', signal: sig }))
242
- );
243
- };
244
- this._ws.onmessage = (e) => {
245
- const msg = JSON.parse(e.data);
246
- if (this._handlers[msg.type]) this._handlers[msg.type](msg.data);
247
- };
248
- this._ws.onclose = () => {
249
- clearTimeout(timeout);
250
- if (this._status === 'connected') {
251
- this._status = 'disconnected-simulated';
252
- this._notifyStatus('disconnected-simulated');
253
- this._startMock();
254
- }
255
- };
256
- this._ws.onerror = () => {
257
- clearTimeout(timeout);
258
- this._startMock();
259
- };
260
- } catch (_) {
261
- clearTimeout(timeout);
262
- this._startMock();
263
- }
246
+ constructor() {
247
+ this._handlers = {};
248
+ this._subscriptions = new Set(); // subscription record — replayed on reconnect
249
+ this._ws = null;
250
+ this._mode = 'manual';
251
+ this._connState = 'idle';
252
+ this._pollTimer = null;
253
+ this._reconnectTimer = null;
254
+ this._backoffIdx = 0;
255
+ this._suppressReconnect = false;
256
+ this._connToken = 0; // incremented on every openSocket() to cancel stale callbacks
264
257
  }
265
258
 
266
- /** Register a handler for a signal type. Call before subscribe(). */
267
- on(signal, fn) {
268
- this._handlers[signal] = fn;
269
- }
259
+ /** Register a handler for a signal type or _status events. Call before subscribe(). */
260
+ on(signal, fn) { this._handlers[signal] = fn; }
270
261
 
271
- /** Subscribe to a signal. Safe to call before the WebSocket opens. */
262
+ /** Add signal to subscription record. If WS is open, send subscribe immediately. */
272
263
  subscribe(signal) {
273
264
  this._subscriptions.add(signal);
274
265
  if (this._ws && this._ws.readyState === WebSocket.OPEN) {
@@ -276,19 +267,41 @@ class MudraClient {
276
267
  }
277
268
  }
278
269
 
279
- /** Send an arbitrary command to the band service. */
270
+ /** Remove signal from subscription record. If WS is open, send unsubscribe. */
271
+ unsubscribe(signal) {
272
+ this._subscriptions.delete(signal);
273
+ if (this._ws && this._ws.readyState === WebSocket.OPEN) {
274
+ this._ws.send(JSON.stringify({ command: 'unsubscribe', signal }));
275
+ }
276
+ }
277
+
278
+ /** Send an arbitrary command. In Manual mode, trigger_gesture dispatches via _emit(). */
280
279
  send(cmd) {
281
280
  if (this._ws && this._ws.readyState === WebSocket.OPEN) {
282
281
  this._ws.send(JSON.stringify(cmd));
283
282
  } else if (cmd.command === 'trigger_gesture') {
284
- this._dispatchMockGesture(cmd.data.type);
283
+ this._dispatchMockGesture(cmd.data?.type ?? 'tap');
285
284
  }
286
285
  }
287
286
 
288
- /** Current connection status string. */
289
- get status() { return this._status; }
287
+ /** Switch mode. setMode("mudra") opens the socket; setMode("manual") closes it. */
288
+ setMode(mode) {
289
+ if (this._mode === mode) return;
290
+ this._mode = mode;
291
+ if (mode === 'manual') {
292
+ this._closeSocket();
293
+ this._setState('idle');
294
+ } else {
295
+ this._suppressReconnect = false;
296
+ this._backoffIdx = 0;
297
+ this._openSocket();
298
+ }
299
+ }
300
+
301
+ get status() { return this._connState; }
290
302
 
291
- _notifyStatus(s) {
303
+ _setState(s) {
304
+ this._connState = s;
292
305
  if (this._handlers['_status']) this._handlers['_status'](s);
293
306
  }
294
307
 
@@ -296,27 +309,122 @@ class MudraClient {
296
309
  if (this._handlers[payload.type]) this._handlers[payload.type](payload.data);
297
310
  }
298
311
 
312
+ /** Called by sim-panel gesture buttons and keyboard Space shortcut. */
299
313
  _dispatchMockGesture(type) {
300
- this._emit({ type: 'gesture', data: { type, confidence: 0.99, timestamp: Date.now() } });
314
+ this._emit({ type: 'gesture', data: { type, timestamp: Date.now() } });
301
315
  }
302
316
 
303
- _startMock() {
304
- if (this._status === 'simulated' || this._status === 'disconnected-simulated') return;
305
- const wasPreviouslyConnected = this._status === 'connected';
306
- this._status = wasPreviouslyConnected ? 'disconnected-simulated' : 'simulated';
307
- this._notifyStatus(this._status);
308
- // Passive mock: no auto-firing. Signals fire ONLY from sim-panel clicks,
309
- // keyboard shortcuts, or real WebSocket messages. This keeps the scene
310
- // stable and makes the sim panel the single, explicit source of motion.
317
+ _openSocket() {
318
+ const token = ++this._connToken;
319
+ this._setState('connecting');
320
+ try {
321
+ const ws = new WebSocket(MudraClient._WS_URL);
322
+ this._ws = ws;
323
+
324
+ ws.onopen = () => {
325
+ if (this._connToken !== token) return;
326
+ this._backoffIdx = 0;
327
+ this._replaySubscriptions();
328
+ ws.send(JSON.stringify({ command: 'get_status' }));
329
+ this._startPoll();
330
+ };
331
+
332
+ ws.onmessage = (e) => {
333
+ if (this._connToken !== token) return;
334
+ let msg;
335
+ try { msg = JSON.parse(e.data); } catch { return; }
336
+
337
+ if (msg.type === 'error' && msg.data?.error === 'client_already_connected') {
338
+ this._suppressReconnect = true;
339
+ this._stopPoll();
340
+ this._setState('already-in-use');
341
+ return;
342
+ }
343
+
344
+ if (msg.type === 'status') {
345
+ const d = msg.data?.device;
346
+ const bandConnected = !!(d?.firmware && d?.serial_number);
347
+ if (bandConnected) {
348
+ if (this._connState !== 'connected') {
349
+ this._setState('connected');
350
+ this._replaySubscriptions(); // FR-012: re-subscribe on band reconnect
351
+ }
352
+ // expose hand via a synthetic 'hand' event for the UI chip
353
+ if (this._handlers['_hand']) {
354
+ const raw = d.hand ?? '';
355
+ const hand = /^(LEFT|RIGHT)$/i.test(raw) ? raw.toUpperCase() : 'None';
356
+ this._handlers['_hand'](hand);
357
+ }
358
+ } else {
359
+ if (this._connState !== 'ws-only') this._setState('ws-only');
360
+ if (this._handlers['_hand']) this._handlers['_hand']('None');
361
+ }
362
+ return;
363
+ }
364
+
365
+ if (this._handlers[msg.type]) this._handlers[msg.type](msg.data);
366
+ };
367
+
368
+ ws.onerror = () => { /* onclose fires after onerror; handle there */ };
369
+
370
+ ws.onclose = () => {
371
+ if (this._connToken !== token) return;
372
+ this._stopPoll();
373
+ if (this._suppressReconnect || this._mode !== 'mudra') return;
374
+ this._setState('reconnecting');
375
+ const delay = MudraClient._BACKOFF[Math.min(this._backoffIdx, MudraClient._BACKOFF.length - 1)];
376
+ this._backoffIdx = Math.min(this._backoffIdx + 1, MudraClient._BACKOFF.length - 1);
377
+ this._reconnectTimer = setTimeout(() => {
378
+ if (this._mode === 'mudra' && !this._suppressReconnect) this._openSocket();
379
+ }, delay);
380
+ };
381
+ } catch (_) {
382
+ this._setState('reconnecting');
383
+ }
384
+ }
385
+
386
+ _closeSocket() {
387
+ this._stopPoll();
388
+ clearTimeout(this._reconnectTimer);
389
+ this._reconnectTimer = null;
390
+ this._connToken++; // invalidate all callbacks from the old socket
391
+ if (this._ws) { try { this._ws.close(); } catch (_) {} this._ws = null; }
392
+ }
393
+
394
+ _startPoll() {
395
+ this._stopPoll();
396
+ this._pollTimer = setInterval(() => {
397
+ if (this._ws && this._ws.readyState === WebSocket.OPEN) {
398
+ this._ws.send(JSON.stringify({ command: 'get_status' }));
399
+ }
400
+ }, 2000);
401
+ }
402
+
403
+ _stopPoll() {
404
+ clearInterval(this._pollTimer);
405
+ this._pollTimer = null;
406
+ }
407
+
408
+ /** Replay the subscription record. Called on open and on band reconnect. */
409
+ _replaySubscriptions() {
410
+ if (!this._ws || this._ws.readyState !== WebSocket.OPEN) return;
411
+ this._subscriptions.forEach(sig =>
412
+ this._ws.send(JSON.stringify({ command: 'subscribe', signal: sig }))
413
+ );
311
414
  }
312
415
 
313
416
  destroy() {
314
- this._timers.forEach(t => clearInterval(t));
315
- if (this._ws) this._ws.close();
417
+ this._closeSocket();
316
418
  }
317
419
  }
318
420
  ```
319
421
 
422
+ ### Section 4 note on "no fallback"
423
+
424
+ **"No fallback"** means: no automatic transition to a "Simulated" or "Disconnected — simulated" state when the WebSocket is unavailable. If the socket fails, the pill shows `Reconnecting…` and retries — it never auto-activates synthetic data injection.
425
+
426
+ Manual mode signal injection (sim panel + keyboard shortcuts calling `_emit()`) is **preserved** — this is Principle IV compliance, not a fallback. Sim-panel signals use the same `_emit()` path as real WebSocket messages, so the app does not branch on whether it is simulated or not.
427
+
320
428
  ---
321
429
 
322
430
  ## Section 5 — Simulator Panel
@@ -379,7 +487,7 @@ checklist failure (see Section 10, item 6).
379
487
  // Example: gesture sim button wires to the same handler as real signals
380
488
  function simGesture(type) {
381
489
  mudra.send({ command: 'trigger_gesture', data: { type } });
382
- handleGesture({ type, confidence: 1.0, timestamp: Date.now() });
490
+ handleGesture({ type, timestamp: Date.now() });
383
491
  }
384
492
 
385
493
  // Example: pressure slider
@@ -486,49 +594,11 @@ those, especially the reserved set above (WASD, arrows, Q/E/R, mouse).
486
594
 
487
595
  ## Section 7 — Connection-Status Indicator
488
596
 
489
- ### Required DOM element
490
-
491
- Every generated app must include exactly one visible element that reflects
492
- the current `MudraClient` status:
493
-
494
- ```html
495
- <div id="mudra-status" style="
496
- position: fixed; top: 8px; right: 12px;
497
- padding: 4px 10px; border-radius: 999px;
498
- font-size: 0.8rem; font-family: system-ui, sans-serif;
499
- background: rgba(0,0,0,0.6); color: #fff;
500
- z-index: 9999;">Connecting…</div>
501
- ```
502
-
503
- ### Text states
504
-
505
- | MudraClient status | textContent |
506
- |-------------------|-------------|
507
- | `connecting` | `Connecting…` |
508
- | `connected` | `Connected` |
509
- | `simulated` | `Simulated` |
510
- | `disconnected-simulated` | `Disconnected — simulated` |
511
-
512
- ### Wiring
513
-
514
- ```js
515
- mudra.on('_status', (s) => {
516
- const chip = document.getElementById('mudra-status');
517
- const labels = {
518
- 'connecting': 'Connecting…',
519
- 'connected': 'Connected',
520
- 'simulated': 'Simulated',
521
- 'disconnected-simulated': 'Disconnected — simulated',
522
- };
523
- chip.textContent = labels[s] ?? s;
524
- });
525
- ```
526
-
527
- ### Visibility
528
-
529
- - Visible in flat-screen mode at all times.
530
- - Disappears automatically in immersive XR (native DOM suppression).
531
- - Place in the top-right corner by default; adapt if the template uses that space.
597
+ > **Superseded by Section 15.** The status pill is now part of the Mode Toggle chrome and follows the 6-state machine defined there. Read Section 15 for the canonical DOM sketch (`#mudra-status` + `#mudra-hand`), state labels, colour hints, and wiring.
598
+ >
599
+ > The 4-state vocabulary (`connecting` / `connected` / `simulated` / `disconnected-simulated`) is **removed**. New vocabulary: `idle` (Manual) / `Connecting…` / `Connected` / `WebSocket Only` (orange) / `Reconnecting…` / `Companion already in use`.
600
+ >
601
+ > The `#mudra-status` `<div>` still lives at `position: fixed; top: 8px; right: 12px` and disappears automatically in immersive WebXR. See Section 15 for full wiring.
532
602
 
533
603
  ---
534
604
 
@@ -753,20 +823,24 @@ Before calling `Write` to emit a generated app, verify all items:
753
823
  | 1 | Single file | Exactly one `<html>` document; all CSS in `<style>`; all JS in `<script>` or `<script type="module">` |
754
824
  | 2 | Import map | One `<script type="importmap">` block; contents match canonical pins (Section 2) exactly; no unused entries |
755
825
  | 3 | xb.Script entry | Top-level logic inside `class <Name> extends xb.Script`; `xb.add(new <Name>())` + `xb.init(new xb.Options())` on `DOMContentLoaded` |
756
- | 4 | MudraClient | One `MudraClient` instance at module scope; URL = `ws://127.0.0.1:8766`; does NOT auto-connect; `setMode()` drives connect/disconnect |
826
+ | 4 | MudraClient | One `MudraClient` instance at module scope; constructor does NOT auto-connect; `setMode()` drives connect/disconnect; no `setTimeout(_, 1500)` timer; no `_startMock()` auto-firing; no `'simulated'` / `'disconnected-simulated'` strings |
757
827
  | 5 | Subscribe commands | Every used signal has exactly one `mudra.subscribe('<signal>')` call; none outside the signal set |
758
828
  | 6 | Simulator panel | `<div id="mudra-sim">` present; ONLY buttons for sub-actions actually handled by the app (no extras like Roll L/R or Twist if unused); buttons fire via handler, not inline `onclick` |
759
829
  | 7 | Keyboard bindings | `window.addEventListener('keydown', …, { capture: true })` present; `event.stopPropagation()` on every Mudra-claimed key |
760
- | 8 | Status indicator | `<div id="mudra-status">` present; text states are `Manual` / `Connecting…` / `Connected` / `Disconnected` (Section 15); no `simulated` strings |
830
+ | 8 | Status indicator | `<div id="mudra-status">` + `<span id="mudra-hand">` present; text states are `Manual` / `Connecting…` / `Connected` / `WebSocket Only` / `Reconnecting…` / `Companion already in use` (Section 15); no `simulated` / `disconnected-simulated` strings; `#mudra-hand` shows `LEFT`/`RIGHT` when connected, `None` otherwise |
761
831
  | 9 | AI key gating | If `usesAI`: the AI-Setup fragment (Section 9) is present inside `.mudra-onb__body`; key is read from `sessionStorage.getItem('mudra.gemini.apiKey')` only; ZERO `prompt(` calls for the key; ZERO `localStorage` references; ZERO baked keys (regex scan) |
762
832
  | 9a | Visible AI chat I/O | If `usesAI`: the scene renders BOTH the latest user input AND the AI response as visible text (xb.ScrollingTroikaTextView, troika `Text`, or xb.SpatialPanel rows). The visible "Purpose" line states what the app does in one sentence. TTS may exist but is never the only output (Section 18) |
763
833
  | 10 | Background lockdown | ZERO `applyBackground_*` methods in the class; ZERO calls to a background helper from `init()`; no `options.simulator.scenePath` line anywhere. Generated apps use the XR Blocks default room only (Section 14) |
764
834
  | 11 | Mode toggle | `<div id="mode-toggle">` with **Manual** + **Mudra** buttons; Manual is the default on load; toggle remains clickable when disconnected; flipping atomically opens/closes the socket per Section 15 |
765
- | 12 | Band-state polling | In Mudra mode the app sends `{command:"get_status"}` on `ws.onopen` and every 2000 ms thereafter; pill flips to `Connected` ONLY when `data.device.state === "connected"` |
835
+ | 12 | Band-state polling | In Mudra mode the app sends `{command:"get_status"}` on `ws.onopen` and every 2000 ms thereafter; pill flips to `Connected` ONLY when `data.device.firmware && data.device.serial_number` (both truthy); `WebSocket Only` (orange) when socket open but band not connected |
766
836
  | 13 | No disconnect overlay | No banner / toast / modal / inline alert ever rendered for disconnect — pill is the only indicator |
767
837
  | 14 | Footer | Exactly one `<div id="mudra-badge">` containing the literal text `Created by Mudra` (no variants) |
768
- | 15 | Mock is passive | `MudraClient._startMock()` (or equivalent) starts NO intervals — synthetic signals come only from sim-panel clicks and keyboard shortcuts |
838
+ | 15 | Mock is passive | No `setTimeout(_, 1500)` timer; no `_startMock()` auto-firing; Manual-mode signals come only from sim-panel clicks and keyboard shortcuts calling `_emit()` directly |
769
839
  | 16 | Gemini model pin | If the app calls `generativelanguage.googleapis.com/v1beta/models/<id>:generateContent`, the captured `<id>` MUST equal `gemini-2.5-flash`. No preview / dated / latest aliases. Live-API and image-gen exceptions per Section 9 |
840
+ | 17 | Subscription record | `MudraClient._subscriptions` is a `Set<string>` that tracks subscribed signals; `_replaySubscriptions()` fires on `ws.onopen` AND on any `ws-only` → `connected` transition; signals in the record are never resurrected if `unsubscribe()` was called explicitly |
841
+ | 18 | client_already_connected | The `{type:"error",data:{error:"client_already_connected"}}` error frame is handled: pill transitions to `Companion already in use`; `_suppressReconnect = true`; the backoff timer does NOT fire on the subsequent `onclose` |
842
+ | 19 | Reconnect backoff | WS `onclose` (non-conflict) schedules reconnect with `[1000, 2000, 5000, 10000]` ms schedule; `_backoffIdx` resets to 0 on every successful `onopen` |
843
+ | 20 | No forbidden strings | Generated source contains zero occurrences of: `confidence` as a gesture field, `enable`/`disable`/`get_docs` as commands, `'simulated'`/`'disconnected-simulated'` as status strings, `8765` as a port |
770
844
 
771
845
  ### Retry policy
772
846
 
@@ -976,11 +1050,9 @@ not written.
976
1050
  ---
977
1051
  ## Section 15 — Mode Toggle (Manual / Mudra) — Required
978
1052
 
979
- **This section supersedes Section 4's auto-fallback "simulated" status and
980
- Section 7's `simulated` / `disconnected-simulated` states for all new
981
- apps.** Every XR app generated by this skill MUST implement the Mode
982
- Toggle exactly as specified here. Canonical protocol:
983
- `references/agent_protocol.json` (v2.0).
1053
+ **This section is the canonical source for the connection-status state machine and status pill wiring.** Section 4 defines the `MudraClient` class; Section 7 is superseded by this section.
1054
+
1055
+ Every XR app generated by this skill MUST implement the Mode Toggle exactly as specified here.
984
1056
 
985
1057
  ### Summary
986
1058
 
@@ -1009,113 +1081,86 @@ default on first load.
1009
1081
  ```text
1010
1082
  type Mode = "manual" | "mudra" // default "manual"
1011
1083
  type ConnectionState =
1012
- | "idle" // No socket open. Always the case in Manual.
1013
- | "connecting" // Socket opening, OR socket open but band-pairing not yet confirmed via get_status.
1014
- | "connected" // Socket open AND last get_status response had data.device.state === "connected".
1015
- | "disconnected" // Socket closed/errored, OR socket open but data.device.state !== "connected".
1084
+ | "idle" // Manual mode. No socket open.
1085
+ | "connecting" // Socket opening, OR socket open but first get_status not yet received.
1086
+ | "connected" // Socket open AND device.firmware && device.serial_number both truthy.
1087
+ | "ws-only" // Socket open BUT device.firmware or device.serial_number is null/falsy (band not paired).
1088
+ | "reconnecting" // Socket closed for non-conflict reason; backoff retry in progress.
1089
+ | "already-in-use" // Server sent client_already_connected; terminal until mode→Manual or page reload.
1016
1090
  ```
1017
1091
 
1018
- Lazy-WS lifecycle (mandatory):
1092
+ ### Lazy-WS lifecycle (mandatory)
1019
1093
 
1020
1094
  | Transition | Action |
1021
1095
  |------------|--------|
1022
1096
  | page load | `mode = "manual"`, `connectionState = "idle"`, NO socket |
1023
- | Manual → Mudra | open new socket, set `connectionState = "connecting"` |
1024
- | Mudra → Manual | close socket, cancel any in-flight reconnect timer, set `connectionState = "idle"` |
1025
- | WS `open` (in Mudra) | keep `connectionState = "connecting"`, send all `subscribe` commands, send `{command:"get_status"}`, start status-poll timer |
1026
- | inbound `status` with `data.device.state === "connected"` (in Mudra) | `connectionState = "connected"` |
1027
- | inbound `status` with `data.device.state !== "connected"` (in Mudra) | `connectionState = "disconnected"`, **keep socket open** — do NOT closeSocket, do NOT schedule WS reconnect; status-poll will surface the band coming back |
1028
- | inbound `connection_status: connected` (in Mudra) | request a fresh `get_status`; do not flip the pill on this alone |
1029
- | inbound `connection_status: disconnected` (in Mudra) | `connectionState = "disconnected"` |
1030
- | WS error / WS close (in Mudra) | `connectionState = "disconnected"`, stop status-poll, schedule socket reconnect |
1031
- | reconnect tick (in Mudra & socket dead) | open new socket → `connecting` |
1032
-
1033
- **Single-socket guarantee:** never have two `WebSocket` instances open at
1034
- once. Use a connection token to neutralise rapid-toggle races (see the
1035
- `MudraClient` extensions below).
1036
-
1037
- ### MudraClient changes vs. Section 4
1038
-
1039
- The `MudraClient` from Section 4 must be extended (or replaced) so it:
1040
-
1041
- 1. Does NOT auto-connect in its constructor. Connection is driven by
1042
- `setMode("mudra")` only.
1043
- 2. Exposes `setMode(mode)` to flip between `"manual"` and `"mudra"`.
1044
- 3. In Manual mode, the `_startMock()` interval generators (if any) are
1045
- NOT started. Mock signals must be **passive**: emitted only when the
1046
- sim panel is clicked or a keyboard shortcut fires. (Memory:
1047
- `MudraClient mock must be passive` — strip auto-firing intervals from
1048
- `_startMock()`; sim panel clicks and keys are the only signal source.)
1049
- 4. On entering Mudra mode, opens one WebSocket, sends `subscribe` for
1050
- every signal in its subscription list, sends `{command:"get_status"}`
1051
- immediately, and starts a 2 s `get_status` poll while the socket is
1052
- `OPEN`.
1053
- 5. Emits `_status` events with the new four-state vocabulary:
1054
- `"idle" | "connecting" | "connected" | "disconnected"`. The legacy
1055
- `"simulated"` / `"disconnected-simulated"` strings are removed.
1056
-
1057
- ### Disconnect detection — band state via `get_status` polling (mandatory)
1058
-
1059
- **The WebSocket handshake to `127.0.0.1:8766` only proves the Companion
1060
- service is up. It does NOT prove the user's Mudra Band is paired.** The
1061
- Companion accepts socket connections even when no band is bonded — so
1062
- flipping the pill to "Connected" on `ws.onopen` is wrong. The pill MUST
1063
- reflect the band itself, not the socket.
1064
-
1065
- The source of truth is the `status` response to `{command:"get_status"}`:
1097
+ | Manual → Mudra | call `_openSocket()`; `connectionState = "connecting"` |
1098
+ | Mudra → Manual | call `_closeSocket()`; cancel reconnect + poll timers; `connectionState = "idle"` |
1099
+ | WS `onopen` (in Mudra) | reset backoff; replay subscription record; send `{command:"get_status"}`; start 2 s poll timer; `connectionState` stays `"connecting"` until first status response |
1100
+ | inbound `{type:"status"}` — `device.firmware && device.serial_number` truthy (in Mudra) | `connectionState = "connected"`; update hand chip from `device.hand`; if transitioning from `"ws-only"`, call `_replaySubscriptions()` (FR-012) |
1101
+ | inbound `{type:"status"}` — firmware or serial_number falsy (in Mudra) | `connectionState = "ws-only"`; hand chip = `None` |
1102
+ | inbound `{type:"error", data:{error:"client_already_connected"}}` (in Mudra) | `_suppressReconnect = true`; stop poll; `connectionState = "already-in-use"` |
1103
+ | WS `onclose` when `_suppressReconnect = true` | no-op (the error frame already set state) |
1104
+ | WS `onerror` / `onclose` (in Mudra, non-conflict) | stop poll; `connectionState = "reconnecting"`; advance backoff; schedule `_openSocket()` after backoff delay |
1105
+ | reconnect tick fires (in Mudra, non-conflict) | call `_openSocket()` → `"connecting"` |
1106
+ | Mudra → Manual at any point | call `_closeSocket()`; clear `_suppressReconnect`; `connectionState = "idle"` |
1066
1107
 
1067
- ```json
1068
- > {"command":"get_status"}
1069
- < {"type":"status","data":{"device":{"state":"connected", ... }, ...}, "timestamp": ...}
1070
- < {"type":"status","data":{"device":{"state":"disconnected", ...}, ...}, "timestamp": ...}
1108
+ **Single-socket guarantee:** The `_connToken` counter (Section 4) ensures callbacks from a stale socket are discarded if mode is toggled rapidly.
1109
+
1110
+ ### Band-connected predicate (mandatory)
1111
+
1112
+ **The WebSocket handshake to `127.0.0.1:8766` only proves the Companion service is up. It does NOT prove the Mudra Band is paired.**
1113
+
1114
+ The band-connected check uses both `device.firmware` AND `device.serial_number`:
1115
+
1116
+ ```js
1117
+ const bandConnected = !!(msg.data?.device?.firmware && msg.data?.device?.serial_number);
1071
1118
  ```
1072
1119
 
1073
- Rules:
1074
-
1075
- 1. On `ws.onopen` (in Mudra mode): stay in `connecting`; send all
1076
- `subscribe` commands; send `{command:"get_status"}`; start a
1077
- **status-poll timer** that sends `{command:"get_status"}` every
1078
- **2000 ms** while `mode === "mudra"` and the socket is `OPEN`.
1079
- 2. On inbound `{type:"status"}` (in Mudra mode):
1080
- - `data?.device?.state === "connected"` → `setState("connected")`.
1081
- - Else → `setState("disconnected")`. Keep the socket open. Do NOT
1082
- `closeSocket()`. The next poll tick picks the band up after pairing.
1083
- 3. On inbound `{type:"connection_status"}`: hint only. On `disconnected`,
1084
- flip the pill. On `connected`, send a fresh `{command:"get_status"}`
1085
- and let the `status` handler do the actual transition.
1086
- 4. On WS `error` / `close` (in Mudra mode): `setState("disconnected")`,
1087
- stop the status-poll timer, `scheduleReconnect()`.
1088
- 5. On Manual mode: stop the status-poll timer in `closeSocket()`.
1089
-
1090
- Do not poll faster than 1 s; do not poll slower than 5 s. 2 s is
1091
- mandated. The pill MAY sit on `Connecting…` for up to one poll cycle
1092
- (~2 s) after entering Mudra mode while the first `status` round-trips —
1093
- that is correct behaviour.
1120
+ - Both truthy → `connectionState = "connected"`.
1121
+ - Either null/falsy → `connectionState = "ws-only"`.
1122
+
1123
+ Poll every **2000 ms** while the socket is `OPEN`. The pill MAY sit on `Connecting…` for up to one poll cycle (~2 s) after entering Mudra mode — that is correct behaviour.
1124
+
1125
+ ### Hand chip
1126
+
1127
+ Every generated app MUST render a hand chip (`<span id="mudra-hand">`) beside the status pill:
1128
+
1129
+ - Shows `LEFT` or `RIGHT` (raw value from `device.hand`, uppercased) when `connectionState === "connected"`.
1130
+ - Shows `None` in all other states.
1131
+ - Normalization: if `device.hand` arrives as an enum string (e.g., `HandType.right`), strip the type prefix and uppercase. If normalization fails, display `None`.
1132
+
1133
+ ```js
1134
+ mudra.on('_hand', (hand) => {
1135
+ document.getElementById('mudra-hand').textContent = hand;
1136
+ });
1137
+ ```
1094
1138
 
1095
1139
  ### Reconnect backoff
1096
1140
 
1097
- While `mode === "mudra" && connectionState === "disconnected"` AND the
1098
- socket itself is dead (not just the band):
1141
+ Non-conflict WS close while `mode === "mudra"`:
1099
1142
 
1100
1143
  ```js
1101
- const RECONNECT_DELAYS_MS = [1000, 2000, 5000, 5000, 5000]; // capped at 5s
1144
+ // MudraClient._BACKOFF = [1000, 2000, 5000, 10000] // ms; last value is the cap
1102
1145
  ```
1103
1146
 
1104
- Reset the index on every successful `connected` transition.
1147
+ - `_backoffIdx` advances on every non-conflict close; resets to 0 on every `onopen`.
1148
+ - The `client_already_connected` path sets `_suppressReconnect = true` which bypasses the backoff timer entirely.
1105
1149
 
1106
- ### Status pill text states (replaces Section 7)
1150
+ ### Status pill text states
1107
1151
 
1108
- | connectionState | textContent | colour hint |
1109
- |-----------------|-------------|-------------|
1110
- | `idle` (Manual) | `Manual` | neutral |
1111
- | `connecting` | `Connecting…` | amber |
1112
- | `connected` | `Connected` | green |
1113
- | `disconnected` | `Disconnected` | red |
1152
+ | connectionState | `#mudra-status` text | `#mudra-hand` text | Colour hint |
1153
+ |---|---|---|---|
1154
+ | `idle` (Manual) | `Manual` | `None` | neutral |
1155
+ | `connecting` | `Connecting…` | `None` | amber |
1156
+ | `connected` | `Connected` | `LEFT` or `RIGHT` | green |
1157
+ | `ws-only` | `WebSocket Only` | `None` | **orange** (`#eab308`) |
1158
+ | `reconnecting` | `Reconnecting…` | `None` | amber (muted) |
1159
+ | `already-in-use` | `Companion already in use — close the other tab first` | `None` | red/error |
1114
1160
 
1115
- The pill is the **only** disconnect indicator. No banner, toast, or
1116
- modal. The simulator panel is greyed (reduced opacity,
1117
- `pointer-events: none`) when in Mudra + `disconnected`, but the pill is
1118
- still the only textual disconnect cue.
1161
+ The pill + hand chip are the **only** connection indicators. No banner, toast, or modal.
1162
+
1163
+ The simulator panel is greyed (`pointer-events: none`, reduced opacity) in every state except `idle` (Manual).
1119
1164
 
1120
1165
  ### Mode-toggle DOM sketch
1121
1166
 
@@ -1124,12 +1169,56 @@ still the only textual disconnect cue.
1124
1169
  <button id="mode-manual" role="tab" aria-selected="true">Manual</button>
1125
1170
  <button id="mode-mudra" role="tab" aria-selected="false">Mudra</button>
1126
1171
  </div>
1127
- <div id="mudra-status" class="conn-manual">Manual</div>
1172
+ <div style="display:flex;align-items:center;gap:6px;position:fixed;top:8px;right:12px;z-index:9999;">
1173
+ <div id="mudra-status" style="padding:4px 10px;border-radius:999px;font-size:0.8rem;font-family:system-ui,sans-serif;background:rgba(0,0,0,0.6);color:#fff;">Manual</div>
1174
+ <span id="mudra-hand" style="padding:2px 8px;border-radius:999px;font-size:0.75rem;font-family:system-ui,sans-serif;background:rgba(255,255,255,0.12);color:#fff;">None</span>
1175
+ </div>
1176
+ ```
1177
+
1178
+ Both `#mudra-status` and `#mudra-hand` are 2D DOM elements — they disappear automatically in immersive WebXR sessions via native DOM suppression. No extra JS wiring needed for this.
1179
+
1180
+ ### Status pill wiring
1181
+
1182
+ ```js
1183
+ mudra.on('_status', (s) => {
1184
+ const pill = document.getElementById('mudra-status');
1185
+ const labels = {
1186
+ 'idle': 'Manual',
1187
+ 'connecting': 'Connecting…',
1188
+ 'connected': 'Connected',
1189
+ 'ws-only': 'WebSocket Only',
1190
+ 'reconnecting': 'Reconnecting…',
1191
+ 'already-in-use':'Companion already in use — close the other tab first',
1192
+ };
1193
+ const colors = {
1194
+ 'idle': 'rgba(0,0,0,0.6)',
1195
+ 'connecting': 'rgba(180,120,0,0.8)',
1196
+ 'connected': 'rgba(22,163,74,0.8)',
1197
+ 'ws-only': '#eab308',
1198
+ 'reconnecting': 'rgba(120,80,0,0.7)',
1199
+ 'already-in-use':'rgba(185,28,28,0.85)',
1200
+ };
1201
+ pill.textContent = labels[s] ?? s;
1202
+ pill.style.background = colors[s] ?? 'rgba(0,0,0,0.6)';
1203
+
1204
+ // Grey the simulator panel in every non-Manual state
1205
+ const sim = document.getElementById('mudra-sim');
1206
+ if (sim) {
1207
+ sim.style.opacity = s === 'idle' ? '1' : '0.35';
1208
+ sim.style.pointerEvents = s === 'idle' ? '' : 'none';
1209
+ }
1210
+
1211
+ // Manage mode-toggle button aria-selected
1212
+ document.getElementById('mode-manual').setAttribute('aria-selected', s === 'idle' ? 'true' : 'false');
1213
+ document.getElementById('mode-mudra').setAttribute('aria-selected', s !== 'idle' ? 'true' : 'false');
1214
+ });
1215
+
1216
+ mudra.on('_hand', (hand) => {
1217
+ document.getElementById('mudra-hand').textContent = hand;
1218
+ });
1128
1219
  ```
1129
1220
 
1130
- On every mode change, atomically: cancel any reconnect timer, stop the
1131
- status-poll, close any open socket, reset `connToken`, then either
1132
- (Manual) leave `connectionState = "idle"` OR (Mudra) call `openSocket()`.
1221
+ On every mode button click, atomically: call `mudra.setMode(newMode)`. `MudraClient.setMode()` handles all internal cleanup (cancel timers, close socket, reset tokens).
1133
1222
 
1134
1223
  ---
1135
1224