ai-agent-session-center 2.0.2 → 2.0.3

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (58) hide show
  1. package/README.md +484 -429
  2. package/docs/3D/ADAPTATION_GUIDE.md +592 -0
  3. package/docs/3D/index.html +754 -0
  4. package/docs/AGENT_TEAM_TASKS.md +716 -0
  5. package/docs/CYBERDROME_V2_SPEC.md +531 -0
  6. package/docs/ERROR_185_ANALYSIS.md +263 -0
  7. package/docs/PLATFORM_FEATURES_PROMPT.md +296 -0
  8. package/docs/SESSION_DETAIL_FEATURES.md +98 -0
  9. package/docs/_3d_multimedia_features.md +1080 -0
  10. package/docs/_frontend_features.md +1057 -0
  11. package/docs/_server_features.md +1077 -0
  12. package/docs/session-duplication-fixes.md +271 -0
  13. package/docs/session-terminal-linkage.md +412 -0
  14. package/package.json +63 -5
  15. package/public/apple-touch-icon.svg +21 -0
  16. package/public/css/dashboard.css +0 -161
  17. package/public/css/detail-panel.css +25 -0
  18. package/public/css/layout.css +18 -1
  19. package/public/css/modals.css +0 -26
  20. package/public/css/settings.css +0 -150
  21. package/public/css/terminal.css +34 -0
  22. package/public/favicon.svg +18 -0
  23. package/public/index.html +6 -26
  24. package/public/js/alarmManager.js +0 -21
  25. package/public/js/app.js +21 -7
  26. package/public/js/detailPanel.js +63 -64
  27. package/public/js/historyPanel.js +61 -55
  28. package/public/js/quickActions.js +132 -48
  29. package/public/js/sessionCard.js +5 -20
  30. package/public/js/sessionControls.js +8 -0
  31. package/public/js/settingsManager.js +0 -142
  32. package/server/apiRouter.js +60 -15
  33. package/server/apiRouter.ts +774 -0
  34. package/server/approvalDetector.ts +94 -0
  35. package/server/authManager.ts +144 -0
  36. package/server/autoIdleManager.ts +110 -0
  37. package/server/config.ts +121 -0
  38. package/server/constants.ts +150 -0
  39. package/server/db.ts +475 -0
  40. package/server/hookInstaller.d.ts +3 -0
  41. package/server/hookProcessor.ts +108 -0
  42. package/server/hookRouter.ts +18 -0
  43. package/server/hookStats.ts +116 -0
  44. package/server/index.js +15 -1
  45. package/server/index.ts +230 -0
  46. package/server/logger.ts +75 -0
  47. package/server/mqReader.ts +349 -0
  48. package/server/portManager.ts +55 -0
  49. package/server/processMonitor.ts +239 -0
  50. package/server/serverConfig.ts +29 -0
  51. package/server/sessionMatcher.js +17 -6
  52. package/server/sessionMatcher.ts +403 -0
  53. package/server/sessionStore.js +109 -3
  54. package/server/sessionStore.ts +1145 -0
  55. package/server/sshManager.js +167 -24
  56. package/server/sshManager.ts +671 -0
  57. package/server/teamManager.ts +289 -0
  58. package/server/wsManager.ts +200 -0
@@ -0,0 +1,1080 @@
1
+ # 3D Scene + Multimedia Features
2
+
3
+ ---
4
+
5
+ ## 15. 3D Cyberdrome Scene
6
+
7
+ The Cyberdrome is a fully interactive 3D office environment rendered with React Three Fiber (R3F) and Three.js. Each active session is represented by an animated 3D robot character that navigates the scene, sits at desks, and reacts to session state changes in real time.
8
+
9
+ ### 15.1 Architecture
10
+
11
+ **Zero Zustand inside Canvas**
12
+
13
+ The core architectural decision is that all Zustand store subscriptions live in the DOM layer (`CyberdromeScene` component), never inside the `<Canvas>`. This prevents cross-reconciler state cascades that cause React Error #185 (illegal store update during render).
14
+
15
+ All data flows into the Canvas via props:
16
+ - Sessions are extracted from `sessionStore` as a plain array before the Canvas.
17
+ - Room configs, workstations, wall collision rects, and casual areas are computed in the DOM layer with `useMemo`.
18
+ - Room assignments per session are precomputed as a `Map<string, number | undefined>` to eliminate `useRoomStore` subscriptions inside Canvas.
19
+ - Subagent connection data is computed as a plain `ConnectionData[]` array.
20
+ - Scene theme colors and fog density are resolved outside Canvas and passed as props.
21
+
22
+ **CustomEvent pattern for click handling**
23
+
24
+ When a user clicks a robot inside R3F, the click handler dispatches a `CustomEvent('robot-select')` with a `setTimeout(0)` delay. This ensures the store update (`selectSession`) fires after R3F's pointer event cycle completes, fully decoupling the React reconciler used by R3F from the Zustand store. The DOM-side `useEffect` listener in `CyberdromeScene` catches the event and calls `selectSession` + `flyTo`.
25
+
26
+ **Ref-based animation (no useState in render loop)**
27
+
28
+ All per-robot animation state is stored in refs:
29
+ - `nav` ref (NavState) holds all movement state — position, rotation, speed, waypoints, mode, desk index.
30
+ - `seatedRef` tracks seated state for animation, updated in `useFrame`, never triggers re-render.
31
+ - `dialogueRef` holds the current dialogue message, updated in `useEffect`, read in `useFrame`.
32
+ - Direct `groupRef.current.position.set()` calls in `useFrame` move the robot group without React prop updates.
33
+
34
+ **Memoized SessionRobot**
35
+
36
+ `SessionRobot` is wrapped in `React.memo` with a custom equality check comparing 16 specific session fields. This prevents re-renders when unrelated session data changes, which would cascade Html portal updates for every robot.
37
+
38
+ **Position persistence**
39
+
40
+ Robot world positions and nav state are saved to `sessionStorage` every 2 seconds via `saveRobotPositions`. On page reload, `loadRobotPositions` restores each robot's position, rotation, navigation mode, and desk index. `NAV_GOTO` mode is reset to `NAV_WALK` on restore since it's a transient state.
41
+
42
+ ### 15.2 Canvas Setup
43
+
44
+ The Three.js canvas is configured with:
45
+
46
+ | Setting | Value |
47
+ |---------|-------|
48
+ | Camera position | `[18, 16, 18]` (isometric-style view) |
49
+ | Camera FOV | 50 degrees |
50
+ | Near clip | 0.1 |
51
+ | Far clip | 150 |
52
+ | Shadow type | `PCFSoftShadowMap` |
53
+ | Tone mapping | `ACESFilmicToneMapping` |
54
+ | Tone mapping exposure | 1.2 |
55
+ | Antialiasing | Enabled |
56
+ | Fog | `FogExp2` (density varies by theme, default 0.008) |
57
+
58
+ **OrbitControls** are configured with:
59
+ - Damping factor: 0.06 (smooth momentum)
60
+ - Max polar angle: `PI / 2.1` (cannot orbit below floor)
61
+ - Min distance: 6 (close zoom)
62
+ - Max distance: 80 (wide overview)
63
+ - Initial target: `[0, 1, 0]`
64
+
65
+ **`SceneThemeSync`** is an internal Canvas component that uses `useThree` to directly update `scene.fog` color/density and `gl.clearColor` when the theme changes — no re-render, pure imperative update.
66
+
67
+ ### 15.3 Map Controls Overlay
68
+
69
+ A DOM overlay positioned `bottom: 20, left: 20` provides three navigation buttons:
70
+
71
+ | Button | Action | Implementation |
72
+ |--------|--------|----------------|
73
+ | + (zoom in) | Zoom factor 0.65 | Scales camera offset from target by 0.65 |
74
+ | - (zoom out) | Zoom factor 1.5 | Scales camera offset from target by 1.5 |
75
+ | Top-down view | Bird's-eye | Flies to `[t.x + 0.01, t.y + 30, t.z + 0.01]` targeting current look-at |
76
+ | Reset view | Default position | Flies to `[18, 16, 18]` targeting `[0, 1, 0]` |
77
+
78
+ Buttons use inline style hover effects: `background: rgba(0,240,255,0.15)` on hover, `backdrop-filter: blur(8px)`, monospace font, 34×34px size.
79
+
80
+ ### 15.4 Dynamic Room System
81
+
82
+ Rooms are created and destroyed dynamically based on `roomStore.rooms`. The layout engine in `src/lib/cyberdromeScene.ts` computes geometry from room indices.
83
+
84
+ **Layout constants:**
85
+
86
+ | Constant | Value | Description |
87
+ |----------|-------|-------------|
88
+ | `ROOM_SIZE` | 12 | Internal room dimension (12×12 units) |
89
+ | `ROOM_GAP` | 5 | Corridor width between rooms |
90
+ | `ROOM_CELL` | 17 | Grid cell size (ROOM_SIZE + ROOM_GAP) |
91
+ | `ROOM_HALF` | 6 | Half of room size |
92
+ | `ROOM_COLS` | 4 | Maximum rooms per row before wrapping |
93
+ | `WALL_H` | 2.8 | Wall height |
94
+ | `WALL_T` | 0.12 | Wall thickness |
95
+ | `DOOR_GAP` | 4 | Doorway width (centered on each wall) |
96
+
97
+ **Room grid positioning:**
98
+
99
+ Rooms are placed in a grid where column is `roomIndex % 4` and row is `floor(roomIndex / 4)`. Columns are centered around X=0. Center of room at index `i`:
100
+ ```
101
+ col = i % 4
102
+ row = floor(i / 4)
103
+ x = (col - 1.5) * 17
104
+ z = row * 17
105
+ ```
106
+
107
+ **`RoomConfig` interface** (computed per room):
108
+ - `index: number` — grid index
109
+ - `roomId: string` — store room ID
110
+ - `name: string` — display name
111
+ - `center: [number, number, number]` — world center
112
+ - `bounds: RoomBound` — min/max X/Z extents
113
+ - `stripColor: 0 | 1` — alternates cyan/magenta strips per even/odd index
114
+
115
+ ### 15.5 Room Walls
116
+
117
+ Each room has 4 walls rendered in `RoomWalls`. North and south walls (along Z edges) each have a 4-unit door gap in the center, splitting into two segments per wall. East and west walls are solid. Each wall has a glowing strip at the top edge.
118
+
119
+ **Wall geometry:**
120
+ - Wall segment: `BoxGeometry(length, WALL_H, WALL_T)` — metallic, semi-transparent
121
+ - Top strip: `BoxGeometry(len, 0.04, WALL_T + 0.06)` — emissive (2.0 intensity), alternates theme `stripPrimary` or `stripSecondary`
122
+ - Wall material: `roughness: 0.2`, `metalness: 0.7`, `transparent: true`, opacity from theme
123
+ - Doors only on north and south walls; east and west walls are solid
124
+
125
+ **Collision rects** for wall physics are built by `buildDynamicWallRects`: each wall segment produces a `WallRect` with 0.25-unit half-thickness on the thin axis.
126
+
127
+ ### 15.6 Room Desks Layout (8 desks per room)
128
+
129
+ Each room has exactly 8 desks placed at fixed positions relative to the room center `[cx, cz]`:
130
+
131
+ | Position | Facing | Notes |
132
+ |----------|--------|-------|
133
+ | `[cx-3.5, cz-4.5]` | South (rot=0) | North wall, left of door |
134
+ | `[cx+3.5, cz-4.5]` | South (rot=0) | North wall, right of door |
135
+ | `[cx+3.5, cz+4.5]` | North (rot=PI) | South wall, offset from door |
136
+ | `[cx-5, cz-1.5]` | East (rot=PI/2) | West wall |
137
+ | `[cx-5, cz+1.5]` | East (rot=PI/2) | West wall |
138
+ | `[cx+5, cz-1.5]` | West (rot=-PI/2) | East wall |
139
+ | `[cx+5, cz+1.5]` | West (rot=-PI/2) | East wall |
140
+ | `[cx+5, cz-3.5]` | West (rot=-PI/2) | East wall, additional |
141
+
142
+ Each desk consists of:
143
+ - **Tabletop**: `BoxGeometry(1.5, 0.05, 0.65)` at Y=0.7
144
+ - **Two side legs**: `BoxGeometry(0.04, 0.66, 0.58)` at X ± 0.72
145
+ - **Monitor frame**: `BoxGeometry(0.48, 0.32, 0.025)` at Y=0.92
146
+ - **Monitor screen**: `BoxGeometry(0.44, 0.28, 0.005)` with emissive color from `PALETTE`
147
+ - **Keyboard**: `BoxGeometry(0.32, 0.012, 0.1)` at Y=0.72
148
+ - **Chair seat**: `BoxGeometry(0.36, 0.03, 0.36)` at Y=0.4
149
+ - **Chair back**: `BoxGeometry(0.34, 0.28, 0.03)` at Y=0.57
150
+ - **Chair stem**: `CylinderGeometry(0.025, 0.025, 0.36, 6)`
151
+ - **Chair base**: `CylinderGeometry(0.16, 0.16, 0.025, 6)`
152
+
153
+ The chair is positioned 0.65 units toward the robot's facing direction from the desk center. Monitor screen color cycles through `PALETTE` using `(deskOffset + di) * 3 + 1`.
154
+
155
+ ### 15.7 Corridor Workstations
156
+
157
+ For robots not assigned to any room (`zone === -1`), a dedicated "common area" with 10 workstations is placed south of all rooms. The layout is a 2-row × 5-column grid:
158
+
159
+ - **No rooms**: Centered at origin, rows at Z=-3 and Z=+3, spacing X=3.5
160
+ - **With rooms**: Placed at `southmostRoomCenter.z + ROOM_HALF + ROOM_GAP + 5`, horizontally centered across the room span. Spacing X=4, Z=4 between rows.
161
+
162
+ Row 0 faces south (rot=0), row 1 faces north (rot=PI). These workstations share the same desk geometry as room desks.
163
+
164
+ ### 15.8 Casual Areas: Coffee Lounge and Gym
165
+
166
+ Two casual areas are built north of the room grid (most negative Z side), separated by a 3-unit gap. Each area is 14×14 units (`CASUAL_AREA_SIZE = 14`).
167
+
168
+ **Placement:**
169
+ - If rooms exist: `baseZ = min room edge - ROOM_GAP - 7 - 2` (7 = CASUAL_HALF)
170
+ - Area centers: `coffeeX = centerX - 10` (CASUAL_HALF + gap/2), `gymX = centerX + 10`
171
+
172
+ **Coffee Lounge** (`zone === -2`):
173
+ - 14×14 floor pad at Y=0.004 with `theme.coffeeFloor` material
174
+ - AreaBorderGlow (14-unit) in `theme.coffeeAccent` color
175
+ - 6 coffee tables in a 2×3 grid (rows at Z±2.5, cols at X-3/0/+3 relative to area center)
176
+ - Each table: round cylinder top (`CylinderGeometry(0.5, 0.5, 0.04, 12)` at Y=0.5), stem, base
177
+ - Two stools per table at X±0.6
178
+ - Counter bar along north edge: `BoxGeometry(8, 1.1, 0.5)` at Y=0.55, with accent top strip
179
+ - Coffee machine: box body + screen panel + nozzle cylinder + cup platform
180
+ - Coffee pot: tapered cylinder + torus handle
181
+ - 3 coffee cups on tables
182
+ - Warm point light: `color: theme.coffeeAccent`, intensity 6, distance 14
183
+
184
+ **Gym Area** (`zone === -3`) — 10 equipment types:
185
+
186
+ | Equipment | Position (relative) | Details |
187
+ |-----------|---------------------|---------|
188
+ | Bench press | `[-5, -5]` | Bench box + legs + barbell cylinder + weight discs (torusGeometry) |
189
+ | Treadmill | `[0, -5]` | Angled running platform + handlebar uprights + console |
190
+ | Rowing machine | `[+5, -5]` | Horizontal rail + sliding seat + foot pads + handle |
191
+ | Stationary bike | `[-5, 0]` | Frame box + seat + handlebar post + wheel cylinder |
192
+ | Pull-up bar | `[0, 0]` | Two uprights (height 2.4) + crossbar cylinder + diagonal braces |
193
+ | Leg press | `[+5, 0]` | Angled platform + seat + back rest + guide rails |
194
+ | Punching bag | `[-5, +5]` | Ceiling mount + vertical support + cylindrical bag |
195
+ | Cable machine | `[0, +5]` | Frame + top crossbar + pulley wheel + vertical cable + weight stack |
196
+ | Kettlebell rack | `[+5, +5]` | Shelf frame + 3 kettlebells (sphere + torus handle) |
197
+ | Dumbbell rack | `[-2.5, +2.5]` | Rack frame + 2 shelf levels + 5 dumbbells (sphereGeometry) |
198
+ | Medicine ball | `[+2.5, +2.5]` | `SphereGeometry(0.13, 8, 8)` in accent color |
199
+
200
+ Cool point light: `color: theme.gymAccent`, intensity 8, distance 18.
201
+
202
+ **Casual workstation zones**: Coffee lounge stations use `zone === -2`, gym stations use `zone === -3`. Both are part of the `workstations` array. Idle robots seek coffee (zone -2), waiting robots seek gym (zone -3).
203
+
204
+ ### 15.9 Floor and Environment
205
+
206
+ **DynamicFloor:**
207
+ - Main floor: `PlaneGeometry(floorSize, floorSize)` at Y=0, `roughness: 0.7, metalness: 0.3`
208
+ - Per-room floor panels: `PlaneGeometry(12, 12)` at Y=0.003, slightly brighter `theme.roomFloor`
209
+ - Room border glow: 4 thin planes (0.06-unit wide) forming a square outline at Y=0.015, emissive intensity 1.5, opacity 0.35
210
+ - Grid overlay 1: `GridHelper(floorSize, floorSize/1)` at Y=0.005, theme `grid1` color, opacity 0.04
211
+ - Grid overlay 2: `GridHelper(floorSize, floorSize/5)` at Y=0.008, theme `grid2` color, opacity 0.03
212
+ - Floor size: `max(30, sceneBounds * 2 + 10)`
213
+
214
+ **Circuit traces**: 14 random L-shaped polylines on the floor at Y=0.011. Each has 4-7 axis-aligned segments of random length 1-4 units. Colors cycle through `[theme.particle1, theme.particle2, theme.trace3]`. Opacity 0.08-0.16. Built with raw `THREE.BufferGeometry` + `lineBasicMaterial`.
215
+
216
+ **Data particles** (animated):
217
+ - 140 cyan particles (`theme.particle1`) rising at speed 0.2-0.8 units/sec
218
+ - 80 magenta particles (`theme.particle2`) at the same speed
219
+ - Size: 0.04, opacity: 0.4, additive blending
220
+ - When a particle reaches Y=10, it resets to Y=0 at a random X/Z position
221
+ - `Float32Array` buffers updated every frame via `points.geometry.attributes.position.needsUpdate = true`
222
+
223
+ **Stars**: 400 random points at Y=8-43, X/Z ±50, size 0.05, opacity 0.4, `theme.stars` color.
224
+
225
+ ### 15.10 Room Lighting
226
+
227
+ Each room has dedicated interior lighting:
228
+
229
+ **Visual sconces**: 9 emissive boxes per room (3 along north wall, 2 on each side wall, 2 along south wall):
230
+ - Bracket: `BoxGeometry(0.8, 0.12, 0.05)` + mount `BoxGeometry(0.1, 0.08, 0.22)` in dark metallic
231
+ - Light tube: `BoxGeometry(1.2, 0.25, 0.12)` emissive in `theme.sconceColor`, intensity 2.5
232
+
233
+ **Point lights** (2 per room, GPU-friendly):
234
+ - Primary: `color: theme.roomLight1`, intensity 10, distance 16, decay 1.5, at ceiling level
235
+ - Secondary: `color: theme.roomLight2`, intensity 4, distance 12, decay 2, at mid-height
236
+
237
+ **Global lighting** (scene-wide):
238
+ - Ambient: `theme.ambientColor`, intensity varies by theme (4-10)
239
+ - Directional (shadows): `position: [8, 20, 6]`, shadow map 2048×2048, bias -0.0004
240
+ - Fill directional: `position: [-6, 15, -8]`
241
+ - 3 point lights at corners: `[-10, 8, -10]`, `[10, 7, 10]`, `[0, 10, 0]`
242
+ - Hemisphere: sky color, ground color, intensity from theme
243
+
244
+ ### 15.11 Robot Navigation AI
245
+
246
+ Each robot has 4 navigation modes (stored in `NavState.mode`):
247
+
248
+ | Constant | Value | Description |
249
+ |----------|-------|-------------|
250
+ | `NAV_WALK` | 0 | Wandering to random target |
251
+ | `NAV_GOTO` | 1 | Navigating to a specific desk via waypoints |
252
+ | `NAV_SIT` | 2 | Seated at a desk, not moving |
253
+ | `NAV_IDLE` | 3 | Frozen in place (alert/input/offline/connecting) |
254
+
255
+ **Navigation state (`NavState`):**
256
+ - `mode` — current nav mode
257
+ - `target` — current waypoint target (Vector3)
258
+ - `deskIdx` — occupied workstation index (-1 if none)
259
+ - `speed` — base speed: 3.0 + random(0, 1.5) units/sec
260
+ - `walkHz` — walk bounce frequency: 12 + random(0, 4) Hz
261
+ - `phase` — random phase offset for animation desync
262
+ - `decisionTimer` — countdown to desk-seek attempt
263
+ - `posX, posY, posZ` — current world position
264
+ - `rotY` — current Y rotation
265
+ - `waypoints` — ordered array of intermediate points
266
+ - `waypointIdx` — current waypoint index
267
+
268
+ **Status-to-navigation mapping:**
269
+
270
+ | Status | Robot State | Navigation behavior |
271
+ |--------|-------------|---------------------|
272
+ | idle | idle | Seek coffee workstation (zone -2); wander if full |
273
+ | prompting | thinking | Seek desk in assigned room or corridor; sit when arrived |
274
+ | working | working | Seek desk; speed multiplier 1.5 |
275
+ | waiting | waiting | Seek gym workstation (zone -3); wander if full |
276
+ | approval | alert | Freeze (NAV_IDLE or stay seated) |
277
+ | input | input | Freeze (NAV_IDLE or stay seated) |
278
+ | ended | offline | Freeze |
279
+ | connecting | connecting | Freeze |
280
+
281
+ **Desk seeking logic:**
282
+
283
+ When entering a desk-seeking state (`thinking` or `working`):
284
+ 1. Determine zone: room assignment → current position zone → fallback
285
+ 2. Find empty workstations in that zone
286
+ 3. Claim one randomly, set `ws.occupantId = sessionId`, go `NAV_GOTO`
287
+ 4. If all desks full: find nearest occupied desk, stand 0.5 units behind it (overflow)
288
+ 5. During `NAV_WALK`, robot periodically checks for empty desks (every 3-8 seconds via `decisionTimer`)
289
+
290
+ **Wall collision** (`collidesAnyWall`):
291
+ - Checks 0.25-unit robot radius against all `WallRect` entries
292
+ - On X-axis collision only: slide along Z, pick new wander target
293
+ - On Z-axis collision only: slide along X, pick new wander target
294
+ - On full collision: pick new wander target
295
+ - Position clamped to `[-sceneBound, sceneBound]` on both axes
296
+
297
+ **Walk bounce:** `posY = abs(sin(time * walkHz + phase)) * 0.03` — small up-down bounce while moving.
298
+
299
+ **Facing:** `rotY` lerps toward `atan2(dx, dz)` at rate `min(1, 10 * dt)` per frame.
300
+
301
+ **Seated position:** When `NAV_SIT` is reached, robot snaps to `ws.seatPos` with `posY = -0.12` (slightly sunken) and faces `ws.faceRot`.
302
+
303
+ ### 15.12 Cross-Room Pathfinding
304
+
305
+ When a robot needs to navigate between zones (room → corridor → room), it uses a door-waypoint system.
306
+
307
+ **Door waypoints:** Built by `buildDoorWaypoints`. Each room gets 2 waypoints per wall (north and south):
308
+ - `outside`: 1 unit past the wall exterior (in the corridor)
309
+ - `inside`: 1 unit past the wall interior (inside the room)
310
+
311
+ **`computePathWaypoints(fromX, fromZ, target, fromZone, targetZone, doors)`:**
312
+ 1. Same zone → direct path (single waypoint = target)
313
+ 2. Both in corridor/casual (`< 0`) → direct path
314
+ 3. Exiting a room: pick the door closest to the target. Add `inside` → `outside` waypoints.
315
+ 4. Entering a room: pick the door closest to the robot's current position. Add `outside` → `inside` waypoints.
316
+ 5. Append final target.
317
+
318
+ **Nearest-door selection:** Euclidean distance from the door's `outside` point to the robot's current position (for entering) or to the target (for exiting).
319
+
320
+ **`setNavTarget` helper:** Calls `computePathWaypoints`, sets `nav.waypoints`, initializes `nav.waypointIdx = 0`, sets `nav.target` to the first waypoint.
321
+
322
+ ### 15.13 Robot 3D Model
323
+
324
+ **Geometry** (shared across all instances, defined in `src/lib/robot3DGeometry.ts`):
325
+
326
+ | Part | Geometry | Dimensions |
327
+ |------|----------|------------|
328
+ | Head | `BoxGeometry` | 0.28 × 0.24 × 0.26 |
329
+ | Visor | `BoxGeometry` | 0.24 × 0.065 × 0.02 |
330
+ | Antenna | `CylinderGeometry` | r=0.007, h=0.14, 4-sided |
331
+ | Antenna tip | `SphereGeometry` | r=0.02, 6 segments |
332
+ | Torso | `BoxGeometry` | 0.32 × 0.38 × 0.2 |
333
+ | Core | `SphereGeometry` | r=0.032, 8 segments |
334
+ | Joint | `SphereGeometry` | r=0.035, 8 segments |
335
+ | Arm | `BoxGeometry` | 0.08 × 0.26 × 0.08 |
336
+ | Leg | `BoxGeometry` | 0.09 × 0.28 × 0.09 |
337
+ | Foot | `BoxGeometry` | 0.1 × 0.045 × 0.12 |
338
+
339
+ Each body part also has an `EdgesGeometry` wireframe overlay (lineSegments) with a neon color material at opacity 0.3.
340
+
341
+ **Materials:**
342
+ - `metalMat`: `MeshStandardMaterial`, color `#2a2a3e`, roughness 0.3, metalness 0.85 (shared)
343
+ - `darkMat`: color `#1c1c2c`, roughness 0.4, metalness 0.7 (shared)
344
+ - `neonMats[i]`: per-palette emissive material, intensity 2.0 (pool of 16, shared)
345
+ - `edgeMats[i]`: `LineBasicMaterial`, color = palette color, opacity 0.3 (pool of 16, shared)
346
+ - `bodyMat`: cloned from `metalMat` per robot (animations mutate emissive each frame)
347
+ - `bodyEdgeMat`: cloned from `edgeMat` per robot
348
+
349
+ **PALETTE** — 16 cyberpunk neon colors:
350
+ `#00f0ff`, `#ff00aa`, `#a855f7`, `#00ff88`, `#ff4444`, `#ffaa00`, `#00aaff`, `#ff66ff`, `#44ff44`, `#ff8800`, `#8855ff`, `#00ffcc`, `#ff0066`, `#ccff00`, `#ff5577`, `#33ddff`
351
+
352
+ **Model variants** (6 types, defined in `src/lib/robot3DModels.ts`):
353
+
354
+ | Type | Description | Distinctive geometry |
355
+ |------|-------------|---------------------|
356
+ | `robot` | Standard humanoid | Default geometry |
357
+ | `mech` | Bulkier, wider stance | Head 0.34×0.2×0.3, Torso 0.42×0.44×0.26, wider arms/legs |
358
+ | `drone` | Hovering unit | Spherical head (r=0.14), flat arms (0.22×0.04×0.06), no legs, baseY=0.3 |
359
+ | `spider` | Low-slung 4-legged | Spherical head (r=0.12), wide flat torso, all 4 limbs stubby at corners, baseY=-0.15 |
360
+ | `orb` | Spherical body | Spherical head (r=0.10), spherical torso (r=0.22, 12-seg), short arms and legs |
361
+ | `tank` | Wide, one-armed | Compact head, wide torso (0.44×0.3×0.26), no left arm, thick right arm, tread-shaped legs, baseY=-0.05 |
362
+
363
+ **Per-robot model selection:** `session.characterModel` overrides the global `settingsStore.characterModel` (default `'robot'`).
364
+
365
+ **CLI source badge:** A Billboard `<Text>` on the robot's chest (position slightly below core) showing a single letter with emissive color:
366
+ - Claude: `'C'`, color `#00f0ff`
367
+ - Gemini: `'G'`, color `#4285f4`
368
+ - Codex: `'X'`, color `#10a37f`
369
+ - OpenClaw: `'O'`, color `#ff6b2b`
370
+ - Unknown: `'?'`, color `#aa66ff`
371
+
372
+ Badge has `outlineWidth: 0.01`, `emissiveIntensity: 0.8`.
373
+
374
+ **CLI color override:** When a session has no explicit `accentColor`, the robot's neon color is set to the CLI badge color rather than the PALETTE color at `session.colorIndex`.
375
+
376
+ ### 15.14 Robot Animations
377
+
378
+ Animations run in `useFrame` by reading `useSettingsStore.getState()` imperatively (no subscription). `animSpeed = animationSpeed / 100`, `ai = animationIntensity / 100`.
379
+
380
+ **Always active:**
381
+ - Antenna tip: scales between 0.8 and 1.2 at 6 Hz with `ai` factor. Web tools: intensity 3-4.5.
382
+ - Core: scales between 0.78 and 1.02 at 3 Hz.
383
+
384
+ **State-specific animations:**
385
+
386
+ | State | Key behaviors |
387
+ |-------|---------------|
388
+ | `idle` | Body bobs at 1.5 Hz (0.02 * ai), arms sway at 0.8 Hz, body tilts at 0.5 Hz |
389
+ | `thinking` (standing) | Body bobs at 1.2 Hz, head tilts at 0.6 Hz, right arm raised (-0.6 + oscillation) |
390
+ | `thinking` (seated) | Head tilts 0.12 + oscillation, right arm raised to chin-scratch pose (-1.1), legs bent (1.2) |
391
+ | `working` | See tool-specific animations below; body bobs at 0.7 Hz; CHARGING EFFECT active |
392
+ | `waiting` | Body bounces (abs sin) at 2 Hz (0.06 * ai), head turns at 0.8 Hz |
393
+ | `alert` | Visor flashes (urgency-scaled); arms raised and shaking at 8 Hz; 30s+ adds lateral shake |
394
+ | `input` | Visor pulses at 4 Hz; right arm fully raised (-1.5); slow body sway |
395
+ | `offline` | Visor/core dim over time; head drooped, arms slack |
396
+ | `connecting` | 1.5-second boot animation: scale 0→scaleProp, Y sinks from 0.5→0 |
397
+
398
+ **Tool-specific working animations (WS7.C):**
399
+
400
+ | Tool category | Tools | Animation |
401
+ |---------------|-------|-----------|
402
+ | `read` | Read, Grep, Glob, NotebookEdit | Head scans left-right at 2.5 Hz (±0.35 rad); arms at -0.4 |
403
+ | `write` | Write, Edit | Rapid arm typing at 14 Hz (±0.07 rad); head stable |
404
+ | `bash` | Bash | Right arm extended forward (-0.9); left arm at -0.3 |
405
+ | `task` | Task | Both arms raised (-0.8) with slow oscillation; head looks around |
406
+ | `web` | WebFetch, WebSearch | Antenna brightness boost (3-4.5 emissive intensity); default arm motion |
407
+ | `default` | All others | Standard rapid arm oscillation at 10 Hz (±0.05 rad) |
408
+
409
+ **Charging body effect** (active during any `working` state):
410
+ - Edge wireframe: opacity surges between 0.8-1.0 at two combined frequencies (12 Hz + 23 Hz)
411
+ - Core glow: emissive intensity 2.0-3.8 at 8 Hz + 19 Hz
412
+ - Visor: emissive intensity 2.0-3.2 at 10 Hz + 17 Hz
413
+ - Antenna tip: emissive intensity 2.5-4.0 at 15 Hz; scale flicker at 20 Hz
414
+ - Body mesh: subtle emissive boost in neon color, intensity 0.05-0.25 at 14 Hz
415
+
416
+ **Alert urgency escalation (WS7.B):** After 15 seconds in `alert` state, visor pulse speed increases from 8 Hz to 12 Hz. After 30 seconds, base intensity rises from 1.5 to 2.5, pulse range from 1.0 to 1.5, and lateral shake is added.
417
+
418
+ **Label completion frame effects (6 types):**
419
+
420
+ Activate when a labeled session transitions to `ended`. Override the normal body animation with:
421
+
422
+ | Effect | Visual character |
423
+ |--------|-----------------|
424
+ | `fire` | Orange/red body emissive with rapid flicker (9 Hz + 17 Hz layered), orange wireframe, fiery core |
425
+ | `electric` | Spike pattern (sin^4 at 20 Hz), white wireframe arc flicker (25 Hz threshold), intense multi-freq core |
426
+ | `chains` | Slow golden aura (2.5 Hz), gold wireframe (3 Hz), golden visor glow |
427
+ | `liquid` | Flowing wave intensity (3 Hz body, 5 Hz edges, 3.5 Hz core), hue-shifted by wave value |
428
+ | `plasma` | Violent magenta oscillations at 12/15/14 Hz body/core/visor; extreme intensities (4.0-6.5 core) |
429
+ | `none` | No frame effect |
430
+
431
+ Frame effect animations run after state animations, taking priority over body charge reset.
432
+
433
+ **Visor material overrides** (static pre-created materials):
434
+ - `alert` state: `ALERT_VISOR_MAT` — neon yellow `#ffdd00`, emissiveIntensity 2
435
+ - `input` state: `INPUT_VISOR_MAT` — purple `#aa66ff`, emissiveIntensity 2
436
+ - `offline` state: `OFFLINE_VISOR_MAT` — dark `#333344`, emissiveIntensity 2
437
+
438
+ ### 15.15 Robot Dialogue
439
+
440
+ `RobotDialogue` shows a floating speech bubble above each robot. The component uses zero React state — all dialogue data flows through a ref (`dialogueRef`) updated in `useEffect` and read in `useFrame`.
441
+
442
+ **Panel geometry:**
443
+ - Billboard positioned at Y=2.8 above robot root
444
+ - Background: `PlaneGeometry(2.2, 0.22)` — dark `#0a0616`, opacity 0.92
445
+ - Border: `PlaneGeometry(2.26, 0.26)` — colored by dialogue type, opacity 0.6
446
+ - Text: `<Text>` via troika-three-text, fontSize 0.09, max-width 2.0, white with black outline
447
+
448
+ **Fade behavior:** Non-persistent dialogues start fading after 5 seconds. Fade speed: 4 units/second toward target opacity. Values below 0.01 clamp to 0.
449
+
450
+ **`TextUpdater`** inner component reads `fillOpacity`, `outlineOpacity`, and `text` from the troika mesh via parent traversal in `useFrame`, without any React state updates.
451
+
452
+ **Dialogue content and trigger rules:**
453
+
454
+ | Trigger | Text | Border color | Persistent |
455
+ |---------|------|--------------|------------|
456
+ | `prompting` status (status change) | First 60 chars of prompt + `...` | `#00e5ff` | No (5s) |
457
+ | `approval` status | `AWAITING APPROVAL` | `#ffdd00` | Yes |
458
+ | `input` status | `NEEDS INPUT` | `#aa66ff` | Yes |
459
+ | `waiting` status (from working) | `Task complete!` | `#00ff88` | No |
460
+ | `ended` status | `OFFLINE` | `#ff4444` | No |
461
+ | `idle` (from non-idle/non-connecting) | `ONLINE` | `#00ff88` | No |
462
+ | Read/Grep/Glob tool | `Reading <filename>...` | `rgba(0,229,255,0.6)` | No |
463
+ | Bash tool | `$ <cmd truncated to 40 chars>` | `#ff9100` | No |
464
+ | Edit/Write tool | `Editing <filename>...` | `#00aaff` | No |
465
+ | Task tool | `Spawning agent...` | `#aa66ff` | No |
466
+ | WebFetch/WebSearch tool | `Fetching...` | `#00e5ff` | No |
467
+
468
+ Tool dialogues are throttled: minimum 500ms between updates unless a status change occurred. Status-based dialogues always take priority over tool dialogues.
469
+
470
+ ### 15.16 Robot Labels
471
+
472
+ `RobotLabel` renders a floating name tag above each robot using drei `<Billboard>` + `<Text>` (pure WebGL, not HTML portals). This avoids the cross-reconciler cascades that `<Html>` would cause.
473
+
474
+ **Layout** (all dimensions scale with `fontSize / 13`):
475
+
476
+ | Element | Base size | Position |
477
+ |---------|-----------|----------|
478
+ | Background panel | 1.8 × 0.14 | Y=0, Z=-0.01 |
479
+ | Border | 1.84 × 0.17 | Y=0, Z=-0.015, opacity 0.15 |
480
+ | Status dot | r=0.025 | X=-0.82, circle with 16 segments |
481
+ | Project name text | 0.065 fontSize | X=0.02, max-width 1.5 |
482
+ | Alert banner | 1.6 × 0.16 | Y=0.18 above label |
483
+ | Alert text | 0.07 fontSize | In alert banner |
484
+ | Billboard Y position | 2.1 (adjusts with scale) | |
485
+
486
+ **Status dot colors:**
487
+ - idle: `#00ff88`, prompting: `#00e5ff`, working: `#ff9100`, waiting: `#00e5ff`, approval: `#ffdd00`, input: `#aa66ff`, ended: `#ff4444`, connecting: `#888888`
488
+
489
+ **Title**: `session.title || session.projectName || 'Unnamed'`, truncated to 28 characters.
490
+
491
+ **Label badge**: When `session.label` is set, appends ` [LABEL]` to the title text.
492
+
493
+ **Alert banner**: Shows pulsing colored banner above the main label for `approval` (yellow) and `input` (purple) states. Opacity pulses via `useFrame` at 0.8+0.2*sin(time/500).
494
+
495
+ **Memoization**: 9-field equality check covering sessionId, status, title, projectName, label, robotState, isSelected, isHovered, fontSize.
496
+
497
+ ### 15.17 Status Particles
498
+
499
+ `StatusParticles` fires brief particle bursts on status transitions. Uses zero React state — always mounted with `bufferGeo.setDrawRange(0, 0)` to hide when inactive.
500
+
501
+ **Pre-allocated geometry**: One `BufferGeometry` with `Float32Array(MAX_PARTICLES * 3)` (MAX=25). Buffer is never recreated, only updated.
502
+
503
+ **Per-instance material**: `PointsMaterial` with `AdditiveBlending`, `depthWrite: false`.
504
+
505
+ **Burst configurations:**
506
+
507
+ | Transition | Color | Count | Pattern | Speed | Gravity | Lifetime |
508
+ |------------|-------|-------|---------|-------|---------|----------|
509
+ | idle/waiting → working/thinking | `#ffdd00` | 20 | up | 2.5 | -1.5 | 1.5s |
510
+ | working/thinking → waiting | `#00ff88` | 20 | confetti | 1.2 | 2.0 | 1.5s |
511
+ | → alert | `#ffdd00` | 25 | ring | 2.0 | 0 | 1.2s |
512
+ | → input | `#aa66ff` | 20 | ring | 1.5 | 0 | 1.2s |
513
+ | → offline | `#666688` | 20 | down | 0.8 | 0.5 | 2.0s |
514
+
515
+ **Patterns:**
516
+ - `up`: velocity mainly upward, small random spread
517
+ - `down`: velocity mainly downward, smaller spread
518
+ - `ring`: particles spread radially in XZ plane at Y=0.15, even angular distribution
519
+ - `confetti`: random XZ spread with upward bias
520
+
521
+ **Fade**: `opacity = 1 - progress^2`. Size: `burstSize * (1 - progress * 0.5)`. Velocity decays at 0.99x per frame tick. Gravity applies to Y velocity each tick.
522
+
523
+ ### 15.18 Subagent Connection Beams
524
+
525
+ `SubagentConnections` renders animated dashed laser-lines between parent sessions and their subagent children. Receives precomputed `ConnectionData[]` from the DOM layer (zero Zustand subscriptions).
526
+
527
+ **Detection**: Sessions where `teamRole === 'member'` and `teamId` exists. The parent ID is extracted from `teamId` by stripping the `team-` prefix. Both parent and child must be non-`ended`.
528
+
529
+ **Line rendering**: Raw `THREE.Line` with `LineDashedMaterial`:
530
+ - `dashSize: 0.3`, `gapSize: 0.2`
531
+ - Opacity: 0.3
532
+ - Color: parent session's `accentColor` or PALETTE color
533
+
534
+ **Animation**: In `useFrame`, the line's endpoint buffer is updated to track both robots' current positions from `robotPositionStore`. The `dashOffset` decrements at `delta * 2` per frame to create a flowing "data flowing from parent to child" visual. `computeLineDistances()` is called each frame (required for dashed lines to render correctly).
535
+
536
+ **Cleanup**: Geometry and material are disposed on unmount.
537
+
538
+ ### 15.19 Camera Controller
539
+
540
+ `CameraController` is a Canvas-side component that smoothly animates OrbitControls to fly-to targets. It reads `cameraStore` imperatively in `useFrame` (no subscription) to avoid cascades.
541
+
542
+ **Constants:**
543
+ - `LERP_FACTOR = 0.04` — smooth camera movement (4% of remaining distance per frame)
544
+ - `ARRIVAL_THRESHOLD = 0.1` — considers animation complete when both position and look-at are within 0.1 units
545
+
546
+ **Fly-to behavior:**
547
+ 1. Detects a new request by comparing `pendingTarget.requestId` to `lastRequestId`.
548
+ 2. Sets `targetPos` and `targetLookAt` from the request.
549
+ 3. Each frame: `camera.position.lerp(targetPos, 0.04)` and `controls.target.lerp(targetLookAt, 0.04)`.
550
+ 4. When arrived: snaps to exact position, calls `completeAnimation()` via `queueMicrotask` (deferred out of R3F render cycle).
551
+
552
+ **Robot click → fly-to:** When a robot is selected, `CyberdromeScene` reads the robot's world position from `robotPositionStore` and calls `flyTo([pos.x + 6, pos.y + 8, pos.z + 10], [pos.x, pos.y + 1, pos.z])`. Offset constants: `FLY_OFFSET_X=6, FLY_OFFSET_Y=8, FLY_OFFSET_Z=10`.
553
+
554
+ **Room zoom:** `computeRoomCameraTarget(roomIndex)` places camera at 45-degree angle, 14 units out, 10 units high from room center.
555
+
556
+ **`cameraStore`:**
557
+ - `DEFAULT_CAMERA_POSITION: [18, 16, 18]`
558
+ - `DEFAULT_CAMERA_TARGET: [0, 1, 0]`
559
+ - `flyTo(position, lookAt)` sets `pendingTarget` with `requestId: Date.now()`
560
+ - `completeAnimation()` clears pending target and sets `isAnimating: false`
561
+
562
+ ### 15.20 Room Labels
563
+
564
+ `RoomLabels` renders 3D text labels on the floor at each room's south door, using drei `<Text>` (SDF-based rendering at any zoom level).
565
+
566
+ **Room labels:**
567
+ - Room name: `fontSize: 0.8`, color alternates cyan (`#00f0ff`) or magenta (`#ff00aa`) based on `stripColor`, `letterSpacing: 0.15`, 2cm outline
568
+ - Unit count: `fontSize: 0.4`, dimmer variant of the strip color, at Z + 0.9 from room name
569
+ - Position: on floor (Y=0.02), rotated flat (`[-PI/2, 0, 0]`), at `cx, cz + ROOM_HALF + 1.5`
570
+
571
+ **Casual area labels:**
572
+ - Coffee Lounge: color `#ff9944`, at south edge of lounge area
573
+ - Gym Area: color `#44ff88`, at south edge of gym area
574
+ - Same fontSize/letterSpacing/outline as room labels
575
+
576
+ ### 15.21 Robot List Sidebar
577
+
578
+ `RobotListSidebar` is a DOM overlay panel (top-right of scene) listing all active robots.
579
+
580
+ - Width: 280px, max-height: `calc(100vh - 100px)`, scrollable
581
+ - Backdrop blur, panel background, neon border with glow box-shadow
582
+ - Hidden when no active sessions
583
+ - Header: "Agents (N)" in monospace uppercase
584
+
585
+ **Entry sorting:** working → prompting/thinking → approval/input → waiting → idle → connecting
586
+
587
+ **Each entry shows:**
588
+ - Status dot (10×10px circle with glow box-shadow)
589
+ - Session title or project name (truncated, ellipsis)
590
+ - Status text (uppercase, status color)
591
+ - Close button (✕) — calls `removeSession` + `DELETE /api/sessions/:id`
592
+
593
+ **Selection:** Clicking an entry dispatches `CustomEvent('robot-select')` — same path as in-scene robot clicks, triggers session selection + camera fly-to.
594
+
595
+ **Selected state:** Border and background tint match the status color.
596
+
597
+ ### 15.22 Scene Themes (9 themes)
598
+
599
+ Each theme provides a `Scene3DTheme` with 35+ color/density properties covering every visual element. Themes are applied via `getScene3DTheme(themeName)` which maps `ThemeName` to the palette:
600
+
601
+ | Theme | Background | Primary strip | Secondary strip | Character |
602
+ |-------|-----------|---------------|-----------------|-----------|
603
+ | `command-center` | `#0e0c1a` (dark navy) | `#00f0ff` (cyan) | `#ff00aa` (magenta) | Classic cyberpunk |
604
+ | `cyberpunk` | `#0d0221` (deep purple) | `#ff00ff` (magenta) | `#00ffff` (cyan) | High contrast neon |
605
+ | `warm` | `#f5ede0` (cream) | `#d97706` (amber) | `#b87333` (copper) | Daylight office |
606
+ | `dracula` | `#282a36` | `#bd93f9` (purple) | `#50fa7b` (green) | Dracula scheme |
607
+ | `solarized` | `#002b36` (dark teal) | `#2aa198` (teal) | `#cb4b16` (orange) | Solarized dark |
608
+ | `nord` | `#2e3440` (slate) | `#88c0d0` (sky) | `#d08770` (peach) | Nordic calm |
609
+ | `monokai` | `#272822` (dark) | `#66d9ef` (cyan) | `#f92672` (pink) | Monokai |
610
+ | `light` | `#e8eaef` (light gray) | `#3b82f6` (blue) | `#0ea5e9` (sky) | Light mode |
611
+ | `blonde` | `#f0e8d8` (warm white) | `#ca8a04` (gold) | `#a16207` (dark gold) | Warm blonde |
612
+
613
+ Theme properties include: `background`, `fogDensity`, `floor`, `roomFloor`, `borderGlow`, `grid1/2`, `wall`, `wallOpacity`, `stripPrimary/Secondary`, `desk`, `monitorFrame`, `chair`, `particle1/2`, `trace3`, `stars`, ambient/directional/fill/point/hemisphere lighting, `sconceColor`, `roomLight1/2`, `coffeeFloor/Accent/Furniture`, `gymFloor/Accent/Equipment`.
614
+
615
+ ---
616
+
617
+ ## 22. Sound System
618
+
619
+ ### 22.1 Architecture
620
+
621
+ The sound system has two layers:
622
+
623
+ 1. **`SoundEngine`** (singleton `soundEngine`) — event-driven sound effects using Web Audio API synthesis
624
+ 2. **`AmbientEngine`** (singleton `ambientEngine`) — continuous procedurally generated ambient presets
625
+
626
+ Both engines use lazy `AudioContext` creation, only initialized after user interaction.
627
+
628
+ ### 22.2 Sound Library (16 sounds)
629
+
630
+ All sounds are synthesized from Web Audio API primitives — no audio files:
631
+
632
+ | Name | Synthesis | Character |
633
+ |------|-----------|-----------|
634
+ | `chirp` | 1200 Hz sine, 80ms | Short high blip |
635
+ | `ping` | 660 Hz sine, 200ms | Medium tone |
636
+ | `chime` | Sequence [523, 659, 784] Hz, 80ms spacing | Major triad ascending |
637
+ | `ding` | 800 Hz triangle, 250ms | Bell-like |
638
+ | `blip` | 880 Hz square at 0.5 vol, 50ms | Short digital beep |
639
+ | `swoosh` | Sine 300→1200 Hz ramp over 250ms | Rising sweep |
640
+ | `click` | 1200 Hz square at 0.2 vol, 30ms | Crisp tap |
641
+ | `beep` | 440 Hz square at 0.4 vol, 150ms | Classic beep |
642
+ | `warble` | 600 Hz sine + 12 Hz LFO (±50 Hz), 300ms | Trembling tone |
643
+ | `buzz` | 200 Hz sawtooth at 0.4 vol, 120ms | Buzzy low tone |
644
+ | `cascade` | Sequence [784, 659, 523, 392], 100ms spacing | Descending arpeggio |
645
+ | `fanfare` | Sequence [523, 659, 784, 1047, 1319], 80ms spacing | Ascending fanfare |
646
+ | `alarm` | Square sequence [880, 660, 880, 660], 150ms each | Alert pattern |
647
+ | `thud` | Sine 80→30 Hz exponential ramp, 350ms | Bass impact |
648
+ | `urgentAlarm` | 3 bursts: square 1000↔800↔1000 Hz + sawtooth 200 Hz undertone | Triple urgent alarm |
649
+ | `none` | No-op | Silence |
650
+
651
+ ### 22.3 Sound Actions (20 actions)
652
+
653
+ Actions are organized in 3 categories:
654
+
655
+ **Session Events** (4): `sessionStart`, `sessionEnd`, `promptSubmit`, `taskComplete`
656
+
657
+ **Tool Calls** (9): `toolRead`, `toolWrite`, `toolEdit`, `toolBash`, `toolGrep`, `toolGlob`, `toolWebFetch`, `toolTask`, `toolOther`
658
+
659
+ **System** (7): `approvalNeeded`, `inputNeeded`, `alert`, `kill`, `archive`, `subagentStart`, `subagentStop`
660
+
661
+ **Default action → sound mapping:**
662
+
663
+ | Action | Default sound |
664
+ |--------|--------------|
665
+ | sessionStart | chime |
666
+ | sessionEnd | cascade |
667
+ | promptSubmit | ping |
668
+ | taskComplete | fanfare |
669
+ | toolRead | click |
670
+ | toolWrite | blip |
671
+ | toolEdit | blip |
672
+ | toolBash | buzz |
673
+ | toolGrep | click |
674
+ | toolGlob | click |
675
+ | toolWebFetch | swoosh |
676
+ | toolTask | ding |
677
+ | toolOther | click |
678
+ | approvalNeeded | alarm |
679
+ | inputNeeded | chime |
680
+ | alert | alarm |
681
+ | kill | thud |
682
+ | archive | ding |
683
+ | subagentStart | chirp |
684
+ | subagentStop | ping |
685
+
686
+ ### 22.4 Per-CLI Sound Profiles
687
+
688
+ Each CLI has an independent sound profile with its own volume and per-action sound mappings:
689
+
690
+ | CLI | Volume | Character |
691
+ |-----|--------|-----------|
692
+ | Claude | 0.7 | Standard with fanfare on task complete |
693
+ | Gemini | 0.7 | Heavier on swoosh, dings |
694
+ | Codex | 0.5 | Quieter, minimal sounds (blips and clicks) |
695
+ | OpenClaw | 0.7 | Dramatic — urgentAlarm for approvals, fanfares |
696
+
697
+ `AlarmEngine.playForCli()` detects the CLI via `detectCli()`, looks up the per-CLI volume and action mapping, temporarily overrides the sound engine volume for the play, then restores it.
698
+
699
+ ### 22.5 CLI Detection
700
+
701
+ `detectCli(session)` determines CLI from:
702
+ 1. `session.model` string: contains `claude`/`opus`/`sonnet`/`haiku` → Claude; `gemini`/`gemma` → Gemini; `gpt`/`codex`/`o1`/`o3`/`o4` → Codex; `openclaw`/`claw` → OpenClaw
703
+ 2. Event type fallback: `BeforeAgent`/`AfterAgent`/`BeforeTool`/`AfterTool` → Gemini; `agent-turn-complete` → Codex; `SessionStart`/`PreToolUse`/`PostToolUse`/`UserPromptSubmit` → Claude
704
+
705
+ ### 22.6 Ambient Presets (6 presets)
706
+
707
+ All ambient sounds are synthesized from oscillators and filtered noise:
708
+
709
+ | Preset | Synthesis technique |
710
+ |--------|---------------------|
711
+ | `off` | Silent (no audio) |
712
+ | `rain` | Bandpass-filtered noise (3000 Hz) + highshelf cutoff + random droplet oscillators every 80-200ms |
713
+ | `lofi` | 60 Hz sine with 0.3 Hz LFO (±5 Hz) + lowpass-filtered noise (400 Hz cutoff) |
714
+ | `serverRoom` | Bandpass noise (500 Hz) + 120 Hz triangle fan hum (0.1 Hz LFO ±3 Hz) + 8000 Hz whine |
715
+ | `deepSpace` | 40 Hz sine with 0.05 Hz LFO (±8 Hz) + convolver reverb (3s exponential impulse) + 80 Hz harmonic |
716
+ | `coffeeShop` | Lowpass+highpass filtered noise (200-1200 Hz) + random triangle dings every 2-6s |
717
+
718
+ ---
719
+
720
+ ## 23. Movement Effects
721
+
722
+ ### 23.1 CSS Data-Attribute System
723
+
724
+ Movement effects in the legacy CSS frontend are applied by setting `data-effect="<effectName>"` on session card DOM elements. A central `movementManager.js` module applies effects and schedules their removal.
725
+
726
+ **Effect lifecycle:**
727
+ 1. Effect applied: `element.dataset.effect = effectName`
728
+ 2. CSS animation triggers via `[data-effect="name"]` selector
729
+ 3. Auto-clear: `setTimeout(() => delete element.dataset.effect, duration)` removes the attribute
730
+
731
+ ### 23.2 Effect Library (18 effects)
732
+
733
+ | Effect | CSS animation | Trigger scenario |
734
+ |--------|---------------|-----------------|
735
+ | `walk` | Horizontal movement | UserPromptSubmit |
736
+ | `run` | Fast horizontal movement | PreToolUse (working) |
737
+ | `bounce` | Vertical bouncing | Task complete |
738
+ | `shake` | Lateral shake | Alarm / ONEOFF label |
739
+ | `flash` | Opacity flicker | HEAVY label complete |
740
+ | `spin` | 360-degree rotation | Session end |
741
+ | `wave` | Wave oscillation | Prompting |
742
+ | `pulse` | Scale pulse | Tool use |
743
+ | `dance` | Celebratory multi-move | Waiting / task complete |
744
+ | `jump` | Vertical leap | Subagent spawn |
745
+ | `slide` | Horizontal slide | Session start |
746
+ | `wobble` | Irregular wobble | Alert state |
747
+ | `flip` | Vertical flip | Kill action |
748
+ | `zoom` | Scale zoom | Selection |
749
+ | `fade` | Opacity fade | Archive |
750
+ | `glow` | Glow pulse | Idle state |
751
+ | `twitch` | Rapid small movements | Approval needed |
752
+ | `none` | No animation | Default |
753
+
754
+ ### 23.3 Action-to-Movement Mapping
755
+
756
+ Default movement mappings (configurable in settings):
757
+
758
+ | Action | Default movement |
759
+ |--------|-----------------|
760
+ | sessionStart | slide |
761
+ | sessionEnd | spin |
762
+ | promptSubmit | wave |
763
+ | taskComplete | bounce |
764
+ | toolRead | pulse |
765
+ | toolWrite | pulse |
766
+ | toolBash | run |
767
+ | toolWebFetch | walk |
768
+ | toolTask | jump |
769
+ | approvalNeeded | twitch |
770
+ | inputNeeded | wobble |
771
+ | kill | flip |
772
+ | archive | fade |
773
+ | subagentStart | jump |
774
+
775
+ ---
776
+
777
+ ## 24. Alarm System
778
+
779
+ ### 24.1 Approval Alarm (repeating)
780
+
781
+ When a session enters `approval` status:
782
+ 1. `soundEngine.play('approvalNeeded')` fires immediately
783
+ 2. A `setInterval` is created for that session, firing every **10 seconds**
784
+ 3. Each interval tick: re-checks if session is still in `approval` and not muted
785
+ 4. If status changes or session is muted, the interval is cleared and removed from `approvalTimers` map
786
+
787
+ Multiple sessions can have simultaneous approval alarms. Each has an independent timer stored in `approvalTimers: Map<string, intervalId>`.
788
+
789
+ ### 24.2 Input Notification (one-shot)
790
+
791
+ When a session enters `input` status:
792
+ 1. Checks `inputFired` map — if this session hasn't fired yet, plays `soundEngine.play('inputNeeded')`
793
+ 2. Sets `inputFired.set('input-' + sessionId, true)` to prevent repeat
794
+ 3. When session leaves `input` status, clears the fired flag so it can fire again next time
795
+
796
+ ### 24.3 Mute Per Session
797
+
798
+ Sessions can be individually muted:
799
+ - `muteSession(sessionId)` — adds to `mutedSessions` set
800
+ - `unmuteSession(sessionId)` — removes from set
801
+ - All alarm checks and sound plays respect `mutedSessions.has(sessionId)`
802
+
803
+ ### 24.4 Label Completion Alerts
804
+
805
+ When a labeled session transitions to `ended`, `handleLabelAlerts(session, labelSettings)` is called:
806
+ - Looks up `labelSettings[session.label.toUpperCase()]`
807
+ - If a `sound` is configured, plays it via `soundEngine.preview(sound)`
808
+ - Respects mute state
809
+
810
+ Default label alarm configurations:
811
+ - `ONEOFF`: sound `alarm`, movement `shake`, frame effect `none`
812
+ - `HEAVY`: sound `urgentAlarm`, movement `flash`, frame effect `electric`
813
+ - `IMPORTANT`: sound `fanfare`, movement `bounce`, frame effect `liquid`
814
+
815
+ ### 24.5 Event-Based Sounds
816
+
817
+ `handleEventSounds(session)` processes the last event in `session.events` and maps it to a sound action:
818
+
819
+ | Event type | Action |
820
+ |------------|--------|
821
+ | `SessionStart` | sessionStart |
822
+ | `UserPromptSubmit` | promptSubmit |
823
+ | `PreToolUse` | toolRead/Write/Edit/Bash/Grep/Glob/WebFetch/Task/Other (by tool_name) |
824
+ | `Stop` | taskComplete |
825
+ | `SessionEnd` | sessionEnd |
826
+ | `SubagentStart` | subagentStart |
827
+ | `SubagentStop` | subagentStop |
828
+
829
+ Tool name → action mapping:
830
+ - Read → toolRead, Write → toolWrite, Edit → toolEdit, Bash → toolBash, Grep → toolGrep, Glob → toolGlob, WebFetch → toolWebFetch, Task → toolTask, all others → toolOther
831
+
832
+ ---
833
+
834
+ ## 25. Settings
835
+
836
+ ### 25.1 Settings Store
837
+
838
+ `settingsStore` (Zustand) manages all user preferences with automatic `IndexedDB` persistence via `db.settings.put()`. Each setter calls `persistSetting(key, value)` which writes to the database and triggers a 2-second "autosave" flash indicator.
839
+
840
+ ### 25.2 Complete Settings Reference
841
+
842
+ **Appearance:**
843
+
844
+ | Setting | Default | Type | Effect |
845
+ |---------|---------|------|--------|
846
+ | `themeName` | `'command-center'` | ThemeName | Sets `data-theme` on `document.body`; changes 3D scene colors |
847
+ | `fontSize` | `13` | number (px) | Sets `document.documentElement.style.fontSize` |
848
+ | `scanlineEnabled` | `true` | boolean | Toggles `no-scanlines` class on body |
849
+ | `animationIntensity` | `100` | number (0-200) | Sets `--anim-intensity` CSS variable (intensity/100) |
850
+ | `animationSpeed` | `100` | number (0-200) | Sets `--anim-speed` CSS variable (speed/100) |
851
+ | `characterModel` | `'robot'` | RobotModelType | Global default robot model for all sessions |
852
+
853
+ **Sound:**
854
+
855
+ | Setting | Default | Type |
856
+ |---------|---------|------|
857
+ | `soundSettings.enabled` | `true` | boolean |
858
+ | `soundSettings.volume` | `0.5` | 0-1 |
859
+ | `soundSettings.muteApproval` | `false` | boolean |
860
+ | `soundSettings.muteInput` | `false` | boolean |
861
+ | `soundSettings.perCli.claude` | Full profile at 0.7 | CliSoundConfig |
862
+ | `soundSettings.perCli.gemini` | Full profile at 0.7 | CliSoundConfig |
863
+ | `soundSettings.perCli.codex` | Full profile at 0.5 | CliSoundConfig |
864
+ | `soundSettings.perCli.openclaw` | Full profile at 0.7 | CliSoundConfig |
865
+
866
+ **Ambient:**
867
+
868
+ | Setting | Default | Type |
869
+ |---------|---------|------|
870
+ | `ambientSettings.enabled` | `false` | boolean |
871
+ | `ambientSettings.volume` | `0.3` | 0-1 |
872
+ | `ambientSettings.preset` | `'off'` | AmbientPreset |
873
+ | `ambientSettings.roomSounds` | `false` | boolean |
874
+ | `ambientSettings.roomVolume` | `0.2` | 0-1 |
875
+
876
+ **UI/UX:**
877
+
878
+ | Setting | Default | Type |
879
+ |---------|---------|------|
880
+ | `hookDensity` | `'medium'` | 'high'/'medium'/'low'/'off' |
881
+ | `activityFeedVisible` | `true` | boolean |
882
+ | `toastEnabled` | `true` | boolean |
883
+ | `autoSendQueue` | `false` | boolean |
884
+ | `defaultTerminalTheme` | `'auto'` | string |
885
+ | `compactMode` | `false` | boolean |
886
+ | `showArchived` | `false` | boolean |
887
+ | `groupBy` | `'none'` | BrowserSettings['groupBy'] |
888
+ | `sortBy` | `'activity'` | BrowserSettings['sortBy'] |
889
+
890
+ **Label settings** (per label): `{ sound, movement, frame }` — all configurable.
891
+
892
+ **API Keys** (persisted): `anthropicApiKey`, `openaiApiKey`, `geminiApiKey`
893
+
894
+ ### 25.3 Theme System (9 themes)
895
+
896
+ | Name | Label | Preview colors |
897
+ |------|-------|---------------|
898
+ | command-center | Command Center | navy, cyan, orange |
899
+ | cyberpunk | Cyberpunk | deep purple, magenta, cyan |
900
+ | warm | Warm | cream, amber, copper |
901
+ | dracula | Dracula | dark, purple, green |
902
+ | solarized | Solarized | dark teal, teal, orange |
903
+ | nord | Nord | slate, sky blue, peach |
904
+ | monokai | Monokai | dark, cyan, pink |
905
+ | light | Light | light gray, blue, sky |
906
+ | blonde | Blonde | warm white, gold, dark gold |
907
+
908
+ ### 25.4 Label Frame Effects
909
+
910
+ | Effect key | Display name |
911
+ |-----------|-------------|
912
+ | `none` | None |
913
+ | `fire` | Burning Fire |
914
+ | `electric` | Electric Surge |
915
+ | `chains` | Golden Aura |
916
+ | `liquid` | Liquid Energy |
917
+ | `plasma` | Plasma Overload |
918
+
919
+ ### 25.5 Settings Panel UI
920
+
921
+ The settings panel has 6 tabs (legacy CSS frontend):
922
+ 1. **Appearance** — theme picker, font size, scanlines, animation speed/intensity, character model
923
+ 2. **Sound** — master volume, enable/disable, per-action sound dropdowns, per-CLI profiles
924
+ 3. **Ambient** — preset picker, volume, room sounds toggle
925
+ 4. **Labels** — per-label sound/movement/frame configuration
926
+ 5. **Hooks** — hook density selector, install/uninstall buttons
927
+ 6. **API Keys** — Anthropic, OpenAI, Gemini key inputs
928
+
929
+ The React frontend (settings route) mirrors this structure using `settingsStore` actions directly.
930
+
931
+ **Import/Export**: Settings can be exported as JSON and imported to restore a configuration.
932
+
933
+ **Reset Defaults**: `resetDefaults()` restores all settings to defaults and persists them all.
934
+
935
+ ---
936
+
937
+ ## 26. Terminal Manager (Frontend)
938
+
939
+ ### 26.1 xterm.js Integration
940
+
941
+ The terminal manager uses xterm.js with addons for full terminal emulation in the browser:
942
+ - `@xterm/xterm` — core terminal
943
+ - `@xterm/addon-fit` — auto-resize to container
944
+ - `@xterm/addon-web-links` — clickable URL detection
945
+ - `@xterm/addon-search` — text search within terminal
946
+
947
+ ### 26.2 Terminal Themes (8 named + auto)
948
+
949
+ | Theme | Character |
950
+ |-------|-----------|
951
+ | `auto` | Matches the dashboard theme (default) |
952
+ | `dark` | Standard dark terminal |
953
+ | `light` | White background |
954
+ | `cyberpunk` | Magenta/cyan on deep purple |
955
+ | `dracula` | Dracula color scheme |
956
+ | `solarized` | Solarized dark |
957
+ | `nord` | Nord color scheme |
958
+ | `monokai` | Monokai colors |
959
+ | `warm` | Amber on cream |
960
+
961
+ Theme is applied on terminal creation and when the setting changes. `auto` resolves the theme from the current `themeName` in settings.
962
+
963
+ ### 26.3 Canvas Repaint Workaround
964
+
965
+ xterm.js uses canvas rendering. A known issue causes the canvas to appear blank when the terminal panel is first opened or when the panel is resized. The workaround:
966
+ 1. After terminal creation: `setTimeout(() => terminal.refresh(0, terminal.rows - 1), 50)` — forces a full repaint
967
+ 2. On panel resize: `fitAddon.fit()` followed by another `refresh` call
968
+ 3. On tab switch back to terminal: another `refresh` call
969
+
970
+ ### 26.4 Fullscreen Mode
971
+
972
+ The terminal tab in the detail panel has a fullscreen toggle button. When activated:
973
+ - The terminal container expands to cover the full viewport
974
+ - `fitAddon.fit()` is called after the transition
975
+ - Escape key or the toggle button exits fullscreen
976
+ - Panel resize handles are hidden in fullscreen
977
+
978
+ ### 26.5 Team Terminal
979
+
980
+ For team sessions (leader + members), a special "Team Terminal" view is available:
981
+ - Shows a split-view with all team member terminals
982
+ - Each terminal pane shows the member's session title and status
983
+ - Switching between member terminals uses the same WebSocket relay as individual terminals
984
+
985
+ ### 26.6 WebSocket Relay
986
+
987
+ Terminal I/O is relayed through the WebSocket connection using message types:
988
+ - `terminal_input`: keystrokes from browser → server → PTY
989
+ - `terminal_output`: PTY output → server → browser (base64-encoded)
990
+ - `terminal_resize`: `{cols, rows}` to resize the PTY
991
+
992
+ The `sshManager.js` on the server creates `node-pty` processes for SSH/local sessions.
993
+
994
+ ---
995
+
996
+ ## 33. Testing
997
+
998
+ ### 33.1 Test Framework
999
+
1000
+ The project uses two testing frameworks:
1001
+ - **Vitest** for unit and integration tests (TypeScript/browser-compatible)
1002
+ - **Playwright** for E2E tests (critical user flows)
1003
+
1004
+ ### 33.2 Test Suite Size
1005
+
1006
+ **407 tests passing across 24 test files** (as of Feb 2025).
1007
+
1008
+ ### 33.3 Test File Coverage
1009
+
1010
+ **Frontend/lib tests (Vitest):**
1011
+
1012
+ | File | What it tests |
1013
+ |------|--------------|
1014
+ | `src/lib/soundEngine.test.ts` | SoundEngine class — unlock, volume, action overrides, play/preview, dispose; ACTION_LABELS completeness; ACTION_CATEGORIES structure |
1015
+ | `src/lib/wsClient.test.ts` | WebSocket client — connection, reconnect, message dispatch |
1016
+ | `src/stores/sessionStore.test.ts` | Session CRUD, status transitions, team tracking |
1017
+ | `src/stores/settingsStore.test.ts` | Settings read/write, persistence, defaults |
1018
+ | `src/stores/roomStore.test.ts` | Room CRUD, session assignment/removal |
1019
+ | `src/stores/uiStore.test.ts` | UI state management |
1020
+ | `src/stores/queueStore.test.ts` | Prompt queue operations |
1021
+ | `src/stores/wsStore.test.ts` | WebSocket store state |
1022
+ | `src/hooks/useAuth.test.ts` | Auth hook behavior |
1023
+
1024
+ **Server tests (Vitest with Node environment):**
1025
+
1026
+ | File | What it tests |
1027
+ |------|--------------|
1028
+ | `test/hookProcessor.test.js` | Hook validation (null, non-object, missing fields, invalid types), processing SessionStart/Stop, latency calculation, alias fields |
1029
+ | `test/sessionStore.test.js` | Session creation, matching, status transitions, deduplication |
1030
+ | `test/sessionMatcher.test.js` | 5-priority session matching logic |
1031
+ | `test/mqReader.test.js` | File-based message queue reading, partial line handling, truncation |
1032
+ | `test/apiRouter.test.js` | REST API endpoints |
1033
+ | `test/hookInstaller.test.js` | Hook script installation/uninstall |
1034
+ | `test/portManager.test.js` | Port conflict resolution |
1035
+ | `test/teamManager.test.js` | Team/subagent tracking |
1036
+ | `test/approvalDetector.test.js` | Approval heuristic timeouts |
1037
+ | `test/autoIdleManager.test.js` | Auto-idle timer logic |
1038
+ | `test/processMonitor.test.js` | PID liveness checking |
1039
+ | `test/wsManager.test.js` | WebSocket broadcast, ring buffer |
1040
+ | `test/config.test.js` | Config loading and defaults |
1041
+ | `test/constants.test.js` | Constants completeness |
1042
+ | `test/hookStats.test.js` | Performance stats tracking |
1043
+ | `test/serverConfig.test.js` | Server config file loading |
1044
+
1045
+ ### 33.4 Sound Engine Test Details
1046
+
1047
+ The `soundEngine.test.ts` file demonstrates the testing pattern for Web Audio API code:
1048
+
1049
+ - `AudioContext` is mocked using a factory function (must use `function` keyword, not arrow, so `new AudioContext()` works)
1050
+ - Mock provides: `createOscillator`, `createGain`, `resume`, `close`, with `vi.fn()` spies
1051
+ - `vi.useFakeTimers()` controls `setTimeout` calls used by `playSequence`
1052
+ - Tests verify: unlock/lock cycle, volume clamping (0-1), action override/restore, play returns true/false, sequence fires correct number of oscillators after `vi.runAllTimers()`, preview bypasses unlock check, dispose closes context
1053
+
1054
+ ### 33.5 Running Tests
1055
+
1056
+ ```bash
1057
+ # Run all tests
1058
+ npm test
1059
+
1060
+ # Watch mode
1061
+ npm run test:watch
1062
+
1063
+ # With verbose reporter
1064
+ npm test -- --reporter=verbose
1065
+
1066
+ # Coverage (if configured)
1067
+ npm test -- --coverage
1068
+ ```
1069
+
1070
+ ### 33.6 E2E Tests (Playwright)
1071
+
1072
+ Playwright E2E tests cover critical user flows:
1073
+ - Dashboard load and session display
1074
+ - Session card selection and detail panel
1075
+ - Terminal session creation and interaction
1076
+ - Settings persistence across reload
1077
+ - Theme switching
1078
+ - Keyboard shortcuts
1079
+
1080
+ E2E tests are located in the `test/e2e/` directory and configured in `playwright.config.ts`.