ai-agent-session-center 2.0.2 → 2.0.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +484 -429
- package/docs/3D/ADAPTATION_GUIDE.md +592 -0
- package/docs/3D/index.html +754 -0
- package/docs/AGENT_TEAM_TASKS.md +716 -0
- package/docs/CYBERDROME_V2_SPEC.md +531 -0
- package/docs/ERROR_185_ANALYSIS.md +263 -0
- package/docs/PLATFORM_FEATURES_PROMPT.md +296 -0
- package/docs/SESSION_DETAIL_FEATURES.md +98 -0
- package/docs/_3d_multimedia_features.md +1080 -0
- package/docs/_frontend_features.md +1057 -0
- package/docs/_server_features.md +1077 -0
- package/docs/session-duplication-fixes.md +271 -0
- package/docs/session-terminal-linkage.md +412 -0
- package/package.json +63 -5
- package/public/apple-touch-icon.svg +21 -0
- package/public/css/dashboard.css +0 -161
- package/public/css/detail-panel.css +25 -0
- package/public/css/layout.css +18 -1
- package/public/css/modals.css +0 -26
- package/public/css/settings.css +0 -150
- package/public/css/terminal.css +34 -0
- package/public/favicon.svg +18 -0
- package/public/index.html +6 -26
- package/public/js/alarmManager.js +0 -21
- package/public/js/app.js +21 -7
- package/public/js/detailPanel.js +63 -64
- package/public/js/historyPanel.js +61 -55
- package/public/js/quickActions.js +132 -48
- package/public/js/sessionCard.js +5 -20
- package/public/js/sessionControls.js +8 -0
- package/public/js/settingsManager.js +0 -142
- package/server/apiRouter.js +60 -15
- package/server/apiRouter.ts +774 -0
- package/server/approvalDetector.ts +94 -0
- package/server/authManager.ts +144 -0
- package/server/autoIdleManager.ts +110 -0
- package/server/config.ts +121 -0
- package/server/constants.ts +150 -0
- package/server/db.ts +475 -0
- package/server/hookInstaller.d.ts +3 -0
- package/server/hookProcessor.ts +108 -0
- package/server/hookRouter.ts +18 -0
- package/server/hookStats.ts +116 -0
- package/server/index.js +15 -1
- package/server/index.ts +230 -0
- package/server/logger.ts +75 -0
- package/server/mqReader.ts +349 -0
- package/server/portManager.ts +55 -0
- package/server/processMonitor.ts +239 -0
- package/server/serverConfig.ts +29 -0
- package/server/sessionMatcher.js +17 -6
- package/server/sessionMatcher.ts +403 -0
- package/server/sessionStore.js +109 -3
- package/server/sessionStore.ts +1145 -0
- package/server/sshManager.js +167 -24
- package/server/sshManager.ts +671 -0
- package/server/teamManager.ts +289 -0
- package/server/wsManager.ts +200 -0
|
@@ -0,0 +1,1080 @@
|
|
|
1
|
+
# 3D Scene + Multimedia Features
|
|
2
|
+
|
|
3
|
+
---
|
|
4
|
+
|
|
5
|
+
## 15. 3D Cyberdrome Scene
|
|
6
|
+
|
|
7
|
+
The Cyberdrome is a fully interactive 3D office environment rendered with React Three Fiber (R3F) and Three.js. Each active session is represented by an animated 3D robot character that navigates the scene, sits at desks, and reacts to session state changes in real time.
|
|
8
|
+
|
|
9
|
+
### 15.1 Architecture
|
|
10
|
+
|
|
11
|
+
**Zero Zustand inside Canvas**
|
|
12
|
+
|
|
13
|
+
The core architectural decision is that all Zustand store subscriptions live in the DOM layer (`CyberdromeScene` component), never inside the `<Canvas>`. This prevents cross-reconciler state cascades that cause React Error #185 (illegal store update during render).
|
|
14
|
+
|
|
15
|
+
All data flows into the Canvas via props:
|
|
16
|
+
- Sessions are extracted from `sessionStore` as a plain array before the Canvas.
|
|
17
|
+
- Room configs, workstations, wall collision rects, and casual areas are computed in the DOM layer with `useMemo`.
|
|
18
|
+
- Room assignments per session are precomputed as a `Map<string, number | undefined>` to eliminate `useRoomStore` subscriptions inside Canvas.
|
|
19
|
+
- Subagent connection data is computed as a plain `ConnectionData[]` array.
|
|
20
|
+
- Scene theme colors and fog density are resolved outside Canvas and passed as props.
|
|
21
|
+
|
|
22
|
+
**CustomEvent pattern for click handling**
|
|
23
|
+
|
|
24
|
+
When a user clicks a robot inside R3F, the click handler dispatches a `CustomEvent('robot-select')` with a `setTimeout(0)` delay. This ensures the store update (`selectSession`) fires after R3F's pointer event cycle completes, fully decoupling the React reconciler used by R3F from the Zustand store. The DOM-side `useEffect` listener in `CyberdromeScene` catches the event and calls `selectSession` + `flyTo`.
|
|
25
|
+
|
|
26
|
+
**Ref-based animation (no useState in render loop)**
|
|
27
|
+
|
|
28
|
+
All per-robot animation state is stored in refs:
|
|
29
|
+
- `nav` ref (NavState) holds all movement state — position, rotation, speed, waypoints, mode, desk index.
|
|
30
|
+
- `seatedRef` tracks seated state for animation, updated in `useFrame`, never triggers re-render.
|
|
31
|
+
- `dialogueRef` holds the current dialogue message, updated in `useEffect`, read in `useFrame`.
|
|
32
|
+
- Direct `groupRef.current.position.set()` calls in `useFrame` move the robot group without React prop updates.
|
|
33
|
+
|
|
34
|
+
**Memoized SessionRobot**
|
|
35
|
+
|
|
36
|
+
`SessionRobot` is wrapped in `React.memo` with a custom equality check comparing 16 specific session fields. This prevents re-renders when unrelated session data changes, which would cascade Html portal updates for every robot.
|
|
37
|
+
|
|
38
|
+
**Position persistence**
|
|
39
|
+
|
|
40
|
+
Robot world positions and nav state are saved to `sessionStorage` every 2 seconds via `saveRobotPositions`. On page reload, `loadRobotPositions` restores each robot's position, rotation, navigation mode, and desk index. `NAV_GOTO` mode is reset to `NAV_WALK` on restore since it's a transient state.
|
|
41
|
+
|
|
42
|
+
### 15.2 Canvas Setup
|
|
43
|
+
|
|
44
|
+
The Three.js canvas is configured with:
|
|
45
|
+
|
|
46
|
+
| Setting | Value |
|
|
47
|
+
|---------|-------|
|
|
48
|
+
| Camera position | `[18, 16, 18]` (isometric-style view) |
|
|
49
|
+
| Camera FOV | 50 degrees |
|
|
50
|
+
| Near clip | 0.1 |
|
|
51
|
+
| Far clip | 150 |
|
|
52
|
+
| Shadow type | `PCFSoftShadowMap` |
|
|
53
|
+
| Tone mapping | `ACESFilmicToneMapping` |
|
|
54
|
+
| Tone mapping exposure | 1.2 |
|
|
55
|
+
| Antialiasing | Enabled |
|
|
56
|
+
| Fog | `FogExp2` (density varies by theme, default 0.008) |
|
|
57
|
+
|
|
58
|
+
**OrbitControls** are configured with:
|
|
59
|
+
- Damping factor: 0.06 (smooth momentum)
|
|
60
|
+
- Max polar angle: `PI / 2.1` (cannot orbit below floor)
|
|
61
|
+
- Min distance: 6 (close zoom)
|
|
62
|
+
- Max distance: 80 (wide overview)
|
|
63
|
+
- Initial target: `[0, 1, 0]`
|
|
64
|
+
|
|
65
|
+
**`SceneThemeSync`** is an internal Canvas component that uses `useThree` to directly update `scene.fog` color/density and `gl.clearColor` when the theme changes — no re-render, pure imperative update.
|
|
66
|
+
|
|
67
|
+
### 15.3 Map Controls Overlay
|
|
68
|
+
|
|
69
|
+
A DOM overlay positioned `bottom: 20, left: 20` provides three navigation buttons:
|
|
70
|
+
|
|
71
|
+
| Button | Action | Implementation |
|
|
72
|
+
|--------|--------|----------------|
|
|
73
|
+
| + (zoom in) | Zoom factor 0.65 | Scales camera offset from target by 0.65 |
|
|
74
|
+
| - (zoom out) | Zoom factor 1.5 | Scales camera offset from target by 1.5 |
|
|
75
|
+
| Top-down view | Bird's-eye | Flies to `[t.x + 0.01, t.y + 30, t.z + 0.01]` targeting current look-at |
|
|
76
|
+
| Reset view | Default position | Flies to `[18, 16, 18]` targeting `[0, 1, 0]` |
|
|
77
|
+
|
|
78
|
+
Buttons use inline style hover effects: `background: rgba(0,240,255,0.15)` on hover, `backdrop-filter: blur(8px)`, monospace font, 34×34px size.
|
|
79
|
+
|
|
80
|
+
### 15.4 Dynamic Room System
|
|
81
|
+
|
|
82
|
+
Rooms are created and destroyed dynamically based on `roomStore.rooms`. The layout engine in `src/lib/cyberdromeScene.ts` computes geometry from room indices.
|
|
83
|
+
|
|
84
|
+
**Layout constants:**
|
|
85
|
+
|
|
86
|
+
| Constant | Value | Description |
|
|
87
|
+
|----------|-------|-------------|
|
|
88
|
+
| `ROOM_SIZE` | 12 | Internal room dimension (12×12 units) |
|
|
89
|
+
| `ROOM_GAP` | 5 | Corridor width between rooms |
|
|
90
|
+
| `ROOM_CELL` | 17 | Grid cell size (ROOM_SIZE + ROOM_GAP) |
|
|
91
|
+
| `ROOM_HALF` | 6 | Half of room size |
|
|
92
|
+
| `ROOM_COLS` | 4 | Maximum rooms per row before wrapping |
|
|
93
|
+
| `WALL_H` | 2.8 | Wall height |
|
|
94
|
+
| `WALL_T` | 0.12 | Wall thickness |
|
|
95
|
+
| `DOOR_GAP` | 4 | Doorway width (centered on each wall) |
|
|
96
|
+
|
|
97
|
+
**Room grid positioning:**
|
|
98
|
+
|
|
99
|
+
Rooms are placed in a grid where column is `roomIndex % 4` and row is `floor(roomIndex / 4)`. Columns are centered around X=0. Center of room at index `i`:
|
|
100
|
+
```
|
|
101
|
+
col = i % 4
|
|
102
|
+
row = floor(i / 4)
|
|
103
|
+
x = (col - 1.5) * 17
|
|
104
|
+
z = row * 17
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
**`RoomConfig` interface** (computed per room):
|
|
108
|
+
- `index: number` — grid index
|
|
109
|
+
- `roomId: string` — store room ID
|
|
110
|
+
- `name: string` — display name
|
|
111
|
+
- `center: [number, number, number]` — world center
|
|
112
|
+
- `bounds: RoomBound` — min/max X/Z extents
|
|
113
|
+
- `stripColor: 0 | 1` — alternates cyan/magenta strips per even/odd index
|
|
114
|
+
|
|
115
|
+
### 15.5 Room Walls
|
|
116
|
+
|
|
117
|
+
Each room has 4 walls rendered in `RoomWalls`. North and south walls (along Z edges) each have a 4-unit door gap in the center, splitting into two segments per wall. East and west walls are solid. Each wall has a glowing strip at the top edge.
|
|
118
|
+
|
|
119
|
+
**Wall geometry:**
|
|
120
|
+
- Wall segment: `BoxGeometry(length, WALL_H, WALL_T)` — metallic, semi-transparent
|
|
121
|
+
- Top strip: `BoxGeometry(len, 0.04, WALL_T + 0.06)` — emissive (2.0 intensity), alternates theme `stripPrimary` or `stripSecondary`
|
|
122
|
+
- Wall material: `roughness: 0.2`, `metalness: 0.7`, `transparent: true`, opacity from theme
|
|
123
|
+
- Doors only on north and south walls; east and west walls are solid
|
|
124
|
+
|
|
125
|
+
**Collision rects** for wall physics are built by `buildDynamicWallRects`: each wall segment produces a `WallRect` with 0.25-unit half-thickness on the thin axis.
|
|
126
|
+
|
|
127
|
+
### 15.6 Room Desks Layout (8 desks per room)
|
|
128
|
+
|
|
129
|
+
Each room has exactly 8 desks placed at fixed positions relative to the room center `[cx, cz]`:
|
|
130
|
+
|
|
131
|
+
| Position | Facing | Notes |
|
|
132
|
+
|----------|--------|-------|
|
|
133
|
+
| `[cx-3.5, cz-4.5]` | South (rot=0) | North wall, left of door |
|
|
134
|
+
| `[cx+3.5, cz-4.5]` | South (rot=0) | North wall, right of door |
|
|
135
|
+
| `[cx+3.5, cz+4.5]` | North (rot=PI) | South wall, offset from door |
|
|
136
|
+
| `[cx-5, cz-1.5]` | East (rot=PI/2) | West wall |
|
|
137
|
+
| `[cx-5, cz+1.5]` | East (rot=PI/2) | West wall |
|
|
138
|
+
| `[cx+5, cz-1.5]` | West (rot=-PI/2) | East wall |
|
|
139
|
+
| `[cx+5, cz+1.5]` | West (rot=-PI/2) | East wall |
|
|
140
|
+
| `[cx+5, cz-3.5]` | West (rot=-PI/2) | East wall, additional |
|
|
141
|
+
|
|
142
|
+
Each desk consists of:
|
|
143
|
+
- **Tabletop**: `BoxGeometry(1.5, 0.05, 0.65)` at Y=0.7
|
|
144
|
+
- **Two side legs**: `BoxGeometry(0.04, 0.66, 0.58)` at X ± 0.72
|
|
145
|
+
- **Monitor frame**: `BoxGeometry(0.48, 0.32, 0.025)` at Y=0.92
|
|
146
|
+
- **Monitor screen**: `BoxGeometry(0.44, 0.28, 0.005)` with emissive color from `PALETTE`
|
|
147
|
+
- **Keyboard**: `BoxGeometry(0.32, 0.012, 0.1)` at Y=0.72
|
|
148
|
+
- **Chair seat**: `BoxGeometry(0.36, 0.03, 0.36)` at Y=0.4
|
|
149
|
+
- **Chair back**: `BoxGeometry(0.34, 0.28, 0.03)` at Y=0.57
|
|
150
|
+
- **Chair stem**: `CylinderGeometry(0.025, 0.025, 0.36, 6)`
|
|
151
|
+
- **Chair base**: `CylinderGeometry(0.16, 0.16, 0.025, 6)`
|
|
152
|
+
|
|
153
|
+
The chair is positioned 0.65 units toward the robot's facing direction from the desk center. Monitor screen color cycles through `PALETTE` using `(deskOffset + di) * 3 + 1`.
|
|
154
|
+
|
|
155
|
+
### 15.7 Corridor Workstations
|
|
156
|
+
|
|
157
|
+
For robots not assigned to any room (`zone === -1`), a dedicated "common area" with 10 workstations is placed south of all rooms. The layout is a 2-row × 5-column grid:
|
|
158
|
+
|
|
159
|
+
- **No rooms**: Centered at origin, rows at Z=-3 and Z=+3, spacing X=3.5
|
|
160
|
+
- **With rooms**: Placed at `southmostRoomCenter.z + ROOM_HALF + ROOM_GAP + 5`, horizontally centered across the room span. Spacing X=4, Z=4 between rows.
|
|
161
|
+
|
|
162
|
+
Row 0 faces south (rot=0), row 1 faces north (rot=PI). These workstations share the same desk geometry as room desks.
|
|
163
|
+
|
|
164
|
+
### 15.8 Casual Areas: Coffee Lounge and Gym
|
|
165
|
+
|
|
166
|
+
Two casual areas are built north of the room grid (most negative Z side), separated by a 3-unit gap. Each area is 14×14 units (`CASUAL_AREA_SIZE = 14`).
|
|
167
|
+
|
|
168
|
+
**Placement:**
|
|
169
|
+
- If rooms exist: `baseZ = min room edge - ROOM_GAP - 7 - 2` (7 = CASUAL_HALF)
|
|
170
|
+
- Area centers: `coffeeX = centerX - 10` (CASUAL_HALF + gap/2), `gymX = centerX + 10`
|
|
171
|
+
|
|
172
|
+
**Coffee Lounge** (`zone === -2`):
|
|
173
|
+
- 14×14 floor pad at Y=0.004 with `theme.coffeeFloor` material
|
|
174
|
+
- AreaBorderGlow (14-unit) in `theme.coffeeAccent` color
|
|
175
|
+
- 6 coffee tables in a 2×3 grid (rows at Z±2.5, cols at X-3/0/+3 relative to area center)
|
|
176
|
+
- Each table: round cylinder top (`CylinderGeometry(0.5, 0.5, 0.04, 12)` at Y=0.5), stem, base
|
|
177
|
+
- Two stools per table at X±0.6
|
|
178
|
+
- Counter bar along north edge: `BoxGeometry(8, 1.1, 0.5)` at Y=0.55, with accent top strip
|
|
179
|
+
- Coffee machine: box body + screen panel + nozzle cylinder + cup platform
|
|
180
|
+
- Coffee pot: tapered cylinder + torus handle
|
|
181
|
+
- 3 coffee cups on tables
|
|
182
|
+
- Warm point light: `color: theme.coffeeAccent`, intensity 6, distance 14
|
|
183
|
+
|
|
184
|
+
**Gym Area** (`zone === -3`) — 10 equipment types:
|
|
185
|
+
|
|
186
|
+
| Equipment | Position (relative) | Details |
|
|
187
|
+
|-----------|---------------------|---------|
|
|
188
|
+
| Bench press | `[-5, -5]` | Bench box + legs + barbell cylinder + weight discs (torusGeometry) |
|
|
189
|
+
| Treadmill | `[0, -5]` | Angled running platform + handlebar uprights + console |
|
|
190
|
+
| Rowing machine | `[+5, -5]` | Horizontal rail + sliding seat + foot pads + handle |
|
|
191
|
+
| Stationary bike | `[-5, 0]` | Frame box + seat + handlebar post + wheel cylinder |
|
|
192
|
+
| Pull-up bar | `[0, 0]` | Two uprights (height 2.4) + crossbar cylinder + diagonal braces |
|
|
193
|
+
| Leg press | `[+5, 0]` | Angled platform + seat + back rest + guide rails |
|
|
194
|
+
| Punching bag | `[-5, +5]` | Ceiling mount + vertical support + cylindrical bag |
|
|
195
|
+
| Cable machine | `[0, +5]` | Frame + top crossbar + pulley wheel + vertical cable + weight stack |
|
|
196
|
+
| Kettlebell rack | `[+5, +5]` | Shelf frame + 3 kettlebells (sphere + torus handle) |
|
|
197
|
+
| Dumbbell rack | `[-2.5, +2.5]` | Rack frame + 2 shelf levels + 5 dumbbells (sphereGeometry) |
|
|
198
|
+
| Medicine ball | `[+2.5, +2.5]` | `SphereGeometry(0.13, 8, 8)` in accent color |
|
|
199
|
+
|
|
200
|
+
Cool point light: `color: theme.gymAccent`, intensity 8, distance 18.
|
|
201
|
+
|
|
202
|
+
**Casual workstation zones**: Coffee lounge stations use `zone === -2`, gym stations use `zone === -3`. Both are part of the `workstations` array. Idle robots seek coffee (zone -2), waiting robots seek gym (zone -3).
|
|
203
|
+
|
|
204
|
+
### 15.9 Floor and Environment
|
|
205
|
+
|
|
206
|
+
**DynamicFloor:**
|
|
207
|
+
- Main floor: `PlaneGeometry(floorSize, floorSize)` at Y=0, `roughness: 0.7, metalness: 0.3`
|
|
208
|
+
- Per-room floor panels: `PlaneGeometry(12, 12)` at Y=0.003, slightly brighter `theme.roomFloor`
|
|
209
|
+
- Room border glow: 4 thin planes (0.06-unit wide) forming a square outline at Y=0.015, emissive intensity 1.5, opacity 0.35
|
|
210
|
+
- Grid overlay 1: `GridHelper(floorSize, floorSize/1)` at Y=0.005, theme `grid1` color, opacity 0.04
|
|
211
|
+
- Grid overlay 2: `GridHelper(floorSize, floorSize/5)` at Y=0.008, theme `grid2` color, opacity 0.03
|
|
212
|
+
- Floor size: `max(30, sceneBounds * 2 + 10)`
|
|
213
|
+
|
|
214
|
+
**Circuit traces**: 14 random L-shaped polylines on the floor at Y=0.011. Each has 4-7 axis-aligned segments of random length 1-4 units. Colors cycle through `[theme.particle1, theme.particle2, theme.trace3]`. Opacity 0.08-0.16. Built with raw `THREE.BufferGeometry` + `lineBasicMaterial`.
|
|
215
|
+
|
|
216
|
+
**Data particles** (animated):
|
|
217
|
+
- 140 cyan particles (`theme.particle1`) rising at speed 0.2-0.8 units/sec
|
|
218
|
+
- 80 magenta particles (`theme.particle2`) at the same speed
|
|
219
|
+
- Size: 0.04, opacity: 0.4, additive blending
|
|
220
|
+
- When a particle reaches Y=10, it resets to Y=0 at a random X/Z position
|
|
221
|
+
- `Float32Array` buffers updated every frame via `points.geometry.attributes.position.needsUpdate = true`
|
|
222
|
+
|
|
223
|
+
**Stars**: 400 random points at Y=8-43, X/Z ±50, size 0.05, opacity 0.4, `theme.stars` color.
|
|
224
|
+
|
|
225
|
+
### 15.10 Room Lighting
|
|
226
|
+
|
|
227
|
+
Each room has dedicated interior lighting:
|
|
228
|
+
|
|
229
|
+
**Visual sconces**: 9 emissive boxes per room (3 along north wall, 2 on each side wall, 2 along south wall):
|
|
230
|
+
- Bracket: `BoxGeometry(0.8, 0.12, 0.05)` + mount `BoxGeometry(0.1, 0.08, 0.22)` in dark metallic
|
|
231
|
+
- Light tube: `BoxGeometry(1.2, 0.25, 0.12)` emissive in `theme.sconceColor`, intensity 2.5
|
|
232
|
+
|
|
233
|
+
**Point lights** (2 per room, GPU-friendly):
|
|
234
|
+
- Primary: `color: theme.roomLight1`, intensity 10, distance 16, decay 1.5, at ceiling level
|
|
235
|
+
- Secondary: `color: theme.roomLight2`, intensity 4, distance 12, decay 2, at mid-height
|
|
236
|
+
|
|
237
|
+
**Global lighting** (scene-wide):
|
|
238
|
+
- Ambient: `theme.ambientColor`, intensity varies by theme (4-10)
|
|
239
|
+
- Directional (shadows): `position: [8, 20, 6]`, shadow map 2048×2048, bias -0.0004
|
|
240
|
+
- Fill directional: `position: [-6, 15, -8]`
|
|
241
|
+
- 3 point lights at corners: `[-10, 8, -10]`, `[10, 7, 10]`, `[0, 10, 0]`
|
|
242
|
+
- Hemisphere: sky color, ground color, intensity from theme
|
|
243
|
+
|
|
244
|
+
### 15.11 Robot Navigation AI
|
|
245
|
+
|
|
246
|
+
Each robot has 4 navigation modes (stored in `NavState.mode`):
|
|
247
|
+
|
|
248
|
+
| Constant | Value | Description |
|
|
249
|
+
|----------|-------|-------------|
|
|
250
|
+
| `NAV_WALK` | 0 | Wandering to random target |
|
|
251
|
+
| `NAV_GOTO` | 1 | Navigating to a specific desk via waypoints |
|
|
252
|
+
| `NAV_SIT` | 2 | Seated at a desk, not moving |
|
|
253
|
+
| `NAV_IDLE` | 3 | Frozen in place (alert/input/offline/connecting) |
|
|
254
|
+
|
|
255
|
+
**Navigation state (`NavState`):**
|
|
256
|
+
- `mode` — current nav mode
|
|
257
|
+
- `target` — current waypoint target (Vector3)
|
|
258
|
+
- `deskIdx` — occupied workstation index (-1 if none)
|
|
259
|
+
- `speed` — base speed: 3.0 + random(0, 1.5) units/sec
|
|
260
|
+
- `walkHz` — walk bounce frequency: 12 + random(0, 4) Hz
|
|
261
|
+
- `phase` — random phase offset for animation desync
|
|
262
|
+
- `decisionTimer` — countdown to desk-seek attempt
|
|
263
|
+
- `posX, posY, posZ` — current world position
|
|
264
|
+
- `rotY` — current Y rotation
|
|
265
|
+
- `waypoints` — ordered array of intermediate points
|
|
266
|
+
- `waypointIdx` — current waypoint index
|
|
267
|
+
|
|
268
|
+
**Status-to-navigation mapping:**
|
|
269
|
+
|
|
270
|
+
| Status | Robot State | Navigation behavior |
|
|
271
|
+
|--------|-------------|---------------------|
|
|
272
|
+
| idle | idle | Seek coffee workstation (zone -2); wander if full |
|
|
273
|
+
| prompting | thinking | Seek desk in assigned room or corridor; sit when arrived |
|
|
274
|
+
| working | working | Seek desk; speed multiplier 1.5 |
|
|
275
|
+
| waiting | waiting | Seek gym workstation (zone -3); wander if full |
|
|
276
|
+
| approval | alert | Freeze (NAV_IDLE or stay seated) |
|
|
277
|
+
| input | input | Freeze (NAV_IDLE or stay seated) |
|
|
278
|
+
| ended | offline | Freeze |
|
|
279
|
+
| connecting | connecting | Freeze |
|
|
280
|
+
|
|
281
|
+
**Desk seeking logic:**
|
|
282
|
+
|
|
283
|
+
When entering a desk-seeking state (`thinking` or `working`):
|
|
284
|
+
1. Determine zone: room assignment → current position zone → fallback
|
|
285
|
+
2. Find empty workstations in that zone
|
|
286
|
+
3. Claim one randomly, set `ws.occupantId = sessionId`, go `NAV_GOTO`
|
|
287
|
+
4. If all desks full: find nearest occupied desk, stand 0.5 units behind it (overflow)
|
|
288
|
+
5. During `NAV_WALK`, robot periodically checks for empty desks (every 3-8 seconds via `decisionTimer`)
|
|
289
|
+
|
|
290
|
+
**Wall collision** (`collidesAnyWall`):
|
|
291
|
+
- Checks 0.25-unit robot radius against all `WallRect` entries
|
|
292
|
+
- On X-axis collision only: slide along Z, pick new wander target
|
|
293
|
+
- On Z-axis collision only: slide along X, pick new wander target
|
|
294
|
+
- On full collision: pick new wander target
|
|
295
|
+
- Position clamped to `[-sceneBound, sceneBound]` on both axes
|
|
296
|
+
|
|
297
|
+
**Walk bounce:** `posY = abs(sin(time * walkHz + phase)) * 0.03` — small up-down bounce while moving.
|
|
298
|
+
|
|
299
|
+
**Facing:** `rotY` lerps toward `atan2(dx, dz)` at rate `min(1, 10 * dt)` per frame.
|
|
300
|
+
|
|
301
|
+
**Seated position:** When `NAV_SIT` is reached, robot snaps to `ws.seatPos` with `posY = -0.12` (slightly sunken) and faces `ws.faceRot`.
|
|
302
|
+
|
|
303
|
+
### 15.12 Cross-Room Pathfinding
|
|
304
|
+
|
|
305
|
+
When a robot needs to navigate between zones (room → corridor → room), it uses a door-waypoint system.
|
|
306
|
+
|
|
307
|
+
**Door waypoints:** Built by `buildDoorWaypoints`. Each room gets 2 waypoints per wall (north and south):
|
|
308
|
+
- `outside`: 1 unit past the wall exterior (in the corridor)
|
|
309
|
+
- `inside`: 1 unit past the wall interior (inside the room)
|
|
310
|
+
|
|
311
|
+
**`computePathWaypoints(fromX, fromZ, target, fromZone, targetZone, doors)`:**
|
|
312
|
+
1. Same zone → direct path (single waypoint = target)
|
|
313
|
+
2. Both in corridor/casual (`< 0`) → direct path
|
|
314
|
+
3. Exiting a room: pick the door closest to the target. Add `inside` → `outside` waypoints.
|
|
315
|
+
4. Entering a room: pick the door closest to the robot's current position. Add `outside` → `inside` waypoints.
|
|
316
|
+
5. Append final target.
|
|
317
|
+
|
|
318
|
+
**Nearest-door selection:** Euclidean distance from the door's `outside` point to the robot's current position (for entering) or to the target (for exiting).
|
|
319
|
+
|
|
320
|
+
**`setNavTarget` helper:** Calls `computePathWaypoints`, sets `nav.waypoints`, initializes `nav.waypointIdx = 0`, sets `nav.target` to the first waypoint.
|
|
321
|
+
|
|
322
|
+
### 15.13 Robot 3D Model
|
|
323
|
+
|
|
324
|
+
**Geometry** (shared across all instances, defined in `src/lib/robot3DGeometry.ts`):
|
|
325
|
+
|
|
326
|
+
| Part | Geometry | Dimensions |
|
|
327
|
+
|------|----------|------------|
|
|
328
|
+
| Head | `BoxGeometry` | 0.28 × 0.24 × 0.26 |
|
|
329
|
+
| Visor | `BoxGeometry` | 0.24 × 0.065 × 0.02 |
|
|
330
|
+
| Antenna | `CylinderGeometry` | r=0.007, h=0.14, 4-sided |
|
|
331
|
+
| Antenna tip | `SphereGeometry` | r=0.02, 6 segments |
|
|
332
|
+
| Torso | `BoxGeometry` | 0.32 × 0.38 × 0.2 |
|
|
333
|
+
| Core | `SphereGeometry` | r=0.032, 8 segments |
|
|
334
|
+
| Joint | `SphereGeometry` | r=0.035, 8 segments |
|
|
335
|
+
| Arm | `BoxGeometry` | 0.08 × 0.26 × 0.08 |
|
|
336
|
+
| Leg | `BoxGeometry` | 0.09 × 0.28 × 0.09 |
|
|
337
|
+
| Foot | `BoxGeometry` | 0.1 × 0.045 × 0.12 |
|
|
338
|
+
|
|
339
|
+
Each body part also has an `EdgesGeometry` wireframe overlay (lineSegments) with a neon color material at opacity 0.3.
|
|
340
|
+
|
|
341
|
+
**Materials:**
|
|
342
|
+
- `metalMat`: `MeshStandardMaterial`, color `#2a2a3e`, roughness 0.3, metalness 0.85 (shared)
|
|
343
|
+
- `darkMat`: color `#1c1c2c`, roughness 0.4, metalness 0.7 (shared)
|
|
344
|
+
- `neonMats[i]`: per-palette emissive material, intensity 2.0 (pool of 16, shared)
|
|
345
|
+
- `edgeMats[i]`: `LineBasicMaterial`, color = palette color, opacity 0.3 (pool of 16, shared)
|
|
346
|
+
- `bodyMat`: cloned from `metalMat` per robot (animations mutate emissive each frame)
|
|
347
|
+
- `bodyEdgeMat`: cloned from `edgeMat` per robot
|
|
348
|
+
|
|
349
|
+
**PALETTE** — 16 cyberpunk neon colors:
|
|
350
|
+
`#00f0ff`, `#ff00aa`, `#a855f7`, `#00ff88`, `#ff4444`, `#ffaa00`, `#00aaff`, `#ff66ff`, `#44ff44`, `#ff8800`, `#8855ff`, `#00ffcc`, `#ff0066`, `#ccff00`, `#ff5577`, `#33ddff`
|
|
351
|
+
|
|
352
|
+
**Model variants** (6 types, defined in `src/lib/robot3DModels.ts`):
|
|
353
|
+
|
|
354
|
+
| Type | Description | Distinctive geometry |
|
|
355
|
+
|------|-------------|---------------------|
|
|
356
|
+
| `robot` | Standard humanoid | Default geometry |
|
|
357
|
+
| `mech` | Bulkier, wider stance | Head 0.34×0.2×0.3, Torso 0.42×0.44×0.26, wider arms/legs |
|
|
358
|
+
| `drone` | Hovering unit | Spherical head (r=0.14), flat arms (0.22×0.04×0.06), no legs, baseY=0.3 |
|
|
359
|
+
| `spider` | Low-slung 4-legged | Spherical head (r=0.12), wide flat torso, all 4 limbs stubby at corners, baseY=-0.15 |
|
|
360
|
+
| `orb` | Spherical body | Spherical head (r=0.10), spherical torso (r=0.22, 12-seg), short arms and legs |
|
|
361
|
+
| `tank` | Wide, one-armed | Compact head, wide torso (0.44×0.3×0.26), no left arm, thick right arm, tread-shaped legs, baseY=-0.05 |
|
|
362
|
+
|
|
363
|
+
**Per-robot model selection:** `session.characterModel` overrides the global `settingsStore.characterModel` (default `'robot'`).
|
|
364
|
+
|
|
365
|
+
**CLI source badge:** A Billboard `<Text>` on the robot's chest (position slightly below core) showing a single letter with emissive color:
|
|
366
|
+
- Claude: `'C'`, color `#00f0ff`
|
|
367
|
+
- Gemini: `'G'`, color `#4285f4`
|
|
368
|
+
- Codex: `'X'`, color `#10a37f`
|
|
369
|
+
- OpenClaw: `'O'`, color `#ff6b2b`
|
|
370
|
+
- Unknown: `'?'`, color `#aa66ff`
|
|
371
|
+
|
|
372
|
+
Badge has `outlineWidth: 0.01`, `emissiveIntensity: 0.8`.
|
|
373
|
+
|
|
374
|
+
**CLI color override:** When a session has no explicit `accentColor`, the robot's neon color is set to the CLI badge color rather than the PALETTE color at `session.colorIndex`.
|
|
375
|
+
|
|
376
|
+
### 15.14 Robot Animations
|
|
377
|
+
|
|
378
|
+
Animations run in `useFrame` by reading `useSettingsStore.getState()` imperatively (no subscription). `animSpeed = animationSpeed / 100`, `ai = animationIntensity / 100`.
|
|
379
|
+
|
|
380
|
+
**Always active:**
|
|
381
|
+
- Antenna tip: scales between 0.8 and 1.2 at 6 Hz with `ai` factor. Web tools: intensity 3-4.5.
|
|
382
|
+
- Core: scales between 0.78 and 1.02 at 3 Hz.
|
|
383
|
+
|
|
384
|
+
**State-specific animations:**
|
|
385
|
+
|
|
386
|
+
| State | Key behaviors |
|
|
387
|
+
|-------|---------------|
|
|
388
|
+
| `idle` | Body bobs at 1.5 Hz (0.02 * ai), arms sway at 0.8 Hz, body tilts at 0.5 Hz |
|
|
389
|
+
| `thinking` (standing) | Body bobs at 1.2 Hz, head tilts at 0.6 Hz, right arm raised (-0.6 + oscillation) |
|
|
390
|
+
| `thinking` (seated) | Head tilts 0.12 + oscillation, right arm raised to chin-scratch pose (-1.1), legs bent (1.2) |
|
|
391
|
+
| `working` | See tool-specific animations below; body bobs at 0.7 Hz; CHARGING EFFECT active |
|
|
392
|
+
| `waiting` | Body bounces (abs sin) at 2 Hz (0.06 * ai), head turns at 0.8 Hz |
|
|
393
|
+
| `alert` | Visor flashes (urgency-scaled); arms raised and shaking at 8 Hz; 30s+ adds lateral shake |
|
|
394
|
+
| `input` | Visor pulses at 4 Hz; right arm fully raised (-1.5); slow body sway |
|
|
395
|
+
| `offline` | Visor/core dim over time; head drooped, arms slack |
|
|
396
|
+
| `connecting` | 1.5-second boot animation: scale 0→scaleProp, Y sinks from 0.5→0 |
|
|
397
|
+
|
|
398
|
+
**Tool-specific working animations (WS7.C):**
|
|
399
|
+
|
|
400
|
+
| Tool category | Tools | Animation |
|
|
401
|
+
|---------------|-------|-----------|
|
|
402
|
+
| `read` | Read, Grep, Glob, NotebookEdit | Head scans left-right at 2.5 Hz (±0.35 rad); arms at -0.4 |
|
|
403
|
+
| `write` | Write, Edit | Rapid arm typing at 14 Hz (±0.07 rad); head stable |
|
|
404
|
+
| `bash` | Bash | Right arm extended forward (-0.9); left arm at -0.3 |
|
|
405
|
+
| `task` | Task | Both arms raised (-0.8) with slow oscillation; head looks around |
|
|
406
|
+
| `web` | WebFetch, WebSearch | Antenna brightness boost (3-4.5 emissive intensity); default arm motion |
|
|
407
|
+
| `default` | All others | Standard rapid arm oscillation at 10 Hz (±0.05 rad) |
|
|
408
|
+
|
|
409
|
+
**Charging body effect** (active during any `working` state):
|
|
410
|
+
- Edge wireframe: opacity surges between 0.8-1.0 at two combined frequencies (12 Hz + 23 Hz)
|
|
411
|
+
- Core glow: emissive intensity 2.0-3.8 at 8 Hz + 19 Hz
|
|
412
|
+
- Visor: emissive intensity 2.0-3.2 at 10 Hz + 17 Hz
|
|
413
|
+
- Antenna tip: emissive intensity 2.5-4.0 at 15 Hz; scale flicker at 20 Hz
|
|
414
|
+
- Body mesh: subtle emissive boost in neon color, intensity 0.05-0.25 at 14 Hz
|
|
415
|
+
|
|
416
|
+
**Alert urgency escalation (WS7.B):** After 15 seconds in `alert` state, visor pulse speed increases from 8 Hz to 12 Hz. After 30 seconds, base intensity rises from 1.5 to 2.5, pulse range from 1.0 to 1.5, and lateral shake is added.
|
|
417
|
+
|
|
418
|
+
**Label completion frame effects (6 types):**
|
|
419
|
+
|
|
420
|
+
Activate when a labeled session transitions to `ended`. Override the normal body animation with:
|
|
421
|
+
|
|
422
|
+
| Effect | Visual character |
|
|
423
|
+
|--------|-----------------|
|
|
424
|
+
| `fire` | Orange/red body emissive with rapid flicker (9 Hz + 17 Hz layered), orange wireframe, fiery core |
|
|
425
|
+
| `electric` | Spike pattern (sin^4 at 20 Hz), white wireframe arc flicker (25 Hz threshold), intense multi-freq core |
|
|
426
|
+
| `chains` | Slow golden aura (2.5 Hz), gold wireframe (3 Hz), golden visor glow |
|
|
427
|
+
| `liquid` | Flowing wave intensity (3 Hz body, 5 Hz edges, 3.5 Hz core), hue-shifted by wave value |
|
|
428
|
+
| `plasma` | Violent magenta oscillations at 12/15/14 Hz body/core/visor; extreme intensities (4.0-6.5 core) |
|
|
429
|
+
| `none` | No frame effect |
|
|
430
|
+
|
|
431
|
+
Frame effect animations run after state animations, taking priority over body charge reset.
|
|
432
|
+
|
|
433
|
+
**Visor material overrides** (static pre-created materials):
|
|
434
|
+
- `alert` state: `ALERT_VISOR_MAT` — neon yellow `#ffdd00`, emissiveIntensity 2
|
|
435
|
+
- `input` state: `INPUT_VISOR_MAT` — purple `#aa66ff`, emissiveIntensity 2
|
|
436
|
+
- `offline` state: `OFFLINE_VISOR_MAT` — dark `#333344`, emissiveIntensity 2
|
|
437
|
+
|
|
438
|
+
### 15.15 Robot Dialogue
|
|
439
|
+
|
|
440
|
+
`RobotDialogue` shows a floating speech bubble above each robot. The component uses zero React state — all dialogue data flows through a ref (`dialogueRef`) updated in `useEffect` and read in `useFrame`.
|
|
441
|
+
|
|
442
|
+
**Panel geometry:**
|
|
443
|
+
- Billboard positioned at Y=2.8 above robot root
|
|
444
|
+
- Background: `PlaneGeometry(2.2, 0.22)` — dark `#0a0616`, opacity 0.92
|
|
445
|
+
- Border: `PlaneGeometry(2.26, 0.26)` — colored by dialogue type, opacity 0.6
|
|
446
|
+
- Text: `<Text>` via troika-three-text, fontSize 0.09, max-width 2.0, white with black outline
|
|
447
|
+
|
|
448
|
+
**Fade behavior:** Non-persistent dialogues start fading after 5 seconds. Fade speed: 4 units/second toward target opacity. Values below 0.01 clamp to 0.
|
|
449
|
+
|
|
450
|
+
**`TextUpdater`** inner component reads `fillOpacity`, `outlineOpacity`, and `text` from the troika mesh via parent traversal in `useFrame`, without any React state updates.
|
|
451
|
+
|
|
452
|
+
**Dialogue content and trigger rules:**
|
|
453
|
+
|
|
454
|
+
| Trigger | Text | Border color | Persistent |
|
|
455
|
+
|---------|------|--------------|------------|
|
|
456
|
+
| `prompting` status (status change) | First 60 chars of prompt + `...` | `#00e5ff` | No (5s) |
|
|
457
|
+
| `approval` status | `AWAITING APPROVAL` | `#ffdd00` | Yes |
|
|
458
|
+
| `input` status | `NEEDS INPUT` | `#aa66ff` | Yes |
|
|
459
|
+
| `waiting` status (from working) | `Task complete!` | `#00ff88` | No |
|
|
460
|
+
| `ended` status | `OFFLINE` | `#ff4444` | No |
|
|
461
|
+
| `idle` (from non-idle/non-connecting) | `ONLINE` | `#00ff88` | No |
|
|
462
|
+
| Read/Grep/Glob tool | `Reading <filename>...` | `rgba(0,229,255,0.6)` | No |
|
|
463
|
+
| Bash tool | `$ <cmd truncated to 40 chars>` | `#ff9100` | No |
|
|
464
|
+
| Edit/Write tool | `Editing <filename>...` | `#00aaff` | No |
|
|
465
|
+
| Task tool | `Spawning agent...` | `#aa66ff` | No |
|
|
466
|
+
| WebFetch/WebSearch tool | `Fetching...` | `#00e5ff` | No |
|
|
467
|
+
|
|
468
|
+
Tool dialogues are throttled: minimum 500ms between updates unless a status change occurred. Status-based dialogues always take priority over tool dialogues.
|
|
469
|
+
|
|
470
|
+
### 15.16 Robot Labels
|
|
471
|
+
|
|
472
|
+
`RobotLabel` renders a floating name tag above each robot using drei `<Billboard>` + `<Text>` (pure WebGL, not HTML portals). This avoids the cross-reconciler cascades that `<Html>` would cause.
|
|
473
|
+
|
|
474
|
+
**Layout** (all dimensions scale with `fontSize / 13`):
|
|
475
|
+
|
|
476
|
+
| Element | Base size | Position |
|
|
477
|
+
|---------|-----------|----------|
|
|
478
|
+
| Background panel | 1.8 × 0.14 | Y=0, Z=-0.01 |
|
|
479
|
+
| Border | 1.84 × 0.17 | Y=0, Z=-0.015, opacity 0.15 |
|
|
480
|
+
| Status dot | r=0.025 | X=-0.82, circle with 16 segments |
|
|
481
|
+
| Project name text | 0.065 fontSize | X=0.02, max-width 1.5 |
|
|
482
|
+
| Alert banner | 1.6 × 0.16 | Y=0.18 above label |
|
|
483
|
+
| Alert text | 0.07 fontSize | In alert banner |
|
|
484
|
+
| Billboard Y position | 2.1 (adjusts with scale) | |
|
|
485
|
+
|
|
486
|
+
**Status dot colors:**
|
|
487
|
+
- idle: `#00ff88`, prompting: `#00e5ff`, working: `#ff9100`, waiting: `#00e5ff`, approval: `#ffdd00`, input: `#aa66ff`, ended: `#ff4444`, connecting: `#888888`
|
|
488
|
+
|
|
489
|
+
**Title**: `session.title || session.projectName || 'Unnamed'`, truncated to 28 characters.
|
|
490
|
+
|
|
491
|
+
**Label badge**: When `session.label` is set, appends ` [LABEL]` to the title text.
|
|
492
|
+
|
|
493
|
+
**Alert banner**: Shows pulsing colored banner above the main label for `approval` (yellow) and `input` (purple) states. Opacity pulses via `useFrame` at 0.8+0.2*sin(time/500).
|
|
494
|
+
|
|
495
|
+
**Memoization**: 9-field equality check covering sessionId, status, title, projectName, label, robotState, isSelected, isHovered, fontSize.
|
|
496
|
+
|
|
497
|
+
### 15.17 Status Particles
|
|
498
|
+
|
|
499
|
+
`StatusParticles` fires brief particle bursts on status transitions. Uses zero React state — always mounted with `bufferGeo.setDrawRange(0, 0)` to hide when inactive.
|
|
500
|
+
|
|
501
|
+
**Pre-allocated geometry**: One `BufferGeometry` with `Float32Array(MAX_PARTICLES * 3)` (MAX=25). Buffer is never recreated, only updated.
|
|
502
|
+
|
|
503
|
+
**Per-instance material**: `PointsMaterial` with `AdditiveBlending`, `depthWrite: false`.
|
|
504
|
+
|
|
505
|
+
**Burst configurations:**
|
|
506
|
+
|
|
507
|
+
| Transition | Color | Count | Pattern | Speed | Gravity | Lifetime |
|
|
508
|
+
|------------|-------|-------|---------|-------|---------|----------|
|
|
509
|
+
| idle/waiting → working/thinking | `#ffdd00` | 20 | up | 2.5 | -1.5 | 1.5s |
|
|
510
|
+
| working/thinking → waiting | `#00ff88` | 20 | confetti | 1.2 | 2.0 | 1.5s |
|
|
511
|
+
| → alert | `#ffdd00` | 25 | ring | 2.0 | 0 | 1.2s |
|
|
512
|
+
| → input | `#aa66ff` | 20 | ring | 1.5 | 0 | 1.2s |
|
|
513
|
+
| → offline | `#666688` | 20 | down | 0.8 | 0.5 | 2.0s |
|
|
514
|
+
|
|
515
|
+
**Patterns:**
|
|
516
|
+
- `up`: velocity mainly upward, small random spread
|
|
517
|
+
- `down`: velocity mainly downward, smaller spread
|
|
518
|
+
- `ring`: particles spread radially in XZ plane at Y=0.15, even angular distribution
|
|
519
|
+
- `confetti`: random XZ spread with upward bias
|
|
520
|
+
|
|
521
|
+
**Fade**: `opacity = 1 - progress^2`. Size: `burstSize * (1 - progress * 0.5)`. Velocity decays at 0.99x per frame tick. Gravity applies to Y velocity each tick.
|
|
522
|
+
|
|
523
|
+
### 15.18 Subagent Connection Beams
|
|
524
|
+
|
|
525
|
+
`SubagentConnections` renders animated dashed laser-lines between parent sessions and their subagent children. Receives precomputed `ConnectionData[]` from the DOM layer (zero Zustand subscriptions).
|
|
526
|
+
|
|
527
|
+
**Detection**: Sessions where `teamRole === 'member'` and `teamId` exists. The parent ID is extracted from `teamId` by stripping the `team-` prefix. Both parent and child must be non-`ended`.
|
|
528
|
+
|
|
529
|
+
**Line rendering**: Raw `THREE.Line` with `LineDashedMaterial`:
|
|
530
|
+
- `dashSize: 0.3`, `gapSize: 0.2`
|
|
531
|
+
- Opacity: 0.3
|
|
532
|
+
- Color: parent session's `accentColor` or PALETTE color
|
|
533
|
+
|
|
534
|
+
**Animation**: In `useFrame`, the line's endpoint buffer is updated to track both robots' current positions from `robotPositionStore`. The `dashOffset` decrements at `delta * 2` per frame to create a flowing "data flowing from parent to child" visual. `computeLineDistances()` is called each frame (required for dashed lines to render correctly).
|
|
535
|
+
|
|
536
|
+
**Cleanup**: Geometry and material are disposed on unmount.
|
|
537
|
+
|
|
538
|
+
### 15.19 Camera Controller
|
|
539
|
+
|
|
540
|
+
`CameraController` is a Canvas-side component that smoothly animates OrbitControls to fly-to targets. It reads `cameraStore` imperatively in `useFrame` (no subscription) to avoid cascades.
|
|
541
|
+
|
|
542
|
+
**Constants:**
|
|
543
|
+
- `LERP_FACTOR = 0.04` — smooth camera movement (4% of remaining distance per frame)
|
|
544
|
+
- `ARRIVAL_THRESHOLD = 0.1` — considers animation complete when both position and look-at are within 0.1 units
|
|
545
|
+
|
|
546
|
+
**Fly-to behavior:**
|
|
547
|
+
1. Detects a new request by comparing `pendingTarget.requestId` to `lastRequestId`.
|
|
548
|
+
2. Sets `targetPos` and `targetLookAt` from the request.
|
|
549
|
+
3. Each frame: `camera.position.lerp(targetPos, 0.04)` and `controls.target.lerp(targetLookAt, 0.04)`.
|
|
550
|
+
4. When arrived: snaps to exact position, calls `completeAnimation()` via `queueMicrotask` (deferred out of R3F render cycle).
|
|
551
|
+
|
|
552
|
+
**Robot click → fly-to:** When a robot is selected, `CyberdromeScene` reads the robot's world position from `robotPositionStore` and calls `flyTo([pos.x + 6, pos.y + 8, pos.z + 10], [pos.x, pos.y + 1, pos.z])`. Offset constants: `FLY_OFFSET_X=6, FLY_OFFSET_Y=8, FLY_OFFSET_Z=10`.
|
|
553
|
+
|
|
554
|
+
**Room zoom:** `computeRoomCameraTarget(roomIndex)` places camera at 45-degree angle, 14 units out, 10 units high from room center.
|
|
555
|
+
|
|
556
|
+
**`cameraStore`:**
|
|
557
|
+
- `DEFAULT_CAMERA_POSITION: [18, 16, 18]`
|
|
558
|
+
- `DEFAULT_CAMERA_TARGET: [0, 1, 0]`
|
|
559
|
+
- `flyTo(position, lookAt)` sets `pendingTarget` with `requestId: Date.now()`
|
|
560
|
+
- `completeAnimation()` clears pending target and sets `isAnimating: false`
|
|
561
|
+
|
|
562
|
+
### 15.20 Room Labels
|
|
563
|
+
|
|
564
|
+
`RoomLabels` renders 3D text labels on the floor at each room's south door, using drei `<Text>` (SDF-based rendering at any zoom level).
|
|
565
|
+
|
|
566
|
+
**Room labels:**
|
|
567
|
+
- Room name: `fontSize: 0.8`, color alternates cyan (`#00f0ff`) or magenta (`#ff00aa`) based on `stripColor`, `letterSpacing: 0.15`, 2cm outline
|
|
568
|
+
- Unit count: `fontSize: 0.4`, dimmer variant of the strip color, at Z + 0.9 from room name
|
|
569
|
+
- Position: on floor (Y=0.02), rotated flat (`[-PI/2, 0, 0]`), at `cx, cz + ROOM_HALF + 1.5`
|
|
570
|
+
|
|
571
|
+
**Casual area labels:**
|
|
572
|
+
- Coffee Lounge: color `#ff9944`, at south edge of lounge area
|
|
573
|
+
- Gym Area: color `#44ff88`, at south edge of gym area
|
|
574
|
+
- Same fontSize/letterSpacing/outline as room labels
|
|
575
|
+
|
|
576
|
+
### 15.21 Robot List Sidebar
|
|
577
|
+
|
|
578
|
+
`RobotListSidebar` is a DOM overlay panel (top-right of scene) listing all active robots.
|
|
579
|
+
|
|
580
|
+
- Width: 280px, max-height: `calc(100vh - 100px)`, scrollable
|
|
581
|
+
- Backdrop blur, panel background, neon border with glow box-shadow
|
|
582
|
+
- Hidden when no active sessions
|
|
583
|
+
- Header: "Agents (N)" in monospace uppercase
|
|
584
|
+
|
|
585
|
+
**Entry sorting:** working → prompting/thinking → approval/input → waiting → idle → connecting
|
|
586
|
+
|
|
587
|
+
**Each entry shows:**
|
|
588
|
+
- Status dot (10×10px circle with glow box-shadow)
|
|
589
|
+
- Session title or project name (truncated, ellipsis)
|
|
590
|
+
- Status text (uppercase, status color)
|
|
591
|
+
- Close button (✕) — calls `removeSession` + `DELETE /api/sessions/:id`
|
|
592
|
+
|
|
593
|
+
**Selection:** Clicking an entry dispatches `CustomEvent('robot-select')` — same path as in-scene robot clicks, triggers session selection + camera fly-to.
|
|
594
|
+
|
|
595
|
+
**Selected state:** Border and background tint match the status color.
|
|
596
|
+
|
|
597
|
+
### 15.22 Scene Themes (9 themes)
|
|
598
|
+
|
|
599
|
+
Each theme provides a `Scene3DTheme` with 35+ color/density properties covering every visual element. Themes are applied via `getScene3DTheme(themeName)` which maps `ThemeName` to the palette:
|
|
600
|
+
|
|
601
|
+
| Theme | Background | Primary strip | Secondary strip | Character |
|
|
602
|
+
|-------|-----------|---------------|-----------------|-----------|
|
|
603
|
+
| `command-center` | `#0e0c1a` (dark navy) | `#00f0ff` (cyan) | `#ff00aa` (magenta) | Classic cyberpunk |
|
|
604
|
+
| `cyberpunk` | `#0d0221` (deep purple) | `#ff00ff` (magenta) | `#00ffff` (cyan) | High contrast neon |
|
|
605
|
+
| `warm` | `#f5ede0` (cream) | `#d97706` (amber) | `#b87333` (copper) | Daylight office |
|
|
606
|
+
| `dracula` | `#282a36` | `#bd93f9` (purple) | `#50fa7b` (green) | Dracula scheme |
|
|
607
|
+
| `solarized` | `#002b36` (dark teal) | `#2aa198` (teal) | `#cb4b16` (orange) | Solarized dark |
|
|
608
|
+
| `nord` | `#2e3440` (slate) | `#88c0d0` (sky) | `#d08770` (peach) | Nordic calm |
|
|
609
|
+
| `monokai` | `#272822` (dark) | `#66d9ef` (cyan) | `#f92672` (pink) | Monokai |
|
|
610
|
+
| `light` | `#e8eaef` (light gray) | `#3b82f6` (blue) | `#0ea5e9` (sky) | Light mode |
|
|
611
|
+
| `blonde` | `#f0e8d8` (warm white) | `#ca8a04` (gold) | `#a16207` (dark gold) | Warm blonde |
|
|
612
|
+
|
|
613
|
+
Theme properties include: `background`, `fogDensity`, `floor`, `roomFloor`, `borderGlow`, `grid1/2`, `wall`, `wallOpacity`, `stripPrimary/Secondary`, `desk`, `monitorFrame`, `chair`, `particle1/2`, `trace3`, `stars`, ambient/directional/fill/point/hemisphere lighting, `sconceColor`, `roomLight1/2`, `coffeeFloor/Accent/Furniture`, `gymFloor/Accent/Equipment`.
|
|
614
|
+
|
|
615
|
+
---
|
|
616
|
+
|
|
617
|
+
## 22. Sound System
|
|
618
|
+
|
|
619
|
+
### 22.1 Architecture
|
|
620
|
+
|
|
621
|
+
The sound system has two layers:
|
|
622
|
+
|
|
623
|
+
1. **`SoundEngine`** (singleton `soundEngine`) — event-driven sound effects using Web Audio API synthesis
|
|
624
|
+
2. **`AmbientEngine`** (singleton `ambientEngine`) — continuous procedurally generated ambient presets
|
|
625
|
+
|
|
626
|
+
Both engines use lazy `AudioContext` creation, only initialized after user interaction.
|
|
627
|
+
|
|
628
|
+
### 22.2 Sound Library (16 sounds)
|
|
629
|
+
|
|
630
|
+
All sounds are synthesized from Web Audio API primitives — no audio files:
|
|
631
|
+
|
|
632
|
+
| Name | Synthesis | Character |
|
|
633
|
+
|------|-----------|-----------|
|
|
634
|
+
| `chirp` | 1200 Hz sine, 80ms | Short high blip |
|
|
635
|
+
| `ping` | 660 Hz sine, 200ms | Medium tone |
|
|
636
|
+
| `chime` | Sequence [523, 659, 784] Hz, 80ms spacing | Major triad ascending |
|
|
637
|
+
| `ding` | 800 Hz triangle, 250ms | Bell-like |
|
|
638
|
+
| `blip` | 880 Hz square at 0.5 vol, 50ms | Short digital beep |
|
|
639
|
+
| `swoosh` | Sine 300→1200 Hz ramp over 250ms | Rising sweep |
|
|
640
|
+
| `click` | 1200 Hz square at 0.2 vol, 30ms | Crisp tap |
|
|
641
|
+
| `beep` | 440 Hz square at 0.4 vol, 150ms | Classic beep |
|
|
642
|
+
| `warble` | 600 Hz sine + 12 Hz LFO (±50 Hz), 300ms | Trembling tone |
|
|
643
|
+
| `buzz` | 200 Hz sawtooth at 0.4 vol, 120ms | Buzzy low tone |
|
|
644
|
+
| `cascade` | Sequence [784, 659, 523, 392], 100ms spacing | Descending arpeggio |
|
|
645
|
+
| `fanfare` | Sequence [523, 659, 784, 1047, 1319], 80ms spacing | Ascending fanfare |
|
|
646
|
+
| `alarm` | Square sequence [880, 660, 880, 660], 150ms each | Alert pattern |
|
|
647
|
+
| `thud` | Sine 80→30 Hz exponential ramp, 350ms | Bass impact |
|
|
648
|
+
| `urgentAlarm` | 3 bursts: square 1000↔800↔1000 Hz + sawtooth 200 Hz undertone | Triple urgent alarm |
|
|
649
|
+
| `none` | No-op | Silence |
|
|
650
|
+
|
|
651
|
+
### 22.3 Sound Actions (20 actions)
|
|
652
|
+
|
|
653
|
+
Actions are organized in 3 categories:
|
|
654
|
+
|
|
655
|
+
**Session Events** (4): `sessionStart`, `sessionEnd`, `promptSubmit`, `taskComplete`
|
|
656
|
+
|
|
657
|
+
**Tool Calls** (9): `toolRead`, `toolWrite`, `toolEdit`, `toolBash`, `toolGrep`, `toolGlob`, `toolWebFetch`, `toolTask`, `toolOther`
|
|
658
|
+
|
|
659
|
+
**System** (7): `approvalNeeded`, `inputNeeded`, `alert`, `kill`, `archive`, `subagentStart`, `subagentStop`
|
|
660
|
+
|
|
661
|
+
**Default action → sound mapping:**
|
|
662
|
+
|
|
663
|
+
| Action | Default sound |
|
|
664
|
+
|--------|--------------|
|
|
665
|
+
| sessionStart | chime |
|
|
666
|
+
| sessionEnd | cascade |
|
|
667
|
+
| promptSubmit | ping |
|
|
668
|
+
| taskComplete | fanfare |
|
|
669
|
+
| toolRead | click |
|
|
670
|
+
| toolWrite | blip |
|
|
671
|
+
| toolEdit | blip |
|
|
672
|
+
| toolBash | buzz |
|
|
673
|
+
| toolGrep | click |
|
|
674
|
+
| toolGlob | click |
|
|
675
|
+
| toolWebFetch | swoosh |
|
|
676
|
+
| toolTask | ding |
|
|
677
|
+
| toolOther | click |
|
|
678
|
+
| approvalNeeded | alarm |
|
|
679
|
+
| inputNeeded | chime |
|
|
680
|
+
| alert | alarm |
|
|
681
|
+
| kill | thud |
|
|
682
|
+
| archive | ding |
|
|
683
|
+
| subagentStart | chirp |
|
|
684
|
+
| subagentStop | ping |
|
|
685
|
+
|
|
686
|
+
### 22.4 Per-CLI Sound Profiles
|
|
687
|
+
|
|
688
|
+
Each CLI has an independent sound profile with its own volume and per-action sound mappings:
|
|
689
|
+
|
|
690
|
+
| CLI | Volume | Character |
|
|
691
|
+
|-----|--------|-----------|
|
|
692
|
+
| Claude | 0.7 | Standard with fanfare on task complete |
|
|
693
|
+
| Gemini | 0.7 | Heavier on swoosh, dings |
|
|
694
|
+
| Codex | 0.5 | Quieter, minimal sounds (blips and clicks) |
|
|
695
|
+
| OpenClaw | 0.7 | Dramatic — urgentAlarm for approvals, fanfares |
|
|
696
|
+
|
|
697
|
+
`AlarmEngine.playForCli()` detects the CLI via `detectCli()`, looks up the per-CLI volume and action mapping, temporarily overrides the sound engine volume for the play, then restores it.
|
|
698
|
+
|
|
699
|
+
### 22.5 CLI Detection
|
|
700
|
+
|
|
701
|
+
`detectCli(session)` determines CLI from:
|
|
702
|
+
1. `session.model` string: contains `claude`/`opus`/`sonnet`/`haiku` → Claude; `gemini`/`gemma` → Gemini; `gpt`/`codex`/`o1`/`o3`/`o4` → Codex; `openclaw`/`claw` → OpenClaw
|
|
703
|
+
2. Event type fallback: `BeforeAgent`/`AfterAgent`/`BeforeTool`/`AfterTool` → Gemini; `agent-turn-complete` → Codex; `SessionStart`/`PreToolUse`/`PostToolUse`/`UserPromptSubmit` → Claude
|
|
704
|
+
|
|
705
|
+
### 22.6 Ambient Presets (6 presets)
|
|
706
|
+
|
|
707
|
+
All ambient sounds are synthesized from oscillators and filtered noise:
|
|
708
|
+
|
|
709
|
+
| Preset | Synthesis technique |
|
|
710
|
+
|--------|---------------------|
|
|
711
|
+
| `off` | Silent (no audio) |
|
|
712
|
+
| `rain` | Bandpass-filtered noise (3000 Hz) + highshelf cutoff + random droplet oscillators every 80-200ms |
|
|
713
|
+
| `lofi` | 60 Hz sine with 0.3 Hz LFO (±5 Hz) + lowpass-filtered noise (400 Hz cutoff) |
|
|
714
|
+
| `serverRoom` | Bandpass noise (500 Hz) + 120 Hz triangle fan hum (0.1 Hz LFO ±3 Hz) + 8000 Hz whine |
|
|
715
|
+
| `deepSpace` | 40 Hz sine with 0.05 Hz LFO (±8 Hz) + convolver reverb (3s exponential impulse) + 80 Hz harmonic |
|
|
716
|
+
| `coffeeShop` | Lowpass+highpass filtered noise (200-1200 Hz) + random triangle dings every 2-6s |
|
|
717
|
+
|
|
718
|
+
---
|
|
719
|
+
|
|
720
|
+
## 23. Movement Effects
|
|
721
|
+
|
|
722
|
+
### 23.1 CSS Data-Attribute System
|
|
723
|
+
|
|
724
|
+
Movement effects in the legacy CSS frontend are applied by setting `data-effect="<effectName>"` on session card DOM elements. A central `movementManager.js` module applies effects and schedules their removal.
|
|
725
|
+
|
|
726
|
+
**Effect lifecycle:**
|
|
727
|
+
1. Effect applied: `element.dataset.effect = effectName`
|
|
728
|
+
2. CSS animation triggers via `[data-effect="name"]` selector
|
|
729
|
+
3. Auto-clear: `setTimeout(() => delete element.dataset.effect, duration)` removes the attribute
|
|
730
|
+
|
|
731
|
+
### 23.2 Effect Library (18 effects)
|
|
732
|
+
|
|
733
|
+
| Effect | CSS animation | Trigger scenario |
|
|
734
|
+
|--------|---------------|-----------------|
|
|
735
|
+
| `walk` | Horizontal movement | UserPromptSubmit |
|
|
736
|
+
| `run` | Fast horizontal movement | PreToolUse (working) |
|
|
737
|
+
| `bounce` | Vertical bouncing | Task complete |
|
|
738
|
+
| `shake` | Lateral shake | Alarm / ONEOFF label |
|
|
739
|
+
| `flash` | Opacity flicker | HEAVY label complete |
|
|
740
|
+
| `spin` | 360-degree rotation | Session end |
|
|
741
|
+
| `wave` | Wave oscillation | Prompting |
|
|
742
|
+
| `pulse` | Scale pulse | Tool use |
|
|
743
|
+
| `dance` | Celebratory multi-move | Waiting / task complete |
|
|
744
|
+
| `jump` | Vertical leap | Subagent spawn |
|
|
745
|
+
| `slide` | Horizontal slide | Session start |
|
|
746
|
+
| `wobble` | Irregular wobble | Alert state |
|
|
747
|
+
| `flip` | Vertical flip | Kill action |
|
|
748
|
+
| `zoom` | Scale zoom | Selection |
|
|
749
|
+
| `fade` | Opacity fade | Archive |
|
|
750
|
+
| `glow` | Glow pulse | Idle state |
|
|
751
|
+
| `twitch` | Rapid small movements | Approval needed |
|
|
752
|
+
| `none` | No animation | Default |
|
|
753
|
+
|
|
754
|
+
### 23.3 Action-to-Movement Mapping
|
|
755
|
+
|
|
756
|
+
Default movement mappings (configurable in settings):
|
|
757
|
+
|
|
758
|
+
| Action | Default movement |
|
|
759
|
+
|--------|-----------------|
|
|
760
|
+
| sessionStart | slide |
|
|
761
|
+
| sessionEnd | spin |
|
|
762
|
+
| promptSubmit | wave |
|
|
763
|
+
| taskComplete | bounce |
|
|
764
|
+
| toolRead | pulse |
|
|
765
|
+
| toolWrite | pulse |
|
|
766
|
+
| toolBash | run |
|
|
767
|
+
| toolWebFetch | walk |
|
|
768
|
+
| toolTask | jump |
|
|
769
|
+
| approvalNeeded | twitch |
|
|
770
|
+
| inputNeeded | wobble |
|
|
771
|
+
| kill | flip |
|
|
772
|
+
| archive | fade |
|
|
773
|
+
| subagentStart | jump |
|
|
774
|
+
|
|
775
|
+
---
|
|
776
|
+
|
|
777
|
+
## 24. Alarm System
|
|
778
|
+
|
|
779
|
+
### 24.1 Approval Alarm (repeating)
|
|
780
|
+
|
|
781
|
+
When a session enters `approval` status:
|
|
782
|
+
1. `soundEngine.play('approvalNeeded')` fires immediately
|
|
783
|
+
2. A `setInterval` is created for that session, firing every **10 seconds**
|
|
784
|
+
3. Each interval tick: re-checks if session is still in `approval` and not muted
|
|
785
|
+
4. If status changes or session is muted, the interval is cleared and removed from `approvalTimers` map
|
|
786
|
+
|
|
787
|
+
Multiple sessions can have simultaneous approval alarms. Each has an independent timer stored in `approvalTimers: Map<string, intervalId>`.
|
|
788
|
+
|
|
789
|
+
### 24.2 Input Notification (one-shot)
|
|
790
|
+
|
|
791
|
+
When a session enters `input` status:
|
|
792
|
+
1. Checks `inputFired` map — if this session hasn't fired yet, plays `soundEngine.play('inputNeeded')`
|
|
793
|
+
2. Sets `inputFired.set('input-' + sessionId, true)` to prevent repeat
|
|
794
|
+
3. When session leaves `input` status, clears the fired flag so it can fire again next time
|
|
795
|
+
|
|
796
|
+
### 24.3 Mute Per Session
|
|
797
|
+
|
|
798
|
+
Sessions can be individually muted:
|
|
799
|
+
- `muteSession(sessionId)` — adds to `mutedSessions` set
|
|
800
|
+
- `unmuteSession(sessionId)` — removes from set
|
|
801
|
+
- All alarm checks and sound plays respect `mutedSessions.has(sessionId)`
|
|
802
|
+
|
|
803
|
+
### 24.4 Label Completion Alerts
|
|
804
|
+
|
|
805
|
+
When a labeled session transitions to `ended`, `handleLabelAlerts(session, labelSettings)` is called:
|
|
806
|
+
- Looks up `labelSettings[session.label.toUpperCase()]`
|
|
807
|
+
- If a `sound` is configured, plays it via `soundEngine.preview(sound)`
|
|
808
|
+
- Respects mute state
|
|
809
|
+
|
|
810
|
+
Default label alarm configurations:
|
|
811
|
+
- `ONEOFF`: sound `alarm`, movement `shake`, frame effect `none`
|
|
812
|
+
- `HEAVY`: sound `urgentAlarm`, movement `flash`, frame effect `electric`
|
|
813
|
+
- `IMPORTANT`: sound `fanfare`, movement `bounce`, frame effect `liquid`
|
|
814
|
+
|
|
815
|
+
### 24.5 Event-Based Sounds
|
|
816
|
+
|
|
817
|
+
`handleEventSounds(session)` processes the last event in `session.events` and maps it to a sound action:
|
|
818
|
+
|
|
819
|
+
| Event type | Action |
|
|
820
|
+
|------------|--------|
|
|
821
|
+
| `SessionStart` | sessionStart |
|
|
822
|
+
| `UserPromptSubmit` | promptSubmit |
|
|
823
|
+
| `PreToolUse` | toolRead/Write/Edit/Bash/Grep/Glob/WebFetch/Task/Other (by tool_name) |
|
|
824
|
+
| `Stop` | taskComplete |
|
|
825
|
+
| `SessionEnd` | sessionEnd |
|
|
826
|
+
| `SubagentStart` | subagentStart |
|
|
827
|
+
| `SubagentStop` | subagentStop |
|
|
828
|
+
|
|
829
|
+
Tool name → action mapping:
|
|
830
|
+
- Read → toolRead, Write → toolWrite, Edit → toolEdit, Bash → toolBash, Grep → toolGrep, Glob → toolGlob, WebFetch → toolWebFetch, Task → toolTask, all others → toolOther
|
|
831
|
+
|
|
832
|
+
---
|
|
833
|
+
|
|
834
|
+
## 25. Settings
|
|
835
|
+
|
|
836
|
+
### 25.1 Settings Store
|
|
837
|
+
|
|
838
|
+
`settingsStore` (Zustand) manages all user preferences with automatic `IndexedDB` persistence via `db.settings.put()`. Each setter calls `persistSetting(key, value)` which writes to the database and triggers a 2-second "autosave" flash indicator.
|
|
839
|
+
|
|
840
|
+
### 25.2 Complete Settings Reference
|
|
841
|
+
|
|
842
|
+
**Appearance:**
|
|
843
|
+
|
|
844
|
+
| Setting | Default | Type | Effect |
|
|
845
|
+
|---------|---------|------|--------|
|
|
846
|
+
| `themeName` | `'command-center'` | ThemeName | Sets `data-theme` on `document.body`; changes 3D scene colors |
|
|
847
|
+
| `fontSize` | `13` | number (px) | Sets `document.documentElement.style.fontSize` |
|
|
848
|
+
| `scanlineEnabled` | `true` | boolean | Toggles `no-scanlines` class on body |
|
|
849
|
+
| `animationIntensity` | `100` | number (0-200) | Sets `--anim-intensity` CSS variable (intensity/100) |
|
|
850
|
+
| `animationSpeed` | `100` | number (0-200) | Sets `--anim-speed` CSS variable (speed/100) |
|
|
851
|
+
| `characterModel` | `'robot'` | RobotModelType | Global default robot model for all sessions |
|
|
852
|
+
|
|
853
|
+
**Sound:**
|
|
854
|
+
|
|
855
|
+
| Setting | Default | Type |
|
|
856
|
+
|---------|---------|------|
|
|
857
|
+
| `soundSettings.enabled` | `true` | boolean |
|
|
858
|
+
| `soundSettings.volume` | `0.5` | 0-1 |
|
|
859
|
+
| `soundSettings.muteApproval` | `false` | boolean |
|
|
860
|
+
| `soundSettings.muteInput` | `false` | boolean |
|
|
861
|
+
| `soundSettings.perCli.claude` | Full profile at 0.7 | CliSoundConfig |
|
|
862
|
+
| `soundSettings.perCli.gemini` | Full profile at 0.7 | CliSoundConfig |
|
|
863
|
+
| `soundSettings.perCli.codex` | Full profile at 0.5 | CliSoundConfig |
|
|
864
|
+
| `soundSettings.perCli.openclaw` | Full profile at 0.7 | CliSoundConfig |
|
|
865
|
+
|
|
866
|
+
**Ambient:**
|
|
867
|
+
|
|
868
|
+
| Setting | Default | Type |
|
|
869
|
+
|---------|---------|------|
|
|
870
|
+
| `ambientSettings.enabled` | `false` | boolean |
|
|
871
|
+
| `ambientSettings.volume` | `0.3` | 0-1 |
|
|
872
|
+
| `ambientSettings.preset` | `'off'` | AmbientPreset |
|
|
873
|
+
| `ambientSettings.roomSounds` | `false` | boolean |
|
|
874
|
+
| `ambientSettings.roomVolume` | `0.2` | 0-1 |
|
|
875
|
+
|
|
876
|
+
**UI/UX:**
|
|
877
|
+
|
|
878
|
+
| Setting | Default | Type |
|
|
879
|
+
|---------|---------|------|
|
|
880
|
+
| `hookDensity` | `'medium'` | 'high'/'medium'/'low'/'off' |
|
|
881
|
+
| `activityFeedVisible` | `true` | boolean |
|
|
882
|
+
| `toastEnabled` | `true` | boolean |
|
|
883
|
+
| `autoSendQueue` | `false` | boolean |
|
|
884
|
+
| `defaultTerminalTheme` | `'auto'` | string |
|
|
885
|
+
| `compactMode` | `false` | boolean |
|
|
886
|
+
| `showArchived` | `false` | boolean |
|
|
887
|
+
| `groupBy` | `'none'` | BrowserSettings['groupBy'] |
|
|
888
|
+
| `sortBy` | `'activity'` | BrowserSettings['sortBy'] |
|
|
889
|
+
|
|
890
|
+
**Label settings** (per label): `{ sound, movement, frame }` — all configurable.
|
|
891
|
+
|
|
892
|
+
**API Keys** (persisted): `anthropicApiKey`, `openaiApiKey`, `geminiApiKey`
|
|
893
|
+
|
|
894
|
+
### 25.3 Theme System (9 themes)
|
|
895
|
+
|
|
896
|
+
| Name | Label | Preview colors |
|
|
897
|
+
|------|-------|---------------|
|
|
898
|
+
| command-center | Command Center | navy, cyan, orange |
|
|
899
|
+
| cyberpunk | Cyberpunk | deep purple, magenta, cyan |
|
|
900
|
+
| warm | Warm | cream, amber, copper |
|
|
901
|
+
| dracula | Dracula | dark, purple, green |
|
|
902
|
+
| solarized | Solarized | dark teal, teal, orange |
|
|
903
|
+
| nord | Nord | slate, sky blue, peach |
|
|
904
|
+
| monokai | Monokai | dark, cyan, pink |
|
|
905
|
+
| light | Light | light gray, blue, sky |
|
|
906
|
+
| blonde | Blonde | warm white, gold, dark gold |
|
|
907
|
+
|
|
908
|
+
### 25.4 Label Frame Effects
|
|
909
|
+
|
|
910
|
+
| Effect key | Display name |
|
|
911
|
+
|-----------|-------------|
|
|
912
|
+
| `none` | None |
|
|
913
|
+
| `fire` | Burning Fire |
|
|
914
|
+
| `electric` | Electric Surge |
|
|
915
|
+
| `chains` | Golden Aura |
|
|
916
|
+
| `liquid` | Liquid Energy |
|
|
917
|
+
| `plasma` | Plasma Overload |
|
|
918
|
+
|
|
919
|
+
### 25.5 Settings Panel UI
|
|
920
|
+
|
|
921
|
+
The settings panel has 6 tabs (legacy CSS frontend):
|
|
922
|
+
1. **Appearance** — theme picker, font size, scanlines, animation speed/intensity, character model
|
|
923
|
+
2. **Sound** — master volume, enable/disable, per-action sound dropdowns, per-CLI profiles
|
|
924
|
+
3. **Ambient** — preset picker, volume, room sounds toggle
|
|
925
|
+
4. **Labels** — per-label sound/movement/frame configuration
|
|
926
|
+
5. **Hooks** — hook density selector, install/uninstall buttons
|
|
927
|
+
6. **API Keys** — Anthropic, OpenAI, Gemini key inputs
|
|
928
|
+
|
|
929
|
+
The React frontend (settings route) mirrors this structure using `settingsStore` actions directly.
|
|
930
|
+
|
|
931
|
+
**Import/Export**: Settings can be exported as JSON and imported to restore a configuration.
|
|
932
|
+
|
|
933
|
+
**Reset Defaults**: `resetDefaults()` restores all settings to defaults and persists them all.
|
|
934
|
+
|
|
935
|
+
---
|
|
936
|
+
|
|
937
|
+
## 26. Terminal Manager (Frontend)
|
|
938
|
+
|
|
939
|
+
### 26.1 xterm.js Integration
|
|
940
|
+
|
|
941
|
+
The terminal manager uses xterm.js with addons for full terminal emulation in the browser:
|
|
942
|
+
- `@xterm/xterm` — core terminal
|
|
943
|
+
- `@xterm/addon-fit` — auto-resize to container
|
|
944
|
+
- `@xterm/addon-web-links` — clickable URL detection
|
|
945
|
+
- `@xterm/addon-search` — text search within terminal
|
|
946
|
+
|
|
947
|
+
### 26.2 Terminal Themes (8 named + auto)
|
|
948
|
+
|
|
949
|
+
| Theme | Character |
|
|
950
|
+
|-------|-----------|
|
|
951
|
+
| `auto` | Matches the dashboard theme (default) |
|
|
952
|
+
| `dark` | Standard dark terminal |
|
|
953
|
+
| `light` | White background |
|
|
954
|
+
| `cyberpunk` | Magenta/cyan on deep purple |
|
|
955
|
+
| `dracula` | Dracula color scheme |
|
|
956
|
+
| `solarized` | Solarized dark |
|
|
957
|
+
| `nord` | Nord color scheme |
|
|
958
|
+
| `monokai` | Monokai colors |
|
|
959
|
+
| `warm` | Amber on cream |
|
|
960
|
+
|
|
961
|
+
Theme is applied on terminal creation and when the setting changes. `auto` resolves the theme from the current `themeName` in settings.
|
|
962
|
+
|
|
963
|
+
### 26.3 Canvas Repaint Workaround
|
|
964
|
+
|
|
965
|
+
xterm.js uses canvas rendering. A known issue causes the canvas to appear blank when the terminal panel is first opened or when the panel is resized. The workaround:
|
|
966
|
+
1. After terminal creation: `setTimeout(() => terminal.refresh(0, terminal.rows - 1), 50)` — forces a full repaint
|
|
967
|
+
2. On panel resize: `fitAddon.fit()` followed by another `refresh` call
|
|
968
|
+
3. On tab switch back to terminal: another `refresh` call
|
|
969
|
+
|
|
970
|
+
### 26.4 Fullscreen Mode
|
|
971
|
+
|
|
972
|
+
The terminal tab in the detail panel has a fullscreen toggle button. When activated:
|
|
973
|
+
- The terminal container expands to cover the full viewport
|
|
974
|
+
- `fitAddon.fit()` is called after the transition
|
|
975
|
+
- Escape key or the toggle button exits fullscreen
|
|
976
|
+
- Panel resize handles are hidden in fullscreen
|
|
977
|
+
|
|
978
|
+
### 26.5 Team Terminal
|
|
979
|
+
|
|
980
|
+
For team sessions (leader + members), a special "Team Terminal" view is available:
|
|
981
|
+
- Shows a split-view with all team member terminals
|
|
982
|
+
- Each terminal pane shows the member's session title and status
|
|
983
|
+
- Switching between member terminals uses the same WebSocket relay as individual terminals
|
|
984
|
+
|
|
985
|
+
### 26.6 WebSocket Relay
|
|
986
|
+
|
|
987
|
+
Terminal I/O is relayed through the WebSocket connection using message types:
|
|
988
|
+
- `terminal_input`: keystrokes from browser → server → PTY
|
|
989
|
+
- `terminal_output`: PTY output → server → browser (base64-encoded)
|
|
990
|
+
- `terminal_resize`: `{cols, rows}` to resize the PTY
|
|
991
|
+
|
|
992
|
+
The `sshManager.js` on the server creates `node-pty` processes for SSH/local sessions.
|
|
993
|
+
|
|
994
|
+
---
|
|
995
|
+
|
|
996
|
+
## 33. Testing
|
|
997
|
+
|
|
998
|
+
### 33.1 Test Framework
|
|
999
|
+
|
|
1000
|
+
The project uses two testing frameworks:
|
|
1001
|
+
- **Vitest** for unit and integration tests (TypeScript/browser-compatible)
|
|
1002
|
+
- **Playwright** for E2E tests (critical user flows)
|
|
1003
|
+
|
|
1004
|
+
### 33.2 Test Suite Size
|
|
1005
|
+
|
|
1006
|
+
**407 tests passing across 24 test files** (as of Feb 2025).
|
|
1007
|
+
|
|
1008
|
+
### 33.3 Test File Coverage
|
|
1009
|
+
|
|
1010
|
+
**Frontend/lib tests (Vitest):**
|
|
1011
|
+
|
|
1012
|
+
| File | What it tests |
|
|
1013
|
+
|------|--------------|
|
|
1014
|
+
| `src/lib/soundEngine.test.ts` | SoundEngine class — unlock, volume, action overrides, play/preview, dispose; ACTION_LABELS completeness; ACTION_CATEGORIES structure |
|
|
1015
|
+
| `src/lib/wsClient.test.ts` | WebSocket client — connection, reconnect, message dispatch |
|
|
1016
|
+
| `src/stores/sessionStore.test.ts` | Session CRUD, status transitions, team tracking |
|
|
1017
|
+
| `src/stores/settingsStore.test.ts` | Settings read/write, persistence, defaults |
|
|
1018
|
+
| `src/stores/roomStore.test.ts` | Room CRUD, session assignment/removal |
|
|
1019
|
+
| `src/stores/uiStore.test.ts` | UI state management |
|
|
1020
|
+
| `src/stores/queueStore.test.ts` | Prompt queue operations |
|
|
1021
|
+
| `src/stores/wsStore.test.ts` | WebSocket store state |
|
|
1022
|
+
| `src/hooks/useAuth.test.ts` | Auth hook behavior |
|
|
1023
|
+
|
|
1024
|
+
**Server tests (Vitest with Node environment):**
|
|
1025
|
+
|
|
1026
|
+
| File | What it tests |
|
|
1027
|
+
|------|--------------|
|
|
1028
|
+
| `test/hookProcessor.test.js` | Hook validation (null, non-object, missing fields, invalid types), processing SessionStart/Stop, latency calculation, alias fields |
|
|
1029
|
+
| `test/sessionStore.test.js` | Session creation, matching, status transitions, deduplication |
|
|
1030
|
+
| `test/sessionMatcher.test.js` | 5-priority session matching logic |
|
|
1031
|
+
| `test/mqReader.test.js` | File-based message queue reading, partial line handling, truncation |
|
|
1032
|
+
| `test/apiRouter.test.js` | REST API endpoints |
|
|
1033
|
+
| `test/hookInstaller.test.js` | Hook script installation/uninstall |
|
|
1034
|
+
| `test/portManager.test.js` | Port conflict resolution |
|
|
1035
|
+
| `test/teamManager.test.js` | Team/subagent tracking |
|
|
1036
|
+
| `test/approvalDetector.test.js` | Approval heuristic timeouts |
|
|
1037
|
+
| `test/autoIdleManager.test.js` | Auto-idle timer logic |
|
|
1038
|
+
| `test/processMonitor.test.js` | PID liveness checking |
|
|
1039
|
+
| `test/wsManager.test.js` | WebSocket broadcast, ring buffer |
|
|
1040
|
+
| `test/config.test.js` | Config loading and defaults |
|
|
1041
|
+
| `test/constants.test.js` | Constants completeness |
|
|
1042
|
+
| `test/hookStats.test.js` | Performance stats tracking |
|
|
1043
|
+
| `test/serverConfig.test.js` | Server config file loading |
|
|
1044
|
+
|
|
1045
|
+
### 33.4 Sound Engine Test Details
|
|
1046
|
+
|
|
1047
|
+
The `soundEngine.test.ts` file demonstrates the testing pattern for Web Audio API code:
|
|
1048
|
+
|
|
1049
|
+
- `AudioContext` is mocked using a factory function (must use `function` keyword, not arrow, so `new AudioContext()` works)
|
|
1050
|
+
- Mock provides: `createOscillator`, `createGain`, `resume`, `close`, with `vi.fn()` spies
|
|
1051
|
+
- `vi.useFakeTimers()` controls `setTimeout` calls used by `playSequence`
|
|
1052
|
+
- Tests verify: unlock/lock cycle, volume clamping (0-1), action override/restore, play returns true/false, sequence fires correct number of oscillators after `vi.runAllTimers()`, preview bypasses unlock check, dispose closes context
|
|
1053
|
+
|
|
1054
|
+
### 33.5 Running Tests
|
|
1055
|
+
|
|
1056
|
+
```bash
|
|
1057
|
+
# Run all tests
|
|
1058
|
+
npm test
|
|
1059
|
+
|
|
1060
|
+
# Watch mode
|
|
1061
|
+
npm run test:watch
|
|
1062
|
+
|
|
1063
|
+
# With verbose reporter
|
|
1064
|
+
npm test -- --reporter=verbose
|
|
1065
|
+
|
|
1066
|
+
# Coverage (if configured)
|
|
1067
|
+
npm test -- --coverage
|
|
1068
|
+
```
|
|
1069
|
+
|
|
1070
|
+
### 33.6 E2E Tests (Playwright)
|
|
1071
|
+
|
|
1072
|
+
Playwright E2E tests cover critical user flows:
|
|
1073
|
+
- Dashboard load and session display
|
|
1074
|
+
- Session card selection and detail panel
|
|
1075
|
+
- Terminal session creation and interaction
|
|
1076
|
+
- Settings persistence across reload
|
|
1077
|
+
- Theme switching
|
|
1078
|
+
- Keyboard shortcuts
|
|
1079
|
+
|
|
1080
|
+
E2E tests are located in the `test/e2e/` directory and configured in `playwright.config.ts`.
|