@vib3code/sdk 2.0.3-canary.60bc0f0 → 2.0.3-canary.74aebb4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/DOCS/EXPANSION_DESIGN.md +977 -0
- package/DOCS/EXPANSION_DESIGN_ULTRA.md +387 -0
- package/DOCS/MASTER_PLAN_2026-01-31.md +2 -2
- package/DOCS/OPTIMIZATION_PLAN_MATH.md +118 -0
- package/DOCS/SYSTEM_INVENTORY.md +2 -2
- package/DOCS/WEBGPU_STATUS.md +119 -38
- package/DOCS/archive/WEBGPU_STATUS_2026-02-15_STALE.md +38 -0
- package/DOCS/dev-tracks/DEV_TRACK_SESSION_2026-02-16.md +108 -0
- package/DOCS/dev-tracks/PERF_UPGRADE_2026-02-16.md +308 -0
- package/docs/webgpu-live.html +1 -1
- package/package.json +1 -1
- package/src/agent/mcp/MCPServer.js +195 -136
- package/src/agent/mcp/tools.js +45 -32
- package/src/experimental/GameLoop.js +72 -0
- package/src/experimental/LatticePhysics.js +100 -0
- package/src/experimental/LiveDirector.js +143 -0
- package/src/experimental/PlayerController4D.js +154 -0
- package/src/experimental/VIB3Actor.js +138 -0
- package/src/experimental/VIB3Compositor.js +117 -0
- package/src/experimental/VIB3Link.js +122 -0
- package/src/experimental/VIB3Orchestrator.js +146 -0
- package/src/experimental/VIB3Universe.js +109 -0
- package/src/experimental/demos/CrystalLabyrinth.js +202 -0
- package/src/faceted/FacetedSystem.js +19 -6
- package/src/geometry/generators/Crystal.js +2 -2
- package/src/holograms/HolographicVisualizer.js +58 -89
- package/src/math/Mat4x4.js +122 -6
- package/src/math/Rotor4D.js +93 -39
- package/src/math/Vec4.js +119 -78
- package/src/quantum/QuantumVisualizer.js +24 -20
- package/src/render/ShaderLoader.js +38 -0
- package/src/render/ShaderProgram.js +4 -4
- package/src/render/UnifiedRenderBridge.js +1 -1
- package/src/render/backends/WebGPUBackend.js +8 -4
- package/src/shaders/common/geometry24.glsl +65 -0
- package/src/shaders/common/geometry24.wgsl +54 -0
- package/src/shaders/common/rotation4d.glsl +4 -4
- package/src/shaders/common/rotation4d.wgsl +2 -2
- package/src/shaders/common/uniforms.wgsl +15 -8
- package/src/shaders/faceted/faceted.frag.wgsl +19 -6
- package/src/shaders/holographic/holographic.frag.wgsl +7 -5
- package/src/shaders/quantum/quantum.frag.wgsl +7 -5
- package/src/ui/adaptive/renderers/webgpu/WebGPURenderer.ts +2 -2
- package/tools/shader-sync-verify.js +6 -4
|
@@ -0,0 +1,387 @@
|
|
|
1
|
+
# VIB3+ Ultra — The Emergent Media Engine
|
|
2
|
+
|
|
3
|
+
**Purpose**: This document defines the **Ultra Tier** expansion for VIB3+. While the Premium Tier (`EXPANSION_DESIGN.md`) focuses on professional visualization tools, the Ultra Tier transforms the SDK into a **full-stack emergent media engine** capable of powering 4D video games, generative cartoons, and multi-user hallucinations.
|
|
4
|
+
|
|
5
|
+
**Status**: Vision Document / Technical Specification.
|
|
6
|
+
**Dependencies**: Requires `@vib3code/sdk` (core) and `@vib3code/premium` (for fine-grained control).
|
|
7
|
+
|
|
8
|
+
---
|
|
9
|
+
|
|
10
|
+
## I. Architecture: The VIB3 Universe
|
|
11
|
+
|
|
12
|
+
The core shift is from **Single Instance** to **Orchestrated Universe**.
|
|
13
|
+
|
|
14
|
+
### The Concept
|
|
15
|
+
Instead of "a canvas on a page," we treat the browser window as a **Universe** containing multiple **Entities** (VIB3 instances). These entities share a clock, a physics lattice, and a narrative context.
|
|
16
|
+
|
|
17
|
+
### The `VIB3Orchestrator`
|
|
18
|
+
A new singleton that manages the lifecycle and coordination of all VIB3 instances.
|
|
19
|
+
|
|
20
|
+
```javascript
|
|
21
|
+
class VIB3Orchestrator {
|
|
22
|
+
constructor() {
|
|
23
|
+
this.entities = new Map(); // id -> VIB3Entity
|
|
24
|
+
this.clock = new MasterClock({ bpm: 120 }); // Shared beat sync
|
|
25
|
+
this.physics = new LatticePhysicsEngine(); // 4D collision detection
|
|
26
|
+
this.link = new VIB3Link(); // WebRTC multi-user sync
|
|
27
|
+
}
|
|
28
|
+
|
|
29
|
+
// Spawn a new entity (actor, prop, or environment)
|
|
30
|
+
spawn(type, config) { ... }
|
|
31
|
+
|
|
32
|
+
// Global tick loop
|
|
33
|
+
tick(deltaTime) {
|
|
34
|
+
this.physics.update(this.entities, deltaTime);
|
|
35
|
+
this.entities.forEach(e => e.update(deltaTime));
|
|
36
|
+
this.link.sync(this.entities);
|
|
37
|
+
}
|
|
38
|
+
}
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
---
|
|
42
|
+
|
|
43
|
+
## II. HyperNarrative: Animating Stories & Cartoons
|
|
44
|
+
|
|
45
|
+
VIB3+ can tell stories. Characters are not mesh rigs but **VIB3 Actors** — distinct visualization instances with "souls" (parameter personalities) that express emotion through math.
|
|
46
|
+
|
|
47
|
+
### 1. The `VIB3Actor`
|
|
48
|
+
Wraps a `VIB3Engine` instance with identity, role, and emotional state.
|
|
49
|
+
|
|
50
|
+
```javascript
|
|
51
|
+
class VIB3Actor {
|
|
52
|
+
constructor(engine, personalityProfile) {
|
|
53
|
+
this.engine = engine;
|
|
54
|
+
this.profile = personalityProfile; // e.g., 'AnxiousGlitch', 'StoicCrystal'
|
|
55
|
+
this.emotion = { valence: 0, arousal: 0 }; // -1 to 1
|
|
56
|
+
this.voice = new AudioEmitter(); // Spatial audio source
|
|
57
|
+
}
|
|
58
|
+
|
|
59
|
+
// Express an emotion (modulates parameters based on profile)
|
|
60
|
+
emote(emotionName, intensity) {
|
|
61
|
+
const params = this.profile.mapEmotion(emotionName, intensity);
|
|
62
|
+
this.engine.transition(params, 500, 'easeOut');
|
|
63
|
+
}
|
|
64
|
+
|
|
65
|
+
// "Speak" with audio-reactive mouth modulation
|
|
66
|
+
speak(audioBuffer) {
|
|
67
|
+
this.voice.play(audioBuffer);
|
|
68
|
+
this.voice.onAnalysis((amplitude) => {
|
|
69
|
+
// Modulate geometry to simulate speech/expression
|
|
70
|
+
this.engine.setParameter('morphFactor', 0.5 + amplitude * 1.5);
|
|
71
|
+
this.engine.setParameter('intensity', 0.6 + amplitude * 0.4);
|
|
72
|
+
});
|
|
73
|
+
}
|
|
74
|
+
}
|
|
75
|
+
```
|
|
76
|
+
|
|
77
|
+
### 2. The Script Format (`.vib3script`)
|
|
78
|
+
A JSON-based screenplay format that controls actors, cameras, and environment.
|
|
79
|
+
|
|
80
|
+
```json
|
|
81
|
+
{
|
|
82
|
+
"title": "The Geometric Encounter",
|
|
83
|
+
"bpm": 110,
|
|
84
|
+
"cast": {
|
|
85
|
+
"Hero": { "system": "faceted", "geometry": 0, "profile": "heroic" },
|
|
86
|
+
"Villain": { "system": "quantum", "geometry": 16, "profile": "chaotic" }
|
|
87
|
+
},
|
|
88
|
+
"sequence": [
|
|
89
|
+
{
|
|
90
|
+
"time": "0:00",
|
|
91
|
+
"action": "camera_cut",
|
|
92
|
+
"target": "Hero",
|
|
93
|
+
"shot": "close_up" // Adjusts zoom/projection
|
|
94
|
+
},
|
|
95
|
+
{
|
|
96
|
+
"time": "0:02",
|
|
97
|
+
"actor": "Hero",
|
|
98
|
+
"action": "speak",
|
|
99
|
+
"text": "Why do you disturb the lattice?",
|
|
100
|
+
"audio": "hero_line_1.mp3",
|
|
101
|
+
"emotion": "stern"
|
|
102
|
+
},
|
|
103
|
+
{
|
|
104
|
+
"time": "0:05",
|
|
105
|
+
"actor": "Villain",
|
|
106
|
+
"action": "emote",
|
|
107
|
+
"emotion": "rage",
|
|
108
|
+
"intensity": 0.8 // Triggers red hue, high chaos, spike geometry
|
|
109
|
+
}
|
|
110
|
+
]
|
|
111
|
+
}
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
### 3. LipSync & Performance
|
|
115
|
+
Use `WebAudioAPI` analysis on dialogue tracks to drive `morphFactor` (mouth opening) and `intensity` (energy).
|
|
116
|
+
- **Consonants (High Freq)**: Trigger `chaos` spikes (sharp edges).
|
|
117
|
+
- **Vowels (Mid Freq)**: Trigger `morphFactor` swells (round shapes).
|
|
118
|
+
- **Volume**: Drives `gridDensity` (louder = larger/closer).
|
|
119
|
+
|
|
120
|
+
---
|
|
121
|
+
|
|
122
|
+
## III. HyperGame: FPV Gameplay Engine
|
|
123
|
+
|
|
124
|
+
Transform VIB3+ from a passive visualizer into an active **4D Game Engine**.
|
|
125
|
+
|
|
126
|
+
### 1. 4D Player Controller (`FPVController`)
|
|
127
|
+
Maps WASD + Mouse to 6D motion, allowing navigation *through* the hyperspace lattice.
|
|
128
|
+
|
|
129
|
+
- **W/S**: Move forward/back in Z (depth).
|
|
130
|
+
- **A/D**: Strafe in X.
|
|
131
|
+
- **Space/Shift**: Move in Y (up/down).
|
|
132
|
+
- **Q/E**: Rotate in XW plane (portal strafe).
|
|
133
|
+
- **Mouse**: Rotate XY (look left/right) and YZ (look up/down).
|
|
134
|
+
|
|
135
|
+
```javascript
|
|
136
|
+
// Inside game loop
|
|
137
|
+
if (input.forward) {
|
|
138
|
+
// Move "camera" through 4D noise space
|
|
139
|
+
// We don't move geometry; we offset the noise coordinate system!
|
|
140
|
+
uniforms.u_noiseOffset.z += speed * dt;
|
|
141
|
+
}
|
|
142
|
+
if (input.portalLeft) {
|
|
143
|
+
// Rotate the entire world in 4D
|
|
144
|
+
uniforms.u_rot4dXW += rotationSpeed * dt;
|
|
145
|
+
}
|
|
146
|
+
```
|
|
147
|
+
|
|
148
|
+
### 2. Lattice Physics (`LatticeCollider`)
|
|
149
|
+
Collision detection is not mesh-based (too expensive/abstract). It is **Function-Based**.
|
|
150
|
+
Since VIB3+ geometries are defined by math functions (SDFs or lattice grids), we check the player's coordinate against the active function.
|
|
151
|
+
|
|
152
|
+
```javascript
|
|
153
|
+
function checkCollision(playerPos, activeGeometry) {
|
|
154
|
+
// Sample the density function at player position
|
|
155
|
+
const density = getDensityAt(playerPos, activeGeometry);
|
|
156
|
+
if (density > 0.8) {
|
|
157
|
+
// Hit a "solid" part of the fractal/lattice
|
|
158
|
+
return true;
|
|
159
|
+
}
|
|
160
|
+
return false;
|
|
161
|
+
}
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
### 3. Entity System
|
|
165
|
+
Game objects are lightweight VIB3 instances or simplified shader particles.
|
|
166
|
+
- **Projectiles**: Small `Quantum` instances (single particle geometry) moving in XYZW.
|
|
167
|
+
- **Pickups**: `Faceted` geometries (spinning crystals) that trigger events on collision.
|
|
168
|
+
- **Enemies**: `Holographic` instances that track player position.
|
|
169
|
+
|
|
170
|
+
### 4. Game State Manager
|
|
171
|
+
Tracks score, health, and inventory.
|
|
172
|
+
- **Health**: Low health = high `chromaticAberration` + desaturation + `glitch` effect.
|
|
173
|
+
- **Powerup**: Ingesting a "hypercube" powerup transitions the player's view to `HyperMode` (geometry 17, max color).
|
|
174
|
+
|
|
175
|
+
---
|
|
176
|
+
|
|
177
|
+
## IV. Emergent Media: AI & Multi-User Sync
|
|
178
|
+
|
|
179
|
+
The final frontier: VIB3+ experiences that are alive, shared, and intelligent.
|
|
180
|
+
|
|
181
|
+
### 1. VIB3Link (Multi-User Hallucination)
|
|
182
|
+
Uses WebRTC/WebSockets to synchronize the `VIB3Orchestrator` state across clients.
|
|
183
|
+
- **Shared Seed**: All clients start with same RNG seed.
|
|
184
|
+
- **Input Sync**: Player A's rotation is broadcast to Player B.
|
|
185
|
+
- **State Consensus**: "If Player A explodes the star, Player B sees it explode."
|
|
186
|
+
|
|
187
|
+
**Use Case**: A shared "VR" room (on flat screens) where users float in a 4D chatroom, represented by their `VIB3Actor` avatars.
|
|
188
|
+
|
|
189
|
+
### 2. The Live Director (AI Agent)
|
|
190
|
+
An autonomous agent (MCP-connected) that watches the "audience" (user input, webcam, mic) and adjusts the show.
|
|
191
|
+
|
|
192
|
+
- **Sentiment Analysis**: If audience audio is loud/happy -> increase `speed` and `saturation`.
|
|
193
|
+
- **Attention Tracking**: If user stops interacting -> trigger `FocusLock` or `Explosion` to regain attention.
|
|
194
|
+
- **Generative Scripting**: The AI writes the `.vib3script` in real-time based on a prompt ("Make it scary", "Make it romantic").
|
|
195
|
+
|
|
196
|
+
---
|
|
197
|
+
|
|
198
|
+
## V. New MCP Tools for Ultra Tier
|
|
199
|
+
|
|
200
|
+
To empower agents to build these experiences, we add the following tools:
|
|
201
|
+
|
|
202
|
+
### `spawn_actor`
|
|
203
|
+
Creates a `VIB3Actor` with a specific personality and role.
|
|
204
|
+
```json
|
|
205
|
+
{ "role": "protagonist", "personality": "glitch_witch", "system": "holographic" }
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
### `direct_scene`
|
|
209
|
+
Issues a high-level direction to the `LiveDirector`.
|
|
210
|
+
```json
|
|
211
|
+
{ "mood": "increasing_tension", "pacing": "frenetic", "focus_target": "villain" }
|
|
212
|
+
```
|
|
213
|
+
|
|
214
|
+
### `configure_game_rules`
|
|
215
|
+
Sets up the `HyperGame` mechanics.
|
|
216
|
+
```json
|
|
217
|
+
{ "physics": "lattice_heavy", "collision_damage": 10, "goal": "collect_shards" }
|
|
218
|
+
```
|
|
219
|
+
|
|
220
|
+
### `sync_session`
|
|
221
|
+
Initializes `VIB3Link` for a multi-user session.
|
|
222
|
+
```json
|
|
223
|
+
{ "room_id": "lobby_1", "max_users": 4, "sync_mode": "strict" }
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
---
|
|
227
|
+
|
|
228
|
+
## VI. Deep Multilayer Architecture
|
|
229
|
+
|
|
230
|
+
To support `VIB3Universe`, we need a robust system for managing interactions between multiple VIB3 instances. This is not just visual layering; it's logic layering.
|
|
231
|
+
|
|
232
|
+
### 1. Visual Compositing (`VIB3Compositor`)
|
|
233
|
+
Managing 10+ layers (e.g., 2 instances × 5 holographic layers) requires a dedicated compositor to avoid DOM explosion and Z-fighting.
|
|
234
|
+
|
|
235
|
+
* **Render Targets**: Instead of rendering directly to the DOM, each VIB3 instance renders to an offscreen canvas.
|
|
236
|
+
* **Unified Stage**: The Compositor draws these offscreen buffers onto a single `GlobalCanvas` using WebGL blending.
|
|
237
|
+
* **Depth Sorting**: Instances are sorted by their "World Z" coordinate.
|
|
238
|
+
* **Masking**: Instances can mask each other (e.g., a character standing behind a portal).
|
|
239
|
+
|
|
240
|
+
```javascript
|
|
241
|
+
class VIB3Compositor {
|
|
242
|
+
registerInstance(instanceId, textureSource) { ... }
|
|
243
|
+
setLayerOrder(instanceIds) { ... }
|
|
244
|
+
render() {
|
|
245
|
+
// Draw background instances
|
|
246
|
+
// Draw foreground instances with blending
|
|
247
|
+
// Apply global post-processing (unifying the look)
|
|
248
|
+
}
|
|
249
|
+
}
|
|
250
|
+
```
|
|
251
|
+
|
|
252
|
+
### 2. Logic Layering (`LoopCoordinator`)
|
|
253
|
+
We have three distinct loops running at different frequencies:
|
|
254
|
+
1. **Physics Loop (Fixed Step, 60hz)**: Collision detection, movement integration. Deterministic for multiplayer sync.
|
|
255
|
+
2. **Narrative Loop (Event Driven)**: Script execution, state machine transitions.
|
|
256
|
+
3. **Reactive Loop (Frame Rate)**: Audio analysis, visual parameter smoothing.
|
|
257
|
+
|
|
258
|
+
The `VIB3Orchestrator` must prioritize these:
|
|
259
|
+
* Physics updates *must* happen before Visual updates.
|
|
260
|
+
* Narrative events trigger Physics state changes.
|
|
261
|
+
|
|
262
|
+
### 3. State Hydration (`UniverseSerializer`)
|
|
263
|
+
Saving a multi-instance universe is complex. We need a schema that captures the relationships, not just individual parameters.
|
|
264
|
+
|
|
265
|
+
```json
|
|
266
|
+
{
|
|
267
|
+
"universe_id": "u_8823",
|
|
268
|
+
"timestamp": 12044,
|
|
269
|
+
"entities": [
|
|
270
|
+
{ "id": "hero", "type": "actor", "pos": [0,0,0], "params": {...} },
|
|
271
|
+
{ "id": "world", "type": "environment", "params": {...} }
|
|
272
|
+
],
|
|
273
|
+
"global_state": {
|
|
274
|
+
"gravity": 9.8,
|
|
275
|
+
"tension": 0.4
|
|
276
|
+
}
|
|
277
|
+
}
|
|
278
|
+
```
|
|
279
|
+
|
|
280
|
+
This allows "save games" for HyperGame and "bookmarks" for HyperNarrative.
|
|
281
|
+
|
|
282
|
+
---
|
|
283
|
+
|
|
284
|
+
## VII. VIB3Link Protocol
|
|
285
|
+
|
|
286
|
+
VIB3Link uses WebRTC DataChannels for low-latency state synchronization between clients in a `VIB3Universe`.
|
|
287
|
+
|
|
288
|
+
### Protocol Message Format
|
|
289
|
+
|
|
290
|
+
Messages are JSON-encoded packets:
|
|
291
|
+
```json
|
|
292
|
+
{
|
|
293
|
+
"t": "type", // 'update', 'event', 'sync'
|
|
294
|
+
"s": 1023, // sequence number (for ordering)
|
|
295
|
+
"p": { ... } // payload
|
|
296
|
+
}
|
|
297
|
+
```
|
|
298
|
+
|
|
299
|
+
### Core Message Types
|
|
300
|
+
|
|
301
|
+
1. **Entity Update (`upd`)**: High-frequency (20Hz) updates of position/rotation.
|
|
302
|
+
```json
|
|
303
|
+
{ "t": "upd", "id": "actor_1", "pos": [x,y,z], "rot": [x,y,z,w] }
|
|
304
|
+
```
|
|
305
|
+
2. **Parameter Delta (`prm`)**: When a visual parameter changes.
|
|
306
|
+
```json
|
|
307
|
+
{ "t": "prm", "id": "actor_1", "k": "chaos", "v": 0.8 }
|
|
308
|
+
```
|
|
309
|
+
3. **Universe Event (`evt`)**: Narrative triggers or game events.
|
|
310
|
+
```json
|
|
311
|
+
{ "t": "evt", "n": "explosion", "loc": [10, 0, 5], "pow": 0.9 }
|
|
312
|
+
```
|
|
313
|
+
|
|
314
|
+
### Synchronization Strategy
|
|
315
|
+
|
|
316
|
+
* **Authority**: One client is the **Host** (orchestrator). Others are **Peers**.
|
|
317
|
+
* **Prediction**: Peers predict entity movement based on velocity.
|
|
318
|
+
* **Reconciliation**: If Peer state diverges > threshold from Host state, snap to Host.
|
|
319
|
+
* **Interpolation**: Visuals render at `t - buffer` to ensure smooth interpolation between network packets.
|
|
320
|
+
|
|
321
|
+
---
|
|
322
|
+
|
|
323
|
+
## VIII. HyperGame Engine
|
|
324
|
+
|
|
325
|
+
The VIB3+ Ultra HyperGame Engine enables 4D First-Person View (FPV) experiences. It decouples the physics simulation from the render loop and provides a 4D-native player controller.
|
|
326
|
+
|
|
327
|
+
### 1. The Game Loop (`GameLoop`)
|
|
328
|
+
A fixed-timestep loop for physics and logic, separate from the variable-timestep render loop. This ensures deterministic physics and smooth rendering even under load.
|
|
329
|
+
|
|
330
|
+
```javascript
|
|
331
|
+
class GameLoop {
|
|
332
|
+
constructor(updateFn, renderFn) {
|
|
333
|
+
this.accumulator = 0;
|
|
334
|
+
this.step = 1 / 60; // 60hz physics
|
|
335
|
+
}
|
|
336
|
+
// ... accumulates time and calls update() multiple times if needed
|
|
337
|
+
}
|
|
338
|
+
```
|
|
339
|
+
|
|
340
|
+
### 2. Lattice Physics (`LatticePhysics`)
|
|
341
|
+
Physics in VIB3+ is not about mesh-mesh intersection. It is about **Point-Field intersection**. We define the world as a scalar field (density) and check if the player's 4D point is inside a high-density region.
|
|
342
|
+
|
|
343
|
+
* **Collision**: `getDensity(pos) > threshold`
|
|
344
|
+
* **Gravity**: Constant force in -Y (or arbitrary 4D vector)
|
|
345
|
+
* **Friction**: Velocity decay
|
|
346
|
+
|
|
347
|
+
### 3. Player Controller 4D (`PlayerController4D`)
|
|
348
|
+
Handles input mapping for 6-degree-of-freedom movement.
|
|
349
|
+
|
|
350
|
+
* **Input**: WASD (Translation), Mouse (Rotation), QE (Portal/4D Rotation).
|
|
351
|
+
* **State**: Position (Vec4), Rotation (Rotor4D/Quat).
|
|
352
|
+
* **Smoothing**: Inputs are smoothed to prevent motion sickness in 4D space.
|
|
353
|
+
|
|
354
|
+
```javascript
|
|
355
|
+
update(dt) {
|
|
356
|
+
// Apply inputs to velocity
|
|
357
|
+
// Apply physics (gravity, friction)
|
|
358
|
+
// Integrate position: pos += vel * dt
|
|
359
|
+
// Resolve collisions: if (collision) pos -= normal * penetration
|
|
360
|
+
}
|
|
361
|
+
```
|
|
362
|
+
|
|
363
|
+
---
|
|
364
|
+
|
|
365
|
+
## IX. Demo: The Crystal Labyrinth
|
|
366
|
+
|
|
367
|
+
A vertical slice demo showcasing all Ultra capabilities in one cohesive experience.
|
|
368
|
+
|
|
369
|
+
### Story & Premise
|
|
370
|
+
The player is a **Lattice Runner** navigating a fractured 4D hyperspace maze. The goal is to collect **Resonance Crystals** (Facet engines) while avoiding **Void Shadows** (Holographic engines) that hunt based on noise (movement speed).
|
|
371
|
+
|
|
372
|
+
### Gameplay Loop
|
|
373
|
+
1. **Explore**: Navigate the 4D maze using WASD+Mouse+QE.
|
|
374
|
+
2. **Collect**: Find blue crystals. Touching one triggers a `VIB3Actor` "Collection" animation (implosion + sound).
|
|
375
|
+
3. **Evade**: Red "Shadows" drift towards you. If they touch you, the screen glitches (`chromaticAberration` spike) and health drops.
|
|
376
|
+
4. **Win**: Collect 5 crystals to stabilize the universe (trigger "Victory" state).
|
|
377
|
+
|
|
378
|
+
### Technical Integration
|
|
379
|
+
* **Universe**: One `VIB3Universe` managing the Player, 5 Crystal Actors, and 3 Shadow Actors.
|
|
380
|
+
* **Physics**: `LatticePhysics` handles wall collisions (fractal noise walls).
|
|
381
|
+
* **AI**: `LiveDirector` adjusts Shadow aggression based on player movement speed (stealth mechanic).
|
|
382
|
+
* **Networking**: (Optional) `VIB3Link` allows a spectator to watch the runner.
|
|
383
|
+
|
|
384
|
+
---
|
|
385
|
+
|
|
386
|
+
*VIB3+ Ultra — The Future of Emergent Media*
|
|
387
|
+
*Draft v5.0 — Added Crystal Labyrinth Demo Design — Feb 16, 2026*
|
|
@@ -19,10 +19,10 @@ The codebase is complete. The product isn't. You have 95,000+ lines of working e
|
|
|
19
19
|
- Creative Tooling (color presets, transitions, post-processing, timeline) — complete
|
|
20
20
|
- Platform Integrations (React, Vue, Svelte, Figma, Three.js, TouchDesigner, OBS) — code complete
|
|
21
21
|
- Advanced Features (WebXR, WebGPU Compute, MIDI, AI Presets, OffscreenWorker) — code complete
|
|
22
|
-
- MCP Agentic Control (14
|
|
22
|
+
- MCP Agentic Control (36 tools — 14 core + 22 added in Phases 6.5-8) — working
|
|
23
23
|
- C++ WASM Core with JS fallback — working
|
|
24
24
|
- Export System (SVG, CSS, Lottie, Shader, Trading Cards, VIB3Package) — working
|
|
25
|
-
-
|
|
25
|
+
- 1762 tests passing, 77 test files (as of Feb 15, 2026)
|
|
26
26
|
- 6 CI/CD workflows active
|
|
27
27
|
|
|
28
28
|
**What's not done**: Everything below.
|
|
@@ -0,0 +1,118 @@
|
|
|
1
|
+
# Optimization Plan: Core Math Library
|
|
2
|
+
|
|
3
|
+
## 1. Add `target` Parameters for Allocation-Free Operations
|
|
4
|
+
|
|
5
|
+
**Status:** High Impact / Medium Effort
|
|
6
|
+
**Currently:** `Mat4x4.multiply(m)` and `Mat4x4.multiplyVec4(v)` always return a `new` instance.
|
|
7
|
+
**Proposed:** Add an optional `target` parameter to write the result into an existing object.
|
|
8
|
+
|
|
9
|
+
### Implementation
|
|
10
|
+
```javascript
|
|
11
|
+
// Before
|
|
12
|
+
multiply(m) {
|
|
13
|
+
const out = new Mat4x4();
|
|
14
|
+
// ... compute ...
|
|
15
|
+
return out;
|
|
16
|
+
}
|
|
17
|
+
|
|
18
|
+
// After
|
|
19
|
+
multiply(m, target = null) {
|
|
20
|
+
const out = target || new Mat4x4();
|
|
21
|
+
// ... compute ...
|
|
22
|
+
return out;
|
|
23
|
+
}
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
### Cascading Changes
|
|
27
|
+
* **Scene Graph (`Node4D.js`):** Update `updateWorldMatrix` to reuse a cached matrix instance instead of creating a new one every frame.
|
|
28
|
+
* **Physics/Animation:** Update loops to reuse vector/matrix instances.
|
|
29
|
+
|
|
30
|
+
### Watch Outs
|
|
31
|
+
* **Aliasing:** If `a.multiply(b, a)` is called (writing result back to operand), ensure the implementation handles this correctly. The current `multiplyInPlace` implementation handles this by caching values in local variables before writing to the array. Ensure new methods do the same.
|
|
32
|
+
* **API Consistency:** Ensure `target` is consistently the last argument or follows a predictable pattern.
|
|
33
|
+
|
|
34
|
+
## 2. Implement `Vec4` Object Pooling (or Lightweight Structure)
|
|
35
|
+
|
|
36
|
+
**Status:** High Impact / High Complexity
|
|
37
|
+
**Currently:** `Vec4` allocates a `Float32Array(4)` per instance. This is heavy for the JS engine and GC.
|
|
38
|
+
**Proposed:**
|
|
39
|
+
1. **Object Pool:** `Vec4.create()` grabs from a pool, `Vec4.release(v)` returns it.
|
|
40
|
+
2. **Lightweight Class:** Use plain object `{x, y, z, w}` for intermediate math, only converting to `Float32Array` when uploading to GPU.
|
|
41
|
+
|
|
42
|
+
### Implementation (Object Pool)
|
|
43
|
+
```javascript
|
|
44
|
+
class Vec4Pool {
|
|
45
|
+
static get() { return pool.pop() || new Vec4(); }
|
|
46
|
+
static release(v) { pool.push(v); }
|
|
47
|
+
}
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
### Cascading Changes
|
|
51
|
+
* **Usage:** Requires changing *every* `new Vec4()` call to `Vec4Pool.get()` and ensuring `release()` is called when done.
|
|
52
|
+
* **Lifecycle Management:** Extremely error-prone in JS. Missing a release leaks memory; double-release corrupts data.
|
|
53
|
+
|
|
54
|
+
### Watch Outs
|
|
55
|
+
* **Manual Memory Management:** This fights against the JS GC. Only worth it in extremely hot paths (e.g., particle systems, per-vertex operations).
|
|
56
|
+
* **Alternatives:** Consider simply using `Float32Array` offsets directly for bulk data (Structure of Arrays).
|
|
57
|
+
|
|
58
|
+
## 3. Cache Common Constants
|
|
59
|
+
|
|
60
|
+
**Status:** Medium Impact / Low Effort
|
|
61
|
+
**Currently:** `Mat4x4.identity()` creates a new matrix every call.
|
|
62
|
+
**Proposed:** Add static read-only constants.
|
|
63
|
+
|
|
64
|
+
### Implementation
|
|
65
|
+
```javascript
|
|
66
|
+
class Mat4x4 {
|
|
67
|
+
static get IDENTITY() {
|
|
68
|
+
if (!this._identity) this._identity = new Mat4x4().setIdentity();
|
|
69
|
+
return this._identity;
|
|
70
|
+
}
|
|
71
|
+
}
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
### Cascading Changes
|
|
75
|
+
* **Usage:** Replace `Mat4x4.identity()` with `Mat4x4.IDENTITY` where read-only access is needed.
|
|
76
|
+
* **Cloning:** If modification is needed, use `Mat4x4.IDENTITY.clone()`.
|
|
77
|
+
|
|
78
|
+
### Watch Outs
|
|
79
|
+
* **Accidental Mutation:** If someone does `Mat4x4.IDENTITY.translate(...)`, it corrupts the constant for everyone.
|
|
80
|
+
* *Mitigation:* `Object.freeze()` or similar protections (though this has a perf cost). Better to rely on convention or a `ReadonlyMat4x4` type if using TS.
|
|
81
|
+
|
|
82
|
+
## 4. Optimize Scene Graph with In-Place Operations
|
|
83
|
+
|
|
84
|
+
**Status:** High Impact / Medium Effort
|
|
85
|
+
**Currently:** `Node4D.updateMatrix` often chains operations: `T * R * S`.
|
|
86
|
+
**Proposed:** Use the new `multiplyInPlace` and `rotateXX` methods.
|
|
87
|
+
|
|
88
|
+
### Implementation
|
|
89
|
+
```javascript
|
|
90
|
+
// Node4D.updateLocalMatrix
|
|
91
|
+
this.localMatrix.setIdentity();
|
|
92
|
+
this.localMatrix.translate(this.position); // Needs implementation
|
|
93
|
+
this.localMatrix.rotateFromAngles(this.rotation); // Needs implementation/update
|
|
94
|
+
this.localMatrix.scale(this.scale); // Needs implementation
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
### Cascading Changes
|
|
98
|
+
* **`Mat4x4` Extensions:** Need to implement `translate(v)`, `scale(v)` as in-place methods.
|
|
99
|
+
* **Logic Updates:** Rewrite `Node4D` transform logic to be imperative/stateful rather than functional/immutable.
|
|
100
|
+
|
|
101
|
+
### Watch Outs
|
|
102
|
+
* **Order of Operations:** Ensure `T * R * S` vs `S * R * T` order is preserved correctly when converting to in-place calls.
|
|
103
|
+
* **Dirty Flags:** Ensure `localMatrix` update only happens when `position`, `rotation`, or `scale` changes.
|
|
104
|
+
|
|
105
|
+
## 5. Bulk Operations for Geometry
|
|
106
|
+
|
|
107
|
+
**Status:** High Impact / High Complexity
|
|
108
|
+
**Currently:** `Mat4x4.multiplyVec4` processes one vector at a time.
|
|
109
|
+
**Proposed:** `Mat4x4.multiplyArray(inputArray, outputArray, count)`
|
|
110
|
+
|
|
111
|
+
### Implementation
|
|
112
|
+
Operate directly on flat `Float32Array` buffers.
|
|
113
|
+
|
|
114
|
+
### Cascading Changes
|
|
115
|
+
* **Geometry Generators:** Update to use bulk processing.
|
|
116
|
+
|
|
117
|
+
### Watch Outs
|
|
118
|
+
* **SIMD:** Browsers are starting to support SIMD via WASM. This might be a better target for heavy bulk math than optimizing JS loops.
|
package/DOCS/SYSTEM_INVENTORY.md
CHANGED
|
@@ -31,7 +31,7 @@ The SDK provides 3 active visualization systems with shared 6D rotation mathemat
|
|
|
31
31
|
| **Core Warp Types** | 3 (Base, Hypersphere, Hypertetrahedron) |
|
|
32
32
|
| **Total Geometries** | 24 per system (8 base × 3 cores) |
|
|
33
33
|
| **Canvas Layers** | 5 per system (background, shadow, content, highlight, accent) |
|
|
34
|
-
| **MCP Tools** |
|
|
34
|
+
| **MCP Tools** | 36 agent-accessible tools (14 core + 4 Phase 6.5 + 7 Phase 7 + 5 Phase 7.1 + 5 Phase 8 + 1 reactivity) |
|
|
35
35
|
| **Spatial Input Sources** | 8 (deviceTilt, mouse, gyroscope, gamepad, perspective, programmatic, audio, MIDI) |
|
|
36
36
|
| **Spatial Profiles** | 6 built-in (cardTilt, wearablePerspective, gameAsset, vjAudioSpatial, uiElement, immersiveXR) |
|
|
37
37
|
| **Creative Effects** | 14 post-processing effects, 22 color presets, 14 easing functions |
|
|
@@ -245,7 +245,7 @@ All rotation parameters: `-6.28` to `6.28` (radians, ±2π)
|
|
|
245
245
|
|
|
246
246
|
## MCP Server & Agent Tools
|
|
247
247
|
|
|
248
|
-
### Available Tools (
|
|
248
|
+
### Available Tools (36 total — see CLAUDE.md for full breakdown by phase)
|
|
249
249
|
|
|
250
250
|
| Tool | Description | Key Parameters |
|
|
251
251
|
|------|-------------|----------------|
|