@viji-dev/core 0.8.17 → 0.10.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/dist/docs-api.js CHANGED
@@ -1,7 +1,7 @@
1
1
  export const docsApi = {
2
2
  "version": "1.1.0",
3
- "coreVersion": "0.8.17",
4
- "generatedAt": "2026-06-12T17:55:57.342Z",
3
+ "coreVersion": "0.10.0",
4
+ "generatedAt": "2026-06-13T05:46:26.266Z",
5
5
  "navigation": [
6
6
  {
7
7
  "id": "getting-started",
@@ -1086,7 +1086,7 @@ export const docsApi = {
1086
1086
  "content": [
1087
1087
  {
1088
1088
  "type": "text",
1089
- "markdown": "# Best Practices\n\nThe rules below apply to every Viji scene, no matter which renderer you use. They cover the few topics where the same constraint has to hold across Native, P5, and Shader at once: animation timing, resolution, audio/video guards, and a handful of safety rules the runtime expects. Get them right and your scenes look correct at any resolution, run smoothly at any frame rate, and stay reliable across devices.\n\nLooking for a specific bug? [Common Mistakes](../common-mistakes/) has wrong/right pairs you can scan.\n\n---\n\n## Animation Timing Across Renderers\n\nViji exposes two timing values that are equivalent across all three renderers:\n\n- **`viji.time`** / **`u_time`**: seconds since the scene started. Use it for animation at a **constant** speed (oscillations, hue cycling, periodic events).\n- **`viji.deltaTime`** / **`u_deltaTime`**: seconds since the last frame. Use it when you accumulate values smoothly regardless of frame rate (movement, physics, fading), or when **animation speed is controlled by a parameter**.\n\n```javascript\n// viji.time: animation at a constant speed\nconst angle = viji.time * 2.0; // fixed 2 rad/s, no parameter involved\nconst x = Math.cos(angle) * radius;\n\n// viji.deltaTime: accumulation that stays smooth at any FPS\nposition += velocity * viji.deltaTime;\nopacity -= fadeRate * viji.deltaTime;\n```\n\n### When to use which\n\n| Use Case | Property | Why |\n|----------|----------|-----|\n| Oscillation at **constant** speed | [`viji.time`](/native/timing#vijitime-absolute-time) / [`u_time`](/shader/timing#u_time-absolute-time) | Depends on absolute position in time |\n| Hue cycling at constant rate | [`viji.time`](/native/timing#vijitime-absolute-time) / [`u_time`](/shader/timing#u_time-absolute-time) | Needs a continuous, monotonic input |\n| Oscillation at **parameter-driven** speed | accumulator with [`viji.deltaTime`](/native/timing#vijideltatime-frame-delta) / [`@viji-accumulator`](/shader/parameters/accumulator) | Avoids phase jumps when the slider changes |\n| Position accumulation | [`viji.deltaTime`](/native/timing#vijideltatime-frame-delta) | Must advance the same amount per second, regardless of FPS |\n| Physics / velocity | [`viji.deltaTime`](/native/timing#vijideltatime-frame-delta) | Distance = speed × time elapsed |\n| Fading / easing | [`viji.deltaTime`](/native/timing#vijideltatime-frame-delta) | Progress should be per-second, not per-frame |\n| Periodic events | [`viji.time`](/native/timing#vijitime-absolute-time) / [`u_time`](/shader/timing#u_time-absolute-time) | Trigger at fixed time intervals |\n\n**Rule of thumb:** if the speed multiplier is a **constant**, `viji.time * constant` is fine. If the speed comes from a **parameter** (slider, etc.), accumulate with `viji.deltaTime` to prevent jumps.\n\n### Accumulator Pattern: Parameter-Driven Speed\n\nWhen animation speed is controlled by a user parameter, **never** multiply `viji.time` (or `u_time`) by the parameter directly. Changing the slider recalculates the entire phase history with the new value and causes a visible jump. Instead, accumulate `speed × deltaTime` incrementally so the slider only affects future frames.\n\nThe pattern is the same across all three renderers:\n\n```javascript\n// Native\nconst speed = viji.slider(1, { min: 0.1, max: 5 });\nlet phase = 0;\nfunction render(viji) {\n phase += speed.value * viji.deltaTime;\n const wave = Math.sin(phase);\n}\n```\n\n```javascript\n// @renderer p5\nconst spin = viji.slider(1, { min: 0.1, max: 5, label: 'Spin' });\nlet phase = 0;\nfunction render(viji, p5) {\n phase += spin.value * viji.deltaTime;\n p5.rotate(phase);\n}\n```\n\n```glsl\n// @renderer shader\n// @viji-slider:speed label:\"Speed\" default:1.0 min:0.1 max:5.0\n// @viji-accumulator:phase rate:speed\n\nvoid main() {\n // Use `phase` instead of `u_time * speed`\n float wave = sin(phase);\n gl_FragColor = vec4(vec3(wave * 0.5 + 0.5), 1.0);\n}\n```\n\nFor shaders, the [`@viji-accumulator`](/shader/parameters/accumulator) directive integrates `speed × deltaTime` into the uniform automatically: there is no manual loop variable to maintain.\n\n### Nested Multiplication: The Same Trap After Accumulating\n\nAfter adopting the accumulator pattern, a common follow-up mistake is multiplying the accumulated value by another parameter. This causes the same jump because the full accumulated history is rescaled instantly:\n\n```javascript\n// Wrong: rotation jumps when rotationSpeed slider changes\nphase += speed.value * viji.deltaTime; // accumulates correctly\nconst rotation = phase * rotationSpeed.value; // jumps when rotationSpeed changes\n```\n\n```javascript\n// Right: each parameter-driven value gets its own accumulator\nlet phase = 0;\nlet rotPhase = 0;\nfunction render(viji) {\n phase += speed.value * viji.deltaTime;\n rotPhase += speed.value * rotationSpeed.value * viji.deltaTime;\n const rotation = rotPhase; // smooth at any slider value\n}\n```\n\n**Rule of thumb:** any time a growing value (accumulated phase, elapsed time) is multiplied by a user parameter, the result will jump when that parameter changes. Each such product needs its own `deltaTime`-based accumulator (or its own `@viji-accumulator` in shaders).\n\n> [!NOTE]\n> When animation speed is driven by a parameter, always use the accumulator pattern (`deltaTime` in JS/P5, `@viji-accumulator` in shaders) to prevent phase jumps. The same rule extends to nested multiplications: never multiply an accumulated value by a parameter.\n\n**See also:** [`native/timing`](/native/timing#when-to-use-vijitime-vs-vijideltatime), [`shader/timing`](/shader/timing#speed-controlled-animation-the-accumulator), [`shader/parameters/accumulator`](/shader/parameters/accumulator#basic-usage), [`p5/timing`](/p5/timing#when-to-use-vijitime-vs-vijideltatime).\n\n---\n\n## Resolution Agnosticism Across Renderers\n\nThe host application controls your scene's resolution. It may change at any time (window resize, resolution scaling for performance, high-DPI displays). Never hardcode pixel values or assume a specific canvas size.\n\nThe dimensions are exposed differently per renderer but the rule is identical:\n\n- **Native + P5**: `viji.width` and `viji.height` (also `p5.width` / `p5.height`, which Viji keeps in sync)\n- **Shader**: `u_resolution` (a `vec2`)\n\n### Native and P5\n\n```javascript\n// Good: scales to any resolution\nconst centerX = viji.width / 2;\nconst centerY = viji.height / 2;\nconst radius = Math.min(viji.width, viji.height) * 0.1;\n\n// Bad: breaks at different resolutions\nconst centerX = 960;\nconst centerY = 540;\nconst radius = 50;\n```\n\nFor parameters that control sizes, use normalized values (0-1) and multiply by canvas dimensions:\n\n```javascript\nconst size = viji.slider(0.15, { min: 0.02, max: 0.5, label: 'Size' });\n\nfunction render(viji) {\n const pixelSize = size.value * Math.min(viji.width, viji.height);\n}\n```\n\n### Shader\n\n```glsl\nvoid main() {\n vec2 uv = gl_FragCoord.xy / u_resolution; // normalized 0-1 coordinates\n float aspect = u_resolution.x / u_resolution.y;\n // ...\n}\n```\n\n> [!NOTE]\n> Always use `viji.width` and `viji.height` for positioning and sizing, and `viji.deltaTime` for frame-rate-independent animation. Never hardcode pixel values or assume a specific frame rate.\n\n**See also:** [`native/canvas-context`](/native/canvas-context), [`p5/canvas-resolution`](/p5/canvas-resolution), [`shader/resolution`](/shader/resolution).\n\n---\n\n## Audio and Video `isConnected` Guards\n\nAudio and video streams are provided by the host and may not always be available. Always check `isConnected` before reading audio bands, drawing video frames, or running CV inference. Without the guard your scene reads zeros or undefined values silently, with no indication that the input is missing.\n\nThe rule applies in three places:\n\n1. **Default audio source** (`viji.audio.isConnected`)\n2. **Default video source** (`viji.video.isConnected`)\n3. **Each external device entry** in `viji.devices[]` (`device.audio.isConnected`, `device.video.isConnected`)\n\n```javascript\nfunction render(viji) {\n if (viji.audio.isConnected) {\n const bass = viji.audio.bands.low;\n // ... react to audio\n }\n\n if (viji.video.isConnected && viji.video.currentFrame) {\n ctx.drawImage(viji.video.currentFrame, 0, 0, viji.width, viji.height);\n }\n\n for (const device of viji.devices) {\n if (device.audio?.isConnected) {\n const level = device.audio.volume.current;\n // ... per-device audio reaction\n }\n if (device.video?.isConnected && device.video.currentFrame) {\n // ... per-device video draw\n }\n }\n}\n```\n\nThe same checks work in P5 (replace canvas calls with `p5.image(...)`). In shaders, default audio and video uniforms safely fall back to `0.0` (or black for the `u_video` texture) when nothing is connected, so a typical scene can simply use the values without an explicit guard. When you read from a specific stream or device, gate on the corresponding boolean uniform: `u_audioStream{N}Connected`, `u_videoStream{N}Connected`, or `u_device{N}Connected`.\n\n**See also:** [`native/audio`](/native/audio), [`native/video`](/native/video), [`p5/audio`](/p5/audio), [`p5/video`](/p5/video), [`shader/audio`](/shader/audio), [`shader/video`](/shader/video).\n\n---\n\n## Worker Environment\n\nScenes run inside a Web Worker. There is no DOM. Use `viji.image()` for images, the `viji.audio` / `viji.video` APIs for media, and `fetch()` for external data.\n\n> [!WARNING]\n> Scenes run in a Web Worker. There is no `window`, `document`, `Image()`, `localStorage`, or any DOM API. All inputs (audio, video, images) are provided through the Viji API. One exception: `fetch()` is available, so you can load external data (JSON, etc.) from CDNs.\n\n**See also:** [`native/canvas-context`](/native/canvas-context) for the worker context model and the canonical `useContext()` pattern.\n\n---\n\n## Parameters at Top Level\n\nParameter helpers (`viji.slider()`, `viji.color()`, etc.) register UI controls with the host. They must run **once** during initialization, never inside `render()`.\n\n> [!NOTE]\n> Parameters must be defined at the top level of your scene, not inside `render()`. They are registered once during initialization. Defining them inside `render()` would re-register the parameter every frame, resetting its value to the default and making user changes ineffective.\n\nFor P5, the same rule applies, with one extra constraint: parameters must also not be declared inside `setup()`, since they need to be registered before `setup()` runs.\n\n**See also:** [`native/parameters`](/native/parameters), [`p5/parameters`](/p5/parameters), [`shader/parameters`](/shader/parameters).\n\n---\n\n## Memory in the Render Loop\n\nAllocating objects, arrays, or strings inside `render()` creates garbage-collection pressure and frame drops. Pre-allocate at the top level and mutate in place.\n\n> [!TIP]\n> Avoid allocating objects, arrays, or strings inside `render()`. Pre-allocate at the top level and reuse them:\n> ```javascript\n> // Good: pre-allocated\n> const pos = { x: 0, y: 0 };\n> function render(viji) {\n> pos.x = viji.width / 2;\n> pos.y = viji.height / 2;\n> }\n>\n> // Bad: creates a new object every frame\n> function render(viji) {\n> const pos = { x: viji.width / 2, y: viji.height / 2 };\n> }\n> ```\n\nThis is especially important for particle systems, arrays of positions, or any data structure that persists across frames.\n\n**See also:** [`native/quickstart`](/native/quickstart), [`p5/quickstart`](/p5/quickstart).\n\n---\n\n## Computer Vision Costs\n\nCV features (face detection, hand tracking, pose detection, etc.) are powerful but expensive. Each feature runs ML inference in its own WebGL context. Understand the relative cost before enabling more than one.\n\n| Feature | Relative Cost | WebGL Contexts | Notes |\n|---------|--------------|----------------|-------|\n| Face Detection | Low | 1 | Bounding box + basic landmarks only |\n| Face Mesh | Medium-High | 1 | 468 facial landmarks, requires more processing |\n| Emotion Detection | High | 1 | 7 expressions + 52 blendshape coefficients |\n| Hand Tracking | Medium | 1 | Up to 2 hands, 21 landmarks each |\n| Pose Detection | Medium | 1 | 33 body landmarks |\n| Body Segmentation | High | 1 | Per-pixel mask, large tensor output |\n\n> [!WARNING]\n> **WebGL Context Limits:** Each CV feature requires its own WebGL context for ML inference. Browsers typically allow 8-16 active WebGL contexts. Enabling too many CV features simultaneously can cause context eviction, potentially breaking the scene's own rendering. Use only the CV features you need.\n\n> [!TIP]\n> **Best practice:** Don't enable CV features by default. Instead, expose a toggle parameter so users can activate them on capable devices:\n> ```javascript\n> const useFace = viji.toggle(false, { label: 'Enable Face Detection', category: 'video' });\n> if (useFace.value) {\n> await viji.video.cv.enableFaceDetection(true);\n> }\n> ```\n\nThe cost rule applies in all three renderers. Native and P5 scenes activate CV through `viji.video.cv.enableX()` / `disableX()` from artist code (typically inside a per-frame reconciler that calls `enable*` when a toggle changes). Shader scenes activate CV through the `// @viji-cv:<featureToken>` directive — bare for always-on, or toggleable to synthesize a host-side parameter and a `bool <featureToken>` uniform mirroring the toggle. See [Shader Video & CV](/shader/video) for the directive surface.\n\n**See also:** [`native/video`](/native/video), [`p5/video`](/p5/video), [`shader/video`](/shader/video).\n\n## `currentFrame` vs `analysedFrame`: Pick the Right Trade-off\n\nCV results are computed asynchronously from the video stream. By the time `viji.video.cv.faces` (or `hands` / `pose` / `segmentation`) is available, the camera has moved on. With CV enabled you always have two frames to choose between, and they fail in different ways:\n\n- **`viji.video.currentFrame`** is the just-arrived video frame. It refreshes every frame your scene runs.\n- **`viji.video.cv.analysedFrame`** is the exact frame MediaPipe ran on to produce the current CV results. It refreshes only when an inference completes, which is not every frame.\n\nEach pairing has a different failure mode:\n\n| Choice | Mismatch | What you see |\n|---|---|---|\n| `currentFrame` + CV results | Spatial: landmarks and mask lag the displayed pixels by one inference | Overlay trails the face/body during fast motion |\n| `analysedFrame` + CV results | Temporal: the displayed image freezes briefly between inferences | The video itself stutters or holds, then jumps |\n\nYou cannot avoid both. Pick the mismatch that matters less for your scene.\n\n### How to choose\n\nAsk: *does my effect read pixels from the displayed frame at CV-derived positions?*\n\n- **No.** Drawing landmark dots, particles attached to fingertips, audio-reactive visuals driven by CV positions, generative geometry, abstract scenes that do not display the camera at all: use `currentFrame`. Spatial drift on a small overlay is much easier to live with than the displayed video stuttering.\n- **Yes.** Compositing the segmentation mask onto the body, sampling skin tone under a face landmark, warping the face along its mesh, texture-mapped face filters: use `analysedFrame`. Spatial alignment is the whole point of the effect, and the visible stutter is the cost.\n\nDrawing landmark dots over `currentFrame` is **not** a mistake. It is a deliberate choice to keep the live-camera feel.\n\n```javascript\n// Live camera with landmark dots: currentFrame is the right default\nctx.drawImage(viji.video.currentFrame, 0, 0, w, h);\nviji.video.cv.faces.forEach(face => { /* dots */ });\n\n// Mask compositing or pixel sampling under landmarks: analysedFrame is required\nconst frame = viji.video.cv.analysedFrame ?? viji.video.currentFrame;\nctx.drawImage(frame, 0, 0, w, h);\n// composite mask against the same frame, or sample skin tone at landmark positions...\n```\n\nFor shaders, `u_video` and `u_videoAnalysed` (gated by `u_videoAnalysedAvailable`) follow the same trade-off. Sample `u_videoAnalysed` only when the effect reads pixels at CV-derived positions; otherwise `u_video` is freshest and simplest.\n\n`analysedFrame` is `null` until the first CV inference completes after a feature is enabled (typically under one second). The `?? currentFrame` fallback covers that startup window, but be aware the source switches when the first inference lands; a brief visual hitch is normal.\n\n`viji.video.cv.analysedFrame` exists only on the main stream's CV surface. On `viji.videoStreams[i].cv.analysedFrame` and `viji.devices[i].video?.cv.analysedFrame` it always reads `null`, since CV does not run on those streams.\n\n---\n\n## Drawing Video at the Right Aspect Ratio\n\nCamera frames almost never match the canvas aspect ratio. Drawing the video to `(0, 0, viji.width, viji.height)` stretches it visibly: faces get squashed or elongated, circles become ovals, and CV bounding boxes drift off the actual face.\n\nThe fix is a small helper that returns the destination rectangle for `drawImage` (or `p5.image`) and lets you map CV coordinates into the same rectangle. The recipe and full code live on the per-renderer basics pages: [native](/native/video/basics/#drawing-video), [p5](/p5/video/basics/#drawing-video), [shader](/shader/video/basics/#drawing-video). Use those helpers in every video / CV scene.\n\n### Cover vs Contain\n\n| Mode | What it does | Use when |\n|---|---|---|\n| `cover` (default) | Video fills the canvas, edges cropped when aspects mismatch | Live camera feeds, filter-style scenes, anywhere stretching would look wrong but black bars would too |\n| `contain` | Full video shown, letterboxed when aspects mismatch | CV demos where bounding boxes near frame edges must stay visible, body segmentation where the user might be at the edge of the camera view, severe aspect mismatches |\n\n**Default to `cover`.** It matches the convention from CSS `object-fit: cover` and most camera apps: the camera fills the visible area cleanly, and the small edge crop is normal. Switch to `contain` only when losing the edges of the frame would lose meaning, which is mostly CV-overlay scenes.\n\n**Stretching `(0, 0, viji.width, viji.height)` is allowed only when distortion is intentional** (an artistic effect that wants the squash). On every other scene it is a quality bug.\n\n---\n\n## Parameter Categories\n\nParameters that depend on an external input (audio, video / camera / CV, or user interaction) must include the matching `category` so the host UI can show or hide them based on whether that input is active.\n\n| Input | Required `category` |\n|-------|---------------------|\n| Audio-related parameters (volume sensitivity, bass reactivity, beat response) | `'audio'` |\n| Video/camera/CV-related parameters (video opacity, CV sensitivity, segmentation toggles) | `'video'` |\n| Interaction-related parameters (mouse attraction, keyboard speed, touch sensitivity) | `'interaction'` |\n| Everything else (colors, sizes, speeds, shapes) | `'general'` (default: can be omitted) |\n\n**Native / P5:**\n\n```javascript\n// Wrong\nconst bassSensitivity = viji.slider(1.5, { min: 0, max: 3, label: 'Bass Sensitivity', group: 'audio' });\n\n// Right\nconst bassSensitivity = viji.slider(1.5, { min: 0, max: 3, label: 'Bass Sensitivity', group: 'audio', category: 'audio' });\n```\n\n**Shader:** use the `category:` key in `@viji-*` directives:\n\n```glsl\n// @viji-slider:bassSensitivity label:\"Bass Sensitivity\" default:1.5 min:0 max:3 group:audio category:audio\n```\n\n> [!NOTE]\n> `group` controls layout (which section the parameter appears in). `category` controls visibility (whether the parameter is shown at all). They are orthogonal: use both when appropriate.\n\n**See also:** [`native/parameters/categories`](/native/parameters/categories), [`p5/parameters/categories`](/p5/parameters/categories), [`shader/parameters/categories`](/shader/parameters/categories).\n\n---\n\n## No Redundant Input Toggles\n\nThe host UI controls whether audio, video, and interaction inputs are available to the scene. A scene-level on/off toggle for whether to *react to* an already-available input is redundant: muting host audio achieves the same thing as flipping an \"Audio Reactive\" toggle off, and any creative-strength slider at zero is also already an off-switch.\n\n**The rule.** Don't add `viji.toggle` parameters whose only effect is gating whether the scene reads from `viji.audio`, `viji.video`, `viji.mouse`, `viji.keyboard`, or `viji.touch`. Use `isConnected` / `isInCanvas` guards plus a creative-strength parameter (slider, color, etc.) tagged with the matching `category` instead.\n\n```javascript\n// Wrong: redundant on/off toggles\nconst audioReact = viji.toggle(true, { label: 'Audio Reactive', category: 'audio' });\nconst useCamera = viji.toggle(true, { label: 'Use Camera', category: 'video' });\nconst showMouse = viji.toggle(true, { label: 'Mouse Trail', category: 'interaction' });\n\n// Right: creative-strength parameters with the matching category\nconst bassSensitivity = viji.slider(1.5, { min: 0, max: 3, label: 'Bass Sensitivity', category: 'audio' });\nconst videoOpacity = viji.slider(0.6, { min: 0, max: 1, label: 'Video Opacity', category: 'video' });\nconst mouseAttraction = viji.slider(0.5, { min: 0, max: 1, label: 'Mouse Attraction', category: 'interaction' });\n```\n\n**Why.** The host already exposes input switches in its UI. A scene-level \"Audio Reactive (on/off)\" toggle just duplicates that switch one layer down, and any creative-strength slider going to zero (`bassSensitivity = 0`) achieves the off state without an extra control. Two controls for the same thing is friction for the user and a quality bug for our example scenes.\n\n**The exception: CV feature toggles.** `viji.video.cv.enableFaceDetection`, `enableFaceMesh`, `enableHandTracking`, etc. are *not* input toggles; they activate paid features that consume WebGL contexts and ML inference cycles. Those have to stay opt-in. The line: if the toggle's effect is \"should the scene read an already-available input?\" → remove. If it's \"should the runtime spend resources to enable a feature?\" → keep.\n\nIn shader scenes the same rule applies, expressed as a directive: `// @viji-cv:faceDetection label:\"Face Detection\" default:false` (toggleable form) is the shader equivalent of the JS toggle pattern above. Bare `// @viji-cv:faceDetection` is the always-on equivalent and skips the parameter entirely.\n\n**See also:** [`native/parameters/categories`](/native/parameters/categories), [Common Mistakes: Redundant Input Toggles](../common-mistakes/#redundant-input-onoff-toggles).\n\n---\n\n## Canvas Context Selection\n\nUse `viji.useContext()` to obtain a 2D or WebGL context for the scene canvas, not `viji.canvas.getContext()`. A canvas only supports one context type for its entire lifetime: pick one and stay with it.\n\n> [!WARNING]\n> A canvas only supports one context type. If you call `useContext('2d')` and later call `useContext('webgl')` (or vice versa), the second call returns `null`. Choose one context type and use it for the entire scene.\n\n**See also:** [`native/canvas-context`](/native/canvas-context).\n\n---\n\n## Related\n\n- [Common Mistakes](../common-mistakes/): symptom catalog with wrong/right pairs.\n- [Renderers Overview](../renderers-overview/): choosing the right renderer."
1089
+ "markdown": "# Best Practices\n\nThe rules below apply to every Viji scene, no matter which renderer you use. They cover the few topics where the same constraint has to hold across Native, P5, and Shader at once: animation timing, resolution, audio/video guards, and a handful of safety rules the runtime expects. Get them right and your scenes look correct at any resolution, run smoothly at any frame rate, and stay reliable across devices.\n\nLooking for a specific bug? [Common Mistakes](../common-mistakes/) has wrong/right pairs you can scan.\n\n---\n\n## Animation Timing Across Renderers\n\nViji exposes two timing values that are equivalent across all three renderers:\n\n- **`viji.time`** / **`u_time`**: seconds since the scene started. Use it for animation at a **constant** speed (oscillations, hue cycling, periodic events).\n- **`viji.deltaTime`** / **`u_deltaTime`**: seconds since the last frame. Use it when you accumulate values smoothly regardless of frame rate (movement, physics, fading), or when **animation speed is controlled by a parameter**.\n\n```javascript\n// viji.time: animation at a constant speed\nconst angle = viji.time * 2.0; // fixed 2 rad/s, no parameter involved\nconst x = Math.cos(angle) * radius;\n\n// viji.deltaTime: accumulation that stays smooth at any FPS\nposition += velocity * viji.deltaTime;\nopacity -= fadeRate * viji.deltaTime;\n```\n\n### When to use which\n\n| Use Case | Property | Why |\n|----------|----------|-----|\n| Oscillation at **constant** speed | [`viji.time`](/native/timing#vijitime-absolute-time) / [`u_time`](/shader/timing#u_time-absolute-time) | Depends on absolute position in time |\n| Hue cycling at constant rate | [`viji.time`](/native/timing#vijitime-absolute-time) / [`u_time`](/shader/timing#u_time-absolute-time) | Needs a continuous, monotonic input |\n| Oscillation at **parameter-driven** speed | accumulator with [`viji.deltaTime`](/native/timing#vijideltatime-frame-delta) / [`@viji-accumulator`](/shader/parameters/accumulator) | Avoids phase jumps when the slider changes |\n| Position accumulation | [`viji.deltaTime`](/native/timing#vijideltatime-frame-delta) | Must advance the same amount per second, regardless of FPS |\n| Physics / velocity | [`viji.deltaTime`](/native/timing#vijideltatime-frame-delta) | Distance = speed × time elapsed |\n| Fading / easing | [`viji.deltaTime`](/native/timing#vijideltatime-frame-delta) | Progress should be per-second, not per-frame |\n| Periodic events | [`viji.time`](/native/timing#vijitime-absolute-time) / [`u_time`](/shader/timing#u_time-absolute-time) | Trigger at fixed time intervals |\n\n**Rule of thumb:** if the speed multiplier is a **constant**, `viji.time * constant` is fine. If the speed comes from a **parameter** (slider, etc.), accumulate with `viji.deltaTime` to prevent jumps.\n\n### Accumulator Pattern: Parameter-Driven Speed\n\nWhen animation speed is controlled by a user parameter, **never** multiply `viji.time` (or `u_time`) by the parameter directly. Changing the slider recalculates the entire phase history with the new value and causes a visible jump. Instead, accumulate `speed × deltaTime` incrementally so the slider only affects future frames.\n\nThe pattern is the same across all three renderers:\n\n```javascript\n// Native\nconst speed = viji.slider(1, { min: 0.1, max: 5 });\nlet phase = 0;\nfunction render(viji) {\n phase += speed.value * viji.deltaTime;\n const wave = Math.sin(phase);\n}\n```\n\n```javascript\n// @renderer p5\nconst spin = viji.slider(1, { min: 0.1, max: 5, label: 'Spin' });\nlet phase = 0;\nfunction render(viji, p5) {\n phase += spin.value * viji.deltaTime;\n p5.rotate(phase);\n}\n```\n\n```glsl\n// @renderer shader\n// @viji-slider:speed label:\"Speed\" default:1.0 min:0.1 max:5.0\n// @viji-accumulator:phase rate:speed\n\nvoid main() {\n // Use `phase` instead of `u_time * speed`\n float wave = sin(phase);\n gl_FragColor = vec4(vec3(wave * 0.5 + 0.5), 1.0);\n}\n```\n\nFor shaders, the [`@viji-accumulator`](/shader/parameters/accumulator) directive integrates `speed × deltaTime` into the uniform automatically: there is no manual loop variable to maintain.\n\n### Nested Multiplication: The Same Trap After Accumulating\n\nAfter adopting the accumulator pattern, a common follow-up mistake is multiplying the accumulated value by another parameter. This causes the same jump because the full accumulated history is rescaled instantly:\n\n```javascript\n// Wrong: rotation jumps when rotationSpeed slider changes\nphase += speed.value * viji.deltaTime; // accumulates correctly\nconst rotation = phase * rotationSpeed.value; // jumps when rotationSpeed changes\n```\n\n```javascript\n// Right: each parameter-driven value gets its own accumulator\nlet phase = 0;\nlet rotPhase = 0;\nfunction render(viji) {\n phase += speed.value * viji.deltaTime;\n rotPhase += speed.value * rotationSpeed.value * viji.deltaTime;\n const rotation = rotPhase; // smooth at any slider value\n}\n```\n\n**Rule of thumb:** any time a growing value (accumulated phase, elapsed time) is multiplied by a user parameter, the result will jump when that parameter changes. Each such product needs its own `deltaTime`-based accumulator (or its own `@viji-accumulator` in shaders).\n\n> [!NOTE]\n> When animation speed is driven by a parameter, always use the accumulator pattern (`deltaTime` in JS/P5, `@viji-accumulator` in shaders) to prevent phase jumps. The same rule extends to nested multiplications: never multiply an accumulated value by a parameter.\n\n**See also:** [`native/timing`](/native/timing#when-to-use-vijitime-vs-vijideltatime), [`shader/timing`](/shader/timing#speed-controlled-animation-the-accumulator), [`shader/parameters/accumulator`](/shader/parameters/accumulator#basic-usage), [`p5/timing`](/p5/timing#when-to-use-vijitime-vs-vijideltatime).\n\n---\n\n## Resolution Agnosticism Across Renderers\n\nThe host application controls your scene's resolution. It may change at any time (window resize, resolution scaling for performance, high-DPI displays). Never hardcode pixel values or assume a specific canvas size.\n\nThe dimensions are exposed differently per renderer but the rule is identical:\n\n- **Native + P5**: `viji.width` and `viji.height` (also `p5.width` / `p5.height`, which Viji keeps in sync)\n- **Shader**: `u_resolution` (a `vec2`)\n\n### Native and P5\n\n```javascript\n// Good: scales to any resolution\nconst centerX = viji.width / 2;\nconst centerY = viji.height / 2;\nconst radius = Math.min(viji.width, viji.height) * 0.1;\n\n// Bad: breaks at different resolutions\nconst centerX = 960;\nconst centerY = 540;\nconst radius = 50;\n```\n\nFor parameters that control sizes, use normalized values (0-1) and multiply by canvas dimensions:\n\n```javascript\nconst size = viji.slider(0.15, { min: 0.02, max: 0.5, label: 'Size' });\n\nfunction render(viji) {\n const pixelSize = size.value * Math.min(viji.width, viji.height);\n}\n```\n\n### Shader\n\n```glsl\nvoid main() {\n vec2 uv = gl_FragCoord.xy / u_resolution; // normalized 0-1 coordinates\n float aspect = u_resolution.x / u_resolution.y;\n // ...\n}\n```\n\n> [!NOTE]\n> Always use `viji.width` and `viji.height` for positioning and sizing, and `viji.deltaTime` for frame-rate-independent animation. Never hardcode pixel values or assume a specific frame rate.\n\n**See also:** [`native/canvas-context`](/native/canvas-context), [`p5/canvas-resolution`](/p5/canvas-resolution), [`shader/resolution`](/shader/resolution).\n\n---\n\n## Audio and Video `isConnected` Guards\n\nAudio and video streams are provided by the host and may not always be available. Always check `isConnected` before reading audio bands, drawing video frames, or running CV inference. Without the guard your scene reads zeros or undefined values silently, with no indication that the input is missing.\n\nThe rule applies in three places:\n\n1. **Default audio source** (`viji.audio.isConnected`)\n2. **Default video source** (`viji.video.isConnected`)\n3. **Each external device entry** in `viji.devices[]` (`device.audio.isConnected`, `device.video.isConnected`)\n\n```javascript\nfunction render(viji) {\n if (viji.audio.isConnected) {\n const bass = viji.audio.bands.low;\n // ... react to audio\n }\n\n if (viji.video.isConnected && viji.video.currentFrame) {\n ctx.drawImage(viji.video.currentFrame, 0, 0, viji.width, viji.height);\n }\n\n for (const device of viji.devices) {\n if (device.audio?.isConnected) {\n const level = device.audio.volume.current;\n // ... per-device audio reaction\n }\n if (device.video?.isConnected && device.video.currentFrame) {\n // ... per-device video draw\n }\n }\n}\n```\n\nThe same checks work in P5 (replace canvas calls with `p5.image(...)`). In shaders, default audio and video uniforms safely fall back to `0.0` (or black for the `u_video` texture) when nothing is connected, so a typical scene can simply use the values without an explicit guard. When you read from a specific stream or device, gate on the corresponding boolean uniform: `u_audioStream{N}Connected`, `u_videoStream{N}Connected`, or `u_device{N}Connected`.\n\n**See also:** [`native/audio`](/native/audio), [`native/video`](/native/video), [`p5/audio`](/p5/audio), [`p5/video`](/p5/video), [`shader/audio`](/shader/audio), [`shader/video`](/shader/video).\n\n---\n\n## Worker Environment\n\nScenes run inside a Web Worker. There is no DOM. Use `viji.image()` for images, the `viji.audio` / `viji.video` APIs for media, and `fetch()` for external data.\n\n> [!WARNING]\n> Scenes run in a Web Worker. There is no `window`, `document`, `Image()`, `localStorage`, or any DOM API. All inputs (audio, video, images) are provided through the Viji API. One exception: `fetch()` is available, so you can load external data (JSON, etc.) from CDNs.\n\n**See also:** [`native/canvas-context`](/native/canvas-context) for the worker context model and the canonical `useContext()` pattern.\n\n---\n\n## Parameters at Top Level\n\nParameter helpers (`viji.slider()`, `viji.color()`, etc.) register UI controls with the host. They must run **once** during initialization, never inside `render()`.\n\n> [!NOTE]\n> Parameters must be defined at the top level of your scene, not inside `render()`. They are registered once during initialization. Defining them inside `render()` would re-register the parameter every frame, resetting its value to the default and making user changes ineffective.\n\nFor P5, the same rule applies, with one extra constraint: parameters must also not be declared inside `setup()`, since they need to be registered before `setup()` runs.\n\n**See also:** [`native/parameters`](/native/parameters), [`p5/parameters`](/p5/parameters), [`shader/parameters`](/shader/parameters).\n\n---\n\n## Memory in the Render Loop\n\nAllocating objects, arrays, or strings inside `render()` creates garbage-collection pressure and frame drops. Pre-allocate at the top level and mutate in place.\n\n> [!TIP]\n> Avoid allocating objects, arrays, or strings inside `render()`. Pre-allocate at the top level and reuse them:\n> ```javascript\n> // Good: pre-allocated\n> const pos = { x: 0, y: 0 };\n> function render(viji) {\n> pos.x = viji.width / 2;\n> pos.y = viji.height / 2;\n> }\n>\n> // Bad: creates a new object every frame\n> function render(viji) {\n> const pos = { x: viji.width / 2, y: viji.height / 2 };\n> }\n> ```\n\nThis is especially important for particle systems, arrays of positions, or any data structure that persists across frames.\n\n**See also:** [`native/quickstart`](/native/quickstart), [`p5/quickstart`](/p5/quickstart).\n\n---\n\n## Computer Vision Costs\n\nCV features (face detection, hand tracking, pose detection, etc.) are powerful but expensive. Each feature runs ML inference in its own WebGL context. Understand the relative cost before enabling more than one.\n\n| Feature | Relative Cost | WebGL Contexts | Notes |\n|---------|--------------|----------------|-------|\n| Face Detection | Low | 1 | Bounding box + basic landmarks only |\n| Face Mesh | Medium-High | 1 | 468 facial landmarks, requires more processing |\n| Emotion Detection | High | 1 | 7 expressions + 52 blendshape coefficients |\n| Hand Tracking | Medium | 1 | Up to 2 hands, 21 landmarks each |\n| Pose Detection | Medium | 1 | 33 body landmarks |\n| Body Segmentation | High | 1 | Per-pixel mask, large tensor output |\n\n> [!WARNING]\n> **WebGL Context Limits:** Each CV feature requires its own WebGL context for ML inference. Browsers typically allow 8-16 active WebGL contexts. Enabling too many CV features simultaneously can cause context eviction, potentially breaking the scene's own rendering. Use only the CV features you need.\n\n> [!TIP]\n> **Best practice:** Don't enable CV features by default. Instead, expose a toggle parameter so users can activate them on capable devices:\n> ```javascript\n> const useFace = viji.toggle(false, { label: 'Enable Face Detection', category: 'video' });\n> if (useFace.value) {\n> await viji.video.cv.enableFaceDetection(true);\n> }\n> ```\n\nThe cost rule applies in all three renderers. Native and P5 scenes activate CV through `viji.video.cv.enableX()` / `disableX()` from artist code (typically inside a per-frame reconciler that calls `enable*` when a toggle changes). Shader scenes activate CV through the `// @viji-cv:<featureToken>` directive — bare for always-on, or toggleable to synthesize a host-side parameter and a `bool <featureToken>` uniform mirroring the toggle. See [Shader Video & CV](/shader/video) for the directive surface.\n\n**See also:** [`native/video`](/native/video), [`p5/video`](/p5/video), [`shader/video`](/shader/video).\n\n## `currentFrame` vs `analysedFrame`: Pick the Right Trade-off\n\nCV results are computed asynchronously from the video stream. By the time `viji.video.cv.faces` (or `hands` / `pose` / `segmentation`) is available, the camera has moved on. With CV enabled you always have two frames to choose between, and they fail in different ways:\n\n- **`viji.video.currentFrame`** is the just-arrived video frame. It refreshes every frame your scene runs.\n- **`viji.video.cv.analysedFrame`** is the exact frame MediaPipe ran on to produce the current CV results. It refreshes only when an inference completes, which is not every frame.\n\nEach pairing has a different failure mode:\n\n| Choice | Mismatch | What you see |\n|---|---|---|\n| `currentFrame` + CV results | Spatial: landmarks and mask lag the displayed pixels by one inference | Overlay trails the face/body during fast motion |\n| `analysedFrame` + CV results | Temporal: the displayed image freezes briefly between inferences | The video itself stutters or holds, then jumps |\n\nYou cannot avoid both. Pick the mismatch that matters less for your scene.\n\n### How to choose\n\nAsk: *does my effect read pixels from the displayed frame at CV-derived positions?*\n\n- **No.** Drawing landmark dots, particles attached to fingertips, audio-reactive visuals driven by CV positions, generative geometry, abstract scenes that do not display the camera at all: use `currentFrame`. Spatial drift on a small overlay is much easier to live with than the displayed video stuttering.\n- **Yes.** Compositing the segmentation mask onto the body, sampling skin tone under a face landmark, warping the face along its mesh, texture-mapped face filters: use `analysedFrame`. Spatial alignment is the whole point of the effect, and the visible stutter is the cost.\n\nDrawing landmark dots over `currentFrame` is **not** a mistake. It is a deliberate choice to keep the live-camera feel.\n\n```javascript\n// Live camera with landmark dots: currentFrame is the right default\nctx.drawImage(viji.video.currentFrame, 0, 0, w, h);\nviji.video.cv.faces.forEach(face => { /* dots */ });\n\n// Mask compositing or pixel sampling under landmarks: analysedFrame is required\nconst frame = viji.video.cv.analysedFrame ?? viji.video.currentFrame;\nctx.drawImage(frame, 0, 0, w, h);\n// composite mask against the same frame, or sample skin tone at landmark positions...\n```\n\nFor shaders, `u_video` and `u_videoAnalysed` (gated by `u_videoAnalysedAvailable`) follow the same trade-off. Sample `u_videoAnalysed` only when the effect reads pixels at CV-derived positions; otherwise `u_video` is freshest and simplest.\n\n`analysedFrame` is `null` until the first CV inference completes after a feature is enabled (typically under one second). The `?? currentFrame` fallback covers that startup window, but be aware the source switches when the first inference lands; a brief visual hitch is normal.\n\n`analysedFrame` is populated only on the main stream's CV surface (`viji.video.cv`). CV features can be enabled per stream on `viji.videoStreams[i].cv` with the same verbs, but `analysedFrame` stays `null` there (use `currentFrame`). Device videos (`viji.devices[i].video?.cv`) do not run CV; their `analysedFrame` always reads `null`.\n\n---\n\n## Drawing Video at the Right Aspect Ratio\n\nCamera frames almost never match the canvas aspect ratio. Drawing the video to `(0, 0, viji.width, viji.height)` stretches it visibly: faces get squashed or elongated, circles become ovals, and CV bounding boxes drift off the actual face.\n\nThe fix is a small helper that returns the destination rectangle for `drawImage` (or `p5.image`) and lets you map CV coordinates into the same rectangle. The recipe and full code live on the per-renderer basics pages: [native](/native/video/basics/#drawing-video), [p5](/p5/video/basics/#drawing-video), [shader](/shader/video/basics/#drawing-video). Use those helpers in every video / CV scene.\n\n### Cover vs Contain\n\n| Mode | What it does | Use when |\n|---|---|---|\n| `cover` (default) | Video fills the canvas, edges cropped when aspects mismatch | Live camera feeds, filter-style scenes, anywhere stretching would look wrong but black bars would too |\n| `contain` | Full video shown, letterboxed when aspects mismatch | CV demos where bounding boxes near frame edges must stay visible, body segmentation where the user might be at the edge of the camera view, severe aspect mismatches |\n\n**Default to `cover`.** It matches the convention from CSS `object-fit: cover` and most camera apps: the camera fills the visible area cleanly, and the small edge crop is normal. Switch to `contain` only when losing the edges of the frame would lose meaning, which is mostly CV-overlay scenes.\n\n**Stretching `(0, 0, viji.width, viji.height)` is allowed only when distortion is intentional** (an artistic effect that wants the squash). On every other scene it is a quality bug.\n\n---\n\n## Parameter Categories\n\nParameters that depend on an external input (audio, video / camera / CV, or user interaction) must include the matching `category` so the host UI can show or hide them based on whether that input is active.\n\n| Input | Required `category` |\n|-------|---------------------|\n| Audio-related parameters (volume sensitivity, bass reactivity, beat response) | `'audio'` |\n| Video/camera/CV-related parameters (video opacity, CV sensitivity, segmentation toggles) | `'video'` |\n| Interaction-related parameters (mouse attraction, keyboard speed, touch sensitivity) | `'interaction'` |\n| Everything else (colors, sizes, speeds, shapes) | `'general'` (default: can be omitted) |\n\n**Native / P5:**\n\n```javascript\n// Wrong\nconst bassSensitivity = viji.slider(1.5, { min: 0, max: 3, label: 'Bass Sensitivity', group: 'audio' });\n\n// Right\nconst bassSensitivity = viji.slider(1.5, { min: 0, max: 3, label: 'Bass Sensitivity', group: 'audio', category: 'audio' });\n```\n\n**Shader:** use the `category:` key in `@viji-*` directives:\n\n```glsl\n// @viji-slider:bassSensitivity label:\"Bass Sensitivity\" default:1.5 min:0 max:3 group:audio category:audio\n```\n\n> [!NOTE]\n> `group` controls layout (which section the parameter appears in). `category` controls visibility (whether the parameter is shown at all). They are orthogonal: use both when appropriate.\n\n**See also:** [`native/parameters/categories`](/native/parameters/categories), [`p5/parameters/categories`](/p5/parameters/categories), [`shader/parameters/categories`](/shader/parameters/categories).\n\n---\n\n## No Redundant Input Toggles\n\nThe host UI controls whether audio, video, and interaction inputs are available to the scene. A scene-level on/off toggle for whether to *react to* an already-available input is redundant: muting host audio achieves the same thing as flipping an \"Audio Reactive\" toggle off, and any creative-strength slider at zero is also already an off-switch.\n\n**The rule.** Don't add `viji.toggle` parameters whose only effect is gating whether the scene reads from `viji.audio`, `viji.video`, `viji.mouse`, `viji.keyboard`, or `viji.touch`. Use `isConnected` / `isInCanvas` guards plus a creative-strength parameter (slider, color, etc.) tagged with the matching `category` instead.\n\n```javascript\n// Wrong: redundant on/off toggles\nconst audioReact = viji.toggle(true, { label: 'Audio Reactive', category: 'audio' });\nconst useCamera = viji.toggle(true, { label: 'Use Camera', category: 'video' });\nconst showMouse = viji.toggle(true, { label: 'Mouse Trail', category: 'interaction' });\n\n// Right: creative-strength parameters with the matching category\nconst bassSensitivity = viji.slider(1.5, { min: 0, max: 3, label: 'Bass Sensitivity', category: 'audio' });\nconst videoOpacity = viji.slider(0.6, { min: 0, max: 1, label: 'Video Opacity', category: 'video' });\nconst mouseAttraction = viji.slider(0.5, { min: 0, max: 1, label: 'Mouse Attraction', category: 'interaction' });\n```\n\n**Why.** The host already exposes input switches in its UI. A scene-level \"Audio Reactive (on/off)\" toggle just duplicates that switch one layer down, and any creative-strength slider going to zero (`bassSensitivity = 0`) achieves the off state without an extra control. Two controls for the same thing is friction for the user and a quality bug for our example scenes.\n\n**The exception: CV feature toggles.** `viji.video.cv.enableFaceDetection`, `enableFaceMesh`, `enableHandTracking`, etc. are *not* input toggles; they activate paid features that consume WebGL contexts and ML inference cycles. Those have to stay opt-in. The line: if the toggle's effect is \"should the scene read an already-available input?\" → remove. If it's \"should the runtime spend resources to enable a feature?\" → keep.\n\nIn shader scenes the same rule applies, expressed as a directive: `// @viji-cv:faceDetection label:\"Face Detection\" default:false` (toggleable form) is the shader equivalent of the JS toggle pattern above. Bare `// @viji-cv:faceDetection` is the always-on equivalent and skips the parameter entirely.\n\n**See also:** [`native/parameters/categories`](/native/parameters/categories), [Common Mistakes: Redundant Input Toggles](../common-mistakes/#redundant-input-onoff-toggles).\n\n---\n\n## Canvas Context Selection\n\nUse `viji.useContext()` to obtain a 2D or WebGL context for the scene canvas, not `viji.canvas.getContext()`. A canvas only supports one context type for its entire lifetime: pick one and stay with it.\n\n> [!WARNING]\n> A canvas only supports one context type. If you call `useContext('2d')` and later call `useContext('webgl')` (or vice versa), the second call returns `null`. Choose one context type and use it for the entire scene.\n\n**See also:** [`native/canvas-context`](/native/canvas-context).\n\n---\n\n## Related\n\n- [Common Mistakes](../common-mistakes/): symptom catalog with wrong/right pairs.\n- [Renderers Overview](../renderers-overview/): choosing the right renderer."
1090
1090
  }
1091
1091
  ]
1092
1092
  },
@@ -3157,7 +3157,7 @@ export const docsApi = {
3157
3157
  "content": [
3158
3158
  {
3159
3159
  "type": "text",
3160
- "markdown": "# Video Basics\n\nThe video stream provides a drawable frame each render cycle through `viji.video.currentFrame`. This section covers how to draw the video feed, read its dimensions, and when to use raw pixel data.\n\n## Property Reference\n\n| Property | Type | Description |\n|----------|------|-------------|\n| `viji.video.currentFrame` | `OffscreenCanvas \\| null` | Just-arrived video frame: pass to `ctx.drawImage()` |\n| `viji.video.cv.analysedFrame` | `OffscreenCanvas \\| null` | Frame paired with the current CV results (faces / hands / pose / segmentation). `null` until first CV result. Read-only. |\n| `viji.video.frameWidth` | `number` | Video frame width in pixels |\n| `viji.video.frameHeight` | `number` | Video frame height in pixels |\n| `viji.video.frameRate` | `number` | Video frame rate (Hz) |\n| `viji.video.getFrameData()` | `ImageData \\| null` | Raw RGBA pixels of `currentFrame` for per-pixel analysis |\n| `viji.video.cv.getAnalysedFrameData()` | `ImageData \\| null` | Raw RGBA pixels of `analysedFrame`, paired with the CV results |\n\n### `currentFrame` vs `getFrameData()`\n\n- **`currentFrame`** is an OffscreenCanvas that can be drawn directly with `ctx.drawImage()`. This is fast and GPU-friendly: use it for all rendering.\n- **`getFrameData()`** returns an `ImageData` object for per-pixel CPU analysis. It allocates a new `ImageData` each call and is significantly slower. Use only when you need to read individual pixel values.\n\n> [!TIP]\n> Use `viji.video.currentFrame` for drawing video to canvas (fast, GPU-friendly). Use `viji.video.getFrameData()` only when you need per-pixel access: it is much slower as it reads back pixel data.\n\n### `currentFrame` vs `analysedFrame`\n\nCV results are computed asynchronously from the video stream, so the displayed frame and the CV results are never instantaneously in sync. You always have two frames to choose between, with different failure modes:\n\n- **`currentFrame`** is the just-arrived video frame. It refreshes every frame your scene runs.\n- **`analysedFrame`** is the exact frame MediaPipe ran on to produce the current `viji.video.cv.faces` / `hands` / `pose` / `segmentation` results. It refreshes only when an inference completes, which is not every frame.\n\nPick by asking: *does my effect read pixels from the displayed frame at CV-derived positions?*\n\n- **No** (drawing landmark dots, particles attached to fingertips, geometry driven by CV positions, scenes that don't display the camera): use `currentFrame`. The visible cost is small spatial drift between landmarks and pixels under fast motion. The visible benefit is a smooth live-camera feed.\n- **Yes** (compositing the segmentation mask onto the body, sampling skin tone under a face landmark, warping the face along its mesh, texture-mapped face filters): use `analysedFrame`. The visible cost is that the displayed image stutters or holds briefly between inferences. The visible benefit is that the effect aligns pixel-for-pixel with the CV results.\n\nDrawing landmark dots over `currentFrame` is **not** a mistake; it is the deliberate choice when you want the live-camera feel.\n\n`analysedFrame` is `null` until the first CV inference completes after a feature is enabled. The common pattern is `viji.video.cv.analysedFrame ?? viji.video.currentFrame` to fall back during the brief startup window; expect a small visual hitch when the first inference lands and the source switches. After that it is a stable `OffscreenCanvas` reference until disconnect.\n\n`getAnalysedFrameData()` is the per-pixel-CPU equivalent of `getFrameData()` for `analysedFrame`. Same lazy + cached semantics, useful when you need to sample pixel values aligned with CV landmark positions (e.g. skin tone at a face landmark).\n\n`analysedFrame` is owned by the engine, read-only. Don't mutate it (no painting on it, no `transferToImageBitmap`); use it as a source for `ctx.drawImage`, `gl.texImage2D`, or P5 `image()` only.\n\n> [!NOTE]\n> `analysedFrame` is available only on the main video stream (`viji.video`). Additional streams (`viji.videoStreams[i]`) and device videos (`viji.devices[i].video`) do not run CV and always expose `analysedFrame: null`.\n\n## Drawing Video\n\nCamera frames almost never match the canvas aspect ratio. Drawing video to `(0, 0, viji.width, viji.height)` stretches it: faces get squashed, circles become ovals, and CV bounding boxes drift off the actual face. Use the `videoFit` helper below in every video / CV scene.\n\n```javascript\nfunction videoFit(viji, mode = 'cover') {\n const vw = viji.video.frameWidth, vh = viji.video.frameHeight;\n const w = viji.width, h = viji.height;\n if (!vw || !vh) return { x: 0, y: 0, width: 0, height: 0 };\n const scale = mode === 'cover' ? Math.max(w / vw, h / vh) : Math.min(w / vw, h / vh);\n const dw = vw * scale, dh = vh * scale;\n return { x: (w - dw) / 2, y: (h - dh) / 2, width: dw, height: dh };\n}\n\nfunction render(viji) {\n const ctx = viji.useContext('2d');\n\n ctx.fillStyle = '#111';\n ctx.fillRect(0, 0, viji.width, viji.height);\n\n if (!viji.video.isConnected || !viji.video.currentFrame) return;\n\n const v = videoFit(viji);\n ctx.drawImage(viji.video.currentFrame, v.x, v.y, v.width, v.height);\n}\n```\n\n`mode` is `'cover'` (default) or `'contain'`. Cover fills the canvas with the video, cropping the edges when aspects mismatch. Contain shows the full video with letterbox bars. See [Best Practices](/getting-started/best-practices/#cover-vs-contain) for when each is the right pick. Default to `cover` for live camera feeds; switch to `contain` for CV-overlay scenes where bounding boxes near frame edges must stay visible.\n\n### Aligning CV Coordinates with the Fitted Video\n\n`face.bounds`, `face.landmarks`, `hand.landmarks`, and `pose.landmarks` are normalized to the source video frame, not the canvas. To draw them aligned with the fitted video, multiply by `width` / `height` and offset by `x` / `y`:\n\n```javascript\nconst v = videoFit(viji);\nctx.drawImage(viji.video.currentFrame, v.x, v.y, v.width, v.height);\n\nviji.video.cv.faces.forEach(face => {\n const bx = v.x + face.bounds.x * v.width;\n const by = v.y + face.bounds.y * v.height;\n const bw = face.bounds.width * v.width;\n const bh = face.bounds.height * v.height;\n ctx.strokeRect(bx, by, bw, bh);\n\n face.landmarks.forEach(pt => {\n ctx.fillRect(v.x + pt.x * v.width, v.y + pt.y * v.height, 2, 2);\n });\n});\n```\n\n### Stretching the Frame (Intentional Distortion Only)\n\nIf you actually want to stretch the video as an artistic effect, draw it to `(0, 0, viji.width, viji.height)` directly. Do this only when the distortion is the intent; otherwise it is a quality bug.\n\n```javascript\nctx.drawImage(viji.video.currentFrame, 0, 0, viji.width, viji.height);\n```\n\n## Usage: Raw Pixel Analysis\n\n```javascript\nfunction render(viji) {\n const ctx = viji.useContext('2d');\n const w = viji.width;\n const h = viji.height;\n\n if (!viji.video.isConnected) return;\n\n const frameData = viji.video.getFrameData();\n if (!frameData) return;\n\n const { data, width, height } = frameData;\n let totalBrightness = 0;\n for (let i = 0; i < data.length; i += 4) {\n totalBrightness += (data[i] + data[i + 1] + data[i + 2]) / 3;\n }\n const avg = totalBrightness / (data.length / 4) / 255;\n\n ctx.fillStyle = `hsl(${avg * 360}, 60%, 50%)`;\n ctx.fillRect(0, 0, w, h);\n}\n```\n\n## Coordinate System\n\nAll CV data coordinates are **normalized 0-1**:\n- `x` ranges from 0 (left) to 1 (right)\n- `y` ranges from 0 (top) to 1 (bottom)\n\nTo draw CV data on the canvas, multiply by canvas dimensions: `point.x * viji.width`, `point.y * viji.height`."
3160
+ "markdown": "# Video Basics\n\nThe video stream provides a drawable frame each render cycle through `viji.video.currentFrame`. This section covers how to draw the video feed, read its dimensions, and when to use raw pixel data.\n\n## Property Reference\n\n| Property | Type | Description |\n|----------|------|-------------|\n| `viji.video.currentFrame` | `OffscreenCanvas \\| null` | Just-arrived video frame: pass to `ctx.drawImage()` |\n| `viji.video.cv.analysedFrame` | `OffscreenCanvas \\| null` | Frame paired with the current CV results (faces / hands / pose / segmentation). `null` until first CV result. Read-only. |\n| `viji.video.frameWidth` | `number` | Video frame width in pixels |\n| `viji.video.frameHeight` | `number` | Video frame height in pixels |\n| `viji.video.frameRate` | `number` | Video frame rate (Hz) |\n| `viji.video.getFrameData()` | `ImageData \\| null` | Raw RGBA pixels of `currentFrame` for per-pixel analysis |\n| `viji.video.cv.getAnalysedFrameData()` | `ImageData \\| null` | Raw RGBA pixels of `analysedFrame`, paired with the CV results |\n\n### `currentFrame` vs `getFrameData()`\n\n- **`currentFrame`** is an OffscreenCanvas that can be drawn directly with `ctx.drawImage()`. This is fast and GPU-friendly: use it for all rendering.\n- **`getFrameData()`** returns an `ImageData` object for per-pixel CPU analysis. It allocates a new `ImageData` each call and is significantly slower. Use only when you need to read individual pixel values.\n\n> [!TIP]\n> Use `viji.video.currentFrame` for drawing video to canvas (fast, GPU-friendly). Use `viji.video.getFrameData()` only when you need per-pixel access: it is much slower as it reads back pixel data.\n\n### `currentFrame` vs `analysedFrame`\n\nCV results are computed asynchronously from the video stream, so the displayed frame and the CV results are never instantaneously in sync. You always have two frames to choose between, with different failure modes:\n\n- **`currentFrame`** is the just-arrived video frame. It refreshes every frame your scene runs.\n- **`analysedFrame`** is the exact frame MediaPipe ran on to produce the current `viji.video.cv.faces` / `hands` / `pose` / `segmentation` results. It refreshes only when an inference completes, which is not every frame.\n\nPick by asking: *does my effect read pixels from the displayed frame at CV-derived positions?*\n\n- **No** (drawing landmark dots, particles attached to fingertips, geometry driven by CV positions, scenes that don't display the camera): use `currentFrame`. The visible cost is small spatial drift between landmarks and pixels under fast motion. The visible benefit is a smooth live-camera feed.\n- **Yes** (compositing the segmentation mask onto the body, sampling skin tone under a face landmark, warping the face along its mesh, texture-mapped face filters): use `analysedFrame`. The visible cost is that the displayed image stutters or holds briefly between inferences. The visible benefit is that the effect aligns pixel-for-pixel with the CV results.\n\nDrawing landmark dots over `currentFrame` is **not** a mistake; it is the deliberate choice when you want the live-camera feel.\n\n`analysedFrame` is `null` until the first CV inference completes after a feature is enabled. The common pattern is `viji.video.cv.analysedFrame ?? viji.video.currentFrame` to fall back during the brief startup window; expect a small visual hitch when the first inference lands and the source switches. After that it is a stable `OffscreenCanvas` reference until disconnect.\n\n`getAnalysedFrameData()` is the per-pixel-CPU equivalent of `getFrameData()` for `analysedFrame`. Same lazy + cached semantics, useful when you need to sample pixel values aligned with CV landmark positions (e.g. skin tone at a face landmark).\n\n`analysedFrame` is owned by the engine, read-only. Don't mutate it (no painting on it, no `transferToImageBitmap`); use it as a source for `ctx.drawImage`, `gl.texImage2D`, or P5 `image()` only.\n\n> [!NOTE]\n> `analysedFrame` is available only on the main video stream (`viji.video`). Additional streams (`viji.videoStreams[i]`) support the same CV enable verbs and result fields per stream, but their `analysedFrame` stays `null` (use `currentFrame`). Device videos (`viji.devices[i].video`) do not run CV and always expose `analysedFrame: null`.\n\n## Drawing Video\n\nCamera frames almost never match the canvas aspect ratio. Drawing video to `(0, 0, viji.width, viji.height)` stretches it: faces get squashed, circles become ovals, and CV bounding boxes drift off the actual face. Use the `videoFit` helper below in every video / CV scene.\n\n```javascript\nfunction videoFit(viji, mode = 'cover') {\n const vw = viji.video.frameWidth, vh = viji.video.frameHeight;\n const w = viji.width, h = viji.height;\n if (!vw || !vh) return { x: 0, y: 0, width: 0, height: 0 };\n const scale = mode === 'cover' ? Math.max(w / vw, h / vh) : Math.min(w / vw, h / vh);\n const dw = vw * scale, dh = vh * scale;\n return { x: (w - dw) / 2, y: (h - dh) / 2, width: dw, height: dh };\n}\n\nfunction render(viji) {\n const ctx = viji.useContext('2d');\n\n ctx.fillStyle = '#111';\n ctx.fillRect(0, 0, viji.width, viji.height);\n\n if (!viji.video.isConnected || !viji.video.currentFrame) return;\n\n const v = videoFit(viji);\n ctx.drawImage(viji.video.currentFrame, v.x, v.y, v.width, v.height);\n}\n```\n\n`mode` is `'cover'` (default) or `'contain'`. Cover fills the canvas with the video, cropping the edges when aspects mismatch. Contain shows the full video with letterbox bars. See [Best Practices](/getting-started/best-practices/#cover-vs-contain) for when each is the right pick. Default to `cover` for live camera feeds; switch to `contain` for CV-overlay scenes where bounding boxes near frame edges must stay visible.\n\n### Aligning CV Coordinates with the Fitted Video\n\n`face.bounds`, `face.landmarks`, `hand.landmarks`, and `pose.landmarks` are normalized to the source video frame, not the canvas. To draw them aligned with the fitted video, multiply by `width` / `height` and offset by `x` / `y`:\n\n```javascript\nconst v = videoFit(viji);\nctx.drawImage(viji.video.currentFrame, v.x, v.y, v.width, v.height);\n\nviji.video.cv.faces.forEach(face => {\n const bx = v.x + face.bounds.x * v.width;\n const by = v.y + face.bounds.y * v.height;\n const bw = face.bounds.width * v.width;\n const bh = face.bounds.height * v.height;\n ctx.strokeRect(bx, by, bw, bh);\n\n face.landmarks.forEach(pt => {\n ctx.fillRect(v.x + pt.x * v.width, v.y + pt.y * v.height, 2, 2);\n });\n});\n```\n\n### Stretching the Frame (Intentional Distortion Only)\n\nIf you actually want to stretch the video as an artistic effect, draw it to `(0, 0, viji.width, viji.height)` directly. Do this only when the distortion is the intent; otherwise it is a quality bug.\n\n```javascript\nctx.drawImage(viji.video.currentFrame, 0, 0, viji.width, viji.height);\n```\n\n## Usage: Raw Pixel Analysis\n\n```javascript\nfunction render(viji) {\n const ctx = viji.useContext('2d');\n const w = viji.width;\n const h = viji.height;\n\n if (!viji.video.isConnected) return;\n\n const frameData = viji.video.getFrameData();\n if (!frameData) return;\n\n const { data, width, height } = frameData;\n let totalBrightness = 0;\n for (let i = 0; i < data.length; i += 4) {\n totalBrightness += (data[i] + data[i + 1] + data[i + 2]) / 3;\n }\n const avg = totalBrightness / (data.length / 4) / 255;\n\n ctx.fillStyle = `hsl(${avg * 360}, 60%, 50%)`;\n ctx.fillRect(0, 0, w, h);\n}\n```\n\n## Coordinate System\n\nAll CV data coordinates are **normalized 0-1**:\n- `x` ranges from 0 (left) to 1 (right)\n- `y` ranges from 0 (top) to 1 (bottom)\n\nTo draw CV data on the canvas, multiply by canvas dimensions: `point.x * viji.width`, `point.y * viji.height`."
3161
3161
  },
3162
3162
  {
3163
3163
  "type": "live-example",
@@ -5917,7 +5917,7 @@ export const docsApi = {
5917
5917
  "content": [
5918
5918
  {
5919
5919
  "type": "text",
5920
- "markdown": "# Video Basics\n\nThe video stream provides a drawable frame each render cycle through `viji.video.currentFrame`. In P5.js scenes, draw it using `p5.image()`.\n\n## Property Reference\n\n| Property | Type | Description |\n|----------|------|-------------|\n| `viji.video.currentFrame` | `OffscreenCanvas \\| null` | Just-arrived video frame: pass to `p5.image()` |\n| `viji.video.cv.analysedFrame` | `OffscreenCanvas \\| null` | Frame paired with the current CV results. `null` until first CV result. Read-only. |\n| `viji.video.frameWidth` | `number` | Video frame width in pixels |\n| `viji.video.frameHeight` | `number` | Video frame height in pixels |\n| `viji.video.frameRate` | `number` | Video frame rate (Hz) |\n| `viji.video.getFrameData()` | `ImageData \\| null` | Raw RGBA pixels of `currentFrame` for per-pixel analysis |\n| `viji.video.cv.getAnalysedFrameData()` | `ImageData \\| null` | Raw RGBA pixels of `analysedFrame`, paired with CV results |\n\n### `currentFrame` vs `getFrameData()`\n\n- **`currentFrame`** is an OffscreenCanvas that can be drawn directly with `p5.image()`. This is fast and GPU-friendly: use it for all rendering.\n- **`getFrameData()`** returns an `ImageData` object for per-pixel CPU analysis. It allocates a new `ImageData` each call and is significantly slower. Use only when you need to read individual pixel values.\n\n> [!TIP]\n> Use `viji.video.currentFrame` with `p5.image()` for drawing video (fast, GPU-friendly). Use `viji.video.getFrameData()` only when you need per-pixel access: it is much slower as it reads back pixel data.\n\n### `currentFrame` vs `analysedFrame`\n\nCV results are computed asynchronously, so the displayed frame and the CV results are never instantaneously in sync. You always have two frames to choose between:\n\n- **`currentFrame`** is the just-arrived video frame. It refreshes every frame your scene runs.\n- **`analysedFrame`** is the exact frame MediaPipe ran on to produce the current CV results. It refreshes only when an inference completes, which is not every frame.\n\nPick by asking: *does my effect read pixels from the displayed frame at CV-derived positions?*\n\n- **No** (drawing landmark dots, particles attached to fingertips, geometry driven by CV positions, scenes that don't display the camera): use `currentFrame`. The cost is small spatial drift between landmarks and pixels under fast motion. The benefit is a smooth live-camera feed.\n- **Yes** (compositing the segmentation mask onto the body, sampling skin tone under a face landmark, warping the face along its mesh): use `analysedFrame`. The cost is that the displayed image stutters or holds briefly between inferences. The benefit is pixel-for-pixel alignment with the CV results.\n\n```javascript\n// Live camera with landmark dots: currentFrame is the right default\np5.image(viji.video.currentFrame, 0, 0, viji.width, viji.height);\nviji.video.cv.faces.forEach(face => { /* dots */ });\n\n// Mask compositing or pixel sampling under landmarks: analysedFrame is required\nconst frame = viji.video.cv.analysedFrame ?? viji.video.currentFrame;\np5.image(frame, 0, 0, viji.width, viji.height);\n// composite mask, or sample skin tone at landmark positions...\n```\n\n`analysedFrame` is `null` until the first CV result. The `??` fallback covers the startup window; expect a small visual hitch when the first inference lands and the source switches.\n\n`analysedFrame` is read-only. Don't mutate it. It's available only on `viji.video` (the main stream); additional streams and device videos always expose `null`.\n\n## Drawing Video\n\nCamera frames almost never match the canvas aspect ratio. Drawing video to `(0, 0, viji.width, viji.height)` stretches it: faces get squashed, circles become ovals, and CV bounding boxes drift off the actual face. Use the `videoFit` helper below in every video / CV scene.\n\n```javascript\n// @renderer p5\n\nfunction videoFit(viji, mode = 'cover') {\n const vw = viji.video.frameWidth, vh = viji.video.frameHeight;\n const w = viji.width, h = viji.height;\n if (!vw || !vh) return { x: 0, y: 0, width: 0, height: 0 };\n const scale = mode === 'cover' ? Math.max(w / vw, h / vh) : Math.min(w / vw, h / vh);\n const dw = vw * scale, dh = vh * scale;\n return { x: (w - dw) / 2, y: (h - dh) / 2, width: dw, height: dh };\n}\n\nfunction render(viji, p5) {\n p5.background(17);\n\n if (!viji.video.isConnected || !viji.video.currentFrame) return;\n\n const v = videoFit(viji);\n p5.image(viji.video.currentFrame, v.x, v.y, v.width, v.height);\n}\n```\n\n`mode` is `'cover'` (default) or `'contain'`. Cover fills the canvas with the video, cropping the edges when aspects mismatch. Contain shows the full video with letterbox bars. See [Best Practices](/getting-started/best-practices/#cover-vs-contain) for when each is the right pick. Default to `cover` for live camera feeds; switch to `contain` for CV-overlay scenes where bounding boxes near frame edges must stay visible.\n\n### Aligning CV Coordinates with the Fitted Video\n\n`face.bounds`, `face.landmarks`, `hand.landmarks`, and `pose.landmarks` are normalized to the source video frame, not the canvas. To draw them aligned with the fitted video, multiply by `width` / `height` and offset by `x` / `y`:\n\n```javascript\nconst v = videoFit(viji);\np5.image(viji.video.currentFrame, v.x, v.y, v.width, v.height);\n\nviji.video.cv.faces.forEach(face => {\n const bx = v.x + face.bounds.x * v.width;\n const by = v.y + face.bounds.y * v.height;\n const bw = face.bounds.width * v.width;\n const bh = face.bounds.height * v.height;\n p5.noFill();\n p5.stroke(78, 205, 196);\n p5.rect(bx, by, bw, bh);\n\n face.landmarks.forEach(pt => {\n p5.circle(v.x + pt.x * v.width, v.y + pt.y * v.height, 2);\n });\n});\n```\n\n### Stretching the Frame (Intentional Distortion Only)\n\nIf you actually want to stretch the video as an artistic effect, pass `(0, 0, viji.width, viji.height)` directly. Do this only when the distortion is the intent; otherwise it is a quality bug.\n\n```javascript\np5.image(viji.video.currentFrame, 0, 0, viji.width, viji.height);\n```\n\n## Coordinate System\n\nAll CV data coordinates are **normalized 0-1** relative to the source video frame:\n- `x` ranges from 0 (left) to 1 (right)\n- `y` ranges from 0 (top) to 1 (bottom)\n\nTo draw CV data aligned with the fitted video, use the helper above to map them through the `videoFit` rectangle. Multiplying by `viji.width` / `viji.height` directly is only correct if you stretch the video to fill the canvas, which is rarely the right thing to do."
5920
+ "markdown": "# Video Basics\n\nThe video stream provides a drawable frame each render cycle through `viji.video.currentFrame`. In P5.js scenes, draw it using `p5.image()`.\n\n## Property Reference\n\n| Property | Type | Description |\n|----------|------|-------------|\n| `viji.video.currentFrame` | `OffscreenCanvas \\| null` | Just-arrived video frame: pass to `p5.image()` |\n| `viji.video.cv.analysedFrame` | `OffscreenCanvas \\| null` | Frame paired with the current CV results. `null` until first CV result. Read-only. |\n| `viji.video.frameWidth` | `number` | Video frame width in pixels |\n| `viji.video.frameHeight` | `number` | Video frame height in pixels |\n| `viji.video.frameRate` | `number` | Video frame rate (Hz) |\n| `viji.video.getFrameData()` | `ImageData \\| null` | Raw RGBA pixels of `currentFrame` for per-pixel analysis |\n| `viji.video.cv.getAnalysedFrameData()` | `ImageData \\| null` | Raw RGBA pixels of `analysedFrame`, paired with CV results |\n\n### `currentFrame` vs `getFrameData()`\n\n- **`currentFrame`** is an OffscreenCanvas that can be drawn directly with `p5.image()`. This is fast and GPU-friendly: use it for all rendering.\n- **`getFrameData()`** returns an `ImageData` object for per-pixel CPU analysis. It allocates a new `ImageData` each call and is significantly slower. Use only when you need to read individual pixel values.\n\n> [!TIP]\n> Use `viji.video.currentFrame` with `p5.image()` for drawing video (fast, GPU-friendly). Use `viji.video.getFrameData()` only when you need per-pixel access: it is much slower as it reads back pixel data.\n\n### `currentFrame` vs `analysedFrame`\n\nCV results are computed asynchronously, so the displayed frame and the CV results are never instantaneously in sync. You always have two frames to choose between:\n\n- **`currentFrame`** is the just-arrived video frame. It refreshes every frame your scene runs.\n- **`analysedFrame`** is the exact frame MediaPipe ran on to produce the current CV results. It refreshes only when an inference completes, which is not every frame.\n\nPick by asking: *does my effect read pixels from the displayed frame at CV-derived positions?*\n\n- **No** (drawing landmark dots, particles attached to fingertips, geometry driven by CV positions, scenes that don't display the camera): use `currentFrame`. The cost is small spatial drift between landmarks and pixels under fast motion. The benefit is a smooth live-camera feed.\n- **Yes** (compositing the segmentation mask onto the body, sampling skin tone under a face landmark, warping the face along its mesh): use `analysedFrame`. The cost is that the displayed image stutters or holds briefly between inferences. The benefit is pixel-for-pixel alignment with the CV results.\n\n```javascript\n// Live camera with landmark dots: currentFrame is the right default\np5.image(viji.video.currentFrame, 0, 0, viji.width, viji.height);\nviji.video.cv.faces.forEach(face => { /* dots */ });\n\n// Mask compositing or pixel sampling under landmarks: analysedFrame is required\nconst frame = viji.video.cv.analysedFrame ?? viji.video.currentFrame;\np5.image(frame, 0, 0, viji.width, viji.height);\n// composite mask, or sample skin tone at landmark positions...\n```\n\n`analysedFrame` is `null` until the first CV result. The `??` fallback covers the startup window; expect a small visual hitch when the first inference lands and the source switches.\n\n`analysedFrame` is read-only. Don't mutate it. It's populated only on `viji.video` (the main stream); additional streams support the same CV verbs and results per stream but expose `analysedFrame: null` (use `currentFrame`), and device videos do not run CV.\n\n## Drawing Video\n\nCamera frames almost never match the canvas aspect ratio. Drawing video to `(0, 0, viji.width, viji.height)` stretches it: faces get squashed, circles become ovals, and CV bounding boxes drift off the actual face. Use the `videoFit` helper below in every video / CV scene.\n\n```javascript\n// @renderer p5\n\nfunction videoFit(viji, mode = 'cover') {\n const vw = viji.video.frameWidth, vh = viji.video.frameHeight;\n const w = viji.width, h = viji.height;\n if (!vw || !vh) return { x: 0, y: 0, width: 0, height: 0 };\n const scale = mode === 'cover' ? Math.max(w / vw, h / vh) : Math.min(w / vw, h / vh);\n const dw = vw * scale, dh = vh * scale;\n return { x: (w - dw) / 2, y: (h - dh) / 2, width: dw, height: dh };\n}\n\nfunction render(viji, p5) {\n p5.background(17);\n\n if (!viji.video.isConnected || !viji.video.currentFrame) return;\n\n const v = videoFit(viji);\n p5.image(viji.video.currentFrame, v.x, v.y, v.width, v.height);\n}\n```\n\n`mode` is `'cover'` (default) or `'contain'`. Cover fills the canvas with the video, cropping the edges when aspects mismatch. Contain shows the full video with letterbox bars. See [Best Practices](/getting-started/best-practices/#cover-vs-contain) for when each is the right pick. Default to `cover` for live camera feeds; switch to `contain` for CV-overlay scenes where bounding boxes near frame edges must stay visible.\n\n### Aligning CV Coordinates with the Fitted Video\n\n`face.bounds`, `face.landmarks`, `hand.landmarks`, and `pose.landmarks` are normalized to the source video frame, not the canvas. To draw them aligned with the fitted video, multiply by `width` / `height` and offset by `x` / `y`:\n\n```javascript\nconst v = videoFit(viji);\np5.image(viji.video.currentFrame, v.x, v.y, v.width, v.height);\n\nviji.video.cv.faces.forEach(face => {\n const bx = v.x + face.bounds.x * v.width;\n const by = v.y + face.bounds.y * v.height;\n const bw = face.bounds.width * v.width;\n const bh = face.bounds.height * v.height;\n p5.noFill();\n p5.stroke(78, 205, 196);\n p5.rect(bx, by, bw, bh);\n\n face.landmarks.forEach(pt => {\n p5.circle(v.x + pt.x * v.width, v.y + pt.y * v.height, 2);\n });\n});\n```\n\n### Stretching the Frame (Intentional Distortion Only)\n\nIf you actually want to stretch the video as an artistic effect, pass `(0, 0, viji.width, viji.height)` directly. Do this only when the distortion is the intent; otherwise it is a quality bug.\n\n```javascript\np5.image(viji.video.currentFrame, 0, 0, viji.width, viji.height);\n```\n\n## Coordinate System\n\nAll CV data coordinates are **normalized 0-1** relative to the source video frame:\n- `x` ranges from 0 (left) to 1 (right)\n- `y` ranges from 0 (top) to 1 (bottom)\n\nTo draw CV data aligned with the fitted video, use the helper above to map them through the `videoFit` rectangle. Multiplying by `viji.width` / `viji.height` directly is only correct if you stretch the video to fill the canvas, which is rarely the right thing to do."
5921
5921
  },
5922
5922
  {
5923
5923
  "type": "live-example",
@@ -9849,8 +9849,8 @@ export const docsApi = {
9849
9849
  "ai": {
9850
9850
  "systemPrompts": {
9851
9851
  "base": "# Viji API: Base Reference\n\nThis document is the renderer-agnostic foundation for generating Viji scenes. It is loaded into every AI turn alongside the renderer-specific reference for the renderer in use.\n\n## Architecture\n\nViji scenes run inside a **Web Worker** on an **OffscreenCanvas**. There is no DOM.\n\n- Top-level code runs once when the scene loads (parameter declarations, state, imports).\n- A `render` function (or, for shaders, `void main()`) runs every frame.\n- `fetch()` and `await import()` are available. `window`, `document`, `Image()`, and `localStorage` are not.\n- The global `viji` object exposes everything: canvas, timing, audio, video, computer vision, input, sensors, and parameters. Each renderer also has its own surface (P5's `p5.*` API; the shader's auto-injected uniforms).\n\n## Renderers\n\nThree renderers. Pick by the visual goal, not by perceived difficulty.\n\n| | Native | P5 | Shader |\n|---|---|---|---|\n| Language | JavaScript (Canvas 2D / WebGL) | JavaScript + P5.js 1.9.4 | GLSL fragment shader |\n| Best for | Full control, Three.js, generative art, pixel-perfect Canvas 2D | Creative coding with shapes, colors, transforms; ports of existing P5 sketches | GPU effects, patterns, raymarching, SDF, post-processing |\n| 3D | Yes (WebGL, Three.js via `await import()`) | Yes (`// @renderer p5 webgl`) | Yes (raymarching, SDF) |\n| External libraries | Yes (any ESM via `await import()`) | P5.js built-in | None (GPU only) |\n\n### Decision criteria\n\n- **Per-pixel GPU effects** (raymarching, fractals, distance fields, fullscreen post-processing, patterns that read every pixel): use **Shader**.\n- **3D with Three.js, custom WebGL, pixel-perfect Canvas 2D, or any external ESM library**: use **Native**.\n- **Classic P5 shapes and transforms** (`ellipse`, `line`, `bezier`, color modes, `push`/`pop`), or a port of an existing P5 sketch: use **P5**.\n- **When ambiguous, default to Native**. Audio reactivity, camera/CV, parameters, and input work in every renderer. Pick on visual style, not data sources.\n\n## Scene shape per renderer\n\nEach example below is the minimum viable entry point. Full surface lives in the matching renderer reference.\n\n**Native**:\n```javascript\nconst speed = viji.slider(1, { min: 0.1, max: 5, label: 'Speed' });\nlet angle = 0;\nfunction render(viji) {\n const ctx = viji.useContext('2d');\n angle += speed.value * viji.deltaTime;\n ctx.clearRect(0, 0, viji.width, viji.height);\n // draw with ctx\n}\n```\n\n**P5**:\n```javascript\n// @renderer p5\nconst speed = viji.slider(1, { min: 0.1, max: 5, label: 'Speed' });\nlet angle = 0;\nfunction setup(viji, p5) { p5.colorMode(p5.HSB, 360, 100, 100); }\nfunction render(viji, p5) {\n angle += speed.value * viji.deltaTime;\n p5.background(0);\n // draw with p5.* prefixed calls\n}\n```\n\n**Shader (GLSL)**:\n```glsl\n// @renderer shader\n// @viji-slider:speed label:\"Speed\" default:1.0 min:0.1 max:5.0\n// @viji-accumulator:phase rate:speed\nvoid main() {\n vec2 uv = gl_FragCoord.xy / u_resolution;\n gl_FragColor = vec4(uv, sin(phase) * 0.5 + 0.5, 1.0);\n}\n```\n\n## Cross-cutting rules (apply to every renderer)\n\n1. **Never** touch the DOM. No `window`, `document`, `Image()`, `localStorage`. `fetch()` and `await import()` are available.\n2. **Always** declare parameters at the **top level**, never inside `render()` or `main()`. Parameters declared inside a render call are recreated every frame and break the host UI.\n3. **Always** use `viji.width` / `viji.height` (or `u_resolution` in shaders) for canvas dimensions. Never hardcode pixel sizes.\n4. **Always** use `viji.deltaTime` (or `u_deltaTime` / `@viji-accumulator` in shaders) for animation. Never count frames or assume a fixed frame rate.\n5. **Never** multiply `viji.time` (or `u_time`) by a parameter to drive animation speed. It causes visible jumps when the slider changes. Use a `deltaTime` accumulator at the top level instead. In shaders, use `@viji-accumulator`. This also applies to nested multiplications: never multiply an accumulator by another parameter. Each independent speed gets its own accumulator.\n6. **Never** allocate objects, arrays, or strings inside `render()`. Pre-allocate at the top level and reuse.\n7. **Always** check `viji.audio.isConnected` (or `u_audioVolume > 0.0`) before reading audio data. Check `viji.video.isConnected && viji.video.currentFrame` (or `u_videoConnected`) before reading video.\n8. **CV features are independent**: each `enableX()` populates only its own subset of fields. Face detection populates `face.bounds` / `center` / `confidence`; face mesh populates `face.landmarks` / `headPose`; emotion detection populates `face.expressions` / `blendshapes`. Fields whose source model is not enabled read as `null` (or `[]` for `landmarks`) — always null-check before reading. Enable from module scope for always-on CV scenes, or per-frame inside `render()` gated by a toggle parameter for opt-in CV (canonical pattern). Each active CV feature uses its own WebGL context; 4+ on low-end hardware can hit context limits, so enable only what you need.\n9. **Always** set `category` on parameters that depend on an external input: `category: 'audio'` for audio-driven controls, `category: 'video'` for video / CV-driven controls, `category: 'interaction'` for mouse / keyboard / touch controls. Omit `category` (defaults to `'general'`) for parameters that work without external input. Use creative-strength sliders, not on/off toggles, for inputs the host already gates.\n10. **In P5**: prefix every P5 function and constant with `p5.` (instance mode). Never call `createCanvas()`; Viji creates the canvas.\n11. **In shaders**: never redeclare `precision`, never redeclare built-in uniforms (`u_time`, `u_resolution`, etc.), never redeclare parameter uniforms. Never use the `u_` prefix for your own parameter names; it is reserved for Viji's built-ins.\n\n## Parameter declaration shape (cross-renderer)\n\nNative and P5 use JavaScript calls at the top level:\n\n```javascript\nviji.slider(default, { min, max, step, label, group, category }) // .value: number\nviji.color(default, { label }) // .value: '#rrggbb', .rgb, .hsb\nviji.toggle(default, { label }) // .value: boolean\nviji.select(default, { options, label }) // .value: string | number\nviji.number(default, { min, max, step, label }) // .value: number\nviji.text(default, { label, maxLength }) // .value: string\nviji.image(null, { label }) // .value: ImageBitmap | null\nviji.button({ label }) // .value: boolean (true for 1 frame)\nviji.coordinate({ x: 0, y: 0 }, { step, label, group, category }) // .value: { x, y }, range -1..1\n```\n\n**`slider` / `number` `step` default**: when omitted, `step` auto-derives from `min`/`max` as the largest power of 10 giving ~100 positions across the range (e.g., range `0..1` → `0.01`; range `0.1..5` → `0.01`; range `0..100` → `1`). Specify explicitly only when you need a different grid: `step: 1` for integer counts (`{ min: 3, max: 30, step: 1 }`), `step: 0.001` for fine precision.\n\nShaders use GLSL comment directives that compile to typed uniforms:\n\n```glsl\n// @viji-slider:name label:\"Label\" default:1.0 min:0.0 max:5.0 -> uniform float name;\n// @viji-color:name label:\"Color\" default:#ff6600 -> uniform vec3 name;\n// @viji-toggle:name label:\"Toggle\" default:false -> uniform bool name;\n// @viji-select:name label:\"Mode\" default:0 options:[\"A\",\"B\"] -> uniform int name;\n// @viji-number:name label:\"Count\" default:10.0 min:1.0 max:100.0 -> uniform float name;\n// @viji-image:name label:\"Texture\" -> uniform sampler2D name;\n// @viji-button:name label:\"Reset\" -> uniform bool name;\n// @viji-coordinate:name label:\"Pos\" default:[0.0,0.0] -> uniform vec2 name; // range -1..1\n// @viji-accumulator:name rate:speed -> uniform float name;\n```\n\nDirectives use `//` line comments only. Block comments (`/* */`) are not parsed.\n\n## Where to look next\n\nFor the full API surface of the renderer in use, consult the matching reference: Native, P5, or Shader. Each renderer reference includes the complete data shapes for audio, video, CV, input, sensors, external devices, and renderer-specific patterns (e.g. `viji.useContext` for Native, `videoFit` for P5, the complete uniform table for Shader).\n",
9852
- "native": "# Viji API: Native Renderer Reference\n\nNative scenes run JavaScript on an `OffscreenCanvas` in a Web Worker, with full access to Canvas 2D, WebGL, and WebGL2 contexts. External libraries load via `await import()`. Top-level code runs once; a `render(viji)` function runs every frame.\n\n## Architecture\n\n- The global `viji` object exposes canvas, timing, audio, video, CV, input, sensors, parameters.\n- Call `viji.useContext('2d' | 'webgl' | 'webgl2')` once at the top of `render()` (or top-level) to obtain a rendering context. After the first call, `viji.ctx` / `viji.gl` shortcuts are populated.\n- **Pick one context type for the entire scene.** Calling `useContext` with a different type after the first call returns `null`.\n- There is **no `setup()` function** in Native. All initialization goes at the top level (which supports `await` for dynamic imports).\n- The DOM is unavailable. `fetch()` and `await import()` are available.\n\n## Rules\n\n1. **Never** access `window`, `document`, `Image()`, `localStorage`, or any DOM API. `fetch()` and `await import()` are available.\n2. **Always** declare parameters at the top level, never inside `render()`.\n3. **Always** read parameters via `.value`. Color parameters also expose `.rgb` (`{ r, g, b }` in 0..255) and `.hsb` (`{ h, s, b }` with `h` in 0..360, `s` / `b` in 0..100). Color defaults accept hex (`'#ff6600'`, `'#f60'`), `{ r, g, b }` (0..255), `{ h, s, b }` (0..360 / 0..100), or CSS `'rgb(...)'` / `'hsl(...)'` strings.\n4. **Always** use `viji.width` and `viji.height` for canvas dimensions. Never hardcode pixel sizes.\n5. **Always** use `viji.deltaTime` for animation. Use `viji.time` only for constant-speed oscillations (`Math.sin(viji.time * 2.0)`). For anything driven by a parameter, use a `deltaTime` accumulator.\n6. **Never** multiply `viji.time` by a parameter; it causes visible jumps when the parameter changes. Use a `deltaTime` accumulator at the top level. This also applies to nested multiplications: each independent speed gets its own accumulator.\n\n ```javascript\n // WRONG\n const t = viji.time * speed.value;\n const rot = phase * rotSpeed.value;\n\n // RIGHT\n let phase = 0, rotPhase = 0;\n phase += speed.value * viji.deltaTime;\n rotPhase += speed.value * rotSpeed.value * viji.deltaTime;\n ```\n7. **Never** allocate objects, arrays, or strings inside `render()`. Pre-allocate at the top level and reuse.\n8. **Always** call `viji.useContext()` to get a context. Pick one type and use it for the entire scene.\n9. **Always** check `viji.audio.isConnected` before reading audio data. Check `viji.video.isConnected && viji.video.currentFrame` before drawing video.\n10. **Never** enable CV features by default. Use a toggle parameter so the user opts in.\n11. **Always** set `category` on parameters that depend on an external input: `'audio'`, `'video'`, `'interaction'`. Use creative-strength sliders (sensitivity, intensity), not on/off toggles, for inputs the host already gates. CV feature toggles (`enableFaceDetection` etc.) are the exception and stay opt-in.\n12. **Always** type any function parameter that receives the `viji` instance, including `render(viji)` and any helpers that take `viji`. Monaco's TS checker flags API misuse (wrong nesting, typo'd property names) at edit time so the error can be corrected before runtime. Scene code is transpiled via sucrase before execution (`transforms: ['typescript']`), so TS annotations are stripped at runtime — they are safe to include.\n\n ```javascript\n function render(viji: VijiAPI) { ... }\n function applyGlow(viji: VijiAPI, intensity: number) { ... }\n ```\n\n## API reference\n\n### Canvas and context\n\n| Member | Type | Description |\n|---|---|---|\n| `viji.canvas` | `OffscreenCanvas` | The canvas |\n| `viji.useContext('2d')` | `OffscreenCanvasRenderingContext2D` | Acquire 2D context |\n| `viji.useContext('webgl')` | `WebGLRenderingContext` | Acquire WebGL 1 |\n| `viji.useContext('webgl2')` | `WebGL2RenderingContext` | Acquire WebGL 2 |\n| `viji.ctx` | `OffscreenCanvasRenderingContext2D` | Shortcut after `useContext('2d')` |\n| `viji.gl` | `WebGLRenderingContext \\| WebGL2RenderingContext` | Shortcut after WebGL context acquired |\n| `viji.width`, `viji.height` | `number` | Current canvas dimensions in pixels |\n\n### Timing\n\n| Member | Type | Description |\n|---|---|---|\n| `viji.time` | `number` | Seconds since scene start |\n| `viji.deltaTime` | `number` | Seconds since last frame |\n| `viji.frameCount` | `number` | Total frames rendered |\n| `viji.fps` | `number` | Target frame rate (host-controlled) |\n\n### Parameters\n\nDeclare at top level. All accept `{ label, description?, group?, category? }`. Category values: `'audio'`, `'video'`, `'interaction'`, `'general'` (default).\n\n```javascript\nviji.slider(default, { min?, max?, step?, label, group?, category? }) // .value: number\nviji.color(default, { label, group?, category? }) // .value: '#rrggbb', .rgb: {r,g,b} 0..255, .hsb: {h:0..360, s/b:0..100}\nviji.toggle(default, { label, group?, category? }) // .value: boolean\nviji.select(default, { options, label, group?, category? }) // .value: string | number\nviji.number(default, { min?, max?, step?, label, group?, category? }) // .value: number\nviji.text(default, { label, group?, category?, maxLength? }) // .value: string\nviji.image(null, { label, group?, category? }) // .value: ImageBitmap | null\nviji.button({ label, description?, group?, category? }) // .value: boolean (true for 1 frame)\nviji.coordinate(default, { step?, label, group?, category? }) // .value: { x, y }, range -1..1\n// slider/number `step` defaults to a power of 10 giving ~100 positions across min..max\n// (e.g., range 0..1 → 0.01, range 0..100 → 1). Specify `step: 1` for integer counts.\n```\n\n### Audio: `viji.audio`\n\nCheck `isConnected` first.\n\n| Member | Type | Notes |\n|---|---|---|\n| `isConnected` | `boolean` | Audio source active |\n| `volume.current`, `volume.peak`, `volume.smoothed` | `number` 0..1 | `smoothed` = 200ms decay envelope of `current` |\n| `bands.low`, `lowMid`, `mid`, `highMid`, `high` | `number` 0..1 | Instant band energies (20-120, 120-400, 400-1600, 1600-6000, 6000-16000 Hz) |\n| `bands.lowSmoothed`, `lowMidSmoothed`, `midSmoothed`, `highMidSmoothed`, `highSmoothed` | `number` 0..1 | 150ms decay envelope siblings (not nested) |\n| `beat.kick`, `snare`, `hat`, `any` | `number` 0..1 | Beat energy curves with a 300ms decay; peak on each detected beat then fall off |\n| `beat.kickSmoothed`, `snareSmoothed`, `hatSmoothed`, `anySmoothed` | `number` 0..1 | Smoother 500ms decay envelopes; use for ambient pulses |\n| `beat.triggers.kick`, `snare`, `hat`, `any` | `boolean` | True for exactly one frame on the matching beat, then auto-resets. OR-accumulated between frames so no beat is lost |\n| `beat.events` | `Array<{type, time, strength}>` | All beats detected since the last frame; `type` is `'kick' \\| 'snare' \\| 'hat'` (never `'any'`); `time` in milliseconds; cleared each frame |\n| `beat.bpm`, `beat.confidence`, `beat.isLocked` | `number`, `number`, `boolean` | `bpm` is `0` when no audio is connected; once audio connects it tracks the detected tempo (clamped to 60..240) with `120` as a fallback before lock-on. `confidence` in 0..1. `isLocked` is `true` on stable tempo lock |\n| `spectral.brightness`, `spectral.flatness` | `number` 0..1 | Spectral centroid; spectral flatness |\n| `getFrequencyData()` | `Uint8Array` | Raw FFT bins 0..255 |\n| `getWaveform()` | `Float32Array` | Time-domain waveform -1..1 |\n\nExternal-device audio (`viji.devices[i].audio`) and host-supplied audio streams (`viji.audioStreams[i]`) use the `AudioStreamAPI` shape: `isConnected`, `volume.{current,peak,smoothed}`, `bands.{low...high}` plus each `*Smoothed` sibling, `spectral.{brightness,flatness}`, `getFrequencyData()`, `getWaveform()`. No `beat`, BPM, triggers, or events on streams: those exist only on the main `viji.audio`.\n\n### Video: `viji.video`\n\nCheck `isConnected && currentFrame` before drawing.\n\n| Member | Type | Notes |\n|---|---|---|\n| `isConnected` | `boolean` | Video source active |\n| `currentFrame` | `OffscreenCanvas \\| ImageBitmap \\| null` | Most recent frame |\n| `frameWidth`, `frameHeight` | `number` | Source frame size |\n| `frameRate` | `number` | Source frame rate |\n| `getFrameData()` | `ImageData \\| null` | Pixel data of `currentFrame` |\n| `cv` | `VideoCVAPI` | Computer-vision surface: see below |\n\nCV data outputs (`analysedFrame`, `getAnalysedFrameData()`, `faces`, `hands`, `pose`, `segmentation`) and CV verbs (`enableFaceDetection`, etc.) all live on `viji.video.cv`, not on `viji.video` directly.\n\n**Aspect ratio.** Camera frames almost never match the canvas aspect. Drawing to `(0, 0, viji.width, viji.height)` stretches video and misaligns CV bounds. Use this helper at module scope:\n\n```javascript\nfunction videoFit(viji, mode = 'cover') {\n const vw = viji.video.frameWidth, vh = viji.video.frameHeight;\n const w = viji.width, h = viji.height;\n if (!vw || !vh) return { x: 0, y: 0, width: 0, height: 0 };\n const scale = mode === 'cover' ? Math.max(w / vw, h / vh) : Math.min(w / vw, h / vh);\n const dw = vw * scale, dh = vh * scale;\n return { x: (w - dw) / 2, y: (h - dh) / 2, width: dw, height: dh };\n}\n\nconst v = videoFit(viji); // 'cover' or 'contain'\nctx.drawImage(viji.video.currentFrame, v.x, v.y, v.width, v.height);\n\n// CV coords are normalized 0..1 to the source frame. Map through v to align with the fitted video:\nviji.video.cv.faces.forEach(face => {\n const bx = v.x + face.bounds.x * v.width;\n const by = v.y + face.bounds.y * v.height;\n const bw = face.bounds.width * v.width;\n const bh = face.bounds.height * v.height;\n ctx.strokeRect(bx, by, bw, bh);\n});\n```\n\nDefault to `'cover'` for live cameras. Use `'contain'` for CV-overlay scenes where bounding boxes near frame edges must stay visible. `viji.video.currentFrame` is the right default for displayed video; reach for `viji.video.cv.analysedFrame` only when sampling pixels at CV-derived positions (segmentation compositing, face-mesh texturing, landmark-anchored color sampling). The common fallback pattern is `viji.video.cv.analysedFrame ?? viji.video.currentFrame` during the brief window before the first CV inference lands.\n\n### Computer Vision: `viji.video.cv`\n\nEach CV feature is independent and populates only its own subset of fields. Enable only what you need (each active feature consumes a separate WebGL context for MediaPipe; 4+ on low-end hardware risks context-limit failures).\n\n```javascript\n// Verbs return Promise<void>. await is optional — without it, the model loads\n// asynchronously while render runs and data fields populate when ready.\n// Safe to call from module scope (always-on CV) or per-frame inside render()\n// gated by a viji.toggle(...) (opt-in CV). Idempotent + reference-counted.\nawait viji.video.cv.enableFaceDetection(true | false); // populates face.bounds, face.center, face.confidence\nawait viji.video.cv.enableFaceMesh(true | false); // populates face.landmarks (468 pts) + face.headPose\nawait viji.video.cv.enableEmotionDetection(true | false); // populates face.expressions + face.blendshapes; also loads landmarker (face.landmarks + face.headPose populated)\nawait viji.video.cv.enableHandTracking(true | false); // populates viji.video.cv.hands[]\nawait viji.video.cv.enablePoseDetection(true | false); // populates viji.video.cv.pose\nawait viji.video.cv.enableBodySegmentation(true | false); // populates viji.video.cv.segmentation\nviji.video.cv.getActiveFeatures(); // CVFeature[]\nviji.video.cv.isProcessing(); // boolean\n```\n\nData outputs on `viji.video.cv`:\n- `analysedFrame: OffscreenCanvas | null`: the exact frame paired with the current CV results. `null` until the first inference lands after a feature is enabled. Use the pattern `viji.video.cv.analysedFrame ?? viji.video.currentFrame` to fall back during the startup window.\n- `getAnalysedFrameData(): ImageData | null`: pixel data of `analysedFrame`, cached and re-extracted only on new CV results.\n\n**`viji.video.cv.faces: FaceData[]`**: `id` always present. Other fields are populated only by their source model and read `null` (or empty array for `landmarks`) otherwise — always null-check before reading.\n- `bounds: {x,y,width,height} | null` (normalized 0..1) — populated by face detection\n- `center: {x,y} | null` (normalized 0..1) — populated by face detection\n- `confidence: number | null` (0..1) — populated by face detection\n- `landmarks: {x,y,z?}[]` (empty `[]` unless face mesh enabled; 468 pts when populated)\n- `headPose: { pitch, yaw, roll } | null` (degrees) — populated by face mesh\n- `expressions: { neutral, happy, sad, angry, surprised, disgusted, fearful } | null` (0..1 each) — populated by emotion detection\n- `blendshapes | null` — populated by emotion detection; 52 ARKit coefficients (`browDownLeft`/`Right`, `browInnerUp`, `browOuterUpLeft`/`Right`, `cheekPuff`, `cheekSquintLeft`/`Right`, `eyeBlinkLeft`/`Right`, `eyeLookDown`/`In`/`Out`/`UpLeft`/`Right`, `eyeSquintLeft`/`Right`, `eyeWideLeft`/`Right`, `jawForward`/`Left`/`Open`/`Right`, `mouthClose`/`DimpleLeft`/`Right`/`FrownLeft`/`Right`/`Funnel`/`Left`/`LowerDownLeft`/`Right`/`PressLeft`/`Right`/`Pucker`/`Right`/`RollLower`/`Upper`/`ShrugLower`/`Upper`/`SmileLeft`/`Right`/`StretchLeft`/`Right`/`UpperUpLeft`/`Right`, `noseSneerLeft`/`Right`, `tongueOut`: all 0..1)\n\nIf you need bounds while only running face mesh, either also enable face detection or compute the bounding box from `face.landmarks` min/max yourself.\n\n**`viji.video.cv.hands: HandData[]`**: `id`, `handedness: 'left' | 'right'`, `confidence`, `bounds`, `landmarks: {x,y,z}[]` (21 points), `palm: {x,y,z}`, `gestures: { fist, openPalm, peace, thumbsUp, thumbsDown, pointing, iLoveYou }` (0..1 confidence each).\n\n**`viji.video.cv.pose: PoseData | null`**: `confidence`, `landmarks: {x,y,z,visibility}[]` (33 points), plus body-part arrays `face`, `torso`, `leftArm`, `rightArm`, `leftLeg`, `rightLeg`.\n\n**`viji.video.cv.segmentation: SegmentationData | null`**: `mask: Uint8Array` (each byte is `0` for background or `1` for person; length = `width * height`), `width`, `height`. Note: the shader `u_segmentationMask` sampler is sampled as a normalized `float` (0.0 = background, 1.0 = person); the JS `mask` byte values are 0/1, not 0/255.\n\n### Input\n\n**Pointer** (unified mouse and touch): `viji.pointer`\n| Member | Type | Notes |\n|---|---|---|\n| `x`, `y` | `number` | Position in pixels |\n| `deltaX`, `deltaY` | `number` | Per-frame movement |\n| `isDown` | `boolean` | Pressed or touching |\n| `wasPressed`, `wasReleased` | `boolean` | One-frame edge triggers |\n| `isInCanvas` | `boolean` | Inside canvas bounds |\n| `type` | `'mouse' \\| 'touch' \\| 'none'` | Active input type |\n\n**Mouse**: `viji.mouse`: `x`, `y`, `isInCanvas`, `isPressed`, `leftButton`, `rightButton`, `middleButton`, `deltaX`, `deltaY`, `wheelDelta`, `wheelX`, `wheelY`, `wasPressed`, `wasReleased`, `wasMoved`.\n\n**Keyboard**: `viji.keyboard`: `isPressed(key)`, `wasPressed(key)`, `wasReleased(key)`, `activeKeys: Set<string>`, `pressedThisFrame`, `releasedThisFrame`, `lastKeyPressed`, `lastKeyReleased`, `shift`, `ctrl`, `alt`, `meta`.\n\n**Touch**: `viji.touches`: `count`, `points: TouchPoint[]`, `started`, `moved`, `ended` (each a `TouchPoint[]`), `primary: TouchPoint | null`. Each `TouchPoint`: `id`, `x`, `y`, `pressure`, `radius`, `radiusX`, `radiusY`, `rotationAngle`, `force`, `isInCanvas`, `deltaX`, `deltaY`, `velocity: {x,y}`, `isNew`, `isActive`, `isEnding`.\n\n### Sensors and external devices\n\n**`viji.device`**: host device motion and orientation:\n- `motion: DeviceMotionData | null`: `acceleration: {x,y,z}` (m/s²), `accelerationIncludingGravity`, `rotationRate: {alpha,beta,gamma}` (deg/s), `interval` (ms).\n- `orientation: DeviceOrientationData | null`: `alpha` (0-360° compass), `beta` (-180..180° front-back tilt), `gamma` (-90..90° left-right tilt), `absolute`.\n\n**`viji.devices`**: `DeviceState[]`, externally connected devices. Each: `id`, `name`, `motion`, `orientation`, `video` (`VideoAPI | null`, same shape as `viji.video` but without CV), `audio` (`AudioStreamAPI | null`, lightweight; no beat or BPM).\n\n### Host-supplied streams\n\n- `viji.videoStreams: VideoAPI[]`: extra video sources from the host compositor. Same shape as `viji.video`. May be empty.\n- `viji.audioStreams: AudioStreamAPI[]`: extra audio sources from the host. Lightweight shape (no beat or BPM). May be empty.\n\n### External libraries\n\n```javascript\nconst THREE = await import('https://esm.sh/three@0.160.0');\nconst renderer = new THREE.WebGLRenderer({ canvas: viji.canvas, antialias: true });\nrenderer.setSize(viji.width, viji.height, false); // always pass false to skip CSS styling\n```\n\nPin library versions in the import URL. Always pass `viji.canvas` to the renderer. Always pass `false` as the third argument to Three.js `setSize`. Handle resize by comparing `viji.width` / `viji.height` against previous values inside `render()`.\n\n## Template\n\n```javascript\nconst bgColor = viji.color('#1a1a2e', { label: 'Background' });\nconst speed = viji.slider(1, { min: 0.1, max: 5, label: 'Speed' }); // implicit step → 0.01\nconst count = viji.slider(12, { min: 3, max: 30, step: 1, label: 'Count' }); // explicit step for integer counts\n\nlet angle = 0;\n\nfunction render(viji) {\n const ctx = viji.useContext('2d');\n angle += speed.value * viji.deltaTime;\n\n ctx.fillStyle = bgColor.value;\n ctx.fillRect(0, 0, viji.width, viji.height);\n\n const cx = viji.width / 2;\n const cy = viji.height / 2;\n const radius = Math.min(viji.width, viji.height) * 0.3;\n const dotSize = Math.min(viji.width, viji.height) * 0.02;\n const n = Math.floor(count.value);\n\n for (let i = 0; i < n; i++) {\n const a = angle + (i / n) * Math.PI * 2;\n const x = cx + Math.cos(a) * radius;\n const y = cy + Math.sin(a) * radius;\n const hue = (i / n) * 360;\n ctx.beginPath();\n ctx.arc(x, y, dotSize, 0, Math.PI * 2);\n ctx.fillStyle = `hsl(${hue}, 80%, 60%)`;\n ctx.fill();\n }\n}\n```\n",
9853
- "p5": "# Viji API: P5 Renderer Reference\n\nP5 scenes use P5.js 1.9.4 inside a Web Worker on an `OffscreenCanvas`. P5 runs in **instance mode**: every P5 function and constant requires the `p5.` prefix. The first line of the scene must be `// @renderer p5` (2D) or `// @renderer p5 webgl` (3D / WEBGL).\n\n## Architecture\n\n- The global `viji` object exposes canvas, timing, audio, video, CV, input, sensors, parameters: identical to Native.\n- The canvas and its rendering context are managed by P5. **Do not call `createCanvas()`.** Viji creates the canvas in the mode declared by the `// @renderer` directive.\n- `viji.useContext()` is **not available** in P5 mode (the context belongs to P5).\n- Top-level code runs once. `function setup(viji, p5)` runs once for configuration. `function render(viji, p5)` runs every frame.\n- The `p5` instance passed to `setup` and `render` is the P5 instance. Use `p5.background`, `p5.fill`, `p5.circle`, `p5.PI`, etc. Unprefixed names will not resolve.\n\n## Rules\n\n1. **Always** add `// @renderer p5` (2D) or `// @renderer p5 webgl` (WEBGL) as the first line.\n2. **Always** use `render(viji, p5)` (not `draw()`) and `setup(viji, p5)` (not `setup()`).\n3. **Always** prefix every P5 function and constant with `p5.`:\n - `background(0)` -> `p5.background(0)`\n - `fill(255)` -> `p5.fill(255)`\n - `PI` -> `p5.PI`, `TWO_PI` -> `p5.TWO_PI`, `HSB` -> `p5.HSB`\n - `createVector(1, 0)` -> `p5.createVector(1, 0)`\n - `map(v, 0, 1, 0, 255)` -> `p5.map(v, 0, 1, 0, 255)`\n - `noise(x)` -> `p5.noise(x)`, `random()` -> `p5.random()`\n4. **Never** call `createCanvas()`. The canvas is created and managed by Viji.\n5. **Never** use `preload()`. Use `viji.image(null, { label })` for images, or `fetch()` inside `setup()`.\n6. **Never** use P5 event callbacks (`mousePressed`, `mouseDragged`, `mouseReleased`, `keyPressed`, `keyReleased`, `keyTyped`, `touchStarted`, `touchMoved`, `touchEnded`). Read state in `render()` via `viji.pointer`, `viji.mouse`, `viji.keyboard`, `viji.touches`. Use `wasPressed` / `wasReleased` for one-frame edges.\n7. **Never** use `loadImage()`, `loadFont()`, `loadJSON()`, `loadModel()`, `loadShader()`. Use `viji.image()` or `fetch()`.\n8. **Never** use `p5.frameRate()`, `p5.save()`, `p5.saveCanvas()`, `p5.saveFrames()`. The host controls frame rate and capture.\n9. **Never** use `createCapture()` or `createVideo()`. Use `viji.video.*`.\n10. **Never** use `p5.dom` or `p5.sound`. Use Viji parameters and `viji.audio.*`.\n11. **Never** access `window`, `document`, `Image()`, `localStorage`. `fetch()` is available.\n12. **Always** declare parameters at the top level, never inside `render()` or `setup()`.\n13. **Always** read parameters via `.value`. Color parameters also expose `.rgb` (matches `colorMode(RGB, 255)`) and `.hsb` (matches `colorMode(HSB, 360, 100, 100)`). Color defaults accept hex, `{ r, g, b }` (0..255), `{ h, s, b }` (0..360 / 0..100), or CSS color strings.\n14. **Always** use `viji.width` and `viji.height` for canvas dimensions. Never hardcode pixel sizes.\n15. **Always** use `viji.deltaTime` for animation:\n ```javascript\n let angle = 0;\n function render(viji, p5) { angle += speed.value * viji.deltaTime; }\n ```\n16. **Never** multiply `viji.time` by a parameter; it causes visible jumps when the parameter changes. Use a `deltaTime` accumulator. This also applies to nested multiplications; each independent speed gets its own accumulator.\n17. **Never** allocate objects, arrays, or strings inside `render()`. Pre-allocate at the top level and reuse.\n18. **Image parameters with P5:** use `.p5` (a `P5Image`) instead of `.value` when passing to `p5.image()`:\n ```javascript\n const photo = viji.image(null, { label: 'Photo' });\n function render(viji, p5) {\n if (photo.value) p5.image(photo.p5, 0, 0, viji.width, viji.height);\n }\n ```\n19. **Video drawing.** In 2D, use `p5.image(viji.video.currentFrame, ...)` or `p5.drawingContext.drawImage(...)`. In WEBGL, use `p5.image(viji.video.currentFrame, ...)` only (`p5.drawingContext` is WebGL, not Canvas 2D). **Always preserve aspect ratio** via the `videoFit` helper below.\n20. `p5.createGraphics(w, h)` is **2D only** (`createGraphics(w, h, p5.WEBGL)` is not supported). It creates an OffscreenCanvas internally.\n21. Fonts: `p5.textFont()` only with CSS generic names (`monospace`, `serif`, `sans-serif`). `loadFont()` is not available.\n22. `p5.tint()` and `p5.blendMode()` work normally.\n23. **Canvas mode is declared by the `// @renderer` directive.** Use `// @renderer p5` for 2D, `// @renderer p5 webgl` for WEBGL. Never call `createCanvas()` or `createCanvas(..., p5.WEBGL)`.\n24. In WEBGL scenes, `p5.drawingContext` is a WebGL context. Use P5 3D drawing primitives or `p5.image()` / textures for images and video.\n25. `p5.pixelDensity()` defaults to 1 in the worker. `p5.loadPixels()` and `p5.pixels[]` work in 2D scenes; WEBGL pixel readback follows standard P5.js rules.\n26. **Always** check `viji.audio.isConnected` before reading audio data.\n27. **Always** check `viji.video.isConnected && viji.video.currentFrame` before drawing video.\n28. **Never** enable CV features by default; use toggle parameters.\n29. **Always** set `category` on input-dependent parameters (`'audio'`, `'video'`, `'interaction'`). Use creative-strength sliders, not on/off toggles, for inputs the host already gates. CV feature toggles stay opt-in.\n30. **Always** type any function parameter that receives the `viji` instance or the `p5` instance, including `setup(viji, p5)`, `render(viji, p5)`, and any helpers that take either. Monaco's TS checker flags API misuse (wrong nesting, typo'd property names) at edit time. Scene code is transpiled via sucrase before execution, so TS annotations are stripped at runtime — safe to include. Note: the p5-instance type is the lowercase namespace `p5` (same name as the conventional parameter; TS resolves the value and the type from separate namespaces, so `p5: p5` is legal and common).\n\n ```javascript\n function setup(viji: VijiAPI, p5: p5) { ... }\n function render(viji: VijiAPI, p5: p5) { ... }\n function videoFit(viji: VijiAPI, mode: 'cover' | 'contain') { ... }\n ```\n\n## Video aspect helper\n\n```javascript\nfunction videoFit(viji, mode = 'cover') {\n const vw = viji.video.frameWidth, vh = viji.video.frameHeight;\n const w = viji.width, h = viji.height;\n if (!vw || !vh) return { x: 0, y: 0, width: 0, height: 0 };\n const scale = mode === 'cover' ? Math.max(w / vw, h / vh) : Math.min(w / vw, h / vh);\n const dw = vw * scale, dh = vh * scale;\n return { x: (w - dw) / 2, y: (h - dh) / 2, width: dw, height: dh };\n}\n\nif (viji.video.isConnected && viji.video.currentFrame) {\n const v = videoFit(viji, 'cover');\n p5.image(viji.video.currentFrame, v.x, v.y, v.width, v.height);\n}\n\n// CV bounds and landmarks are normalized 0..1 to the source frame.\n// Map them through v to align with the drawn video.\n```\n\n## P5 to Viji mapping\n\n| Standard P5.js | Viji P5 equivalent |\n|---|---|\n| `width` / `height` | `viji.width` / `viji.height` |\n| `mouseX` / `mouseY` | `viji.pointer.x` / `viji.pointer.y` (or `viji.mouse.x` / `viji.mouse.y`) |\n| `mouseIsPressed` | `viji.pointer.isDown` |\n| `mouseButton === LEFT` | `viji.mouse.leftButton` |\n| `keyIsPressed` | `viji.keyboard.activeKeys.size > 0` |\n| `keyIsDown(code)` | `viji.keyboard.isPressed('keyName')` |\n| `key` | `viji.keyboard.lastKeyPressed` |\n| `frameCount` | `viji.frameCount` (or a `viji.deltaTime` accumulator) |\n| `frameRate(n)` | Remove. Host controls frame rate. |\n| `createCanvas(w, h)` | Remove. Canvas is provided. |\n| `preload()` | Remove. Use `viji.image()` or `fetch()` in `setup()`. |\n| `loadImage(url)` | `viji.image(null, { label: 'Image' })`; pass `.p5` to `p5.image()`. |\n| `save()` / `saveCanvas()` | Remove. Host handles capture. |\n| `mousePressed()` / `mouseReleased()` callbacks | Check `viji.pointer.wasPressed` / `wasReleased` inside `render()`. |\n| `keyPressed()` / `keyReleased()` callbacks | Check `viji.keyboard.wasPressed(key)` / `wasReleased(key)` inside `render()`. |\n\n## API reference\n\nAll `viji.*` members are identical to Native (same object, same types).\n\n### Canvas and timing\n\n| Member | Type | Notes |\n|---|---|---|\n| `viji.canvas` | `OffscreenCanvas` | Managed by P5 |\n| `viji.width`, `viji.height` | `number` | Canvas dimensions |\n| `viji.time` | `number` | Seconds since scene start |\n| `viji.deltaTime` | `number` | Seconds since last frame |\n| `viji.frameCount` | `number` | Total frames rendered |\n| `viji.fps` | `number` | Target frame rate |\n\n### Parameters\n\n```javascript\nviji.slider(default, { min?, max?, step?, label, group?, category? }) // .value: number\nviji.color(default, { label, group?, category? }) // .value, .rgb, .hsb\nviji.toggle(default, { label, group?, category? }) // .value: boolean\nviji.select(default, { options, label, group?, category? }) // .value: string | number\nviji.number(default, { min?, max?, step?, label, group?, category? }) // .value: number\nviji.text(default, { label, group?, category?, maxLength? }) // .value: string\nviji.image(null, { label, group?, category? }) // .value: ImageBitmap | null, .p5: P5Image\nviji.button({ label, description?, group?, category? }) // .value: boolean (1 frame)\nviji.coordinate(default, { step?, label, group?, category? }) // .value: { x, y }, range -1..1\n// slider/number `step` defaults to a power of 10 giving ~100 positions across min..max\n// (e.g., range 0..1 → 0.01, range 0..100 → 1). Specify `step: 1` for integer counts.\n```\n\n### Audio: `viji.audio`\n\nCheck `isConnected` first. Members:\n\n`isConnected` (boolean); `volume.{current, peak, smoothed}` (0..1; `smoothed` 200ms decay); `bands.{low, lowMid, mid, highMid, high}` (0..1; ranges 20-120, 120-400, 400-1600, 1600-6000, 6000-16000 Hz) and the sibling `*Smoothed` envelopes (150ms decay); `beat.{kick, snare, hat, any}` (0..1, 300ms decay curves; peak on each detected beat); `beat.{kickSmoothed, snareSmoothed, hatSmoothed, anySmoothed}` (500ms decay envelopes); `beat.triggers.{kick, snare, hat, any}` (boolean, true for exactly one frame, OR-accumulated between frames); `beat.events: Array<{type, time, strength}>` (`type` is `'kick' | 'snare' | 'hat'`; `time` ms; cleared each frame); `beat.bpm` (`0` when no audio is connected; once audio connects it tracks the detected tempo clamped to 60..240, with `120` as a fallback before lock-on), `beat.confidence` (0..1), `beat.isLocked` (boolean); `spectral.{brightness, flatness}` (0..1); `getFrequencyData(): Uint8Array` (1024 FFT bins, 0..255); `getWaveform(): Float32Array` (2048 samples, -1..1).\n\nExternal-device audio (`viji.devices[i].audio`) and host streams (`viji.audioStreams[i]`) follow the `AudioStreamAPI` shape: `isConnected`, `volume`, `bands` (instant + smoothed siblings), `spectral`, `getFrequencyData`, `getWaveform`. No beat, BPM, triggers, or events on streams.\n\n### Video: `viji.video`\n\nCheck `isConnected && currentFrame` before drawing. Members:\n\n`isConnected` (boolean); `currentFrame` (`OffscreenCanvas | ImageBitmap | null`); `frameWidth`, `frameHeight`, `frameRate`; `getFrameData(): ImageData | null`; `cv: VideoCVAPI` (CV outputs and verbs live here, not on `viji.video` directly).\n\n### Computer Vision: `viji.video.cv`\n\nEach CV feature is independent and populates only its own subset of fields. Enable only what you need (each active feature uses a separate WebGL context for MediaPipe; 4+ on low-end hardware risks context-limit failures).\n\n```javascript\n// Verbs return Promise<void>. await is optional. Safe to call from module\n// scope (always-on CV) or per-frame inside render() gated by a viji.toggle(...)\n// (opt-in CV). Idempotent + reference-counted.\nawait viji.video.cv.enableFaceDetection(true | false); // face.bounds, face.center, face.confidence\nawait viji.video.cv.enableFaceMesh(true | false); // face.landmarks (468 pts) + face.headPose\nawait viji.video.cv.enableEmotionDetection(true | false); // face.expressions + face.blendshapes; also loads landmarker\nawait viji.video.cv.enableHandTracking(true | false); // viji.video.cv.hands[]\nawait viji.video.cv.enablePoseDetection(true | false); // viji.video.cv.pose\nawait viji.video.cv.enableBodySegmentation(true | false); // viji.video.cv.segmentation\nviji.video.cv.getActiveFeatures(); // CVFeature[]\nviji.video.cv.isProcessing(); // boolean\n```\n\nData outputs on `viji.video.cv`:\n- `analysedFrame: OffscreenCanvas | null`: the frame paired with the current CV results. `null` until the first inference lands. Use `viji.video.cv.analysedFrame ?? viji.video.currentFrame` to fall back during startup.\n- `getAnalysedFrameData(): ImageData | null`: pixel data of `analysedFrame`.\n\n**`faces: FaceData[]`**: `id` always present. Other fields populated only by their source model; otherwise `null` (`landmarks` is `[]` when empty). Always null-check.\n- `bounds: {x,y,width,height} | null` — populated by face detection\n- `center: {x,y} | null` — populated by face detection\n- `confidence: number | null` — populated by face detection\n- `landmarks: {x,y,z?}[]` (empty `[]` unless face mesh enabled)\n- `headPose: {pitch, yaw, roll} | null` — populated by face mesh\n- `expressions: {neutral, happy, sad, angry, surprised, disgusted, fearful} | null` (0..1 each) — populated by emotion detection\n- `blendshapes | null` (52 ARKit coefficients 0..1) — populated by emotion detection\n\nIf you need `bounds` while only running face mesh, either also enable face detection or compute it from `face.landmarks` min/max.\n\n**`hands: HandData[]`**: `id`, `handedness: 'left' | 'right'`, `confidence`, `bounds`, `landmarks` (21 points), `palm`, `gestures` (`fist`, `openPalm`, `peace`, `thumbsUp`, `thumbsDown`, `pointing`, `iLoveYou` 0..1).\n\n**`pose: PoseData | null`**: `confidence`, `landmarks` (33 points), body-part arrays `face`, `torso`, `leftArm`, `rightArm`, `leftLeg`, `rightLeg`.\n\n**`segmentation: SegmentationData | null`**: `mask: Uint8Array` (each byte is `0` for background or `1` for person; length = `width * height`), `width`, `height`.\n\n### Input\n\n- `viji.pointer`: `x`, `y`, `deltaX`, `deltaY`, `isDown`, `wasPressed`, `wasReleased`, `isInCanvas`, `type` (`'mouse' | 'touch' | 'none'`).\n- `viji.mouse`: `x`, `y`, `isInCanvas`, `isPressed`, `leftButton`, `rightButton`, `middleButton`, `deltaX`, `deltaY`, `wheelDelta`, `wheelX`, `wheelY`, `wasPressed`, `wasReleased`, `wasMoved`.\n- `viji.keyboard`: `isPressed(key)`, `wasPressed(key)`, `wasReleased(key)`, `activeKeys`, `pressedThisFrame`, `releasedThisFrame`, `lastKeyPressed`, `lastKeyReleased`, `shift`, `ctrl`, `alt`, `meta`.\n- `viji.touches`: `count`, `points`, `started`, `moved`, `ended`, `primary`. Each `TouchPoint`: `id`, `x`, `y`, `pressure`, `radius`, `radiusX`, `radiusY`, `rotationAngle`, `force`, `isInCanvas`, `deltaX`, `deltaY`, `velocity`, `isNew`, `isActive`, `isEnding`.\n\n### Sensors and external devices\n\n`viji.device.motion`: `acceleration {x,y,z}` (m/s²), `accelerationIncludingGravity`, `rotationRate {alpha,beta,gamma}` (deg/s), `interval` (ms).\n`viji.device.orientation`: `alpha` (0-360°), `beta` (-180..180°), `gamma` (-90..90°), `absolute`.\n\n`viji.devices: DeviceState[]`: each entry: `id`, `name`, `motion`, `orientation`, `video` (`VideoAPI | null`, no CV), `audio` (`AudioStreamAPI | null`, lightweight).\n\n### Host streams\n\n`viji.videoStreams: VideoAPI[]`: extra video sources. May be empty.\n`viji.audioStreams: AudioStreamAPI[]`: extra audio sources. May be empty. Lightweight shape (no beat / BPM).\n\n## Template\n\n```javascript\n// @renderer p5\n\nconst bgColor = viji.color('#1a1a2e', { label: 'Background' });\nconst speed = viji.slider(1, { min: 0.1, max: 5, label: 'Speed' }); // implicit step → 0.01\nconst count = viji.slider(8, { min: 3, max: 30, step: 1, label: 'Count' }); // explicit step for integer counts\n\nlet angle = 0;\n\nfunction setup(viji, p5) {\n p5.colorMode(p5.HSB, 360, 100, 100);\n}\n\nfunction render(viji, p5) {\n angle += speed.value * viji.deltaTime;\n p5.background(bgColor.value);\n\n const cx = viji.width / 2;\n const cy = viji.height / 2;\n const radius = p5.min(viji.width, viji.height) * 0.3;\n const dotSize = p5.min(viji.width, viji.height) * 0.04;\n const n = p5.floor(count.value);\n\n p5.noStroke();\n for (let i = 0; i < n; i++) {\n const a = angle + (i / n) * p5.TWO_PI;\n const x = cx + p5.cos(a) * radius;\n const y = cy + p5.sin(a) * radius;\n p5.fill((i / n) * 360, 80, 90);\n p5.circle(x, y, dotSize);\n }\n}\n```\n",
9852
+ "native": "# Viji API: Native Renderer Reference\n\nNative scenes run JavaScript on an `OffscreenCanvas` in a Web Worker, with full access to Canvas 2D, WebGL, and WebGL2 contexts. External libraries load via `await import()`. Top-level code runs once; a `render(viji)` function runs every frame.\n\n## Architecture\n\n- The global `viji` object exposes canvas, timing, audio, video, CV, input, sensors, parameters.\n- Call `viji.useContext('2d' | 'webgl' | 'webgl2')` once at the top of `render()` (or top-level) to obtain a rendering context. After the first call, `viji.ctx` / `viji.gl` shortcuts are populated.\n- **Pick one context type for the entire scene.** Calling `useContext` with a different type after the first call returns `null`.\n- There is **no `setup()` function** in Native. All initialization goes at the top level (which supports `await` for dynamic imports).\n- The DOM is unavailable. `fetch()` and `await import()` are available.\n\n## Rules\n\n1. **Never** access `window`, `document`, `Image()`, `localStorage`, or any DOM API. `fetch()` and `await import()` are available.\n2. **Always** declare parameters at the top level, never inside `render()`.\n3. **Always** read parameters via `.value`. Color parameters also expose `.rgb` (`{ r, g, b }` in 0..255) and `.hsb` (`{ h, s, b }` with `h` in 0..360, `s` / `b` in 0..100). Color defaults accept hex (`'#ff6600'`, `'#f60'`), `{ r, g, b }` (0..255), `{ h, s, b }` (0..360 / 0..100), or CSS `'rgb(...)'` / `'hsl(...)'` strings.\n4. **Always** use `viji.width` and `viji.height` for canvas dimensions. Never hardcode pixel sizes.\n5. **Always** use `viji.deltaTime` for animation. Use `viji.time` only for constant-speed oscillations (`Math.sin(viji.time * 2.0)`). For anything driven by a parameter, use a `deltaTime` accumulator.\n6. **Never** multiply `viji.time` by a parameter; it causes visible jumps when the parameter changes. Use a `deltaTime` accumulator at the top level. This also applies to nested multiplications: each independent speed gets its own accumulator.\n\n ```javascript\n // WRONG\n const t = viji.time * speed.value;\n const rot = phase * rotSpeed.value;\n\n // RIGHT\n let phase = 0, rotPhase = 0;\n phase += speed.value * viji.deltaTime;\n rotPhase += speed.value * rotSpeed.value * viji.deltaTime;\n ```\n7. **Never** allocate objects, arrays, or strings inside `render()`. Pre-allocate at the top level and reuse.\n8. **Always** call `viji.useContext()` to get a context. Pick one type and use it for the entire scene.\n9. **Always** check `viji.audio.isConnected` before reading audio data. Check `viji.video.isConnected && viji.video.currentFrame` before drawing video.\n10. **Never** enable CV features by default. Use a toggle parameter so the user opts in.\n11. **Always** set `category` on parameters that depend on an external input: `'audio'`, `'video'`, `'interaction'`. Use creative-strength sliders (sensitivity, intensity), not on/off toggles, for inputs the host already gates. CV feature toggles (`enableFaceDetection` etc.) are the exception and stay opt-in.\n12. **Always** type any function parameter that receives the `viji` instance, including `render(viji)` and any helpers that take `viji`. Monaco's TS checker flags API misuse (wrong nesting, typo'd property names) at edit time so the error can be corrected before runtime. Scene code is transpiled via sucrase before execution (`transforms: ['typescript']`), so TS annotations are stripped at runtime — they are safe to include.\n\n ```javascript\n function render(viji: VijiAPI) { ... }\n function applyGlow(viji: VijiAPI, intensity: number) { ... }\n ```\n\n## API reference\n\n### Canvas and context\n\n| Member | Type | Description |\n|---|---|---|\n| `viji.canvas` | `OffscreenCanvas` | The canvas |\n| `viji.useContext('2d')` | `OffscreenCanvasRenderingContext2D` | Acquire 2D context |\n| `viji.useContext('webgl')` | `WebGLRenderingContext` | Acquire WebGL 1 |\n| `viji.useContext('webgl2')` | `WebGL2RenderingContext` | Acquire WebGL 2 |\n| `viji.ctx` | `OffscreenCanvasRenderingContext2D` | Shortcut after `useContext('2d')` |\n| `viji.gl` | `WebGLRenderingContext \\| WebGL2RenderingContext` | Shortcut after WebGL context acquired |\n| `viji.width`, `viji.height` | `number` | Current canvas dimensions in pixels |\n\n### Timing\n\n| Member | Type | Description |\n|---|---|---|\n| `viji.time` | `number` | Seconds since scene start |\n| `viji.deltaTime` | `number` | Seconds since last frame |\n| `viji.frameCount` | `number` | Total frames rendered |\n| `viji.fps` | `number` | Target frame rate (host-controlled) |\n\n### Parameters\n\nDeclare at top level. All accept `{ label, description?, group?, category? }`. Category values: `'audio'`, `'video'`, `'interaction'`, `'general'` (default).\n\n```javascript\nviji.slider(default, { min?, max?, step?, label, group?, category? }) // .value: number\nviji.color(default, { label, group?, category? }) // .value: '#rrggbb', .rgb: {r,g,b} 0..255, .hsb: {h:0..360, s/b:0..100}\nviji.toggle(default, { label, group?, category? }) // .value: boolean\nviji.select(default, { options, label, group?, category? }) // .value: string | number\nviji.number(default, { min?, max?, step?, label, group?, category? }) // .value: number\nviji.text(default, { label, group?, category?, maxLength? }) // .value: string\nviji.image(null, { label, group?, category? }) // .value: ImageBitmap | null\nviji.button({ label, description?, group?, category? }) // .value: boolean (true for 1 frame)\nviji.coordinate(default, { step?, label, group?, category? }) // .value: { x, y }, range -1..1\n// slider/number `step` defaults to a power of 10 giving ~100 positions across min..max\n// (e.g., range 0..1 → 0.01, range 0..100 → 1). Specify `step: 1` for integer counts.\n```\n\n### Audio: `viji.audio`\n\nCheck `isConnected` first.\n\n| Member | Type | Notes |\n|---|---|---|\n| `isConnected` | `boolean` | Audio source active |\n| `volume.current`, `volume.peak`, `volume.smoothed` | `number` 0..1 | `smoothed` = 200ms decay envelope of `current` |\n| `bands.low`, `lowMid`, `mid`, `highMid`, `high` | `number` 0..1 | Instant band energies (20-120, 120-400, 400-1600, 1600-6000, 6000-16000 Hz) |\n| `bands.lowSmoothed`, `lowMidSmoothed`, `midSmoothed`, `highMidSmoothed`, `highSmoothed` | `number` 0..1 | 150ms decay envelope siblings (not nested) |\n| `beat.kick`, `snare`, `hat`, `any` | `number` 0..1 | Beat energy curves with a 300ms decay; peak on each detected beat then fall off |\n| `beat.kickSmoothed`, `snareSmoothed`, `hatSmoothed`, `anySmoothed` | `number` 0..1 | Smoother 500ms decay envelopes; use for ambient pulses |\n| `beat.triggers.kick`, `snare`, `hat`, `any` | `boolean` | True for exactly one frame on the matching beat, then auto-resets. OR-accumulated between frames so no beat is lost |\n| `beat.events` | `Array<{type, time, strength}>` | All beats detected since the last frame; `type` is `'kick' \\| 'snare' \\| 'hat'` (never `'any'`); `time` in milliseconds; cleared each frame |\n| `beat.bpm`, `beat.confidence`, `beat.isLocked` | `number`, `number`, `boolean` | `bpm` is `0` when no audio is connected; once audio connects it tracks the detected tempo (clamped to 60..240) with `120` as a fallback before lock-on. `confidence` in 0..1. `isLocked` is `true` on stable tempo lock |\n| `spectral.brightness`, `spectral.flatness` | `number` 0..1 | Spectral centroid; spectral flatness |\n| `getFrequencyData()` | `Uint8Array` | Raw FFT bins 0..255 |\n| `getWaveform()` | `Float32Array` | Time-domain waveform -1..1 |\n\nExternal-device audio (`viji.devices[i].audio`) and host-supplied audio streams (`viji.audioStreams[i]`) use the `AudioStreamAPI` shape: `isConnected`, `volume.{current,peak,smoothed}`, `bands.{low...high}` plus each `*Smoothed` sibling, `spectral.{brightness,flatness}`, `getFrequencyData()`, `getWaveform()`. No `beat`, BPM, triggers, or events on streams: those exist only on the main `viji.audio`.\n\n### Video: `viji.video`\n\nCheck `isConnected && currentFrame` before drawing.\n\n| Member | Type | Notes |\n|---|---|---|\n| `isConnected` | `boolean` | Video source active |\n| `currentFrame` | `OffscreenCanvas \\| ImageBitmap \\| null` | Most recent frame |\n| `frameWidth`, `frameHeight` | `number` | Source frame size |\n| `frameRate` | `number` | Source frame rate |\n| `getFrameData()` | `ImageData \\| null` | Pixel data of `currentFrame` |\n| `cv` | `VideoCVAPI` | Computer-vision surface: see below |\n\nCV data outputs (`analysedFrame`, `getAnalysedFrameData()`, `faces`, `hands`, `pose`, `segmentation`) and CV verbs (`enableFaceDetection`, etc.) all live on `viji.video.cv`, not on `viji.video` directly.\n\n**Aspect ratio.** Camera frames almost never match the canvas aspect. Drawing to `(0, 0, viji.width, viji.height)` stretches video and misaligns CV bounds. Use this helper at module scope:\n\n```javascript\nfunction videoFit(viji, mode = 'cover') {\n const vw = viji.video.frameWidth, vh = viji.video.frameHeight;\n const w = viji.width, h = viji.height;\n if (!vw || !vh) return { x: 0, y: 0, width: 0, height: 0 };\n const scale = mode === 'cover' ? Math.max(w / vw, h / vh) : Math.min(w / vw, h / vh);\n const dw = vw * scale, dh = vh * scale;\n return { x: (w - dw) / 2, y: (h - dh) / 2, width: dw, height: dh };\n}\n\nconst v = videoFit(viji); // 'cover' or 'contain'\nctx.drawImage(viji.video.currentFrame, v.x, v.y, v.width, v.height);\n\n// CV coords are normalized 0..1 to the source frame. Map through v to align with the fitted video:\nviji.video.cv.faces.forEach(face => {\n const bx = v.x + face.bounds.x * v.width;\n const by = v.y + face.bounds.y * v.height;\n const bw = face.bounds.width * v.width;\n const bh = face.bounds.height * v.height;\n ctx.strokeRect(bx, by, bw, bh);\n});\n```\n\nDefault to `'cover'` for live cameras. Use `'contain'` for CV-overlay scenes where bounding boxes near frame edges must stay visible. `viji.video.currentFrame` is the right default for displayed video; reach for `viji.video.cv.analysedFrame` only when sampling pixels at CV-derived positions (segmentation compositing, face-mesh texturing, landmark-anchored color sampling). The common fallback pattern is `viji.video.cv.analysedFrame ?? viji.video.currentFrame` during the brief window before the first CV inference lands.\n\n### Computer Vision: `viji.video.cv`\n\nEach CV feature is independent and populates only its own subset of fields. Enable only what you need (each active feature consumes a separate WebGL context for MediaPipe; 4+ on low-end hardware risks context-limit failures).\n\n```javascript\n// Verbs return Promise<void>. await is optional — without it, the model loads\n// asynchronously while render runs and data fields populate when ready.\n// Safe to call from module scope (always-on CV) or per-frame inside render()\n// gated by a viji.toggle(...) (opt-in CV). Idempotent + reference-counted.\nawait viji.video.cv.enableFaceDetection(true | false); // populates face.bounds, face.center, face.confidence\nawait viji.video.cv.enableFaceMesh(true | false); // populates face.landmarks (468 pts) + face.headPose\nawait viji.video.cv.enableEmotionDetection(true | false); // populates face.expressions + face.blendshapes; also loads landmarker (face.landmarks + face.headPose populated)\nawait viji.video.cv.enableHandTracking(true | false); // populates viji.video.cv.hands[]\nawait viji.video.cv.enablePoseDetection(true | false); // populates viji.video.cv.pose\nawait viji.video.cv.enableBodySegmentation(true | false); // populates viji.video.cv.segmentation\nviji.video.cv.getActiveFeatures(); // CVFeature[]\nviji.video.cv.isProcessing(); // boolean\n```\n\nData outputs on `viji.video.cv`:\n- `analysedFrame: OffscreenCanvas | null`: the exact frame paired with the current CV results. `null` until the first inference lands after a feature is enabled. Use the pattern `viji.video.cv.analysedFrame ?? viji.video.currentFrame` to fall back during the startup window.\n- `getAnalysedFrameData(): ImageData | null`: pixel data of `analysedFrame`, cached and re-extracted only on new CV results.\n\n**`viji.video.cv.faces: FaceData[]`**: `id` always present. Other fields are populated only by their source model and read `null` (or empty array for `landmarks`) otherwise — always null-check before reading.\n- `bounds: {x,y,width,height} | null` (normalized 0..1) — populated by face detection\n- `center: {x,y} | null` (normalized 0..1) — populated by face detection\n- `confidence: number | null` (0..1) — populated by face detection\n- `landmarks: {x,y,z?}[]` (empty `[]` unless face mesh enabled; 468 pts when populated)\n- `headPose: { pitch, yaw, roll } | null` (degrees) — populated by face mesh\n- `expressions: { neutral, happy, sad, angry, surprised, disgusted, fearful } | null` (0..1 each) — populated by emotion detection\n- `blendshapes | null` — populated by emotion detection; 52 ARKit coefficients (`browDownLeft`/`Right`, `browInnerUp`, `browOuterUpLeft`/`Right`, `cheekPuff`, `cheekSquintLeft`/`Right`, `eyeBlinkLeft`/`Right`, `eyeLookDown`/`In`/`Out`/`UpLeft`/`Right`, `eyeSquintLeft`/`Right`, `eyeWideLeft`/`Right`, `jawForward`/`Left`/`Open`/`Right`, `mouthClose`/`DimpleLeft`/`Right`/`FrownLeft`/`Right`/`Funnel`/`Left`/`LowerDownLeft`/`Right`/`PressLeft`/`Right`/`Pucker`/`Right`/`RollLower`/`Upper`/`ShrugLower`/`Upper`/`SmileLeft`/`Right`/`StretchLeft`/`Right`/`UpperUpLeft`/`Right`, `noseSneerLeft`/`Right`, `tongueOut`: all 0..1)\n\nIf you need bounds while only running face mesh, either also enable face detection or compute the bounding box from `face.landmarks` min/max yourself.\n\n**`viji.video.cv.hands: HandData[]`**: `id`, `handedness: 'left' | 'right'`, `confidence`, `bounds`, `landmarks: {x,y,z}[]` (21 points), `palm: {x,y,z}`, `gestures: { fist, openPalm, peace, thumbsUp, thumbsDown, pointing, iLoveYou }` (0..1 confidence each).\n\n**`viji.video.cv.pose: PoseData | null`**: `confidence`, `landmarks: {x,y,z,visibility}[]` (33 points), plus body-part arrays `face`, `torso`, `leftArm`, `rightArm`, `leftLeg`, `rightLeg`.\n\n**`viji.video.cv.segmentation: SegmentationData | null`**: `mask: Uint8Array` (each byte is `0` for background or `1` for person; length = `width * height`), `width`, `height`. Note: the shader `u_segmentationMask` sampler is sampled as a normalized `float` (0.0 = background, 1.0 = person); the JS `mask` byte values are 0/1, not 0/255.\n\n### Input\n\n**Pointer** (unified mouse and touch): `viji.pointer`\n| Member | Type | Notes |\n|---|---|---|\n| `x`, `y` | `number` | Position in pixels |\n| `deltaX`, `deltaY` | `number` | Per-frame movement |\n| `isDown` | `boolean` | Pressed or touching |\n| `wasPressed`, `wasReleased` | `boolean` | One-frame edge triggers |\n| `isInCanvas` | `boolean` | Inside canvas bounds |\n| `type` | `'mouse' \\| 'touch' \\| 'none'` | Active input type |\n\n**Mouse**: `viji.mouse`: `x`, `y`, `isInCanvas`, `isPressed`, `leftButton`, `rightButton`, `middleButton`, `deltaX`, `deltaY`, `wheelDelta`, `wheelX`, `wheelY`, `wasPressed`, `wasReleased`, `wasMoved`.\n\n**Keyboard**: `viji.keyboard`: `isPressed(key)`, `wasPressed(key)`, `wasReleased(key)`, `activeKeys: Set<string>`, `pressedThisFrame`, `releasedThisFrame`, `lastKeyPressed`, `lastKeyReleased`, `shift`, `ctrl`, `alt`, `meta`.\n\n**Touch**: `viji.touches`: `count`, `points: TouchPoint[]`, `started`, `moved`, `ended` (each a `TouchPoint[]`), `primary: TouchPoint | null`. Each `TouchPoint`: `id`, `x`, `y`, `pressure`, `radius`, `radiusX`, `radiusY`, `rotationAngle`, `force`, `isInCanvas`, `deltaX`, `deltaY`, `velocity: {x,y}`, `isNew`, `isActive`, `isEnding`.\n\n### Sensors and external devices\n\n**`viji.device`**: host device motion and orientation:\n- `motion: DeviceMotionData | null`: `acceleration: {x,y,z}` (m/s²), `accelerationIncludingGravity`, `rotationRate: {alpha,beta,gamma}` (deg/s), `interval` (ms).\n- `orientation: DeviceOrientationData | null`: `alpha` (0-360° compass), `beta` (-180..180° front-back tilt), `gamma` (-90..90° left-right tilt), `absolute`.\n\n**`viji.devices`**: `DeviceState[]`, externally connected devices. Each: `id`, `name`, `motion`, `orientation`, `video` (`VideoAPI | null`, same shape as `viji.video` but without CV), `audio` (`AudioStreamAPI | null`, lightweight; no beat or BPM).\n\n### Host-supplied streams\n\n- `viji.videoStreams: VideoAPI[]`: extra video sources from the host compositor. Same shape as `viji.video`, including a per-stream `cv` surface with the same enable verbs and result fields; `cv.analysedFrame` is populated on the main stream only. May be empty.\n- `viji.audioStreams: AudioStreamAPI[]`: extra audio sources from the host. Lightweight shape (no beat or BPM). May be empty.\n\n### External libraries\n\n```javascript\nconst THREE = await import('https://esm.sh/three@0.160.0');\nconst renderer = new THREE.WebGLRenderer({ canvas: viji.canvas, antialias: true });\nrenderer.setSize(viji.width, viji.height, false); // always pass false to skip CSS styling\n```\n\nPin library versions in the import URL. Always pass `viji.canvas` to the renderer. Always pass `false` as the third argument to Three.js `setSize`. Handle resize by comparing `viji.width` / `viji.height` against previous values inside `render()`.\n\n## Template\n\n```javascript\nconst bgColor = viji.color('#1a1a2e', { label: 'Background' });\nconst speed = viji.slider(1, { min: 0.1, max: 5, label: 'Speed' }); // implicit step → 0.01\nconst count = viji.slider(12, { min: 3, max: 30, step: 1, label: 'Count' }); // explicit step for integer counts\n\nlet angle = 0;\n\nfunction render(viji) {\n const ctx = viji.useContext('2d');\n angle += speed.value * viji.deltaTime;\n\n ctx.fillStyle = bgColor.value;\n ctx.fillRect(0, 0, viji.width, viji.height);\n\n const cx = viji.width / 2;\n const cy = viji.height / 2;\n const radius = Math.min(viji.width, viji.height) * 0.3;\n const dotSize = Math.min(viji.width, viji.height) * 0.02;\n const n = Math.floor(count.value);\n\n for (let i = 0; i < n; i++) {\n const a = angle + (i / n) * Math.PI * 2;\n const x = cx + Math.cos(a) * radius;\n const y = cy + Math.sin(a) * radius;\n const hue = (i / n) * 360;\n ctx.beginPath();\n ctx.arc(x, y, dotSize, 0, Math.PI * 2);\n ctx.fillStyle = `hsl(${hue}, 80%, 60%)`;\n ctx.fill();\n }\n}\n```\n",
9853
+ "p5": "# Viji API: P5 Renderer Reference\n\nP5 scenes use P5.js 1.9.4 inside a Web Worker on an `OffscreenCanvas`. P5 runs in **instance mode**: every P5 function and constant requires the `p5.` prefix. The first line of the scene must be `// @renderer p5` (2D) or `// @renderer p5 webgl` (3D / WEBGL).\n\n## Architecture\n\n- The global `viji` object exposes canvas, timing, audio, video, CV, input, sensors, parameters: identical to Native.\n- The canvas and its rendering context are managed by P5. **Do not call `createCanvas()`.** Viji creates the canvas in the mode declared by the `// @renderer` directive.\n- `viji.useContext()` is **not available** in P5 mode (the context belongs to P5).\n- Top-level code runs once. `function setup(viji, p5)` runs once for configuration. `function render(viji, p5)` runs every frame.\n- The `p5` instance passed to `setup` and `render` is the P5 instance. Use `p5.background`, `p5.fill`, `p5.circle`, `p5.PI`, etc. Unprefixed names will not resolve.\n\n## Rules\n\n1. **Always** add `// @renderer p5` (2D) or `// @renderer p5 webgl` (WEBGL) as the first line.\n2. **Always** use `render(viji, p5)` (not `draw()`) and `setup(viji, p5)` (not `setup()`).\n3. **Always** prefix every P5 function and constant with `p5.`:\n - `background(0)` -> `p5.background(0)`\n - `fill(255)` -> `p5.fill(255)`\n - `PI` -> `p5.PI`, `TWO_PI` -> `p5.TWO_PI`, `HSB` -> `p5.HSB`\n - `createVector(1, 0)` -> `p5.createVector(1, 0)`\n - `map(v, 0, 1, 0, 255)` -> `p5.map(v, 0, 1, 0, 255)`\n - `noise(x)` -> `p5.noise(x)`, `random()` -> `p5.random()`\n4. **Never** call `createCanvas()`. The canvas is created and managed by Viji.\n5. **Never** use `preload()`. Use `viji.image(null, { label })` for images, or `fetch()` inside `setup()`.\n6. **Never** use P5 event callbacks (`mousePressed`, `mouseDragged`, `mouseReleased`, `keyPressed`, `keyReleased`, `keyTyped`, `touchStarted`, `touchMoved`, `touchEnded`). Read state in `render()` via `viji.pointer`, `viji.mouse`, `viji.keyboard`, `viji.touches`. Use `wasPressed` / `wasReleased` for one-frame edges.\n7. **Never** use `loadImage()`, `loadFont()`, `loadJSON()`, `loadModel()`, `loadShader()`. Use `viji.image()` or `fetch()`.\n8. **Never** use `p5.frameRate()`, `p5.save()`, `p5.saveCanvas()`, `p5.saveFrames()`. The host controls frame rate and capture.\n9. **Never** use `createCapture()` or `createVideo()`. Use `viji.video.*`.\n10. **Never** use `p5.dom` or `p5.sound`. Use Viji parameters and `viji.audio.*`.\n11. **Never** access `window`, `document`, `Image()`, `localStorage`. `fetch()` is available.\n12. **Always** declare parameters at the top level, never inside `render()` or `setup()`.\n13. **Always** read parameters via `.value`. Color parameters also expose `.rgb` (matches `colorMode(RGB, 255)`) and `.hsb` (matches `colorMode(HSB, 360, 100, 100)`). Color defaults accept hex, `{ r, g, b }` (0..255), `{ h, s, b }` (0..360 / 0..100), or CSS color strings.\n14. **Always** use `viji.width` and `viji.height` for canvas dimensions. Never hardcode pixel sizes.\n15. **Always** use `viji.deltaTime` for animation:\n ```javascript\n let angle = 0;\n function render(viji, p5) { angle += speed.value * viji.deltaTime; }\n ```\n16. **Never** multiply `viji.time` by a parameter; it causes visible jumps when the parameter changes. Use a `deltaTime` accumulator. This also applies to nested multiplications; each independent speed gets its own accumulator.\n17. **Never** allocate objects, arrays, or strings inside `render()`. Pre-allocate at the top level and reuse.\n18. **Image parameters with P5:** use `.p5` (a `P5Image`) instead of `.value` when passing to `p5.image()`:\n ```javascript\n const photo = viji.image(null, { label: 'Photo' });\n function render(viji, p5) {\n if (photo.value) p5.image(photo.p5, 0, 0, viji.width, viji.height);\n }\n ```\n19. **Video drawing.** In 2D, use `p5.image(viji.video.currentFrame, ...)` or `p5.drawingContext.drawImage(...)`. In WEBGL, use `p5.image(viji.video.currentFrame, ...)` only (`p5.drawingContext` is WebGL, not Canvas 2D). **Always preserve aspect ratio** via the `videoFit` helper below.\n20. `p5.createGraphics(w, h)` is **2D only** (`createGraphics(w, h, p5.WEBGL)` is not supported). It creates an OffscreenCanvas internally.\n21. Fonts: `p5.textFont()` only with CSS generic names (`monospace`, `serif`, `sans-serif`). `loadFont()` is not available.\n22. `p5.tint()` and `p5.blendMode()` work normally.\n23. **Canvas mode is declared by the `// @renderer` directive.** Use `// @renderer p5` for 2D, `// @renderer p5 webgl` for WEBGL. Never call `createCanvas()` or `createCanvas(..., p5.WEBGL)`.\n24. In WEBGL scenes, `p5.drawingContext` is a WebGL context. Use P5 3D drawing primitives or `p5.image()` / textures for images and video.\n25. `p5.pixelDensity()` defaults to 1 in the worker. `p5.loadPixels()` and `p5.pixels[]` work in 2D scenes; WEBGL pixel readback follows standard P5.js rules.\n26. **Always** check `viji.audio.isConnected` before reading audio data.\n27. **Always** check `viji.video.isConnected && viji.video.currentFrame` before drawing video.\n28. **Never** enable CV features by default; use toggle parameters.\n29. **Always** set `category` on input-dependent parameters (`'audio'`, `'video'`, `'interaction'`). Use creative-strength sliders, not on/off toggles, for inputs the host already gates. CV feature toggles stay opt-in.\n30. **Always** type any function parameter that receives the `viji` instance or the `p5` instance, including `setup(viji, p5)`, `render(viji, p5)`, and any helpers that take either. Monaco's TS checker flags API misuse (wrong nesting, typo'd property names) at edit time. Scene code is transpiled via sucrase before execution, so TS annotations are stripped at runtime — safe to include. Note: the p5-instance type is the lowercase namespace `p5` (same name as the conventional parameter; TS resolves the value and the type from separate namespaces, so `p5: p5` is legal and common).\n\n ```javascript\n function setup(viji: VijiAPI, p5: p5) { ... }\n function render(viji: VijiAPI, p5: p5) { ... }\n function videoFit(viji: VijiAPI, mode: 'cover' | 'contain') { ... }\n ```\n\n## Video aspect helper\n\n```javascript\nfunction videoFit(viji, mode = 'cover') {\n const vw = viji.video.frameWidth, vh = viji.video.frameHeight;\n const w = viji.width, h = viji.height;\n if (!vw || !vh) return { x: 0, y: 0, width: 0, height: 0 };\n const scale = mode === 'cover' ? Math.max(w / vw, h / vh) : Math.min(w / vw, h / vh);\n const dw = vw * scale, dh = vh * scale;\n return { x: (w - dw) / 2, y: (h - dh) / 2, width: dw, height: dh };\n}\n\nif (viji.video.isConnected && viji.video.currentFrame) {\n const v = videoFit(viji, 'cover');\n p5.image(viji.video.currentFrame, v.x, v.y, v.width, v.height);\n}\n\n// CV bounds and landmarks are normalized 0..1 to the source frame.\n// Map them through v to align with the drawn video.\n```\n\n## P5 to Viji mapping\n\n| Standard P5.js | Viji P5 equivalent |\n|---|---|\n| `width` / `height` | `viji.width` / `viji.height` |\n| `mouseX` / `mouseY` | `viji.pointer.x` / `viji.pointer.y` (or `viji.mouse.x` / `viji.mouse.y`) |\n| `mouseIsPressed` | `viji.pointer.isDown` |\n| `mouseButton === LEFT` | `viji.mouse.leftButton` |\n| `keyIsPressed` | `viji.keyboard.activeKeys.size > 0` |\n| `keyIsDown(code)` | `viji.keyboard.isPressed('keyName')` |\n| `key` | `viji.keyboard.lastKeyPressed` |\n| `frameCount` | `viji.frameCount` (or a `viji.deltaTime` accumulator) |\n| `frameRate(n)` | Remove. Host controls frame rate. |\n| `createCanvas(w, h)` | Remove. Canvas is provided. |\n| `preload()` | Remove. Use `viji.image()` or `fetch()` in `setup()`. |\n| `loadImage(url)` | `viji.image(null, { label: 'Image' })`; pass `.p5` to `p5.image()`. |\n| `save()` / `saveCanvas()` | Remove. Host handles capture. |\n| `mousePressed()` / `mouseReleased()` callbacks | Check `viji.pointer.wasPressed` / `wasReleased` inside `render()`. |\n| `keyPressed()` / `keyReleased()` callbacks | Check `viji.keyboard.wasPressed(key)` / `wasReleased(key)` inside `render()`. |\n\n## API reference\n\nAll `viji.*` members are identical to Native (same object, same types).\n\n### Canvas and timing\n\n| Member | Type | Notes |\n|---|---|---|\n| `viji.canvas` | `OffscreenCanvas` | Managed by P5 |\n| `viji.width`, `viji.height` | `number` | Canvas dimensions |\n| `viji.time` | `number` | Seconds since scene start |\n| `viji.deltaTime` | `number` | Seconds since last frame |\n| `viji.frameCount` | `number` | Total frames rendered |\n| `viji.fps` | `number` | Target frame rate |\n\n### Parameters\n\n```javascript\nviji.slider(default, { min?, max?, step?, label, group?, category? }) // .value: number\nviji.color(default, { label, group?, category? }) // .value, .rgb, .hsb\nviji.toggle(default, { label, group?, category? }) // .value: boolean\nviji.select(default, { options, label, group?, category? }) // .value: string | number\nviji.number(default, { min?, max?, step?, label, group?, category? }) // .value: number\nviji.text(default, { label, group?, category?, maxLength? }) // .value: string\nviji.image(null, { label, group?, category? }) // .value: ImageBitmap | null, .p5: P5Image\nviji.button({ label, description?, group?, category? }) // .value: boolean (1 frame)\nviji.coordinate(default, { step?, label, group?, category? }) // .value: { x, y }, range -1..1\n// slider/number `step` defaults to a power of 10 giving ~100 positions across min..max\n// (e.g., range 0..1 → 0.01, range 0..100 → 1). Specify `step: 1` for integer counts.\n```\n\n### Audio: `viji.audio`\n\nCheck `isConnected` first. Members:\n\n`isConnected` (boolean); `volume.{current, peak, smoothed}` (0..1; `smoothed` 200ms decay); `bands.{low, lowMid, mid, highMid, high}` (0..1; ranges 20-120, 120-400, 400-1600, 1600-6000, 6000-16000 Hz) and the sibling `*Smoothed` envelopes (150ms decay); `beat.{kick, snare, hat, any}` (0..1, 300ms decay curves; peak on each detected beat); `beat.{kickSmoothed, snareSmoothed, hatSmoothed, anySmoothed}` (500ms decay envelopes); `beat.triggers.{kick, snare, hat, any}` (boolean, true for exactly one frame, OR-accumulated between frames); `beat.events: Array<{type, time, strength}>` (`type` is `'kick' | 'snare' | 'hat'`; `time` ms; cleared each frame); `beat.bpm` (`0` when no audio is connected; once audio connects it tracks the detected tempo clamped to 60..240, with `120` as a fallback before lock-on), `beat.confidence` (0..1), `beat.isLocked` (boolean); `spectral.{brightness, flatness}` (0..1); `getFrequencyData(): Uint8Array` (1024 FFT bins, 0..255); `getWaveform(): Float32Array` (2048 samples, -1..1).\n\nExternal-device audio (`viji.devices[i].audio`) and host streams (`viji.audioStreams[i]`) follow the `AudioStreamAPI` shape: `isConnected`, `volume`, `bands` (instant + smoothed siblings), `spectral`, `getFrequencyData`, `getWaveform`. No beat, BPM, triggers, or events on streams.\n\n### Video: `viji.video`\n\nCheck `isConnected && currentFrame` before drawing. Members:\n\n`isConnected` (boolean); `currentFrame` (`OffscreenCanvas | ImageBitmap | null`); `frameWidth`, `frameHeight`, `frameRate`; `getFrameData(): ImageData | null`; `cv: VideoCVAPI` (CV outputs and verbs live here, not on `viji.video` directly).\n\n### Computer Vision: `viji.video.cv`\n\nEach CV feature is independent and populates only its own subset of fields. Enable only what you need (each active feature uses a separate WebGL context for MediaPipe; 4+ on low-end hardware risks context-limit failures).\n\n```javascript\n// Verbs return Promise<void>. await is optional. Safe to call from module\n// scope (always-on CV) or per-frame inside render() gated by a viji.toggle(...)\n// (opt-in CV). Idempotent + reference-counted.\nawait viji.video.cv.enableFaceDetection(true | false); // face.bounds, face.center, face.confidence\nawait viji.video.cv.enableFaceMesh(true | false); // face.landmarks (468 pts) + face.headPose\nawait viji.video.cv.enableEmotionDetection(true | false); // face.expressions + face.blendshapes; also loads landmarker\nawait viji.video.cv.enableHandTracking(true | false); // viji.video.cv.hands[]\nawait viji.video.cv.enablePoseDetection(true | false); // viji.video.cv.pose\nawait viji.video.cv.enableBodySegmentation(true | false); // viji.video.cv.segmentation\nviji.video.cv.getActiveFeatures(); // CVFeature[]\nviji.video.cv.isProcessing(); // boolean\n```\n\nData outputs on `viji.video.cv`:\n- `analysedFrame: OffscreenCanvas | null`: the frame paired with the current CV results. `null` until the first inference lands. Use `viji.video.cv.analysedFrame ?? viji.video.currentFrame` to fall back during startup.\n- `getAnalysedFrameData(): ImageData | null`: pixel data of `analysedFrame`.\n\n**`faces: FaceData[]`**: `id` always present. Other fields populated only by their source model; otherwise `null` (`landmarks` is `[]` when empty). Always null-check.\n- `bounds: {x,y,width,height} | null` — populated by face detection\n- `center: {x,y} | null` — populated by face detection\n- `confidence: number | null` — populated by face detection\n- `landmarks: {x,y,z?}[]` (empty `[]` unless face mesh enabled)\n- `headPose: {pitch, yaw, roll} | null` — populated by face mesh\n- `expressions: {neutral, happy, sad, angry, surprised, disgusted, fearful} | null` (0..1 each) — populated by emotion detection\n- `blendshapes | null` (52 ARKit coefficients 0..1) — populated by emotion detection\n\nIf you need `bounds` while only running face mesh, either also enable face detection or compute it from `face.landmarks` min/max.\n\n**`hands: HandData[]`**: `id`, `handedness: 'left' | 'right'`, `confidence`, `bounds`, `landmarks` (21 points), `palm`, `gestures` (`fist`, `openPalm`, `peace`, `thumbsUp`, `thumbsDown`, `pointing`, `iLoveYou` 0..1).\n\n**`pose: PoseData | null`**: `confidence`, `landmarks` (33 points), body-part arrays `face`, `torso`, `leftArm`, `rightArm`, `leftLeg`, `rightLeg`.\n\n**`segmentation: SegmentationData | null`**: `mask: Uint8Array` (each byte is `0` for background or `1` for person; length = `width * height`), `width`, `height`.\n\n### Input\n\n- `viji.pointer`: `x`, `y`, `deltaX`, `deltaY`, `isDown`, `wasPressed`, `wasReleased`, `isInCanvas`, `type` (`'mouse' | 'touch' | 'none'`).\n- `viji.mouse`: `x`, `y`, `isInCanvas`, `isPressed`, `leftButton`, `rightButton`, `middleButton`, `deltaX`, `deltaY`, `wheelDelta`, `wheelX`, `wheelY`, `wasPressed`, `wasReleased`, `wasMoved`.\n- `viji.keyboard`: `isPressed(key)`, `wasPressed(key)`, `wasReleased(key)`, `activeKeys`, `pressedThisFrame`, `releasedThisFrame`, `lastKeyPressed`, `lastKeyReleased`, `shift`, `ctrl`, `alt`, `meta`.\n- `viji.touches`: `count`, `points`, `started`, `moved`, `ended`, `primary`. Each `TouchPoint`: `id`, `x`, `y`, `pressure`, `radius`, `radiusX`, `radiusY`, `rotationAngle`, `force`, `isInCanvas`, `deltaX`, `deltaY`, `velocity`, `isNew`, `isActive`, `isEnding`.\n\n### Sensors and external devices\n\n`viji.device.motion`: `acceleration {x,y,z}` (m/s²), `accelerationIncludingGravity`, `rotationRate {alpha,beta,gamma}` (deg/s), `interval` (ms).\n`viji.device.orientation`: `alpha` (0-360°), `beta` (-180..180°), `gamma` (-90..90°), `absolute`.\n\n`viji.devices: DeviceState[]`: each entry: `id`, `name`, `motion`, `orientation`, `video` (`VideoAPI | null`, no CV), `audio` (`AudioStreamAPI | null`, lightweight).\n\n### Host streams\n\n`viji.videoStreams: VideoAPI[]`: extra video sources. Same shape as `viji.video`, including a per-stream `cv` surface (same verbs and result fields; `cv.analysedFrame` is main-stream only). May be empty.\n`viji.audioStreams: AudioStreamAPI[]`: extra audio sources. May be empty. Lightweight shape (no beat / BPM).\n\n## Template\n\n```javascript\n// @renderer p5\n\nconst bgColor = viji.color('#1a1a2e', { label: 'Background' });\nconst speed = viji.slider(1, { min: 0.1, max: 5, label: 'Speed' }); // implicit step → 0.01\nconst count = viji.slider(8, { min: 3, max: 30, step: 1, label: 'Count' }); // explicit step for integer counts\n\nlet angle = 0;\n\nfunction setup(viji, p5) {\n p5.colorMode(p5.HSB, 360, 100, 100);\n}\n\nfunction render(viji, p5) {\n angle += speed.value * viji.deltaTime;\n p5.background(bgColor.value);\n\n const cx = viji.width / 2;\n const cy = viji.height / 2;\n const radius = p5.min(viji.width, viji.height) * 0.3;\n const dotSize = p5.min(viji.width, viji.height) * 0.04;\n const n = p5.floor(count.value);\n\n p5.noStroke();\n for (let i = 0; i < n; i++) {\n const a = angle + (i / n) * p5.TWO_PI;\n const x = cx + p5.cos(a) * radius;\n const y = cy + p5.sin(a) * radius;\n p5.fill((i / n) * 360, 80, 90);\n p5.circle(x, y, dotSize);\n }\n}\n```\n",
9854
9854
  "shader": "# Viji API: Shader Renderer Reference\n\nShader scenes are GLSL fragment shaders running on a fullscreen quad. Write helper functions and `void main()`. Precision and all uniforms (built-in plus those generated from `@viji-*` directives) are auto-injected by Viji. The first line must be `// @renderer shader` (or `#version 300 es` followed by `// @renderer shader` for GLSL ES 3.00).\n\n## Architecture\n\n- The shader is **GLSL ES 1.00** by default. Opt into ES 3.00 by making `#version 300 es` the literal first line.\n- Viji auto-injects `precision mediump float;` and every uniform declaration. Never declare them yourself.\n- ES 3.00 requires `out vec4 fragColor;` before `main` and `fragColor = ...` instead of `gl_FragColor`. ES 3.00 uses `texture()` instead of `texture2D()`.\n- If the shader references `fwidth`, Viji auto-injects `#extension GL_OES_standard_derivatives : enable`.\n- The `backbuffer` sampler is auto-enabled when referenced. It carries the previous frame.\n\n## Rules\n\n1. **Always** add `// @renderer shader` as the first line (or as line 2 after `#version 300 es`).\n2. **Never** declare `precision mediump float;` or `precision highp float;`. Viji auto-injects precision.\n3. **Never** redeclare built-in uniforms (`u_time`, `u_resolution`, `u_mouse`, audio / video / CV uniforms, etc.). They are auto-injected.\n4. **Never** redeclare parameter uniforms. They are auto-generated from `@viji-*` directives.\n5. **Never** use the `u_` prefix for your own parameter names. It is reserved for Viji's built-ins. Use descriptive names (`speed`, `colorMix`, `intensity`).\n6. `@viji-*` parameter directives work with `//` comments only. Block comments (`/* */`) are not parsed.\n7. **Always** use `@viji-accumulator` instead of `u_time * speed` for parameter-driven animation. Multiplying `u_time` by a parameter causes visible jumps when sliders change.\n\n ```glsl\n // @viji-slider:speed label:\"Speed\" default:1.0 min:0.1 max:5.0\n // @viji-accumulator:phase rate:speed\n float wave = sin(phase);\n ```\n\n This also applies to **nested** multiplications. Each independent speed needs its own accumulator:\n\n ```glsl\n // @viji-accumulator:phase rate:speed\n // @viji-accumulator:rotPhase rate:rotSpeed\n ```\n8. For `backbuffer` (previous frame): reference it directly in code. Viji auto-detects and enables it. Sample with `texture2D(backbuffer, uv)` (ES 1.00) or `texture(backbuffer, uv)` (ES 3.00).\n9. Remove any `#ifdef GL_ES` / `precision` blocks; Viji handles them.\n10. **Always** set `category` on input-dependent directives: `category:audio`, `category:video`, `category:interaction`. Use creative-strength sliders, not on/off toggles, for inputs the host already gates.\n11. **CV features must be activated via `// @viji-cv:<featureToken>` directives.** Without the matching `@viji-cv:` activation, CV uniforms (`u_faceCount`, `u_face0*`, `u_handCount`, `u_poseDetected`, `u_segmentationMask`, etc.) read zero. Use the bare form (`// @viji-cv:faceDetection`) for always-on detection, or the toggleable form (`// @viji-cv:faceDetection label:\"Face Detection\" default:false`) which synthesizes both a host-side toggle parameter and a `bool <featureToken>` shader uniform mirroring its state. The six tokens: `faceDetection`, `faceMesh`, `emotionDetection`, `handTracking`, `poseDetection`, `bodySegmentation`.\n\n## Parameter directives\n\n```glsl\n// @viji-slider:name label:\"Label\" default:1.0 min:0.0 max:5.0 step:0.1\n// -> uniform float name;\n\n// @viji-color:name label:\"Color\" default:#ff6600\n// -> uniform vec3 name; (RGB 0..1)\n// default: accepts #rrggbb, #rgb, vec3(r,g,b) (0..1), rgb(r,g,b) (0..255),\n// hsl(h, s%, l%) (h: 0..360), hsb(h, s, b) (h: 0..360, s/b: 0..100)\n\n// @viji-toggle:name label:\"Toggle\" default:false\n// -> uniform bool name;\n\n// @viji-select:name label:\"Mode\" default:0 options:[\"Solid\",\"Gradient\",\"Noise\"]\n// -> uniform int name; (0-based)\n\n// @viji-number:name label:\"Count\" default:10.0 min:1.0 max:100.0 step:1.0\n// -> uniform float name;\n\n// @viji-image:name label:\"Texture\"\n// -> uniform sampler2D name;\n\n// @viji-button:name label:\"Reset\"\n// -> uniform bool name; (true for one frame on press)\n\n// @viji-coordinate:name label:\"Origin\" default:[0.0,0.0]\n// -> uniform vec2 name; (each component -1..1)\n\n// @viji-accumulator:name rate:speed\n// -> uniform float name; (CPU-side: += rate * deltaTime each frame)\n```\n\nAll directives support `group:\"GroupName\"` and `category:\"audio|video|interaction|general\"`.\n\n## Uniform reference\n\nAll uniforms below are always available. Do not declare them.\n\n### Core\n\n| Uniform | Type | Description |\n|---|---|---|\n| `u_resolution` | `vec2` | Canvas width and height in pixels |\n| `u_time` | `float` | Seconds since scene start |\n| `u_deltaTime` | `float` | Seconds since last frame |\n| `u_frame` | `int` | Current frame number |\n| `u_fps` | `float` | Target frame rate (host-controlled) |\n\n### Mouse\n\n| Uniform | Type | Description |\n|---|---|---|\n| `u_mouse` | `vec2` | Mouse position in pixels (WebGL coords, bottom-left origin) |\n| `u_mouseInCanvas` | `bool` | Mouse inside canvas |\n| `u_mousePressed` | `bool` | Any button pressed |\n| `u_mouseLeft`, `u_mouseRight`, `u_mouseMiddle` | `bool` | Specific buttons |\n| `u_mouseDelta` | `vec2` | Per-frame movement |\n| `u_mouseWheel` | `float` | Scroll wheel delta |\n| `u_mouseWasPressed`, `u_mouseWasReleased` | `bool` | One-frame edges |\n\n### Pointer (unified mouse / touch)\n\n| Uniform | Type | Description |\n|---|---|---|\n| `u_pointer` | `vec2` | Primary input position in pixels (WebGL coords) |\n| `u_pointerDelta` | `vec2` | Primary input movement delta |\n| `u_pointerDown` | `bool` | Primary input active |\n| `u_pointerWasPressed`, `u_pointerWasReleased` | `bool` | One-frame edges |\n| `u_pointerInCanvas` | `bool` | Inside canvas |\n\n### Keyboard\n\n| Uniform | Type | Description |\n|---|---|---|\n| `u_keySpace`, `u_keyShift`, `u_keyCtrl`, `u_keyAlt` | `bool` | Modifier and space |\n| `u_keyW`, `u_keyA`, `u_keyS`, `u_keyD` | `bool` | WASD |\n| `u_keyUp`, `u_keyDown`, `u_keyLeft`, `u_keyRight` | `bool` | Arrows |\n| `u_keyboard` | `sampler2D` | Full keyboard state (256x3 LUMINANCE). Row 0 = held, row 1 = pressed-this-frame, row 2 = toggle. Access: `texelFetch(u_keyboard, ivec2(keyCode, row), 0).r` |\n\n### Touch\n\n| Uniform | Type | Description |\n|---|---|---|\n| `u_touchCount` | `int` | Active touches (0-5) |\n| `u_touch0`-`u_touch4` | `vec2` | Touch point positions in pixels |\n\n### Audio (scalars)\n\n| Uniform | Type | Description |\n|---|---|---|\n| `u_audioVolume`, `u_audioPeak` | `float` | RMS / peak 0..1 |\n| `u_audioVolumeSmoothed` | `float` | 200ms decay envelope of `u_audioVolume` |\n| `u_audioLow`, `u_audioLowMid`, `u_audioMid`, `u_audioHighMid`, `u_audioHigh` | `float` | Band energies 0..1 (20-120, 120-400, 400-1600, 1600-6000, 6000-16000 Hz) |\n| `u_audioLowSmoothed` - `u_audioHighSmoothed` | `float` | 150ms decay envelopes of the bands |\n| `u_audioKick`, `u_audioSnare`, `u_audioHat`, `u_audioAny` | `float` | Beat energy curves with 300ms decay; peak on each detected beat then fall off |\n| `u_audioKickSmoothed` - `u_audioAnySmoothed` | `float` | Smoother 500ms decay envelopes |\n| `u_audioKickTrigger`, `u_audioSnareTrigger`, `u_audioHatTrigger`, `u_audioAnyTrigger` | `bool` | True for exactly one frame on the matching beat, then auto-resets; OR-accumulated between frames |\n| `u_audioBPM`, `u_audioConfidence`, `u_audioIsLocked` | `float`, `float`, `bool` | `u_audioBPM` is `0.0` when no audio is connected; once audio connects it tracks the detected tempo (clamped to 60..240) with `120.0` as a fallback before lock-on. `u_audioConfidence` 0..1. `u_audioIsLocked` true on stable tempo lock |\n| `u_audioBrightness`, `u_audioFlatness` | `float` | Spectral features 0..1 |\n\n### Audio (textures, main source only)\n\n| Uniform | Type | Description |\n|---|---|---|\n| `u_audioFFT` | `sampler2D` | FFT spectrum (1024 bins, 0..255) |\n| `u_audioWaveform` | `sampler2D` | Time-domain waveform (-1..1) |\n\nAdditional audio streams and device audio expose scalar uniforms only (no FFT / waveform textures): see \"Streams\" below.\n\n### Video\n\n| Uniform | Type | Description |\n|---|---|---|\n| `u_video` | `sampler2D` | Most recent video frame |\n| `u_videoAnalysed` | `sampler2D` | Frame paired with current CV uniforms |\n| `u_videoAnalysedAvailable` | `bool` | True after first CV result lands |\n| `u_videoResolution` | `vec2` | Source frame size in pixels |\n| `u_videoFrameRate` | `float` | Source frame rate |\n| `u_videoConnected` | `bool` | Video source active |\n\n**Aspect handling.** Camera frames almost never match the canvas aspect. Sampling `texture2D(u_video, uv)` with canvas UV stretches the video and misaligns CV uniforms. Use these helpers at the top of every video / CV shader:\n\n```glsl\n// mode: 1 = cover (fills canvas, video edges cropped)\n// 0 = contain (fits video, canvas letterboxed)\nvec2 vijiVideoUV(vec2 uv, int mode) {\n vec2 canvas = u_resolution;\n vec2 video = u_videoResolution;\n if (video.x == 0.0 || video.y == 0.0) return uv;\n float canvasAspect = canvas.x / canvas.y;\n float videoAspect = video.x / video.y;\n vec2 scale = vec2(1.0);\n if (mode == 1) {\n scale = canvasAspect > videoAspect\n ? vec2(1.0, canvasAspect / videoAspect)\n : vec2(videoAspect / canvasAspect, 1.0);\n } else {\n scale = canvasAspect > videoAspect\n ? vec2(videoAspect / canvasAspect, 1.0)\n : vec2(1.0, canvasAspect / videoAspect);\n }\n return (uv - 0.5) / scale + 0.5;\n}\n\nbool vijiInVideo(vec2 uv) {\n return all(greaterThanEqual(uv, vec2(0.0))) && all(lessThanEqual(uv, vec2(1.0)));\n}\n```\n\nDefault to mode `1` (cover) for live cameras. Use `0` (contain) for CV-overlay shaders where features near edges must stay visible. CV uniforms (`u_face0Center`, etc.) share the same bottom-up convention as `videoUV`; compare them directly without flipping.\n\n**`u_video` vs `u_videoAnalysed`.** Default to `u_video` for displayed video. Reach for `u_videoAnalysed` only when sampling pixels at CV-derived positions (segmentation compositing onto the body, sampling skin under face landmarks, face-mesh texturing). For shaders that consume CV uniforms without sampling at CV positions, stay on `u_video` (`u_videoAnalysed` advances only on inference completion and will stutter the displayed video). When `u_videoAnalysed` is the right choice, gate on `u_videoAnalysedAvailable` for the brief startup window:\n\n```glsl\nvec4 source = u_videoAnalysedAvailable\n ? texture2D(u_videoAnalysed, uv)\n : texture2D(u_video, uv);\n```\n\n### CV: face detection\n\n| Uniform | Type | Description |\n|---|---|---|\n| `u_faceCount` | `int` | Detected faces (0-1) |\n| `u_face0Bounds` | `vec4` | `(x, y, width, height)` normalized 0..1 |\n| `u_face0Center` | `vec2` | Center normalized 0..1 |\n| `u_face0HeadPose` | `vec3` | `(pitch, yaw, roll)` in degrees |\n| `u_face0Confidence` | `float` | 0..1 |\n| `u_face0Neutral` - `u_face0Fearful` | `float` | 7 expression scores 0..1 |\n\n**52 blendshape uniforms** (all `float`, 0..1, ARKit names prefixed with `u_face0`):\n`BrowDownLeft`, `BrowDownRight`, `BrowInnerUp`, `BrowOuterUpLeft`, `BrowOuterUpRight`, `CheekPuff`, `CheekSquintLeft`, `CheekSquintRight`, `EyeBlinkLeft`, `EyeBlinkRight`, `EyeLookDownLeft`, `EyeLookDownRight`, `EyeLookInLeft`, `EyeLookInRight`, `EyeLookOutLeft`, `EyeLookOutRight`, `EyeLookUpLeft`, `EyeLookUpRight`, `EyeSquintLeft`, `EyeSquintRight`, `EyeWideLeft`, `EyeWideRight`, `JawForward`, `JawLeft`, `JawOpen`, `JawRight`, `MouthClose`, `MouthDimpleLeft`, `MouthDimpleRight`, `MouthFrownLeft`, `MouthFrownRight`, `MouthFunnel`, `MouthLeft`, `MouthLowerDownLeft`, `MouthLowerDownRight`, `MouthPressLeft`, `MouthPressRight`, `MouthPucker`, `MouthRight`, `MouthRollLower`, `MouthRollUpper`, `MouthShrugLower`, `MouthShrugUpper`, `MouthSmileLeft`, `MouthSmileRight`, `MouthStretchLeft`, `MouthStretchRight`, `MouthUpperUpLeft`, `MouthUpperUpRight`, `NoseSneerLeft`, `NoseSneerRight`, `TongueOut`.\n\n### CV: hands\n\n| Uniform | Type | Description |\n|---|---|---|\n| `u_handCount` | `int` | Detected hands (0-2) |\n| `u_leftHandPalm`, `u_rightHandPalm` | `vec3` | Palm position `(x, y, z)` |\n| `u_leftHandConfidence`, `u_rightHandConfidence` | `float` | 0..1 |\n| `u_leftHandBounds`, `u_rightHandBounds` | `vec4` | Bounds normalized 0..1 |\n| `u_leftHandFist` - `u_leftHandILoveYou` | `float` | 7 left-hand gesture scores |\n| `u_rightHandFist` - `u_rightHandILoveYou` | `float` | 7 right-hand gesture scores |\n\n### CV: pose\n\n| Uniform | Type | Description |\n|---|---|---|\n| `u_poseDetected` | `bool` | Pose detected |\n| `u_poseConfidence` | `float` | 0..1 |\n| `u_nosePosition` | `vec2` | Nose landmark, normalized 0..1 |\n| `u_leftShoulderPosition` … `u_rightAnklePosition` | `vec2` | Joint landmarks (shoulders, elbows, wrists, hips, knees, ankles) |\n\n### CV: body segmentation\n\n| Uniform | Type | Description |\n|---|---|---|\n| `u_segmentationMask` | `sampler2D` | 0 = background, 1 = person |\n| `u_segmentationRes` | `vec2` | Mask resolution in pixels |\n\n### Device sensors\n\n| Uniform | Type | Description |\n|---|---|---|\n| `u_deviceAcceleration` | `vec3` | Acceleration without gravity (m/s²) |\n| `u_deviceAccelerationGravity` | `vec3` | Acceleration including gravity (m/s²) |\n| `u_deviceRotationRate` | `vec3` | Rotation rate (deg/s) |\n| `u_deviceOrientation` | `vec3` | `(alpha, beta, gamma)` in degrees |\n| `u_deviceOrientationAbsolute` | `bool` | Using magnetometer |\n\n### External devices\n\n| Uniform | Type | Description |\n|---|---|---|\n| `u_deviceCount` | `int` | Device video sources (0-8) |\n| `u_externalDeviceCount` | `int` | External devices (0-8) |\n| `u_device0` - `u_device7` | `sampler2D` | Device camera textures |\n| `u_device0Resolution` - `u_device7Resolution` | `vec2` | Per-device resolution |\n| `u_device0Connected` - `u_device7Connected` | `bool` | Per-device connection |\n| `u_device0Acceleration` - `u_device7Acceleration` | `vec3` | Per-device acceleration (no gravity) |\n| `u_device0AccelerationGravity` - `u_device7AccelerationGravity` | `vec3` | Per-device acceleration (with gravity) |\n| `u_device0RotationRate` - `u_device7RotationRate` | `vec3` | Per-device rotation rate |\n| `u_device0Orientation` - `u_device7Orientation` | `vec3` | Per-device orientation |\n\n### Streams (compositor)\n\nVideo streams (host-provided):\n\n| Uniform | Type | Description |\n|---|---|---|\n| `u_videoStreamCount` | `int` | Active streams (0-8) |\n| `u_videoStream0` - `u_videoStream7` | `sampler2D` | Stream textures |\n| `u_videoStream0Resolution` - `u_videoStream7Resolution` | `vec2` | Per-stream resolution |\n| `u_videoStream0Connected` - `u_videoStream7Connected` | `bool` | Per-stream connection |\n\nAudio streams (additional sources, including device audio):\n\n| Uniform | Type | Description |\n|---|---|---|\n| `u_audioStreamCount` | `int` | Active streams (0-8) |\n| `u_audioStream0Connected` - `u_audioStream7Connected` | `bool` | Per-slot active |\n| `u_audioStream{i}Volume` | `float` | Volume 0..1 |\n| `u_audioStream{i}Low` - `u_audioStream{i}High` | `float` | Band energies 0..1 |\n| `u_audioStream{i}Brightness`, `u_audioStream{i}Flatness` | `float` | Spectral features 0..1 |\n\n`i` ranges 0..7. **Scalars only**: no `u_audioFFT` / `u_audioWaveform` per stream. Beat / BPM / trigger uniforms are main audio only.\n\n### Backbuffer\n\n| Uniform | Type | Description |\n|---|---|---|\n| `backbuffer` | `sampler2D` | Previous frame (auto-enabled when referenced) |\n\nNo `u_` prefix. RGBA 8-bit, `LINEAR` filtering, `CLAMP_TO_EDGE` wrapping. First frame samples as black. Clears on canvas resize.\n\n## Template\n\n```glsl\n// @renderer shader\n// @viji-slider:speed label:\"Speed\" default:1.0 min:0.1 max:5.0\n// @viji-color:baseColor label:\"Color\" default:#ff6600\n// @viji-accumulator:phase rate:speed\n\nvoid main() {\n vec2 uv = gl_FragCoord.xy / u_resolution;\n\n float wave = sin(uv.x * 10.0 + phase) * 0.5 + 0.5;\n float pulse = 1.0 + u_audioLow * 0.5;\n vec3 color = baseColor * wave * pulse;\n\n gl_FragColor = vec4(color, 1.0);\n}\n```\n"
9855
9855
  },
9856
9856
  "conversionGuides": {
@@ -1,4 +1,4 @@
1
- import { g as kn } from "./index-Cvk3ZfuY.js";
1
+ import { g as kn } from "./index-C-CGSJSn.js";
2
2
  function jn(Oe, ot) {
3
3
  for (var ve = 0; ve < ot.length; ve++) {
4
4
  const Z = ot[ve];
@@ -3374,4 +3374,4 @@ const xn = /* @__PURE__ */ kn(yi), Ln = /* @__PURE__ */ jn({
3374
3374
  export {
3375
3375
  Ln as e
3376
3376
  };
3377
- //# sourceMappingURL=essentia-wasm.web-DDcSN_G4.js.map
3377
+ //# sourceMappingURL=essentia-wasm.web--e5vDL35.js.map