frameyard 0.2.0 → 0.3.0

package/docs/adr/0006-seek-embedded-media-from-the-frame-clock.md

@@ -0,0 +1,9 @@
+# Seek embedded media from the frame clock
+
+Embedded video and WebGL are supported without giving compositions a second clock. A composition places local `<video>` elements in `mount()`, describes their timeline position with `data-frameyard-*` attributes, and the runtime pauses and seeks them to the frame-derived source time after every synchronous `update()`. The renderer and `check` await that media settle before capturing or advancing.
+
+The browser remains the source of truth for video presentation, while production audio stays offline and deterministic: an unmuted video's audio track is inspected and mixed by FFmpeg alongside declarative audio clips. Video elements are silenced in the browser. Live streams, remote media, and sources that cannot be seeked are rejected because they cannot be reproduced from a frame number.
+
+WebGL canvases are redrawn from the current frame and captured through the existing screenshot path. Production Chromium pins SwiftShader so WebGL pixels do not vary with the host GPU. The runtime waits two animation frames after a media seek to ensure the decoded video frame and WebGL drawing are composited before capture.
+
+`update()` stays synchronous and seek-safe. Media preparation happens during loading, and only the settle step is asynchronous. This preserves the existing lifecycle contract while extending deterministic frame production to embedded video and WebGL.

package/docs/v0.1-design.md

@@ -47,7 +47,9 @@ The only sources of animation state are the integer frame number and immutable p
 - Assets, remote data, images, fonts, generated audio, and transcripts are resolved during an explicit loading phase before frame production.
 - Required network access is limited to the loading phase. Network requests are blocked during frame capture.
 - AI model inference is never part of `mount()`, `update()`, or frame production.
-- v0.1 supports DOM, CSS, SVG, and Canvas 2D that is redrawn from the current frame.
+- v0.1 supports DOM, CSS, SVG, Canvas 2D, WebGL, and frame-seeked embedded video.
+- Embedded video is loaded during the loading phase, paused, and seeked from the frame-derived source time before capture. Its source must be a local, seekable project file. An unmuted embedded video contributes its audio track to the offline mix unless the composition opts out.
+- WebGL 1 and WebGL 2 canvases are redrawn from the current frame and captured through the normal screenshot path. Production sessions pin software WebGL so pixel output does not depend on the host GPU.
 - Composition code is trusted local developer code. Frameyard does not claim to render untrusted compositions safely.
 
 ### Reproducibility guarantee
@@ -74,15 +76,15 @@ Production rendering:
 
 1. Uses a pinned, Playwright-managed Chromium through the Chrome DevTools Protocol.
 2. Seeks explicitly to each requested frame.
-3. Waits for composition readiness.
+3. Waits for composition readiness, embedded media decode, and browser paint.
 4. Captures the frame.
-5. Pipes frames to FFmpeg for encoding and audio muxing.
+5. Pipes frames in timeline order to FFmpeg for encoding and audio muxing.
 
-The reference renderer is sequential and has no frame cache. Its design nevertheless permits future parallel and out-of-order rendering because each frame is independently seekable. Simple scenes target real-time preview, but v0.1 has no production rendering-speed guarantee; it reports progress, current frame, and elapsed time.
+The renderer is sequential by default and has no frame cache. Optional `--concurrency <n>` rendering, with `n` from 1 to 16, opens up to `n` independent browser sessions, each loading the same composition and props from the same preview server. Workers seek and capture assigned frames, while a single ordered writer pipes them to FFmpeg with bounded lookahead. Parallelism therefore changes scheduling only: every frame remains derivable from its integer frame number and immutable props, and output order is unchanged. The default stays at one session for predictable resource use, and the upper bound limits accidental browser-process and memory growth. Simple scenes target real-time preview, but there is no production rendering-speed guarantee; the CLI reports progress, current frame, and elapsed time.
 
 ## 5. Audio and output
 
-Audio is a declarative timeline of clips. Each clip supports a source, start frame, source trim, and volume. Preview synchronizes this timeline in the browser; production mixes it offline with FFmpeg rather than treating live `<audio>` or Web Audio playback as the production source of truth.
+Audio is a declarative timeline of clips. Each clip supports a source, start frame, source trim, and volume. Preview synchronizes this timeline in the browser; production mixes it offline with FFmpeg rather than treating live `<audio>` or Web Audio playback as the production source of truth. Unmuted embedded videos contribute their own audio tracks to the same mix, placed by the video's frame and trim attributes. Muting a video or setting `data-frameyard-audio="off"` keeps it visual-only.
 
 Generated speech behaves like any other audio clip. Frameyard does not automatically time-stretch it, change composition duration, or regenerate it to fit. `check` reports clip durations and overrun warnings so users and agents can revise text, timing, or composition deliberately.
 
@@ -96,7 +98,7 @@ Project and composition commands are:
 
 - `init`: scaffold a minimal project, including an agent guide (`AGENTS.md`) and a private `package.json` with pinned `dev`, `check`, `render`, and `still` scripts
 - `dev`: start the preview
-- `render <composition-id> --props props.json --out video.mp4`: render a composition
+- `render <composition-id> --props props.json --out video.mp4`: render a composition, optionally with `--concurrency <n>` from 1 to 16
 - `check <composition-id>`: validate configuration, props, assets, audio, and transcript requirements, then mount the composition and evaluate every frame without capturing or encoding
 - `still <composition-id> --frame <n>`: capture one requested frame as PNG through the production rendering path
 - `doctor`: inspect core requirements and optional integrations without repairing or downloading anything
@@ -120,7 +122,7 @@ Frameyard ships an optional, host-neutral agent skill with a `SKILL.md`, support
 The skill:
 
 - Uses the normal project format, composition API, and CLI available to humans.
-- Does not introduce an agent-only runtime, MCP server, hosted control plane, or parallel rendering API.
+- Does not introduce an agent-only runtime, MCP server, hosted control plane, or separate rendering API.
 - Triggers for Frameyard authoring and operation rather than claiming to be a general-purpose media-generation skill.
 - Uses Frameyard's audio wrappers instead of invoking `qwen-tts` or `whisper-cli` directly.
 - Includes references for deterministic authoring, audio assets, subtitle and karaoke recipes, JSON command results, and failure recovery.
@@ -198,10 +200,12 @@ Release documentation must identify the exact `qwentts.cpp` and `whisper.cpp` re
 
 Tests prioritize public behavior and end-to-end evidence over implementation-only unit coverage:
 
-- Sequential, individual, and random-order frame renders produce equivalent images.
+- Sequential, individual, random-order, and parallel frame renders produce equivalent images.
 - Determinism checks compare captured pixels or stills; encoded MP4 bytes are not a reproducibility oracle.
 - Preview and renderer use the same runtime.
 - CSS, JavaScript, SVG, and Canvas animations seek correctly.
+- Embedded video frames seek deterministically, including under parallel capture, and their audio appears in the offline mix.
+- WebGL canvases produce deterministic stills and rendered frames under the pinned software renderer.
 - Asset and runtime failures produce nonzero CLI exits.
 - Rendered MP4 files have the expected dimensions, frame count, duration, and audio stream.
 - Golden-frame tests cover representative visual behavior.
@@ -224,9 +228,9 @@ These are scope boundaries, not permanent rejections:
 - React or other UI-framework adapters in the core
 - Public plugin APIs and a broad integration ecosystem
 - Dynamic composition metadata derived from props
-- Embedded video, WebGL, live streams, and other independently timed visual content
+- Live streams, remote media, and other visual sources that cannot be deterministically seeked
 - Dynamic per-frame asset loading or AI inference
-- Parallel rendering and frame caches
+- Frame caches
 - Additional final-video formats, general image-sequence output, and arbitrary FFmpeg argument passthrough
 - Built-in TypeScript transpilation and npm dependency optimization
 - Rendering untrusted compositions

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "frameyard",
-  "version": "0.2.0",
+  "version": "0.3.0",
   "description": "Local-first, deterministic video composition framework for HTML, CSS, and JavaScript.",
   "type": "module",
   "license": "MIT",
@@ -65,12 +65,16 @@
     "build": "tsc -p tsconfig.json",
     "typecheck": "tsc -p tsconfig.json --noEmit",
     "typecheck:test": "tsc -p tsconfig.test.json",
+    "check:scripts": "node scripts/check-scripts.mjs",
     "lint": "oxlint",
     "format": "oxfmt",
     "format:check": "oxfmt --check",
+    "check:static": "pnpm check:scripts && pnpm lint && pnpm format:check && pnpm typecheck && pnpm typecheck:test",
+    "check": "pnpm check:static && pnpm test",
     "test": "pnpm build && vitest run",
-    "test:unit": "vitest run test/contracts.test.ts test/docs.test.ts test/random.test.ts test/scaffold.test.ts test/transcript.test.ts",
+    "test:unit": "vitest run test/contracts.test.ts test/docs.test.ts test/random.test.ts test/release.test.ts test/scaffold.test.ts test/transcript.test.ts",
     "bench": "pnpm build && node scripts/bench-render.mjs",
-    "set-version": "node scripts/set-version.mjs"
+    "set-version": "node scripts/set-version.mjs",
+    "verify-release": "node scripts/verify-release.mjs"
   }
 }

package/skills/frameyard/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: frameyard
 description: Create, validate, inspect, and render deterministic Frameyard videos authored with HTML, CSS, and native JavaScript. Use when a user asks to build or operate a Frameyard project, produce parameterized video, add a local TTS or transcription asset through Frameyard's wrappers, or diagnose a Frameyard render. Do not use for general-purpose media generation outside a Frameyard project.
-frameyard-api-version: 0.2.0
+frameyard-api-version: 0.3.0
 ---
 
 # Frameyard
@@ -17,12 +17,12 @@ Use Frameyard's public project format and CLI. Do not introduce an agent-only ru
 
 ## Workflow
 
-1. **Author** the composition with HTML, CSS, SVG, Canvas 2D, and native JavaScript ES modules. Keep `width`, `height`, `fps`, and `durationInFrames` fixed. Read [deterministic authoring](references/deterministic-authoring.md).
+1. **Author** the composition with HTML, CSS, SVG, Canvas 2D, WebGL, embedded video, and native JavaScript ES modules. Keep `width`, `height`, `fps`, and `durationInFrames` fixed. Read [deterministic authoring](references/deterministic-authoring.md).
 2. **Provide props** as JSON. Validate domain-specific requirements with `validateProps()`.
 3. **Produce optional assets** with `npx frameyard tts` and `npx frameyard transcribe`, never with the upstream executables. Read [audio assets](references/audio-assets.md). Do not overwrite or regenerate existing generated assets without permission.
 4. **Validate** with `npx frameyard check <id> --json`. It evaluates every frame, checks audio and declared transcripts, and does not encode.
 5. **Inspect** representative frames with `npx frameyard still <id> --frame <n> --out <file.png> --json`. View the PNGs when the host supports image viewing. If visual inspection is impossible, say so explicitly.
-6. **Render** with `npx frameyard render <id> --out <file.mp4> [--props <file>] --json`.
+6. **Render** with `npx frameyard render <id> --out <file.mp4> [--props <file>] [--concurrency <n>] --json`.
 7. **Report** the project path, composition entry, props, generated assets, provenance manifest, validation results, still paths and inspection status, final video path, and any failures.
 
 Never claim success when `check` fails, a still is missing, a render exits nonzero, or visual inspection was skipped. A partial or unpublished render is not a successful result.
@@ -32,6 +32,7 @@ Never claim success when `check` fails, a still is missing, a render exits nonze
 - Use `--json` for every non-interactive command. Parse stdout as the structured result; treat stderr as progress and diagnostics.
 - Keep the current output files unless the user explicitly approves replacement. Use `--force` only with that approval.
 - Use `--project <dir>` when not running from the project root.
+- Use `--concurrency <n>` only when faster local rendering is useful; the default is `1` and the maximum is `16`. It changes capture scheduling only, not composition behavior or output frame order.
 - Use `npx frameyard doctor --json` before reporting an environment problem. It inspects without repairing or downloading.
 - Read [JSON results](references/json-results.md) for result and error shapes, and [failure recovery](references/failure-recovery.md) for actionable error codes.
 
@@ -42,6 +43,8 @@ Never claim success when `check` fails, a still is missing, a render exits nonze
 - Never use `Date.now()`, `performance.now()`, timers, independent `requestAnimationFrame` loops, or CSS transitions as animation inputs.
 - Finish remote data, fonts, and image loading during `mount()` or before the first frame. Network requests are blocked during frame capture.
 - Keep `update({ frame, props })` synchronous and seek-safe. Do not carry state between frames.
+- Let the runtime seek embedded `<video>` elements. Do not play, pause, or set `currentTime` in `update()`; use the `data-frameyard-*` attributes described in [deterministic authoring](references/deterministic-authoring.md).
+- Redraw WebGL canvases from the current frame. Production pins software WebGL; do not depend on GPU-specific results or the previous frame's drawing buffer.
 - Do not run model inference inside `mount()`, `update()`, or frame production.
 
 ## Subtitle and karaoke rule

package/skills/frameyard/references/deterministic-authoring.md

@@ -73,20 +73,48 @@ import { random } from "frameyard/runtime";
 const x = random("star:17:x") * width;
 ```
 
-## CSS and Canvas
+## Canvas and WebGL
 
 CSS `@keyframes` and the Web Animations API are paused and seeked by the runtime inside the composition container. They are valid animation drivers. CSS transitions are diagnosed as non-deterministic and should not be used.
 
-Canvas 2D must be redrawn from the current frame:
+Canvas 2D and WebGL must be redrawn from the current frame:
 
 ```js
 update({ frame }) {
   context.clearRect(0, 0, canvas.width, canvas.height);
   context.fillStyle = frame % 2 === 0 ? "#101820" : "#203040";
   context.fillRect(0, 0, canvas.width, canvas.height);
+
+  gl.clearColor(frame % 2 === 0 ? 0.1 : 0.2, 0.1, 0.13, 1);
+  gl.clear(gl.COLOR_BUFFER_BIT);
 }
 ```
 
+Do not preserve and accumulate WebGL drawing across frames unless that accumulation is itself deterministic. Production sessions use software WebGL so output does not depend on the host GPU.
+
+## Embedded video
+
+Place local `<video>` elements in `mount()` and let the runtime seek them:
+
+```html
+<video
+  src="assets/clip.webm"
+  data-frameyard-start="30"
+  data-frameyard-trim="0.5"
+  data-frameyard-duration="4"
+  data-frameyard-volume="0.8"
+></video>
+```
+
+- `data-frameyard-start` is the composition frame where the source starts. Default `0`.
+- `data-frameyard-trim` is seconds trimmed from the source start. Default `0`.
+- `data-frameyard-duration` optionally limits how many source seconds are used.
+- `data-frameyard-volume` sets embedded-audio volume from `0` to `1`. Default `1`.
+
+Do not call `play()`, `pause()`, or set `currentTime` in `update()`. Create video elements and set the `data-frameyard-*` attributes in `mount()`. Videos must be in the composition container's light DOM; shadow roots are not scanned. The runtime pauses every video, loads the complete local file during the loading phase, and seeks it after each `update()` before capture. Video sources must be local, seekable project files; live streams, remote URLs, and videos added or changed during `update()` are not supported.
+
+An unmuted video's audio is extracted and mixed into the output timeline. Set `muted` or `data-frameyard-audio="off"` for a silent visual asset. The same start, trim, duration, and volume attributes place its audio.
+
 ## Loading assets
 
 Remote data, images, fonts, and other assets must resolve during `mount()` or before the first frame. The production browser blocks network requests after loading begins. A fetch from `update()` is a bug.

package/skills/frameyard/references/failure-recovery.md

@@ -16,6 +16,8 @@ Do not claim success after a nonzero exit. Fix the cause, rerun `check`, and onl
 | `PROPS_INVALID` | Props are not JSON or fail `validateProps()` | Fix the props file or validator. |
 | `FRAME_OUT_OF_RANGE` | Frame is outside `0..durationInFrames - 1` | Choose a valid frame. |
 | `AUDIO_INVALID` | Bad clip or unreadable/unsupported audio | Use a local project-relative file with a decodable audio stream. |
+| `MEDIA_INVALID` | Embedded video has invalid attributes, is remote, or cannot be decoded | Use a local seekable video file and valid `data-frameyard-*` values. |
+| `MEDIA_SEEK_FAILED` | Embedded video did not reach the requested frame time | Re-encode the source with a regular keyframe interval or remove the bad source. |
 | `ASSET_NOT_FOUND` | Required file is missing or unreadable | Restore the asset, model, or text file. |
 | `OUTPUT_EXISTS` | Output exists and replacement was not approved | Reuse it, choose another path, or pass `--overwrite`/`--force` with permission. |
 | `OUTPUT_FAILED` | An output or manifest could not be written atomically | Check directory permissions and free space; rerun. |
@@ -38,6 +40,7 @@ Do not claim success after a nonzero exit. Fix the cause, rerun `check`, and onl
 - Existing outputs are replaced only with `--overwrite` (renders/stills) or `--force` (generated assets).
 - Cancellation terminates the child inference process and cleans temporary work.
 - `check` validates every frame and declared transcripts without encoding.
+- `check`, `still`, and `render` seek embedded videos and wait for decode before evaluating or capturing a frame.
 - `doctor` inspects but never repairs, downloads, or modifies the environment.
 
 ## When visual inspection is impossible

package/skills/frameyard/references/json-results.md

@@ -92,6 +92,7 @@ Use the error `code` for decisions, not message text. `command` is `null` only f
   "durationInFrames": 900,
   "durationSeconds": 30,
   "framesRendered": 900,
+  "concurrency": 1,
   "bytes": 1234567,
   "audio": { "clips": 1, "warnings": [] },
   "transcripts": { "clips": 1, "warnings": [] },
@@ -99,7 +100,7 @@ Use the error `code` for decisions, not message text. `command` is `null` only f
 }
 ```
 
-A successful render means the final path exists and was published atomically. A nonzero exit means no new partial video was published.
+A successful render means the final path exists and was published atomically. A nonzero exit means no new partial video was published. `concurrency` is the number of browser sessions actually used; it is capped by `durationInFrames` and by the CLI maximum of 16.
 
 ## tts