frameyard 0.1.0 → 0.3.0

package/{skill → skills/frameyard}/references/deterministic-authoring.md

@@ -73,20 +73,48 @@ import { random } from "frameyard/runtime";
 const x = random("star:17:x") * width;
 ```
 
-## CSS and Canvas
+## Canvas and WebGL
 
 CSS `@keyframes` and the Web Animations API are paused and seeked by the runtime inside the composition container. They are valid animation drivers. CSS transitions are diagnosed as non-deterministic and should not be used.
 
-Canvas 2D must be redrawn from the current frame:
+Canvas 2D and WebGL must be redrawn from the current frame:
 
 ```js
 update({ frame }) {
   context.clearRect(0, 0, canvas.width, canvas.height);
   context.fillStyle = frame % 2 === 0 ? "#101820" : "#203040";
   context.fillRect(0, 0, canvas.width, canvas.height);
+
+  gl.clearColor(frame % 2 === 0 ? 0.1 : 0.2, 0.1, 0.13, 1);
+  gl.clear(gl.COLOR_BUFFER_BIT);
 }
 ```
 
+Do not preserve and accumulate WebGL drawing across frames unless that accumulation is itself deterministic. Production sessions use software WebGL so output does not depend on the host GPU.
+
+## Embedded video
+
+Place local `<video>` elements in `mount()` and let the runtime seek them:
+
+```html
+<video
+  src="assets/clip.webm"
+  data-frameyard-start="30"
+  data-frameyard-trim="0.5"
+  data-frameyard-duration="4"
+  data-frameyard-volume="0.8"
+></video>
+```
+
+- `data-frameyard-start` is the composition frame where the source starts. Default `0`.
+- `data-frameyard-trim` is seconds trimmed from the source start. Default `0`.
+- `data-frameyard-duration` optionally limits how many source seconds are used.
+- `data-frameyard-volume` sets embedded-audio volume from `0` to `1`. Default `1`.
+
+Do not call `play()`, `pause()`, or set `currentTime` in `update()`. Create video elements and set the `data-frameyard-*` attributes in `mount()`. Videos must be in the composition container's light DOM; shadow roots are not scanned. The runtime pauses every video, loads the complete local file during the loading phase, and seeks it after each `update()` before capture. Video sources must be local, seekable project files; live streams, remote URLs, and videos added or changed during `update()` are not supported.
+
+An unmuted video's audio is extracted and mixed into the output timeline. Set `muted` or `data-frameyard-audio="off"` for a silent visual asset. The same start, trim, duration, and volume attributes place its audio.
+
 ## Loading assets
 
 Remote data, images, fonts, and other assets must resolve during `mount()` or before the first frame. The production browser blocks network requests after loading begins. A fetch from `update()` is a bug.
@@ -96,8 +124,8 @@ Images inserted in `mount()` are awaited by the runtime. Static fonts should be
 ## Validation and inspection
 
 ```sh
-frameyard check hello --json
-frameyard still hello --frame 42 --out hello-42.png --json
+npx frameyard check hello --json
+npx frameyard still hello --frame 42 --out hello-42.png --json
 ```
 
 `check` mounts the composition, evaluates every frame, and validates audio and declared transcripts without capturing or encoding. Do not replace it with a single still.

package/{skill → skills/frameyard}/references/failure-recovery.md

@@ -6,7 +6,7 @@ Do not claim success after a nonzero exit. Fix the cause, rerun `check`, and onl
 
 | Code | Meaning | Action |
 | --- | --- | --- |
-| `USAGE` | Invalid flags or missing required arguments | Read `frameyard <command> --help`. |
+| `USAGE` | Invalid flags or missing required arguments | Read `npx frameyard <command> --help`. |
 | `CONFIG_NOT_FOUND` | No `video.config.js` found | Run from the project root or pass `--project`. |
 | `CONFIG_INVALID` | Invalid config or composition entry | Fix `video.config.js`; entries must be explicit, local, and exist. |
 | `COMPOSITION_NOT_FOUND` | ID is not listed | Add the entry to `video.config.js`. |
@@ -16,16 +16,18 @@ Do not claim success after a nonzero exit. Fix the cause, rerun `check`, and onl
 | `PROPS_INVALID` | Props are not JSON or fail `validateProps()` | Fix the props file or validator. |
 | `FRAME_OUT_OF_RANGE` | Frame is outside `0..durationInFrames - 1` | Choose a valid frame. |
 | `AUDIO_INVALID` | Bad clip or unreadable/unsupported audio | Use a local project-relative file with a decodable audio stream. |
+| `MEDIA_INVALID` | Embedded video has invalid attributes, is remote, or cannot be decoded | Use a local seekable video file and valid `data-frameyard-*` values. |
+| `MEDIA_SEEK_FAILED` | Embedded video did not reach the requested frame time | Re-encode the source with a regular keyframe interval or remove the bad source. |
 | `ASSET_NOT_FOUND` | Required file is missing or unreadable | Restore the asset, model, or text file. |
 | `OUTPUT_EXISTS` | Output exists and replacement was not approved | Reuse it, choose another path, or pass `--overwrite`/`--force` with permission. |
 | `OUTPUT_FAILED` | An output or manifest could not be written atomically | Check directory permissions and free space; rerun. |
 | `EXECUTABLE_NOT_FOUND` | `qwen-tts` or `whisper-cli` could not be resolved | Pass `--executable`, set the environment variable, or fix `PATH`. |
 | `TTS_FAILED` | Speech synthesis failed or produced an invalid WAV | Read stderr; verify the talker/codec models and CLI compatibility. |
 | `TRANSCRIBE_FAILED` | Normalization or `whisper-cli` failed | Verify FFmpeg, the source media, and the Whisper model. |
-| `TRANSCRIPT_INVALID` | Canonical transcript JSON is malformed | Regenerate it with `frameyard transcribe`; do not hand-edit timestamps. |
+| `TRANSCRIPT_INVALID` | Canonical transcript JSON is malformed | Regenerate it with `npx frameyard transcribe`; do not hand-edit timestamps. |
 | `TRANSCRIPT_WORDS_UNAVAILABLE` | Karaoke requested without measured word timings | Re-run transcription with `--words` or remove `words: true`. |
 | `MANIFEST_INVALID` | Provenance manifest is corrupt or unsupported | Restore or regenerate the manifest; do not invent hashes. |
-| `FFMPEG_FAILED` | FFmpeg is missing, lacks encoders, or failed | Run `frameyard doctor --json`; install FFmpeg with libx264 and AAC. |
+| `FFMPEG_FAILED` | FFmpeg is missing, lacks encoders, or failed | Run `npx frameyard doctor --json`; install FFmpeg with libx264 and AAC. |
 | `FFPROBE_FAILED` | FFprobe is missing or failed | Install FFmpeg/FFprobe together. |
 | `BROWSER_FAILED` | Chromium could not start | Install the Playwright Chromium build. |
 | `RENDER_FAILED` | Capture or encoding failed | Read stderr; rerun `check` and a representative `still`. |
@@ -38,6 +40,7 @@ Do not claim success after a nonzero exit. Fix the cause, rerun `check`, and onl
 - Existing outputs are replaced only with `--overwrite` (renders/stills) or `--force` (generated assets).
 - Cancellation terminates the child inference process and cleans temporary work.
 - `check` validates every frame and declared transcripts without encoding.
+- `check`, `still`, and `render` seek embedded videos and wait for decode before evaluating or capturing a frame.
 - `doctor` inspects but never repairs, downloads, or modifies the environment.
 
 ## When visual inspection is impossible

package/{skill → skills/frameyard}/references/json-results.md

@@ -92,6 +92,7 @@ Use the error `code` for decisions, not message text. `command` is `null` only f
   "durationInFrames": 900,
   "durationSeconds": 30,
   "framesRendered": 900,
+  "concurrency": 1,
   "bytes": 1234567,
   "audio": { "clips": 1, "warnings": [] },
   "transcripts": { "clips": 1, "warnings": [] },
@@ -99,7 +100,7 @@ Use the error `code` for decisions, not message text. `command` is `null` only f
 }
 ```
 
-A successful render means the final path exists and was published atomically. A nonzero exit means no new partial video was published.
+A successful render means the final path exists and was published atomically. A nonzero exit means no new partial video was published. `concurrency` is the number of browser sessions actually used; it is capped by `durationInFrames` and by the CLI maximum of 16.
 
 ## tts
 

package/{skill → skills/frameyard}/references/subtitles-karaoke.md

@@ -79,8 +79,8 @@ Apply that value to text styling inside `update({ frame })`. Do not use timers,
 ## Check and inspect
 
 ```sh
-frameyard check karaoke --json
-frameyard still karaoke --frame 120 --out karaoke-120.png --json
+npx frameyard check karaoke --json
+npx frameyard still karaoke --frame 120 --out karaoke-120.png --json
 ```
 
 Inspect several frames across a word boundary. A single still cannot prove that highlighting is synchronized.
@@ -88,7 +88,7 @@ Inspect several frames across a word boundary. A single still cannot prove that
 ## Derived exports
 
 ```sh
-frameyard transcribe assets/voice.wav --model models/ggml-base.en.bin \
+npx frameyard transcribe assets/voice.wav --model models/ggml-base.en.bin \
   --out assets/voice.transcript.json --export srt,vtt --json
 ```
 

package/{skill → skills/frameyard}/scripts/frameyard.mjs

@@ -5,7 +5,7 @@ import path from "node:path";
 import { fileURLToPath } from "node:url";
 
 const scriptDir = path.dirname(fileURLToPath(import.meta.url));
-const localCli = path.resolve(scriptDir, "../../dist/cli.js");
+const localCli = path.resolve(scriptDir, "../../../dist/cli.js");
 const configured = process.env.FRAMEYARD_CLI;
 
 let command;
@@ -43,7 +43,7 @@ for (const signal of ["SIGINT", "SIGTERM"]) {
 child.once("error", (error) => {
   if (error.code === "ENOENT") {
     process.stderr.write(
-      "The frameyard CLI was not found. Add frameyard to PATH or set FRAMEYARD_CLI to dist/cli.js.\n",
+      "The frameyard CLI was not found. Run `npx frameyard`, add frameyard to PATH, or set FRAMEYARD_CLI to an installed CLI path.\n",
     );
   } else {
     process.stderr.write(`Could not start the Frameyard CLI: ${error.message}\n`);

package/skill/README.md

@@ -1,29 +0,0 @@
-# Frameyard agent skill
-
-This directory is the optional, host-neutral Frameyard agent skill. Its API version matches the npm package version. The npm package includes this directory.
-
-## Install or reference
-
-Copy or symlink the whole `skill/` directory into the skill location your agent host uses. Examples:
-
-- Claude Code project skills: `.claude/skills/frameyard`
-- Agent hosts that read `.agents/skills`: `.agents/skills/frameyard`
-- Global installations: place it under your host's documented personal skills directory
-
-The skill is host-neutral: it uses only the normal `frameyard` project format and CLI. There is no installer command and no agent-only runtime.
-
-When the skill is copied outside the npm package, `scripts/frameyard.mjs` uses the `frameyard` command on `PATH`. If that command is unavailable, set `FRAMEYARD_CLI` to the installed `dist/cli.js` path.
-
-## Contents
-
-- `SKILL.md`: trigger description and operating workflow
-- `references/deterministic-authoring.md`
-- `references/audio-assets.md`
-- `references/subtitles-karaoke.md`
-- `references/json-results.md`
-- `references/failure-recovery.md`
-- `scripts/frameyard.mjs`: thin fallback wrapper for the installed CLI
-
-## Version matching
-
-`SKILL.md` records `frameyard-api-version`. If it does not match the installed package's version, use the skill files shipped inside that package. Contract changes are documented in the repository README and design baseline.

package/skill/SKILL.md

@@ -1,57 +0,0 @@
----
-name: frameyard
-description: Create, validate, inspect, and render deterministic Frameyard videos authored with HTML, CSS, and native JavaScript. Use when a user asks to build or operate a Frameyard project, produce parameterized video, add a local TTS or transcription asset through Frameyard's wrappers, or diagnose a Frameyard render. Do not use for general-purpose media generation outside a Frameyard project.
-frameyard-api-version: 0.1.0
----
-
-# Frameyard
-
-Use Frameyard's public project format and CLI. Do not introduce an agent-only runtime, a hosted service, or direct calls to `qwen-tts` or `whisper-cli`.
-
-## First decisions
-
-1. Find the project: look for `video.config.js` from the working directory upward, or run `frameyard init <directory>` to scaffold one.
-2. Read the composition entry points listed in `video.config.js`. There is no directory scanning.
-3. Read the existing props file and composition source before changing behavior.
-4. If the task needs speech or transcription, confirm that inference is requested or explicitly approved before producing assets.
-
-## Workflow
-
-1. **Author** the composition with HTML, CSS, SVG, Canvas 2D, and native JavaScript ES modules. Keep `width`, `height`, `fps`, and `durationInFrames` fixed. Read [deterministic authoring](references/deterministic-authoring.md).
-2. **Provide props** as JSON. Validate domain-specific requirements with `validateProps()`.
-3. **Produce optional assets** with `frameyard tts` and `frameyard transcribe`, never with the upstream executables. Read [audio assets](references/audio-assets.md). Do not overwrite or regenerate existing generated assets without permission.
-4. **Validate** with `frameyard check <id> --json`. It evaluates every frame, checks audio and declared transcripts, and does not encode.
-5. **Inspect** representative frames with `frameyard still <id> --frame <n> --out <file.png> --json`. View the PNGs when the host supports image viewing. If visual inspection is impossible, say so explicitly.
-6. **Render** with `frameyard render <id> --out <file.mp4> [--props <file>] --json`.
-7. **Report** the project path, composition entry, props, generated assets, provenance manifest, validation results, still paths and inspection status, final video path, and any failures.
-
-Never claim success when `check` fails, a still is missing, a render exits nonzero, or visual inspection was skipped. A partial or unpublished render is not a successful result.
-
-## Command rules
-
-- Use `--json` for every non-interactive command. Parse stdout as the structured result; treat stderr as progress and diagnostics.
-- Keep the current output files unless the user explicitly approves replacement. Use `--force` only with that approval.
-- Use `--project <dir>` when not running from the project root.
-- Use `frameyard doctor --json` before reporting an environment problem. It inspects without repairing or downloading.
-- Read [JSON results](references/json-results.md) for result and error shapes, and [failure recovery](references/failure-recovery.md) for actionable error codes.
-
-## Determinism guardrails
-
-- A frame may depend only on its integer `frame`, immutable `props`, and assets loaded before frame production.
-- Use `random("stable:key")` from `frameyard/runtime`; never use `Math.random()`.
-- Never use `Date.now()`, `performance.now()`, timers, independent `requestAnimationFrame` loops, or CSS transitions as animation inputs.
-- Finish remote data, fonts, and image loading during `mount()` or before the first frame. Network requests are blocked during frame capture.
-- Keep `update({ frame, props })` synchronous and seek-safe. Do not carry state between frames.
-- Do not run model inference inside `mount()`, `update()`, or frame production.
-
-## Subtitle and karaoke rule
-
-Use canonical transcripts produced by `frameyard transcribe`. Convert integer source-relative milliseconds to composition frames with the audio clip's `startFrame`, `trimStart`, and the composition `fps`. Karaoke highlighting requires actual `words` timings; never interpolate segment timings. Read [subtitles and karaoke](references/subtitles-karaoke.md).
-
-## Host script
-
-`scripts/frameyard.mjs` runs the installed package CLI with inherited stdio. It prefers the package-local `dist/cli.js`, then the `frameyard` command on `PATH`. Prefer the `frameyard` command directly; the script is a fallback for hosts that cannot resolve package binaries. When the skill is copied out of the package, set `FRAMEYARD_CLI` to the installed CLI if `frameyard` is not on `PATH`.
-
-```sh
-node scripts/frameyard.mjs check <id> --json
-```