frameyard 0.4.0 → 0.6.0

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "frameyard",
-  "version": "0.4.0",
+  "version": "0.6.0",
   "description": "Local-first, deterministic video composition framework for HTML, CSS, and JavaScript.",
   "type": "module",
   "license": "MIT",
@@ -66,11 +66,13 @@
     "build": "tsc -p tsconfig.json",
     "typecheck": "tsc -p tsconfig.json --noEmit",
     "typecheck:test": "tsc -p tsconfig.test.json",
+    "typecheck:browser": "tsc -p tsconfig.browser.json",
     "check:scripts": "node scripts/check-scripts.mjs",
     "lint": "oxlint",
     "format": "oxfmt",
     "format:check": "oxfmt --check",
-    "check:static": "pnpm check:scripts && pnpm lint && pnpm format:check && pnpm typecheck && pnpm typecheck:test",
+    "check:static": "pnpm check:scripts && pnpm lint && pnpm format:check && pnpm typecheck && pnpm typecheck:test && pnpm typecheck:browser",
+    "compare-stills": "node scripts/compare-stills.mjs",
     "check": "pnpm check:static && pnpm test",
     "test": "pnpm build && vitest run",
     "test:unit": "vitest run test/contracts.test.ts test/docs.test.ts test/random.test.ts test/release.test.ts test/scaffold.test.ts test/transcript.test.ts",

package/skills/frameyard/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: frameyard
-description: Create, validate, inspect, and render deterministic Frameyard videos authored with HTML, CSS, and an optional JavaScript controller. Use when a user asks to build or operate a Frameyard project, produce parameterized video, add a local TTS or transcription asset through Frameyard's wrappers, or diagnose a Frameyard render. Do not use for general-purpose media generation outside a Frameyard project.
-frameyard-api-version: 0.4.0
+description: Create, validate, inspect, and render deterministic Frameyard videos authored with HTML compositions and JSON timelines. Use when a user asks to build or operate a Frameyard project, produce parameterized video, assemble clips into a timeline, add a local TTS or transcription asset through Frameyard's wrappers, or diagnose a Frameyard render. Do not use for general-purpose media generation outside a Frameyard project.
+frameyard-api-version: 0.6.0
 ---
 
 # Frameyard
@@ -11,19 +11,20 @@ Use Frameyard's public project format and CLI. Do not introduce an agent-only ru
 ## First decisions
 
 1. Find the project: look for `video.config.js` from the working directory upward, or run `npx frameyard init <directory>` to scaffold one.
-2. Read the composition entry points listed in `video.config.js`. They are `.html` documents; there is no directory scanning.
-3. Read the composition HTML, its optional controller, and the props file before changing behavior.
+2. Read the `format`, `compositions`, and `timelines` maps in `video.config.js`. Entries are explicit `.html` and `.json` documents; there is no directory scanning.
+3. Read the timeline JSON, the referenced composition HTML, its optional controller, and the props files before changing behavior.
 4. If the task needs speech or transcription, confirm that inference is requested or explicitly approved before producing assets.
 
 ## Workflow
 
-1. **Author** the composition as a full HTML document with one `<fy-composition>` root. Put metadata, default props, markup, CSS animation, bindings, audio, and transcripts in the document. Use an optional controller for Canvas 2D, WebGL, karaoke, or per-frame computation CSS cannot express. Read [deterministic authoring](references/deterministic-authoring.md), [bindings](references/bindings.md), and [controllers](references/controllers.md).
-2. **Provide props** as JSON. Validate domain-specific requirements with a controller's `validateProps()`.
-3. **Produce optional assets** with `npx frameyard tts` and `npx frameyard transcribe`, never with the upstream executables. Read [audio assets](references/audio-assets.md). Do not overwrite or regenerate existing generated assets without permission.
-4. **Validate** with `npx frameyard check <id> --json`. Static validation runs before Chromium; then every frame is evaluated. Read the structured `diagnostics` array. Read [diagnostics](references/diagnostics.md) and [common mistakes](references/common-mistakes.md).
-5. **Inspect** representative frames with `npx frameyard still <id> --frame <n> --out <file.png> --json`. View the PNGs when the host supports image viewing. If visual inspection is impossible, say so explicitly.
-6. **Render** with `npx frameyard render <id> --out <file.mp4> [--props <file>] [--concurrency <n>] --json`.
-7. **Report** the project path, composition entry, controller, props, generated assets, provenance manifest, validation results, diagnostics, still paths and inspection status, final video path, and any failures.
+1. **Author compositions** as full HTML documents with one `<fy-composition>` root. Put metadata, default props, markup, CSS animation, bindings, audio, and transcripts in the document. Use an optional controller for Canvas 2D, WebGL, karaoke, or per-frame computation CSS cannot express. Read [deterministic authoring](references/deterministic-authoring.md), [bindings](references/bindings.md), and [controllers](references/controllers.md).
+2. **Assemble the timeline** in JSON: project format, video/audio tracks, composition, media, and nested-timeline clips, source ranges, static transforms, and audio. Read [timelines](references/timelines.md).
+3. **Provide props** as JSON. Composition props merge over defaults; timeline render props are a map keyed by composition clip ID. Validate domain-specific requirements with a controller's `validateProps()`.
+4. **Produce optional assets** with `npx frameyard tts` and `npx frameyard transcribe`, never with the upstream executables. Read [audio assets](references/audio-assets.md). Do not overwrite or regenerate existing generated assets without permission.
+5. **Validate** with `npx frameyard check <id> --json`, where `<id>` is a timeline or composition. Static validation runs before Chromium; then every frame is evaluated. Read the structured `diagnostics` array. Read [diagnostics](references/diagnostics.md) and [common mistakes](references/common-mistakes.md).
+6. **Inspect** representative frames with `npx frameyard still <id> --frame <n> --out <file.png> --json`. View the PNGs when the host supports image viewing. If visual inspection is impossible, say so explicitly.
+7. **Render** with `npx frameyard render <id> [--out <file.mp4>] [--props <file>] --json`. Composition renders accept `--concurrency <n>`; timeline renders accept `--workers <n>` and `--scratch <dir>`. The default output is `out/<id>.mp4` in the project root.
+8. **Report** the project path, timeline and composition entries, controller, props, generated assets, provenance manifest, validation results, diagnostics, still paths and inspection status, final video path, and any failures.
 
 Never claim success when `check` fails, a still is missing, a render exits nonzero, or visual inspection was skipped. A partial or unpublished render is not a successful result.
 
@@ -32,13 +33,16 @@ Never claim success when `check` fails, a still is missing, a render exits nonze
 - Use `--json` for every non-interactive command. Parse stdout as the structured result; treat stderr as progress and diagnostics.
 - Keep the current output files unless the user explicitly approves replacement. Use `--force` only with that approval.
 - Use `--project <dir>` when not running from the project root.
-- Use `--concurrency <n>` only when faster local rendering is useful; the default is `1` and the maximum is `16`. It changes capture scheduling only, not composition behavior or output frame order.
+- `render` writes to `out/<id>.mp4` in the project root unless `--out` is passed. Explicit `--out` paths are relative to the current working directory.
+- Use `--concurrency <n>` only for composition renders; the default is `1` and the maximum is `16`. Use `--workers <n>` for timeline renders; the default is `8`. Timeline renders also accept `--scratch <dir>` for FFV1 intermediates.
+- For a timeline, `--props` is a JSON object keyed by top-level composition clip ID. For a composition, it is the props object.
 - Use `npx frameyard doctor --json` before reporting an environment problem. It inspects without repairing or downloading.
 - Read [JSON results](references/json-results.md) for result and error shapes, and [failure recovery](references/failure-recovery.md) for actionable error codes.
 
 ## Authoring rules
 
 - A composition is a full HTML document with one `<fy-composition width height fps duration-in-frames>` root and a `data-fy="props"` default-props block.
+- A timeline is a versioned JSON document with explicit tracks and clips. Sources are composition IDs, nested timeline IDs, or local video/image/audio paths. Timeline frames, sourceIn, and duration are integer frames. No remote URLs.
 - Bindings are explicit: `data-fy-text`, `data-fy-attr-*`, `data-fy-class-*`, `data-fy-if`, and `data-fy-repeat` on `<template>`. Primitive top-level props are also `--fy-prop-<name>` CSS variables.
 - CSS `@keyframes` and the Web Animations API are seeked from the frame clock. `data-fy-anim-start`, `data-fy-anim-duration`, and `data-fy-anim-easing` schedule one animation per element in frames.
 - Frame-state variables on `:root`: `--fy-frame`, `--fy-time`, `--fy-progress`, `--fy-fps`, `--fy-duration`, and `--fy-duration-frames`.
@@ -49,7 +53,7 @@ Never claim success when `check` fails, a still is missing, a render exits nonze
 
 ## Determinism guardrails
 
-- A frame may depend only on its integer `frame`, immutable `props`, and assets loaded before frame production.
+- A composition frame may depend only on its integer `frame`, immutable `props`, and assets loaded before frame production. A timeline frame derives from its active clips and the same composition rules.
 - Use `random("stable:key")` from `frameyard/runtime`; never use `Math.random()`.
 - Never use `Date.now()`, `performance.now()`, timers, independent `requestAnimationFrame` loops, or CSS transitions as animation inputs.
 - Finish remote data, fonts, and image loading during the loading phase. Network requests are blocked during frame capture.
@@ -79,6 +83,7 @@ When the CLI version differs from `frameyard-api-version`, pin it with `npx fram
 
 ## References
 
+- [Timelines](references/timelines.md)
 - [Deterministic HTML authoring](references/deterministic-authoring.md)
 - [Bindings](references/bindings.md)
 - [Controllers](references/controllers.md)

package/skills/frameyard/references/json-results.md

@@ -118,7 +118,7 @@ Static validation diagnostics include source locations. Runtime diagnostics may
   "composition": "hello",
   "entry": "compositions/hello.html",
   "controller": { "kind": "inline" },
-  "path": "/absolute/path/hello.mp4",
+  "path": "/absolute/path/out/hello.mp4",
   "width": 1920,
   "height": 1080,
   "fps": 30,
@@ -134,7 +134,7 @@ Static validation diagnostics include source locations. Runtime diagnostics may
 }
 ```
 
-A successful render means the final path exists and was published atomically. A nonzero exit means no new partial video was published. `concurrency` is the number of browser sessions actually used; it is capped by `durationInFrames` and by the CLI maximum of 16.
+A successful render means the final path exists and was published atomically. A nonzero exit means no new partial video was published. Without `--out`, the final path is `out/<composition-id>.mp4` inside the project root. `concurrency` is the number of browser sessions actually used; it is capped by `durationInFrames` and by the CLI maximum of 16.
 
 ## tts
 

package/skills/frameyard/references/timelines.md

@@ -0,0 +1,141 @@
+# Timelines
+
+A timeline is a project-local JSON document that assembles composition and media clips on tracks and renders as one video. Compositions remain reusable HTML sources; a composition clip is one placement of a composition with its own source range and props.
+
+## Project configuration
+
+```js
+export default {
+  format: { width: 1920, height: 1080, fps: 30 },
+  compositions: {
+    intro: "./compositions/intro.html",
+  },
+  timelines: {
+    main: "./timelines/main.timeline.json",
+  },
+};
+```
+
+- `format` fixes width, height, and fps for every timeline. It is required when the project lists timelines.
+- Composition and timeline IDs share one namespace and must be unique.
+- A composition used in a timeline must match the project fps exactly. A different size is fitted by the clip.
+- Composition-only projects remain valid without `format` or `timelines`.
+
+## Timeline document
+
+```json
+{
+  "version": 1,
+  "name": "Main edit",
+  "tracks": [
+    { "id": "v1", "kind": "video" },
+    { "id": "v2", "kind": "video" },
+    { "id": "a1", "kind": "audio", "gain": 1, "muted": false }
+  ],
+  "clips": [
+    {
+      "id": "intro",
+      "track": "v1",
+      "source": { "kind": "composition", "id": "intro" },
+      "start": 0,
+      "duration": 150,
+      "sourceIn": 0,
+      "props": { "title": "Hello" },
+      "transform": { "fit": "contain", "x": 0, "y": 0, "scale": 1, "rotation": 0, "opacity": 1 },
+      "audio": { "muted": false, "gain": 1 }
+    },
+    {
+      "id": "logo",
+      "track": "v2",
+      "source": { "kind": "image", "src": "assets/logo.png" },
+      "start": 30,
+      "duration": 60,
+      "transform": { "fit": "contain", "width": 480, "height": 270, "x": 640, "y": -320, "rotation": 0, "opacity": 0.9 }
+    },
+    {
+      "id": "music",
+      "track": "a1",
+      "source": { "kind": "audio", "src": "assets/music.wav" },
+      "start": 0,
+      "duration": 210,
+      "audio": { "gain": 0.5 }
+    }
+  ]
+}
+```
+
+- `version` is `1`.
+- `tracks` and `clips` are required. Track order is bottom-to-top for video and irrelevant for audio.
+- `background` is an optional color; the final video composites over black.
+- `durationInFrames` is optional. The duration is derived from the latest clip end unless declared; a declared duration may extend but not cut clips.
+- Unknown fields are errors. No transitions, effects, keyframes, or track solo/visibility in 0.6.
+
+## Sources
+
+| `source.kind` | Fields | Notes |
+| --- | --- | --- |
+| `composition` | `id` | Project composition ID. The clip may set `props`. |
+| `timeline` | `id` | Nested timeline ID. Rendered as a group. |
+| `video` | `src` | Local MP4/WebM, project-relative. |
+| `image` | `src` | Local PNG/JPEG/WebP. `duration` is required. |
+| `audio` | `src` | Local WAV/MP3/M4A/AAC or a video file used only for audio. |
+
+Media paths resolve from the project root. No remote URLs.
+
+## Time and trims
+
+- `start` is the timeline frame where the clip begins.
+- `sourceIn` is the first source frame. It defaults to `0` and must be `0` for images.
+- `duration` is the number of timeline frames. It is optional for compositions, nested timelines, video, and audio; it defaults to the rest of the source. Images require it.
+- A clip that exceeds its source is an error.
+- Clips on one track cannot overlap. Touching endpoints are allowed. Use separate tracks for overlays.
+
+## Presentation
+
+Visual clips accept a static transform:
+
+- `fit`: `contain`, `cover`, or `stretch`.
+- `width`, `height`: optional clip box in output pixels. Both are required together and cannot be combined with `scale`. `fit` applies inside the box, and `x`/`y` offset the box center. Without them, the project frame is the box.
+- `x`, `y`: pixel offsets from the centered fitted result.
+- `scale`: multiplier after fitting to the project frame; an alternative to `width`/`height`.
+- `rotation`: degrees around the center.
+- `opacity`: 0–1.
+
+No keyframes or animated transforms in 0.6.
+
+## Audio
+
+- Composition, video, and nested-timeline clips contribute audio automatically unless `audio.muted` is true.
+- Timeline audio clips live on audio tracks. Audio tracks support `gain` and `muted`.
+- `audio.gain` on a clip is 0–1 and multiplies the source audio.
+- There are no linked A/V pairs. For a J/L cut, mute the visual clip and add an audio clip with a different `sourceIn`.
+- Audio is mixed once for the whole timeline and muxed into the final MP4.
+
+## Nested timelines
+
+A nested timeline clip behaves as one layer: its internal tracks are composited first, then the parent clip's opacity and transform apply to the whole group. Nested audio is included. Cycles are rejected statically. Timeline clips do not override props of compositions inside a nested timeline in 0.6.
+
+## Commands
+
+```sh
+npx frameyard check main --json
+npx frameyard still main --frame 42 --out main-42.png --json
+npx frameyard render main --props props/main.json --workers 8 --scratch /path/to/scratch --json
+```
+
+- `--props` on a timeline is a JSON object keyed by top-level composition clip ID; values merge over the clip's `props`.
+- `--workers` is 1–16. The default is 8.
+- `--scratch` selects the FFV1 intermediate directory. Use a disk with enough space; intermediates are cleaned after the render.
+- `--concurrency` is composition-only.
+- `render` defaults to `out/<timeline-id>.mp4`.
+- `check` evaluates every timeline frame and composition frame. `still` captures one frame through the same clip capture and composite path.
+- `npx frameyard dev` lists timelines in the preview; the timeline view has a program monitor, read-only tracks and clips, playhead, playback, audio controls, and diagnostics.
+
+## Determinism and performance
+
+- A timeline frame derives only from its active clips, their composition frames, props, assets, and the render plan.
+- Composition frames are captured with Chromium; transparent compositions preserve alpha.
+- Timeline production captures active composition and nested-timeline clips to lossless FFV1, composites segments with FFmpeg, concatenates the ordered segments, and mixes audio once.
+- Preview composites in Chromium; production composites with FFmpeg. Small scaling, alpha, and video color differences are expected. Parity is guaranteed at clip capture, not at final encoded pixels.
+- Final MP4 bytes are not a reproducibility guarantee.
+- Long timelines are segmented and bounded. The reference target and measured worst cases are documented in `docs/adr/0012-render-timelines-with-segmented-lossless-clip-intermediates.md`.