frameyard 0.8.0 → 0.9.0

package/skills/frameyard/references/quick-reference.md

@@ -0,0 +1,222 @@
+# Quick reference
+
+Use this page when context is limited or when you need one canonical skeleton.
+Read a topic reference only when the task needs it; the full contract is in
+`SKILL.md` and the linked references.
+
+## Choose the smallest authoring level
+
+1. **Static composition** for a poster, card, or thumbnail: no `duration-in-frames`, no `fps`, renders to PNG.
+2. **CSS animation** for timed reveals, wipes, and simple motion: `@keyframes` plus `data-fy-anim-*`.
+3. **Bindings** for props-driven text, attributes, classes, conditionals, and repeats.
+4. **Controller** for Canvas 2D, WebGL, karaoke, or per-frame math CSS cannot express.
+5. **Timeline** when several compositions, media clips, or audio tracks must share one deliverable.
+
+Do not use a controller when CSS or bindings are enough.
+
+## Animated composition
+
+A composition is a partial HTML fragment, never a full document. Props, the
+controller, import maps, and styles are top-level siblings of the root.
+
+```html
+<script type="application/json" data-fy="props">{ "title": "Hello" }</script>
+<style>
+  html, body { margin: 0; width: 100%; height: 100%; }
+  @keyframes rise { from { opacity: 0; } to { opacity: 1; } }
+  h1 { animation: rise 1s linear both; font: 8vmin system-ui; color: white; }
+  body { background: #101820; display: grid; place-items: center; }
+</style>
+<fy-composition duration-in-frames="150">
+  <h1 data-fy-text="title" data-fy-anim-start="0" data-fy-anim-duration="30"></h1>
+</fy-composition>
+```
+
+`width`, `height`, and `fps` are optional and inherit the project `format`.
+Declare both `width` and `height` together. Never derive metadata from props.
+
+## Static composition
+
+```html
+<script type="application/json" data-fy="props">{ "title": "Poster" }</script>
+<style>
+  body { margin: 0; background: #0c1220; color: #ff7a45; display: grid; place-items: center; }
+  h1 { font: 8vmin system-ui; }
+</style>
+<fy-composition>
+  <h1 data-fy-text="title"></h1>
+</fy-composition>
+```
+
+A static composition has no frame clock: no `data-fy-anim-*`, no `<video>`,
+`<audio>`, or `<fy-transcript>`. Its controller runs `mount()` only.
+
+## Fonts
+
+Fontsource is the default catalog. Search, then install a pinned family:
+
+```sh
+npx frameyard font search "space grotesk" --subset latin --json
+npx frameyard font add space-grotesk --subset latin
+```
+
+Variable families install the `wght` face by default. Use `--static --weight
+400,700` for static files. A local or brand font uses `font add --file`; an
+explicit HTTPS source uses `font add --url` and may pass `--sha256`. A URL
+import without `--license` or `--license-file` is recorded as unknown and
+requires `full`.
+
+Then use the family name normally:
+
+```css
+h1 { font-family: "Space Grotesk", system-ui, sans-serif; font-weight: 700; }
+```
+
+The server injects `@font-face` before author styles, uses `font-display: block`,
+and the host waits for `document.fonts.ready`. Run `font verify` after moving
+files. Remote font URLs produce `FONT_REMOTE`; a used face that fails to load
+produces `FONT_LOAD_FAILED`. See [fonts](fonts.md).
+
+## Timeline
+
+```json
+{
+  "version": 1,
+  "tracks": [
+    { "id": "v1", "kind": "video" },
+    { "id": "a1", "kind": "audio" }
+  ],
+  "clips": [
+    {
+      "id": "intro",
+      "track": "v1",
+      "source": { "kind": "composition", "id": "intro" },
+      "start": 0
+    },
+    {
+      "id": "music",
+      "track": "a1",
+      "source": { "kind": "audio", "src": "assets/music.wav" },
+      "start": 0,
+      "duration": 150,
+      "audio": { "gain": 0.6 }
+    }
+  ]
+}
+```
+
+`start`, `sourceIn`, and `duration` are timeline frames. Image clips and static
+composition clips require an explicit `duration`; a static composition clip
+needs `sourceIn: 0` and contributes no audio. Source kinds are `composition`,
+`timeline`, `video`, `image`, and `audio`.
+
+## Bindings
+
+- `data-fy-text="path"` sets text.
+- `data-fy-attr-src="path"` sets an attribute.
+- `data-fy-class-active="path"` toggles a class.
+- `data-fy-if="path"` removes the element when false.
+- `<template data-fy-repeat="items">` clones content; use `item` and `index`.
+- Paths are dot-separated; missing paths warn and resolve empty.
+
+Bindings apply once per load, before `mount()`. Controller-created DOM is not
+re-bound.
+
+## Controller
+
+```html
+<script type="text/fy-controller">
+export default {
+  mount({ root, props }) {},
+  update({ frame, props }) {},
+  dispose() {},
+};
+</script>
+```
+
+Exactly one controller is allowed, inline or external. `mount` may be async;
+`update` is synchronous and seek-safe. The controller is the only executable
+code. Use `random("stable:key")` from `frameyard/runtime`, never `Math.random()`.
+
+## Commands
+
+| Command | Use |
+| --- | --- |
+| `npx frameyard context --json` | One compact project digest: format, props, timelines, assets, policy, next commands |
+| `npx frameyard lint <id> --json` | Fast edit loop: static validation, scene/keyframe projection, controller determinism |
+| `npx frameyard check <id> --json` | Full gate: props hooks and every animated frame through Chromium |
+| `npx frameyard still <id> --frame <spec> --json` | Inspect representative frames; policy caps the count |
+| `npx frameyard inspect <image> --json` | Blank, alpha, colors, content bbox, luminance grid, edge density |
+| `npx frameyard compare <a> <b> --json` | Pixel and 3×3 region diff; exits 1 when different |
+| `npx frameyard render <id> --dry-run --json` | Handoff plan without Chromium, FFmpeg, or a model |
+| `npx frameyard render <id> --json` | Produce the video when the policy allows it |
+| `npx frameyard add --list --json` | List project templates |
+| `npx frameyard add composition <id> --template <name> --json` | Write a valid fragment and print the config snippet |
+| `npx frameyard font search <query> --json` | Search Fontsource by name, category, subset, style, or variable |
+| `npx frameyard font add <fontsource-id> --json` | Pin and vendor a Fontsource family; variable `wght` by default |
+| `npx frameyard font add <fontsource-id> --static --weight 400,700 --json` | Install explicit static weights |
+| `npx frameyard font add --file <path> --family <name> --json` | Vendor a local font into `assets/fonts/` and update `fy.fonts.json` |
+| `npx frameyard font add --url <https-url> --family <name> --json` | Download and vendor an HTTPS font, optionally with `--sha256` |
+| `npx frameyard font list --json` | Installed families, subsets, licenses, files, and status |
+| `npx frameyard font verify --json` | Manifest paths, hashes, licenses, and orphaned directories |
+| `npx frameyard font remove <id> --json` | Remove a font and its directory; `--keep-files` keeps the binaries |
+| `npx frameyard doctor --json` | Environment, integrations, policy, fonts, and resources |
+
+Every non-interactive command accepts `--json`. Parse stdout; read stderr for
+progress and diagnostics.
+
+## Resource policy
+
+`fy.policy.json` is discovered from the working directory upward,
+independent of `fy.config.js`.
+
+| Tier | `still` | `check` | `tts`/`transcribe`/`render` | overwrite |
+| --- | --- | --- | --- | --- |
+| `light` | 1 frame | blocked | blocked | denied |
+| `authoring` | 4 frames | low priority | blocked | denied |
+| `full` | 32 frames | allowed | allowed | allowed |
+
+`lint`, `context`, `doctor`, `add`, `inspect`, `compare`, and `--dry-run` are
+available in every tier. `font search`, `font list`, and `font verify` are
+available in every tier; `font add` requires `authoring` or `full`; `font
+remove`, `font add --force`, and `font add --url` without
+`--license`/`--license-file` require `full`. When a heavy command is blocked, run it with `--dry-run`, report `deferred: true`, and give the user
+`commandLine`, `prerequisites`, and `blockers`.
+
+## Ten diagnostics that matter
+
+| Code | Fix |
+| --- | --- |
+| `COMPOSITION_DOCUMENT_SHELL` | Remove `<!doctype>`, `<html>`, `<head>`, and `<body>`; compositions are fragments |
+| `COMPOSITION_FORMAT_UNRESOLVED` | Declare `width`/`height` or add project `format` |
+| `COMPOSITION_INVALID` | Check the single `<fy-composition>` root and metadata |
+| `CONTROLLER_INVALID` | Exactly one controller; inline XOR `src`; at least one export |
+| `SCRIPT_NOT_ALLOWED` | Move executable code into the controller |
+| `BINDING_PATH_MISSING` | Fix the props path; use the diagnostic suggestion |
+| `BINDING_INVALID` | Unknown `data-fy-*` attribute; use a documented binding |
+| `TIMELINE_CLIP_OUT_OF_RANGE` | Keyframes cannot extend a clip; lengthen the clip or move the key |
+| `TIMELINE_CLIP_OVERLAP` | Declare a transition or move the clip |
+| `RESOURCE_POLICY_BLOCKED` | Use `--dry-run`, or ask the user for `FRAMEYARD_POLICY=full` |
+
+Font diagnostics: `FONT_HASH_MISMATCH` means the vendored file changed; restore
+it or reinstall with `font add --force`. `FONT_LOAD_FAILED` means a used face
+did not load; check the manifest and `font verify`. `FONT_REMOTE` is a warning
+that the composition still depends on a remote font URL.
+
+## Do not
+
+- Use `Date.now()`, `performance.now()`, timers, `requestAnimationFrame` loops, or CSS transitions as animation inputs.
+- Call `play()`, `pause()`, or set `currentTime` on embedded media.
+- Fetch remote data from `update()`; load during `mount()`.
+- Put visual content outside `<fy-composition>`.
+- Edit `fy.policy.json`, host configuration, or the skill source.
+- Claim success when `check` fails, a still is missing, or a render did not run.
+
+## Next
+
+- Full authoring rules: [deterministic authoring](deterministic-authoring.md)
+- Fonts: [fonts](fonts.md)
+- Timelines and keyframes: [timelines](timelines.md)
+- Error codes: [diagnostics](diagnostics.md)
+- Result shapes: [JSON results](json-results.md)
+- Recovery loops: [failure recovery](failure-recovery.md)

package/skills/frameyard/references/timelines.md

@@ -42,7 +42,13 @@ export default {
       "sourceIn": 0,
       "props": { "title": "Hello" },
       "blend": "screen",
-      "transform": { "fit": "contain", "x": 0, "y": 0, "scale": 1, "rotation": 0, "opacity": 1 },
+      "transform": { "fit": "contain", "x": 0, "y": 0, "scale": 1, "rotation": 0 },
+      "keyframes": {
+        "opacity": [
+          { "frame": 0, "value": 0 },
+          { "frame": 20, "value": 1, "easing": "ease-out" }
+        ]
+      },
       "audio": { "muted": false, "gain": 1 }
     },
     {
@@ -69,7 +75,7 @@ export default {
 - `tracks` and `clips` are required. Track order is bottom-to-top for video and irrelevant for audio.
 - `background` is an optional color; the final video composites over black.
 - `durationInFrames` is optional. The duration is derived from the latest clip end unless declared; a declared duration may extend but not cut clips.
-- Unknown fields are errors. Blend modes are static per clip; no transitions, keyframes, other effects, or track solo/visibility.
+- Unknown fields are errors. Blend modes are static per clip; there are no animated blend modes, other effects, or track solo/visibility. Clip transitions are the one same-track overlap (see [Transitions](#transitions)).
 
 ## Sources
 
@@ -90,7 +96,7 @@ Media paths resolve from the project root. No remote URLs.
 - `duration` is the number of timeline frames. It is optional for animated compositions, nested timelines, video, and audio; it defaults to the rest of the source. Images and static compositions require it.
 - A clip that exceeds its source is an error. Static compositions have no intrinsic duration, so the clip duration is the only bound.
 - A static composition clip is captured once per props variant as a transparent PNG and reused across segments. A clip whose source is static and whose `audio` settings are non-default receives an `AUDIO_WARNING` because there is no source audio.
-- Clips on one track cannot overlap. Touching endpoints are allowed. Use separate tracks for overlays.
+- Clips on one track cannot overlap unless a timeline-level `transitions` entry pairs them. Touching endpoints are allowed but are not a transition. Use separate tracks for overlays that are not transitions.
 
 ## Presentation
 
@@ -104,7 +110,90 @@ Visual clips accept a static transform:
 - `opacity`: 0–1.
 - `blend`: CSS `mix-blend-mode`, defaulting to `normal`. Supported values are `normal`, `multiply`, `screen`, `overlay`, `darken`, `lighten`, `color-dodge`, `color-burn`, `hard-light`, `soft-light`, `difference`, and `exclusion`. `hue`, `saturation`, `color`, and `luminosity` are rejected.
 
-A blend mode composites the clip against everything below it in the stage, including the timeline background, and only where the clip's source alpha is non-zero. A composition or nested timeline that paints an opaque background blocks the lower layers; source alpha is what lets a blend reach them. A top-level timeline with no declared background renders over black; a nested or embedded timeline stays transparent. Blend modes are static; there are no keyframes or animated transforms.
+A blend mode composites the clip against everything below it in the stage, including the timeline background, and only where the clip's source alpha is non-zero. A composition or nested timeline that paints an opaque background blocks the lower layers; source alpha is what lets a blend reach them. A top-level timeline with no declared background renders over black; a nested or embedded timeline stays transparent. Blend modes are static; there are no animated blend modes.
+
+Rotation keeps the full rotated bounds. Production expands the rotate filter's output to a fixed bound: static rotation uses its exact rotated bounding box, while keyed rotation uses the diagonal of the fitted box so every segment uses the same size. The filter fills the padding with transparent pixels, so rotated corners are visible instead of being cut off.
+
+## Keyframes
+
+Visual clips may animate `x`, `y`, `scale`, `rotation`, and `opacity` with a top-level `keyframes` object keyed by channel:
+
+```json
+"keyframes": {
+  "opacity": [
+    { "frame": 0, "value": 0 },
+    { "frame": 24, "value": 1, "easing": "ease-out" },
+    { "frame": 120, "value": 1 },
+    { "frame": 150, "value": 0, "easing": "ease-in" }
+  ],
+  "x": [
+    { "frame": 0, "value": -80, "easing": "ease-out" },
+    { "frame": 24, "value": 0 }
+  ]
+}
+```
+
+- `frame` is clip-local. `0` is the clip's start boundary and `duration` is its end boundary. Moving `start` moves the animation with the clip; `sourceIn` does not shift it.
+- Keys are integers in `0..duration`, strictly increasing, with at least two per channel and a required `value`.
+- A channel holds its first value before the first key and its last value after the last key. Keyframes never extend a clip's duration.
+- `easing` applies to the interval after that key. It defaults to `linear`; the final key's easing has no effect.
+- Supported easings are `linear`, `ease`, `ease-in`, `ease-out`, `ease-in-out`, `step-start`, `step-end`, `hold`, `cubic-bezier(x1, y1, x2, y2)`, and `steps(n, jump-start|jump-end|jump-none|jump-both)`.
+- A channel cannot also be an explicitly declared scalar in `transform`. Defaults do not conflict; unkeyed channels keep their `transform` value.
+- `opacity` values are `0..1` and `scale` values are `>= 0`. Interpolated opacity and scale are clamped after easing; `x`, `y`, and `rotation` may overshoot.
+- `keyframes.scale` cannot be combined with a `transform.width`/`height` box.
+- `keyframes` is only valid on visual clips. A static composition is captured once as an image and then animated like any other visual layer.
+
+Preview evaluates keyframes on every seek with the exact CSS timing function. Production generates per-frame FFmpeg expressions from the same evaluator and uses an alpha mask for animated opacity. Floating-point easing in the filter graph approximates the CSS curve; parity remains at clip capture, not at the final encoded pixel. Fractional `x`/`y` values may differ by a subpixel between preview and production.
+
+The read-only timeline preview draws a marker for each distinct keyframe frame on a clip. Hovering a marker lists the channels at that frame; clicking it seeks the playhead there.
+
+## Transitions
+
+A timeline may declare a `transitions` array. Each entry pairs two clips on one track across their overlap:
+
+```json
+"clips": [
+  { "id": "alpha", "track": "main", "source": { "kind": "composition", "id": "alpha" }, "start": 0, "duration": 60 },
+  { "id": "beta", "track": "main", "source": { "kind": "composition", "id": "beta" }, "start": 45, "duration": 60 }
+],
+"transitions": [
+  { "from": "alpha", "to": "beta", "kind": "dissolve", "easing": "ease-in-out" }
+]
+```
+
+- `from` and `to` must be clip IDs on the same track. A transition never spans tracks.
+- `kind` is required. `dissolve` is a visual fade on a video track and also crossfades the audio the two clips contribute. `crossfade` is an audio gain ramp on an audio track. Using the wrong kind for the track is an error.
+- `easing` defaults to `linear` and accepts the same CSS timing functions as keyframes, including `hold` and `steps(...)`.
+- There is no `duration`. The window is the overlap `to.start .. from.end`. Move `to.start` earlier to lengthen the transition; the outgoing clip's declared duration already covers its side, so no media handles are needed.
+- Valid geometry is `from.start < to.start < from.end < to.end`. The incoming clip must start inside the outgoing clip and end after it. Reversed order, a contained clip, a touching or later start, and cross-track pairs are errors.
+- A clip may be the `to` of one transition and the `from` of the next, so transitions chain. A clip may not participate twice on the same side, and the same pair may not be declared twice.
+- Overlap without a declared transition is still `TIMELINE_CLIP_OVERLAP`. A third clip that overlaps either side is an error, so at most two clips are active on a track.
+- During a dissolve the incoming clip composites directly above the outgoing clip with alpha `clipOpacity × eased progress`. Its static or keyframed `opacity` multiplies the transition. `blend` works normally because transition alpha is part of the source alpha.
+- Both kinds use equal power: the outgoing clip ramps as `sqrt(1 - t)` and the incoming as `sqrt(t)`, after easing. For a dissolve this applies to contributed audio; for a crossfade it applies to the two audio clips. `audio.muted` still zeroes its side. Transitions on a nested-timeline clip fade that group's audio.
+- Preview stacks the incoming visual layer above the outgoing layer, applies the same alpha, ramps `<audio>` volume during playback, and draws a read-only hatched band over the overlap on either track.
+
+`check` exposes the resolved transition count and validates every window. `still` and `render` share the composite path, so a still captured inside a dissolve matches the rendered frame. A crossfade changes no pixels. See `examples/transitions`.
+
+### Audio on the timeline
+
+Audio can live directly on an audio track and still crossfade:
+
+```json
+"tracks": [
+  { "id": "v1", "kind": "video" },
+  { "id": "a1", "kind": "audio" }
+],
+"clips": [
+  { "id": "bed", "track": "v1", "source": { "kind": "composition", "id": "bed" }, "start": 0, "duration": 105 },
+  { "id": "chord-a", "track": "a1", "source": { "kind": "audio", "src": "assets/a.wav" }, "start": 0, "duration": 60 },
+  { "id": "chord-b", "track": "a1", "source": { "kind": "audio", "src": "assets/b.wav" }, "start": 45, "duration": 60 }
+],
+"transitions": [
+  { "from": "chord-a", "to": "chord-b", "kind": "crossfade", "easing": "linear" }
+]
+```
+
+Mute composition audio with `audio.muted: true` when the sound comes from audio-track clips, so it does not play twice. `examples/transitions` has an `audio-crossfade` timeline that pairs a visual dissolve with an audio crossfade over the same 15 frames.
 
 ## Audio
 
@@ -133,15 +222,16 @@ npx frameyard render main --props props/main.json --workers 8 --scratch /path/to
 - `--container <mp4|mov|mkv|webm>` selects the delivery container (default `mp4`); `--codec <h264|hevc|vp9>` selects the codec, and `--encoder <auto|software|nvenc|qsv>` selects how it is encoded. MP4 and MOV hold H.264/HEVC with AAC; MKV holds all three; WebM holds VP9 with Opus and defaults to VP9. `auto` is the default and probes NVENC, then Quick Sync, then software; `software` forces `libx264`, `libx265`, or `libvpx-vp9`. FFV1 intermediates stay lossless and software-encoded, HEVC keeps the `hvc1` tag in MP4/MOV, and the output extension must match the container.
 - `--concurrency` is composition-only.
 - `render` defaults to `out/<timeline-id>.mp4`.
-- `check` evaluates every timeline frame and every animated composition frame; a static composition layer is inspected once and not seeked per frame. `still` captures one or more requested frames through the same clip capture and composite path; a multi-frame still resolves the timeline, starts one server, and reuses one scratch directory. The `--frame` spec and 32-frame cap are the same as for compositions.
+- `check` evaluates every timeline frame and every animated composition frame; a static composition layer is inspected once and not seeked per frame. It also evaluates every keyframe channel at every timeline frame. `still` captures one or more requested frames through the same clip capture and composite path; a multi-frame still resolves the timeline, starts one server, and reuses one scratch directory. The `--frame` spec and 32-frame cap are the same as for compositions.
 - `npx frameyard dev` lists timelines in the preview; the timeline view has a program monitor, read-only tracks and clips, playhead, playback, audio controls, and diagnostics.
 
 ## Determinism and performance
 
-- A timeline frame derives only from its active clips, their composition frames, props, assets, and the render plan.
+- A timeline frame derives only from its active clips, their composition frames, keyframe values, props, assets, and the render plan.
 - Composition frames are captured with Chromium; transparent compositions preserve alpha.
 - Timeline production captures active composition and nested-timeline clips to lossless FFV1, composites segments with FFmpeg, concatenates the ordered segments, and mixes audio once.
-- Preview composites in Chromium with CSS `mix-blend-mode`; production composites with FFmpeg's `blend` and `maskedmerge`. CSS is the reference; small scaling, alpha, color, and blend-math differences are expected. Parity is guaranteed at clip capture, not at final encoded pixels.
+- Keyframe and transition values are evaluated against absolute timeline frames, so segment boundaries do not change them. Position, scale, and rotation use per-frame filter expressions; animated opacity and incoming transition alpha use an alpha mask. Animated opacity, animated scale, and a dissolve add a mask/scale pass per affected layer.
+- Preview composites in Chromium with CSS `mix-blend-mode` and CSS `opacity`; production composites with FFmpeg's `blend`, `maskedmerge`, and an alpha mask. CSS is the reference; small scaling, alpha, color, blend-math, and subpixel-position differences are expected. Easing curves in the filter graph approximate the CSS curve; parity is guaranteed at clip capture, not at final encoded pixels.
 - Alpha-correct blending costs roughly 1.7× per blended 1080p layer in the production composite. Budget for it when a timeline stacks several blended layers.
 - Final output bytes are not a reproducibility guarantee.
 - Long timelines are segmented and bounded. The reference target and measured worst cases are documented in `docs/adr/0012-render-timelines-with-segmented-lossless-clip-intermediates.md`.