frameyard 0.9.0 → 0.10.0

package/docs/adr/0027-encode-two-full-frame-layers-through-dual-pipes.md

@@ -0,0 +1,76 @@
+# Encode a full-frame composition and a static overlay through dual pipes
+
+**Status:** Accepted. Extends [ADR-0026](./0026-reuse-loaded-composition-sessions-across-segments.md).
+**Date:** 2026-11-06
+
+A segment with a full-frame composition base and one static composition overlay
+now captures both layers concurrently and feeds them to one FFmpeg process
+through two image pipes. The base has an identity transform; the overlay may use
+a static fit, box, and position, but no keyframes, blend, transition, or
+opacity. The process composites the background, base, and overlay and encodes
+the delivery segment directly. The FFV1 intermediates and the separate
+composite pass are skipped.
+
+## Context
+
+The two-layer 60-second PIP gate takes 43.78 s; the overlay is a 640×360
+composition on a second track. A two-full-frame-layer workload (1920×1080 base
+plus a 1920×1080 transparent overlay) took 53.27 s with the normal FFV1 and
+composite path. Both captures already ran concurrently after ADR 0025, but the
+segment still paid for two FFV1 encodes, two decodes, and a separate composite
+encode.
+
+FFmpeg can read a second image stream from `pipe:3`, and Node can provide that
+extra stdio pipe on Linux and macOS. The two captures write to independent
+pipes, so neither blocks the other while FFmpeg overlays frames in order.
+
+## Decision
+
+- `startDualPipeSegmentEncoder` spawns one FFmpeg process with:
+  - `pipe:0` for the base composition;
+  - `pipe:3` for the overlay composition;
+  - a `lavfi color` input for the timeline background;
+  - `[2:v][0:v]overlay`, then `[base][1:v]overlay`, then the delivery encoder.
+- `captureTwoLayerDirectComposition` leases two capture sessions, writes each
+  layer to its pipe, and waits for both frame counts before finishing.
+- The fast path requires exactly two composition layers: a base that matches the
+  stage with an identity transform, and an overlay with no keyframes, blend,
+  transition, opacity, scale, or rotation. The overlay may use a static fit,
+  explicit box, and position. An opaque timeline background is required.
+- Windows falls back to the FFV1 and composite path because extra stdio pipe
+  descriptors are not portable there.
+- `FRAMEYARD_DISABLE_DUAL_PIPE=1` forces the fallback for A/B measurements.
+- One-layer direct segments keep `startDirectSegmentEncoder`; PIP, transformed,
+  blended, keyframed, nested, static, and media layers keep the segmented
+  lossless path.
+
+## Considered Options
+
+- **General multi-pipe compositor for any layer count.** Rejected for now:
+  extra pipes multiply the coordination surface and make Windows support
+  harder. Two full-frame layers cover the measured multi-layer case.
+- **Interleave two captures into one pipe.** Rejected: the filter graph would
+  need frame-splitting metadata and the capture loops would have to lockstep.
+- **Buffer the base layer and stream only the overlay.** Rejected: buffering a
+  300-frame 1080p layer costs hundreds of megabytes and adds a copy.
+- **Keep FFV1 and tune the composite.** Measured filters and thread knobs gave
+  no meaningful gain; the intermediate round trip is the cost.
+
+## Consequences
+
+- The two-full-frame-layer 60-second gate fell from 53.27 s to 34.62 s (−35.0%)
+  with byte-identical output. The full 60-minute two-layer gate completed in
+  29.69 minutes with eight workers, 108,000 exact frames, and clean scratch,
+  just inside the 30-minute target.
+- The two-layer PIP 60-second gate fell from 43.78 s to 33.90 s (−22.6%); sampled
+  decoded frames are pixel-identical to the FFV1-and-composite path. The full
+  PIP 60-minute gate completed in 25.75 minutes with eight workers, 108,000
+  exact frames, and clean scratch.
+- The fast path writes no FFV1 files and no PNG sequences for qualifying
+  segments.
+- A dual-pipe failure aborts both FFmpeg inputs and discards both sessions;
+  frame counts are checked per layer before publication.
+- Windows and non-qualifying segments keep the existing deterministic path, so
+  the capture-pixel guarantee is unchanged.
+- Tests cover two full-frame compositions with a transparent overlay over a
+  non-black background, plus the full timeline rendering suite.

package/docs/adr/0028-encode-full-frame-video-segments-directly.md

@@ -0,0 +1,65 @@
+# Encode full-frame video segments directly
+
+**Status:** Accepted. Extends [ADR-0027](./0027-encode-two-full-frame-layers-through-dual-pipes.md).
+**Date:** 2026-11-06
+
+A segment with one full-frame video layer, an identity transform, and an opaque
+timeline background now decodes and re-encodes the source range directly into
+the delivery segment. It skips the background overlay, fit, presentation, and
+composite filter graph, but keeps the same delivery encoder and container as
+every other segment.
+
+## Context
+
+The representative 60-minute 50/50 gate contains 24 seconds per minute of
+full-frame video on a single track. The composite path still built a color
+background, overlaid the video, normalized timestamps, fitted the frame, and
+then encoded. The video source already matches the stage, so those filters are
+identity work except for the fps/format normalization.
+
+Stream-copy was considered and deferred: it requires keyframe alignment, matching
+codec parameters, and a concat-compatible bitstream across composition and media
+segments. A direct re-encode avoids those risks while removing the composite
+overhead.
+
+## Decision
+
+- `encodeVideoSegmentDirect` seeks to the clip's source start, maps the first
+  video stream, drops audio, applies
+  `fps=<project fps>,format=rgba,format=yuv420p`, and encodes with the normal
+  delivery encoder/container settings. The RGBA round trip preserves the
+  composite path's color conversion and keeps decoded pixels identical.
+- The fast path requires one visual layer whose source is a video matching the
+  stage dimensions, an identity transform, and no keyframes, blend, transition,
+  or opacity. The timeline background must be opaque.
+- `FRAMEYARD_DISABLE_VIDEO_DIRECT=1` forces the composite path for A/B
+  measurements.
+- Video with transforms, opacity, overlays, keyframes, transitions, or a
+  non-matching source/stage size keeps the composite path. Composition and
+  nested-timeline segments are unaffected.
+
+## Considered Options
+
+- **Stream-copy the video range.** Deferred: keyframe alignment and mixed-codec
+  concat compatibility need a larger probe and validation surface. A direct
+  re-encode reaches the target without that risk.
+- **Keep the composite graph.** Rejected for this case: the background, fit, and
+  presentation filters are identity work for a full-frame video.
+- **Use the dual-pipe path.** Not applicable: there is only one visual layer.
+
+## Consequences
+
+- Full-frame video segments still re-encode, so every delivered segment shares
+  the same codec, profile, and container settings. The conformance risk is lower
+  than stream-copy.
+- The representative 60-minute 50/50 gate drops the composite path for its
+  single-video seconds while retaining it for overlays, PIP, images, and
+  multi-layer segments.
+- A 60-second full-frame video-only gate fell from 10.65 s to 7.61 s (−28.5%)
+  with pixel-identical decoded frames.
+- The representative 60-minute 50/50 gate fell from 16.82 to 15.57 minutes
+  (−7.4%) with 180 direct video segments, exact 108,000 frames, clean scratch,
+  and a byte-identical output file.
+- Tests cover a full-frame video timeline through the direct path and the
+  existing media/blend/keyframe/timeline suites.
+- The fast path is internal; there is no CLI or JSON contract change.

package/docs/preview-testing.md

@@ -39,6 +39,14 @@ expose the same seek/settle contract the renderer uses.
   `opacity` is `clipOpacity × eased progress` and it stacks above the outgoing
   layer on the same track. The strip draws a `.transition` band with a
   `from → to` tooltip; it is read-only.
+- The strip opens with a `.ruler-scale` carrying five `.ruler-tick` labels at
+  0/25/50/75/100% of the frame range. One `.playhead` line spans the ruler and
+  every track, driven by a `--playhead-factor` custom property on
+  `#timeline-inner`; `--progress` on `#scrub` fills the transport rail.
+- Both preview pages share the chrome in `src/server/preview-style.ts` and its
+  design tokens. The chrome serves its own Figtree and Space Mono from
+  `/__frameyard/framework/preview/fonts/` and does not link the project font
+  sheet. The embedded host page has none of this chrome.
 
 ## Waiting for a frame
 

package/docs/v0.x-design.md

@@ -118,13 +118,13 @@ Production timeline rendering is segmented and parallel:
 
 1. Run static timeline validation, resolve the timeline, and project the scene and render plan.
 2. Split the timeline into segments at active-layer changes or a maximum segment duration. Each segment has a fixed set of visual layers and a fixed filter graph.
-3. In parallel workers, capture the active composition and nested-timeline clip frames for each segment with Chromium and encode them to lossless FFV1 intermediates. Transparent clips are captured with an alpha-preserving background. A static composition is captured once per props variant as a transparent PNG and reused as an image layer across segments and workers.
+3. In parallel workers, capture the active composition and nested-timeline clip frames for each segment with Chromium and stream them into lossless FFV1 intermediates. Independent layers of one segment are captured concurrently under a global FIFO session limit, and loaded composition sessions are reused across segments when their entry, props, and transparency match. Transparent clips are captured with an alpha-preserving background. A static composition is captured once per props variant as a transparent PNG and reused as an image layer across segments and workers. A segment with one full-frame, untransformed composition and no keyframes, blend, transition, or opacity skips the FFV1 intermediate and the composite, encoding the capture directly into the delivery segment over the timeline background. A full-frame composition base plus a static composition overlay (including a static PIP box) uses two image pipes into one FFmpeg process, which composites the background, base, and scaled overlay before the delivery encode; Windows keeps the segmented fallback. A single full-frame video with an identity transform re-encodes its source range directly, skipping the composite filter graph while keeping the delivery encoder settings.
 4. Decode media clips, apply trims, fit, presentation keyframes, rotation into an expanded bound, and opacity, composite the segment with FFmpeg, and encode H.264, HEVC, or VP9.
 5. Join the ordered segments with a stream copy. Segment boundaries never change frame order or timing.
 6. Mix the timeline audio once with FFmpeg and mux it into the final output.
 7. Publish the output atomically; delete scratch intermediates on success and failure.
 
-The default timeline worker count is 8, configurable up to the machine's capacity. With `--encoder nvenc` the count is capped at eight concurrent encode sessions, the consumer driver limit; a larger `--workers` value produces an `ENCODER_WARNING` and the lower count. Scratch is a configurable directory with a bounded budget; it must not depend on `/tmp` alone. The reference long-form target is a representative 60-minute 1920×1080 30 fps timeline with a 50/50 composition/media split by layer-seconds, 60% one active layer, 30% two, and 10% three, rendered in 30 minutes or less on the reference machine. The 10-second benchmark measured 10.7 s for two composition captures, 35.2 MB of FFV1, and 6.1 s to composite and encode. Continuous three-layer stress runs measured 52.4 s and 58.5 s for 60 s of timeline, about 1× real time, and are documented as worst-case rather than the target workload. The representative gate measured 29.39 s for 60 s with four workers, exact 1800-frame output, and a 1080p media fast path, extrapolating to about 29.4 minutes for 60 minutes. The full 60-minute composition-only gate rendered 108,000 frames in 35.35 minutes with exact 108,000-frame / 3,600 s output; it is stricter than the 50/50 target mix because every frame is an HTML composition capture. The fast path copies full-frame, untransformed video whose source matches the project format and whose source keyframes (I-frames) fall on segment boundaries; media that needs scaling or transforms is re-encoded and adds cost not included in that projection. Timeline clip keyframes add per-frame filter expressions and, for animated opacity or scale, an alpha-mask pass per affected layer; the keyframe benchmark reports current timings and is not part of the 60-minute projection.
+The default timeline worker count is 8, configurable up to the machine's capacity. With `--encoder nvenc` the count is capped at eight concurrent encode sessions, the consumer driver limit; a larger `--workers` value produces an `ENCODER_WARNING` and the lower count. Scratch is a configurable directory with a bounded budget; it must not depend on `/tmp` alone. The reference long-form target is a representative 60-minute 1920×1080 30 fps timeline with a 50/50 composition/media split by layer-seconds, 60% one active layer, 30% two, and 10% three, rendered in 30 minutes or less on the reference machine. The 10-second benchmark measured 10.7 s for two composition captures, 35.2 MB of FFV1, and 6.1 s to composite and encode. Continuous three-layer stress runs measured 52.4 s and 58.5 s for 60 s of timeline, about 1× real time, and are documented as worst-case rather than the target workload. The representative gate measured 29.39 s for 60 s with four workers, exact 1800-frame output, and a 1080p media fast path, extrapolating to about 29.4 minutes for 60 minutes. The full 60-minute composition-only gate rendered 108,000 frames in 37.28 minutes with four workers; it is stricter than the 50/50 target mix because every frame is an HTML composition capture. Capture was 48.9% of worker time, FFV1 23.2%, composite 24.7%, and session setup 2.8%. Composition captures now stream PNG buffers into FFV1, and a segment with one full-frame, untransformed composition skips the intermediate and composite. The 60-second one-layer gate fell from 43.27 s to 27.64 s, and the full 60-minute gate completed in 19.83 minutes with eight workers; the direct output was byte-identical to the FFV1-and-composite output. Reusing loaded sessions reduced the 60-minute gate further to 18.78 minutes with eight sessions instead of 360. The two-layer PIP 60-second gate improved from 62.79 s to 59.60 s with streaming and to 43.78 s with parallel layer capture under a global session limit. A full-frame composition base plus a static composition overlay uses dual image pipes; the full-frame 60-second workload fell from 53.27 s to 34.62 s, the PIP 60-second workload from 43.78 s to 33.90 s, and the full two-layer 60-minute gates completed in 29.69 minutes (full-frame) and 25.75 minutes (PIP) with eight workers. The production representative 60-minute gate — 50/50 layer-seconds, 60/30/10 occupancy, 360 clips, 540 segments, and video re-encoded rather than stream-copied — completed in 16.82 minutes with eight workers, exact 108,000 frames, and clean scratch. Direct full-frame video re-encoding later reduced it to 15.57 minutes with a byte-identical file; a 60-second video-only gate fell from 10.65 to 7.61 seconds. The 10-minute one-layer gate improved from 211.8 s to 192.6 s by reusing loaded composition sessions across segments. The fast path copies full-frame, untransformed video whose source matches the project format and whose source keyframes (I-frames) fall on segment boundaries; media that needs scaling or transforms is re-encoded and adds cost not included in that projection. Timeline clip keyframes add per-frame filter expressions and, for animated opacity or scale, an alpha-mask pass per affected layer; the keyframe benchmark reports current timings and is not part of the 60-minute projection.
 
 The renderer uses a bounded segment plan, a worker pool, and a scratch budget. It never publishes a partial output, and it cleans scratch on success and failure. Segment boundaries preserve timeline frame order and timing; the final audio mix is muxed once.
 
@@ -296,7 +296,7 @@ Tests prioritize public behavior and end-to-end evidence over implementation-onl
 - Media clips trim and fit deterministically; images require durations; source overruns fail.
 - Props overrides for timeline composition clips merge over clip props and are reflected in the render.
 - `check` exercises every animated frame and inspects a static composition once; it also evaluates every keyframe channel at every timeline frame. `still` follows the clip capture and timeline composite path. A static composition render captures one PNG; a timeline reuses one static image capture per props variant.
-- The 10-second and 60-second long-form benchmarks record capture, FFV1, composite, encode, disk, worker, and audio numbers. Continuous three-layer runs measured 52.4 s and 58.5 s for 60 s of timeline and are documented as worst-case (~1× real time). The representative 50/50, 60/30/10 occupancy gate measured 29.39 s for 60 s with four workers and a 1080p media fast path, extrapolating to about 29.4 minutes for 60 minutes with exact 1800-frame output. The full 60-minute composition-only gate rendered 108,000 frames in 35.35 minutes with exact output and clean scratch, documenting composition-heavy performance above 30 minutes.
+- The 10-second and 60-second long-form benchmarks record capture, FFV1, composite, encode, disk, worker, and audio numbers. Continuous three-layer runs measured 52.4 s and 58.5 s for 60 s of timeline and are documented as worst-case (~1× real time). The representative 50/50, 60/30/10 occupancy gate measured 29.39 s for 60 s with four workers and a 1080p media fast path, extrapolating to about 29.4 minutes for 60 minutes with exact 1800-frame output. The full 60-minute composition-only gate rendered 108,000 frames in 37.28 minutes with four workers before streaming and the direct-segment fast path; the same gate completed in 19.83 minutes with eight workers after ADR 0024 and 18.78 minutes after session reuse (ADR 0026), with exact output, clean scratch, and a byte-identical file. The ten-minute gate projects about 24 minutes at four workers. Independent layers are captured concurrently under a global session limit (ADR 0025); the two-layer PIP 60-second gate fell from 59.60 s to 43.78 s with parallel captures. A full-frame composition base plus a static composition overlay encodes through dual image pipes (ADR 0027); the full-frame 60-second workload fell from 53.27 s to 34.62 s, the PIP 60-second workload from 43.78 s to 33.90 s, and the full two-layer 60-minute gates completed in 29.69 minutes (full-frame) and 25.75 minutes (PIP) with eight workers. The representative 60-minute 50/50 production gate completed in 16.82 minutes with eight workers, exact 108,000 frames, clean scratch, and no video stream-copy; direct full-frame video re-encoding reduced it to 15.57 minutes with a byte-identical file. Loaded composition sessions are reused across segments (ADR 0026); the 10-minute one-layer gate fell from 211.8 s to 192.6 s, and the full gate opened eight sessions instead of 360.
 - Golden-frame tests cover representative composition and timeline visual behavior.
 - Normal CI uses fake `qwen-tts`, `sherpa-onnx-offline-tts`, and `whisper-cli` executables and small fixtures to verify arguments, artifacts, manifests, cancellation, and errors without model downloads or GPU hardware.
 - Real-model integration tests are explicitly opt-in and use documented upstream revisions.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "frameyard",
-  "version": "0.9.0",
+  "version": "0.10.0",
   "description": "Local-first, deterministic video composition framework for HTML, CSS, and JavaScript.",
   "type": "module",
   "license": "MIT",
@@ -64,7 +64,7 @@
     "vitest": "5.0.5"
   },
   "scripts": {
-    "build": "tsc -p tsconfig.json",
+    "build": "tsc -p tsconfig.json && node scripts/copy-preview-fonts.mjs",
     "typecheck": "tsc -p tsconfig.json --noEmit",
     "typecheck:test": "tsc -p tsconfig.test.json",
     "typecheck:browser": "tsc -p tsconfig.browser.json",

package/skills/frameyard/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: frameyard
 description: Create, validate, inspect, and render deterministic Frameyard videos authored with HTML compositions and JSON timelines. Use when a user asks to build or operate a Frameyard project, produce parameterized video, assemble clips into a timeline, search or install project fonts, add a local TTS or transcription asset through Frameyard's wrappers, or diagnose a Frameyard render. Do not use for general-purpose media generation outside a Frameyard project.
-frameyard-api-version: 0.9.0
+frameyard-api-version: 0.10.0
 ---
 
 # Frameyard