frameyard 0.5.0 → 0.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (128) hide show
  1. package/CHANGELOG.md +100 -0
  2. package/GLOSSARY.md +152 -24
  3. package/README.md +99 -31
  4. package/dist/assets/process.js.map +1 -1
  5. package/dist/assets/provenance.js.map +1 -1
  6. package/dist/cli.js +326 -40
  7. package/dist/cli.js.map +1 -1
  8. package/dist/composition/attributes.d.ts.map +1 -1
  9. package/dist/composition/diagnostics.js.map +1 -1
  10. package/dist/contracts.d.ts +12 -0
  11. package/dist/contracts.d.ts.map +1 -1
  12. package/dist/contracts.js.map +1 -1
  13. package/dist/doctor.d.ts.map +1 -1
  14. package/dist/doctor.js +79 -0
  15. package/dist/doctor.js.map +1 -1
  16. package/dist/errors.d.ts +1 -1
  17. package/dist/errors.d.ts.map +1 -1
  18. package/dist/errors.js.map +1 -1
  19. package/dist/html/validate.js.map +1 -1
  20. package/dist/index.d.ts +2 -0
  21. package/dist/index.d.ts.map +1 -1
  22. package/dist/index.js +1 -0
  23. package/dist/index.js.map +1 -1
  24. package/dist/preview/main.js +96 -11
  25. package/dist/preview/main.js.map +1 -1
  26. package/dist/preview/timeline-data.d.ts +85 -0
  27. package/dist/preview/timeline-data.d.ts.map +1 -0
  28. package/dist/preview/timeline-data.js +2 -0
  29. package/dist/preview/timeline-data.js.map +1 -0
  30. package/dist/preview/timeline-host.d.ts +21 -0
  31. package/dist/preview/timeline-host.d.ts.map +1 -0
  32. package/dist/preview/timeline-host.js +308 -0
  33. package/dist/preview/timeline-host.js.map +1 -0
  34. package/dist/preview/timeline-main.d.ts +17 -0
  35. package/dist/preview/timeline-main.d.ts.map +1 -0
  36. package/dist/preview/timeline-main.js +368 -0
  37. package/dist/preview/timeline-main.js.map +1 -0
  38. package/dist/project.d.ts +16 -1
  39. package/dist/project.d.ts.map +1 -1
  40. package/dist/project.js +81 -26
  41. package/dist/project.js.map +1 -1
  42. package/dist/renderer/browser.d.ts +2 -0
  43. package/dist/renderer/browser.d.ts.map +1 -1
  44. package/dist/renderer/browser.js +6 -1
  45. package/dist/renderer/browser.js.map +1 -1
  46. package/dist/renderer/ffmpeg.d.ts +126 -1
  47. package/dist/renderer/ffmpeg.d.ts.map +1 -1
  48. package/dist/renderer/ffmpeg.js +450 -6
  49. package/dist/renderer/ffmpeg.js.map +1 -1
  50. package/dist/renderer/index.d.ts +28 -3
  51. package/dist/renderer/index.d.ts.map +1 -1
  52. package/dist/renderer/index.js +138 -38
  53. package/dist/renderer/index.js.map +1 -1
  54. package/dist/renderer/timeline.d.ts +146 -0
  55. package/dist/renderer/timeline.d.ts.map +1 -0
  56. package/dist/renderer/timeline.js +1098 -0
  57. package/dist/renderer/timeline.js.map +1 -0
  58. package/dist/runtime/host.js.map +1 -1
  59. package/dist/runtime/media.d.ts.map +1 -1
  60. package/dist/runtime/media.js +3 -6
  61. package/dist/runtime/media.js.map +1 -1
  62. package/dist/runtime/video.d.ts +13 -0
  63. package/dist/runtime/video.d.ts.map +1 -0
  64. package/dist/runtime/video.js +41 -0
  65. package/dist/runtime/video.js.map +1 -0
  66. package/dist/scaffold.d.ts.map +1 -1
  67. package/dist/scaffold.js +90 -24
  68. package/dist/scaffold.js.map +1 -1
  69. package/dist/server/composition-html.d.ts.map +1 -1
  70. package/dist/server/preview-html.d.ts.map +1 -1
  71. package/dist/server/preview-html.js +15 -1
  72. package/dist/server/preview-html.js.map +1 -1
  73. package/dist/server/server.d.ts.map +1 -1
  74. package/dist/server/server.js +39 -1
  75. package/dist/server/server.js.map +1 -1
  76. package/dist/server/timeline-preview-data.d.ts +4 -0
  77. package/dist/server/timeline-preview-data.d.ts.map +1 -0
  78. package/dist/server/timeline-preview-data.js +97 -0
  79. package/dist/server/timeline-preview-data.js.map +1 -0
  80. package/dist/server/timeline-preview-html.d.ts +4 -0
  81. package/dist/server/timeline-preview-html.d.ts.map +1 -0
  82. package/dist/server/timeline-preview-html.js +227 -0
  83. package/dist/server/timeline-preview-html.js.map +1 -0
  84. package/dist/timeline/plan.d.ts +32 -0
  85. package/dist/timeline/plan.d.ts.map +1 -0
  86. package/dist/timeline/plan.js +68 -0
  87. package/dist/timeline/plan.js.map +1 -0
  88. package/dist/timeline/resolve.d.ts +60 -0
  89. package/dist/timeline/resolve.d.ts.map +1 -0
  90. package/dist/timeline/resolve.js +523 -0
  91. package/dist/timeline/resolve.js.map +1 -0
  92. package/dist/timeline/schema.d.ts +71 -0
  93. package/dist/timeline/schema.d.ts.map +1 -0
  94. package/dist/timeline/schema.js +425 -0
  95. package/dist/timeline/schema.js.map +1 -0
  96. package/dist/transcript.js.map +1 -1
  97. package/dist/version.d.ts +1 -1
  98. package/dist/version.js +1 -1
  99. package/docs/adr/0002-make-frame-and-props-the-only-animation-state.md +2 -0
  100. package/docs/adr/0005-keep-parallel-rendering-opt-in-and-frame-ordered.md +2 -0
  101. package/docs/adr/0007-make-html-documents-the-composition-unit.md +1 -1
  102. package/docs/adr/0009-composite-production-frames-in-software.md +1 -1
  103. package/docs/adr/0010-make-timelines-the-render-unit.md +24 -0
  104. package/docs/adr/0011-compose-timeline-frames-in-a-live-browser-host.md +22 -0
  105. package/docs/adr/0012-render-timelines-with-segmented-lossless-clip-intermediates.md +31 -0
  106. package/docs/adr/0013-capture-bounded-multi-frame-stills-for-inspection.md +33 -0
  107. package/docs/preview-testing.md +53 -0
  108. package/docs/v0.1-design.md +138 -87
  109. package/examples/demo/README.md +10 -3
  110. package/examples/demo/timelines/demo.timeline.json +12 -0
  111. package/examples/demo/video.config.js +4 -0
  112. package/examples/video/README.md +50 -6
  113. package/examples/video/compositions/video-overlay.html +350 -0
  114. package/examples/video/props/video-timeline.json +10 -0
  115. package/examples/video/timelines/video-timeline-mp4.timeline.json +44 -0
  116. package/examples/video/timelines/video-timeline.timeline.json +44 -0
  117. package/examples/video/video.config.js +6 -0
  118. package/package.json +5 -3
  119. package/skills/frameyard/README.md +7 -5
  120. package/skills/frameyard/SKILL.md +19 -13
  121. package/skills/frameyard/references/common-mistakes.md +1 -0
  122. package/skills/frameyard/references/deterministic-authoring.md +2 -1
  123. package/skills/frameyard/references/diagnostics.md +1 -1
  124. package/skills/frameyard/references/failure-recovery.md +3 -2
  125. package/skills/frameyard/references/json-results.md +15 -6
  126. package/skills/frameyard/references/subtitles-karaoke.md +2 -2
  127. package/skills/frameyard/references/timelines.md +143 -0
  128. package/skills/frameyard/scripts/frameyard.mjs +20 -3
@@ -0,0 +1,22 @@
1
+ # Compose timeline frames in a live browser host
2
+
3
+ **Status:** Superseded by [ADR-0012](./0012-render-timelines-with-segmented-lossless-clip-intermediates.md). Rejected for production in 0.6; retained as the preview model.
4
+ **Date:** 2026-10-31
5
+
6
+ > **Spike result (2026-10-31):** the host was deterministic (pixel-identical across seeks and sessions) and a one-clip host was pixel-identical to a direct composition capture, but a 1920×1080 30 fps timeline with four active layers averaged 138.8 ms/frame versus 33.6 ms/frame for a direct composition capture (4.1×). The project prioritizes long-timeline rendering throughput, so the live host is kept only as a preview candidate, not as the production compositor.
7
+
8
+ Timeline frames will be produced by a browser timeline host that loads every clip source as a layer and captures the composed stage. Composition and nested-timeline clips are same-origin iframes, media clips are video and image elements, and static clip transforms and opacity are CSS. For each timeline frame the host seeks only the active clips, waits for their settle, and the renderer captures the stage through CDP. Preview and production drive the same host, so the timeline output has one compositing path.
9
+
10
+ ## Considered Options
11
+
12
+ - **Pre-render each clip and assemble with FFmpeg.** Rejected as the primary path because preview must composite live, so preview and render would diverge at scaling, opacity, and blending; intermediates also add generation loss or very large lossless files.
13
+ - **Capture each clip separately and composite PNG frames in Node.** Rejected because it is substantially slower per timeline frame and cannot use browser-native compositing; preview would still need a separate browser compositor.
14
+ - **Composite with CSS in the preview but pre-render for production.** Rejected for the same parity reason; the project's value is that preview and production execute the same behavior.
15
+
16
+ ## Consequences
17
+
18
+ - Composition and nested-timeline clips are isolated iframe surfaces. CSS blend modes and backdrop filters cannot blend across clip boundaries; 0.6 has neither.
19
+ - The timeline host preloads all clip sources, then network is blocked for frame production. Memory and WebGL context use grow with the number of loaded clips; the spike must measure a representative timeline before this decision is accepted.
20
+ - Timeline rendering is sequential in 0.6. Composition rendering keeps opt-in concurrency.
21
+ - Audio is not played by the host; preview mixes timeline audio in the parent and production mixes it offline with FFmpeg.
22
+ - If the spike fails its cost budget, this ADR is revised before implementation; the timeline document contract does not depend on the compositor choice.
@@ -0,0 +1,31 @@
1
+ # Render timelines with segmented lossless clip intermediates
2
+
3
+ **Status:** Accepted
4
+ **Date:** 2026-10-31
5
+
6
+ Timeline rendering will be segmented and parallel. For each segment, Chromium renders the active composition and nested-timeline clips to lossless FFV1 intermediates; FFmpeg decodes media, applies the clip's fit and static transform, composites the segment, encodes H.264, and the ordered segments are concatenated without re-encoding. Audio is mixed once for the whole timeline and muxed at the end. Preview remains a live Chromium timeline host with a bounded pool of active clips; parity is guaranteed at clip capture, not at the final composite.
7
+
8
+ The production target is a representative 60-minute 1920×1080 30 fps timeline with up to three active layers, a roughly 50/50 composition/media split by layer-seconds, 60% one layer, 30% two layers, and 10% three layers. The measured 10-second segment (two compositions, video, image) captured in 10.7 s, produced 35.2 MB of FFV1, and composited plus encoded in 6.1 s. A continuous three-layer composition-heavy 60-second gate took 52.4 s, and a continuous 50/50 full-frame video-overlay gate took 58.5 s; both extrapolate to about 1× real time and are documented as worst-case behavior, not the target workload. The representative 60-second gate — 50/50 layer-seconds, 60/30/10 occupancy, eight segments, four workers, and a 1080p video fast path — took 29.39 s and produced exactly 1800 frames at 30 fps, extrapolating to about 29.4 minutes for 60 minutes. The full 60-minute gate (2026-11-01) rendered a composition-only stress timeline of 108,000 frames, 120 composition clips, 360 segments, and four workers in 2,120.9 s (35.35 minutes). The output was exactly 108,000 frames / 3,600.000 s at 1920×1080, 30 fps, with scratch cleaned. That gate is stricter than the 50/50 target mix because every second is an HTML composition capture; it exceeds 30 minutes by 18%. The target is therefore met for the representative 50/50 workload, with little margin, while composition-heavy timelines run at roughly 1.7× real time.
9
+
10
+ The 1.8 TB USB HDD provides enough scratch capacity; a raw-frame cache would be roughly 670–900 GB, which does not fit, which is why the design uses compressed lossless FFV1 rather than raw frames or PNG sequences.
11
+
12
+ ## Considered Options
13
+
14
+ - **Live browser host (ADR 0011).** Rejected: deterministic and pixel-exact, but 138.8 ms per 1920×1080 frame with four layers versus 33.6 ms for a direct composition capture. It does not meet the long-timeline target.
15
+ - **Skia + FFmpeg streaming compositor.** Rejected for 0.6: it avoids intermediates and preserves capture pixels, but adds a large compositor dependency and a custom frame pipeline. The FFmpeg + FFV1 path already meets the target without it. Skia remains an option if future effects or blend modes outgrow FFmpeg filters.
16
+ - **FFmpeg streaming through pipes.** Rejected: avoids intermediates and dependencies, but requires a fragile cross-platform multi-pipe/filter-graph coordination for a variable set of active layers.
17
+ - **Pre-render every clip for the whole timeline.** Rejected: for 60 minutes the intermediates can be tens to hundreds of gigabytes and the write/read time alone can exceed the render budget. Segmentation bounds peak scratch and lets workers run in parallel.
18
+ - **Lossy intermediates.** Rejected as the default: an extra lossy encode breaks pixel parity at clip capture. FFV1 keeps the captured frames exact; only the final H.264 encode is lossy, as it already is for composition renders.
19
+
20
+ ## Consequences
21
+
22
+ - Composition and nested-timeline clips are captured with Chromium; media clips are decoded directly by FFmpeg. The timeline compositor is FFmpeg, not the browser.
23
+ - Composition clips that are transparent must be captured with a transparent browser background (`Emulation.setDefaultBackgroundColorOverride` with alpha 0); otherwise the top-level canvas produces a white background. FFV1 uses a pixel format that preserves alpha.
24
+ - Segments are bounded by active-layer changes or a maximum duration (10 s in the benchmark) so each segment has a fixed filter graph; segments are encoded with identical settings and concatenated with `-c copy`.
25
+ - Full-frame video clips with no transform or opacity that already match the project format and whose segment boundaries fall on source keyframes may be stream-copied instead of re-encoded. Other media is scaled/transformed/re-encoded. The representative gate's 29.4-minute projection includes this fast path and excludes any source-transcode time.
26
+ - Timeline workers are bounded (8 by default on the reference machine) and scratch is configurable and cleaned on success and failure. A 60-minute render should not depend on the system `/tmp` tmpfs.
27
+ - Nested timelines are rendered recursively to segment intermediates and treated as one layer, so parent opacity and transform apply to the grouped result. Cycles are rejected statically.
28
+ - Audio is mixed globally rather than per segment, avoiding boundary clicks and keeping clip trims/gains exact.
29
+ - `check` and `still` use the clip capture + timeline composite path for the requested frame; the long-form render path is not used to produce a single still.
30
+ - Preview uses a live Chromium timeline host with a small active-clip pool. Scaling, alpha, and video color conversion may differ from the final FFmpeg composite; the reproducibility guarantee is at clip capture.
31
+ - The 60-second, three-layer, audio-enabled benchmark is the gate before production implementation. A continuous three-layer composition-heavy run measured 52.4 s for 60 s, and a continuous 50/50 full-frame video-overlay run measured 58.5 s; both are about 1× real time and are documented as worst-case. The representative 50/50, 60/30/10 occupancy gate measured 29.39 s for 60 s with four workers and a 1080p media fast path, extrapolating to about 29.4 minutes for 60 minutes with exact 1800-frame output. The full 60-minute, composition-only gate completed in 35.35 minutes with exact 108,000-frame output; composition-heavy performance remains above the 30-minute target. If stricter composition-heavy performance is required, the next levers are a faster lossless intermediate codec, persistent capture sessions, and encoder preset/thread tuning.
@@ -0,0 +1,33 @@
1
+ # Capture bounded multi-frame stills for inspection
2
+
3
+ **Status:** Accepted
4
+ **Date:** 2026-11-01
5
+
6
+ `still` will accept a bounded set of frames in one invocation. `--frame` becomes a spec: a frame number (`42`), an inclusive range (`2-12`), or a comma-separated list (`0,118-126,240`). Ranges are only shorthand for lists. Duplicates are removed preserving the first occurrence, order is preserved, and a single invocation captures at most 32 unique frames. There is no step, start/end flag, fps flag, output pattern, or general image-sequence mode. With one frame, `--out` remains a `.png` file. With multiple frames, `--out` is a directory and the files are named `still-<id>-<frame>.png`; when `--out` is omitted the current working directory is used.
7
+
8
+ The batch uses one preview server and one composition session with sequential seeks. Timeline stills resolve the timeline once, start one server, reuse one scratch directory, and call the existing frame composite path for each requested frame. All frames are validated against the target duration and all output paths are checked for conflicts before Chromium starts. Capture is all-or-nothing at the capture boundary: every frame is captured to a temporary file first, and only after all captures succeed are the files published. Publication is atomic per file. A disk or I/O failure during publication can leave a prefix of already-published files; strict batch atomicity is not promised.
9
+
10
+ JSON results always add `frames: [{ frame, path }]`, including for a single frame. `frame` and `path` remain only when exactly one frame was captured. `diagnostics`, `width`, `height`, and `elapsedMs` stay shared at the top level; diagnostics are merged and deduplicated, and timing is for the whole invocation. Grammar and cap failures are `USAGE`; out-of-range frames are `FRAME_OUT_OF_RANGE` with all invalid frames and the valid range; output conflicts are `OUTPUT_EXISTS` with every conflicting path; capture failures keep their underlying code and add the attempted frames and the failing frame. The preflight order is parse/expand, cap, bounds, output conflicts, then Chromium.
11
+
12
+ ## Context
13
+
14
+ Agents and humans inspect several representative frames when validating motion, easing, transitions, video seeking, and visual regressions. The previous `still` contract captured exactly one frame per invocation, so inspecting a short animation required one command per frame or an external loop. That multiplied tool calls, repeated static validation, preview-server startup, and Chromium startup, and fragmented the structured diagnostics across several JSON envelopes. The agent skill already asked for several representative frames, so the CLI fought the documented workflow.
15
+
16
+ At the same time, general image-sequence output is a documented non-goal. The boundary is scale and intent, not syntax: a list or short range for inspection is useful, while exporting an arbitrary sequence should use a video render or a separate downstream tool.
17
+
18
+ ## Considered Options
19
+
20
+ - **Keep one frame per invocation.** Rejected: simple, but every inspection of motion pays one process, server, and browser per frame and forces the agent to loop or make several calls.
21
+ - **Allow unbounded batch capture or a `--step`/`--fps` sequence.** Rejected: this turns `still` into an image-sequence exporter and makes it easy to generate hundreds of files and unreadable inspection workloads. A hard cap keeps the feature in the debugging-artifact category.
22
+ - **Capture a contact sheet (one PNG grid).** Rejected for now: one image is cheaper to inspect, but it loses per-frame resolution and adds layout and compositing decisions that are not part of the rendering contract.
23
+ - **Open a fresh session per frame.** Rejected: slower, and it hides state or media-seek bugs that production sequential seeking would expose. One session also makes batch pixels comparable to single-frame captures, which the tests assert.
24
+ - **Add a separate `--frames` flag or a `stills` command.** Rejected: it duplicates the concept and forces callers to choose a tool based on frame count. Extending `--frame` keeps one grammar and one command.
25
+ - **Strict batch atomicity with a staging directory and a directory rename.** Rejected: it requires a new output-directory protocol and cannot restore files already replaced under `--overwrite` without a swap. Capture-level all-or-nothing plus per-file atomic publication covers the realistic failure mode for an inspection tool.
26
+
27
+ ## Consequences
28
+
29
+ - `still` remains a debugging artifact. The cap is per invocation; repeated invocations can still produce more files, but the CLI does not advertise or support image-sequence export.
30
+ - Single-frame output paths, PNG bytes, human output, and the existing JSON fields are unchanged; `frames` is additive. Multi-frame results omit the singular `frame` and `path`.
31
+ - A failed batch publishes no files and removes temporary files. If the CLI created the output directory, it removes the empty directory on failure. Existing directories are left in place.
32
+ - Output naming is one rule everywhere: `still-<id>-<frame>.png` without zero-padding. Composition and timeline IDs are already valid filename components.
33
+ - The cap, range grammar, output rules, and publication caveat are documented in the design baseline, README, CLI help, glossary, and official skill. Tests cover grammar, dedupe, order, cap boundaries, bounds, conflicts, output modes, all-or-nothing failure, batch-versus-single pixels for compositions and timelines, embedded video, and WebGL.
@@ -0,0 +1,53 @@
1
+ # Preview testing
2
+
3
+ The preview runs the production composition and timeline hosts in a browser. Use
4
+ the embedded hosts for deterministic Playwright probes; they are same-origin and
5
+ expose the same seek/settle contract the renderer uses.
6
+
7
+ ## Composition preview
8
+
9
+ - Page: `/__frameyard/?composition=<id>`.
10
+ - Stage: the `#stage-frame` iframe.
11
+ - API: `window.__frameyardPreview` exposes `load(entry, props?)`, `seek(frame)`,
12
+ `settle()`, and `dispose()`.
13
+ - `seek()` is synchronous. Call `await settle()` before reading pixels; `load()`
14
+ resets the host, so apply props through it.
15
+
16
+ ## Timeline preview
17
+
18
+ - Page: `/__frameyard/timeline/<id>`; add `?__fy-embed=1` for the embedded host.
19
+ - API: `window.__fyTimeline` exposes `ready`, `format`, `durationInFrames`,
20
+ `load()`, `seek(frame)`, `settle()`, and `dispose()`.
21
+ - Order: `await ready`, `await load()`, `await seek(frame)`, `await settle()`.
22
+ - The non-embedded page owns transport and audio; only the embedded host exposes
23
+ `window.__fyTimeline`.
24
+
25
+ ## Waiting for a frame
26
+
27
+ - `settle()` waits for video presentation through `requestVideoFrameCallback`
28
+ and then for two animation frames for composition and iframe paint. Do not add
29
+ `waitForTimeout` to assertions.
30
+ - For a composition layer, wait for a controller-updated marker. The shared
31
+ helper `test/helpers/preview.ts` exports `waitForFrame(page, frame)`.
32
+ - For a video layer, sample the rendered stage after `settle()`; do not trust
33
+ `video.currentTime` alone. `test/timeline-preview-runtime.test.ts` covers this.
34
+ - The same helper exports `openEmbeddedTimeline`, `loadTimeline`, and
35
+ `seekTimeline`.
36
+
37
+ ## Verification
38
+
39
+ For preview-only UI changes, run `pnpm check:preview`. It runs the static checks
40
+ and then builds before running `test/preview-runtime.test.ts`,
41
+ `test/timeline-preview.test.ts`, and `test/timeline-preview-runtime.test.ts`.
42
+ Use the full `pnpm check` when the change reaches shared contracts, the runtime,
43
+ the renderer, or the CLI, and before pushing or publishing.
44
+
45
+ ## Notes
46
+
47
+ - Preview iframes and the composition stage pin `color-scheme: light` to match
48
+ the production capture context. A dark inherited scheme makes Chromium paint a
49
+ white base behind transparent compositions.
50
+ - A running `dev` server keeps the `dist/` modules it loaded. Restart it after
51
+ `pnpm build`.
52
+ - The timeline preview host is not the production compositor; compare pixels at
53
+ the capture boundary, not encoded output.
@@ -2,155 +2,198 @@
2
2
 
3
3
  **Status:** Accepted
4
4
  **Initial acceptance:** 2026-10-27
5
- **Last design revision:** 2026-10-30
5
+ **Last design revision:** 2026-10-31
6
6
 
7
- This document records the decisions confirmed during design sessions. It is the living product and architecture baseline; later changes revise this document explicitly rather than silently broaden the scope. The original v0.1 baseline has been revised through 0.4. Architecture decisions that changed the baseline are recorded in `docs/adr/`.
7
+ This document records the decisions confirmed during design sessions. It is the living product and architecture baseline; later changes revise this document explicitly rather than silently broaden the scope. The original v0.1 baseline has been revised through 0.6. Architecture decisions that changed the baseline are recorded in `docs/adr/`.
8
8
 
9
9
  ## 1. Product contract
10
10
 
11
- Frameyard is a local-first, open-source framework for frontend developers and creative coders who build short, parameterized videos with HTML, CSS, and JavaScript. AI agents can perform the same workflows on a user's behalf through an official skill. Its differentiator is web-native authoring combined with deterministic, frame-accurate output—not a React-centric composition model or a feature-for-feature clone of an existing tool.
11
+ Frameyard is a local-first, open-source framework for frontend developers and creative coders who build parameterized videos with HTML, CSS, and JavaScript. It separates reusable, prop-parameterized compositions from the timelines that assemble compositions and media into one video. AI agents can perform the same workflows on a user's behalf through an official skill.
12
12
 
13
- The initial use cases are product demos, social clips, animated diagrams, data visualizations, karaoke-style subtitles, and personalized videos. Optional local speech synthesis and transcription produce assets for these workflows. Professional long-form editing is outside the target audience.
13
+ Its differentiator is web-native authoring combined with deterministic, frame-accurate output and an explicit timeline model—not a React-centric composition model or a feature-for-feature clone of an existing NLE.
14
+
15
+ The target use cases are product demos, social clips, animated diagrams, data visualizations, karaoke-style subtitles, personalized videos, and long-form edits up to roughly 60 minutes. Professional collaborative editing, cloud production, and broadcast workflows remain outside the target audience.
14
16
 
15
17
  The core success criterion is an end-to-end example in which a developer can:
16
18
 
17
- 1. Create a reusable 30-second, 1920×1080 HTML composition.
18
- 2. Animate it with HTML, CSS, SVG, and, where needed, a seek-safe controller for Canvas 2D and WebGL.
19
+ 1. Create reusable, 1920×1080 HTML compositions with fixed metadata.
20
+ 2. Animate them with HTML, CSS, SVG, and, where needed, a seek-safe controller for Canvas 2D and WebGL.
19
21
  3. Supply JSON props and synchronized audio.
20
- 4. Preview and scrub it interactively.
21
- 5. Render it reproducibly to MP4 without dropped frames or timing drift.
22
+ 4. Assemble compositions and local media into a timeline with tracks, clips, trims, layers, and nested timelines.
23
+ 5. Preview and scrub the timeline interactively.
24
+ 6. Render it reproducibly to MP4 without dropped frames or timing drift.
25
+ 7. Render a 60-minute timeline with up to three active layers within the long-form performance target.
22
26
 
23
- The agent-enabled success criterion additionally requires an agent using the official skill to create and preserve a valid source project, optionally produce local audio and transcript assets, validate and inspect the composition, render it, and report all resulting artifact paths without concealing validation failures or skipped visual review.
27
+ The agent-enabled success criterion additionally requires an agent using the official skill to create and preserve a valid source project, optionally produce local audio and transcript assets, validate and inspect the timeline and its compositions, render it, and report all resulting artifact paths without concealing validation failures or skipped visual review.
24
28
 
25
29
  ## 2. Project and authoring model
26
30
 
31
+ ### Project
32
+
27
33
  - Projects use ordinary HTML, CSS, static assets, and native JavaScript controllers, either inline in a composition document or in external modules.
28
- - `video.config.js` explicitly lists `.html` composition entry points. There is no directory-scanning discovery convention.
29
- - A project may contain multiple compositions, each with a unique ID.
34
+ - `video.config.js` explicitly maps project IDs to source files. There is no directory scanning. It has three optional maps:
35
+ - `format`: the project's fixed `width`, `height`, and `fps`.
36
+ - `compositions`: IDs mapped to `.html` composition documents.
37
+ - `timelines`: IDs mapped to JSON timeline documents.
38
+ - `format` is required when a project has timelines. Every timeline uses it. A composition used in a timeline must share the project frame rate exactly; a different size is fitted and transformed by the clip. A standalone composition render keeps using the composition's own metadata.
39
+ - Composition and timeline IDs share one namespace so `render <id>`, `check <id>`, and `still <id>` are unambiguous. The config rejects duplicates.
40
+ - A project may contain any number of compositions and timelines.
41
+
42
+ ### Composition
43
+
30
44
  - A composition is a full HTML document with one `<fy-composition>` root. The root declares fixed `width`, `height`, `fps`, and `duration-in-frames` metadata. Props cannot change this metadata.
31
- - Default props are a JSON block, `<script type="application/json" data-fy="props">`. A render's JSON props merge shallowly over the defaults, are validated as JSON, deep-frozen, and fixed for the render.
45
+ - Default props are a JSON block, `<script type="application/json" data-fy="props">`. A load's JSON props merge shallowly over the defaults, are validated as JSON, deep-frozen, and fixed for that load.
32
46
  - Props bind into the document through explicit `data-fy-*` attributes: `data-fy-text`, `data-fy-attr-*`, `data-fy-class-*`, `data-fy-if`, and `data-fy-repeat`. Primitive top-level props are also exposed as `--fy-prop-<name>` CSS custom properties. Bindings apply once per load; props are immutable.
33
47
  - CSS and the Web Animations API are the animation language. `data-fy-anim-start`, `data-fy-anim-duration`, and `data-fy-anim-easing` schedule one animation per element in composition frames.
34
- - An optional composition controller is declared with `<script type="text/fy-controller">`. It is either inline source or an external module referenced by `src`; a composition has exactly one controller. It may export `mount({ root, props })`, `update({ frame, props, root })`, `dispose()`, and `validateProps(props)`; at least one is required, and `mount` and `update` are optional for validation-only controllers. It has no metadata, default props, audio, or transcripts. Inline keeps small compositions in one file; external is for reuse, tests, and complex logic.
35
- - Inline controllers are the default for small, single-use controllers. Extract to an external module when the controller grows, is reused, needs separate imports or tests, or drives Canvas 2D, WebGL, or karaoke.
36
- - The controller is the only executable code in a composition document. Other executable `<script>` elements and inline `on*` handlers are rejected. Libraries are imported as ES modules from the controller.
48
+ - An optional composition controller is declared with `<script type="text/fy-controller">`. It is either inline source or an external module referenced by `src`; a composition has exactly one controller. It may export `mount`, `update`, `dispose`, and `validateProps`; at least one is required. The controller is the only executable code in the document.
37
49
  - `fy` is the authoring namespace: `fy-composition`, `fy-transcript`, `data-fy-*`, `--fy-*`, and `text/fy-controller`. `frameyard` remains the product, package, CLI, config, manifest, server-route, import-specifier, and error-code namespace.
38
- - User controllers are native JavaScript. The framework itself is authored in TypeScript and ships declarations, but 0.4 does not transpile user TypeScript.
50
+ - User controllers are native JavaScript. The framework itself is authored in TypeScript and ships declarations, but 0.6 does not transpile user TypeScript.
51
+
52
+ ### Timeline
53
+
54
+ - A timeline is a versioned JSON document. It references compositions and nested timelines by project ID and media clips by project-relative path.
55
+ - A timeline has a project-fixed format, an ordered list of tracks, and a list of clips. It may declare a `background` color and an optional `durationInFrames`.
56
+ - Tracks are `video` or `audio`. Video tracks composite from bottom to top; audio tracks mix. Each clip belongs to exactly one track. Clips on the same track cannot overlap; clips on different tracks can. Touching endpoints are allowed. Transitions are deferred, so overlap within a track would have ambiguous precedence.
57
+ - A clip has an ID, a track, a source, a timeline start, a source in-point, a duration, and clip-specific presentation settings. Source in-point, duration, and timeline start are integer project frames.
58
+ - Clip sources are:
59
+ - a composition, identified by its project ID;
60
+ - a nested timeline, identified by its project ID;
61
+ - a local video, image, or audio asset.
62
+ - A composition clip fixes props for that instance and may trim the composition's source range. The same composition may appear multiple times with different props, source ranges, transforms, and audio settings.
63
+ - A nested timeline clip is a visual and audio source. It uses the project format, trims with the same source in-point and duration, contributes its audio, and is rendered as a group so parent opacity and transform apply to the whole nested result. Cycles are rejected statically. Timeline clips do not override props of compositions inside the nested timeline in 0.6.
64
+ - Media clips use local project files only. Video and audio durations, dimensions, and audio presence are probed before frame production. Images are static and require an explicit duration. A video or audio clip that exceeds its source range is an error; an omitted duration uses the remaining source.
65
+ - Visual clips support a static transform: fit policy (`contain`, `cover`, `stretch`), an optional explicit `width`/`height` box (both fields together, mutually exclusive with `scale`) or a `scale` multiplier, pixel offset, rotation, and opacity. `fit` applies inside the box; without one, the project frame is the box. No keyframes, blend modes, masks, filters, or other effects in 0.6.
66
+ - Visual clips contribute their audio to the timeline mix unless muted: composition audio, embedded video audio, and nested timeline audio. Audio-only clips live on audio tracks and may source audio or video files. Per-clip and per-audio-track gain and mute are supported. There are no linked A/V pairs; a J/L cut is expressed by muting a video clip and placing a separate audio clip with a different source in-point.
67
+ - A timeline's duration is derived from the latest clip end. An explicit `durationInFrames` may extend it but not cut clips. A timeline with no clips is invalid; an audio-only timeline is valid and renders black video with audio.
68
+ - A timeline background is transparent by default. The final video composites it over black; a nested timeline remains transparent so lower layers can show through. A composition that paints no background is transparent and may be used as an overlay.
69
+ - The preview provides a read-only timeline view: tracks, clips, playhead, program monitor, and diagnostics. It is not a visual editor.
39
70
 
40
71
  ## 3. Deterministic execution
41
72
 
42
- The only sources of animation state are the integer frame number and immutable props. Composition frame rate establishes the mapping between frames and time.
73
+ The timeline frame is the master clock. A composition's frame is derived from the clip's timeline start, source in-point, and duration. The only sources of a composition's animation state remain the integer composition frame and its immutable props.
43
74
 
44
- - The runtime owns the frame clock and deterministic randomness. The controller imports `random("stable:key")` from `frameyard/runtime`.
45
- - CSS `@keyframes` and Web Animations API animations are paused and positioned from the shared frame clock. `data-fy-anim-*` attributes schedule one animation per element in composition frames; the runtime maps frames to milliseconds and sets the corresponding animation delay and duration.
46
- - The runtime sets `--fy-frame`, `--fy-time`, `--fy-progress`, `--fy-fps`, `--fy-duration`, and `--fy-duration-frames` on the composition root every seek. Authors use them for simple computed CSS values.
75
+ - The runtime owns the frame clock and deterministic randomness. A composition controller imports `random("stable:key")` from `frameyard/runtime`.
76
+ - CSS `@keyframes` and Web Animations API animations are paused and positioned from the shared composition frame clock. `data-fy-anim-*` schedules one animation per element in composition frames; the runtime maps frames to milliseconds.
77
+ - The runtime sets `--fy-frame`, `--fy-time`, `--fy-progress`, `--fy-fps`, `--fy-duration`, and `--fy-duration-frames` on the composition root every seek.
47
78
  - JavaScript animation runs in the controller's `update()`, which must be synchronous and seek-safe. It cannot depend on previously visited frames.
48
79
  - Declarative bindings apply once per load, before the controller mounts. Controller-created DOM is the controller's responsibility.
49
80
  - Independent `requestAnimationFrame` loops, timers, CSS transitions, wall-clock reads, and unseeded randomness are not supported animation drivers.
50
- - Development mode provides best-effort diagnostics for forbidden clocks and randomness in the controller; it does not claim to police every third-party library perfectly.
51
- - Assets, remote data, images, fonts, generated audio, and transcripts are resolved during an explicit loading phase before frame production. Network requests are blocked during frame capture.
81
+ - Timeline clips use static transforms. Timeline-level animation, transitions, effects, and keyframes are not part of 0.6.
82
+ - Preview and production seek composition clips explicitly. The timeline compositor is deterministic given the captured clip frames, media frames, clip settings, and segment boundaries.
83
+ - Assets, remote data, images, fonts, generated audio, transcripts, and media are resolved before frame production. For a segmented timeline render, each segment's composition and nested-timeline clips are loaded and settled before that segment's frames are captured; no frame production depends on a new network request. Network is blocked during capture.
52
84
  - AI model inference is never part of the loading phase, controller lifecycle, or frame production.
53
85
  - The baseline supports DOM, CSS, SVG, Canvas 2D, WebGL, and frame-seeked embedded video. Canvas 2D and WebGL are drawn from the current frame by the controller.
54
- - Embedded video and audio are declared in the composition HTML with `data-fy-*` attributes. The runtime loads local files during the loading phase, pauses them, and seeks video from the frame-derived source time before capture. An unmuted video contributes its audio track to the offline mix unless the composition opts out.
55
- - Full-scene opacity animations above an embedded video can make interactive preview reuse the load frame after a large seek; fade overlay layers instead. Capture sessions keep the composition hidden until the first seek, so production stills resolve to the requested frame.
86
+ - Embedded video and audio inside compositions are declared with `data-fy-*` attributes. The runtime loads local files, pauses them, and seeks video from the frame-derived source time before capture. An unmuted video contributes its audio track to the timeline mix unless the composition opts out or the clip mutes it.
87
+ - Transparent compositions are captured with an alpha-preserving browser background. Without that step, a top-level Chromium capture paints a white canvas and overlays become white rectangles.
56
88
  - WebGL 1 and WebGL 2 canvases are captured through the normal screenshot path. Production sessions pin software WebGL so pixel output does not depend on the host GPU.
57
89
  - Compositions are trusted local developer code. Frameyard does not claim to render untrusted compositions safely.
58
90
 
59
91
  ### Reproducibility guarantee
60
92
 
61
- For the same composition revision, props, frame, assets, and pinned runtime/browser environment, Frameyard guarantees identical visual state. It does not promise bit-identical MP4 bytes, even across runs on the same machine; encoded output depends on the FFmpeg build, encoder threading, and runtime environment.
93
+ For the same timeline or composition revision, props, assets, segment plan, and frame under a pinned runtime/browser/FFmpeg environment, the captured composition frames and the deterministic timeline composite are identical. The final output is not promised to be bit-identical: encoded bytes depend on the encoder and its runtime.
62
94
 
63
- Generated assets are covered once frozen. Frameyard does not promise that repeating speech synthesis or transcription produces identical artifacts across different binaries, models, parameters, or hardware.
95
+ Preview and production share composition behavior and capture composition frames the same way, but the timeline preview composites layers in Chromium while production composites them with FFmpeg. Small differences in scaling, alpha, and video color conversion are expected and do not violate clip-capture parity.
64
96
 
65
97
  ## 4. Preview and production rendering
66
98
 
67
- Preview and production execute the same composition document and browser host runtime. They differ only in which component drives the frame clock and whether frames are captured.
99
+ Preview and production execute the same composition documents and host runtime. They differ in which component drives the frame clock, which compositor combines timeline layers, and whether frames are captured.
68
100
 
69
101
  The preview provides:
70
102
 
71
- - Composition selection
72
- - JSON props editing
73
- - Play/pause and frame-accurate scrubbing
74
- - Current frame and time display
75
- - Audio controls
103
+ - Composition preview: composition selection, JSON props editing, play/pause, frame-accurate scrubbing, frame-by-frame stepping with buttons or arrow keys, one-second jumps with buttons or Shift+arrow keys, current frame and time, audio controls, and diagnostics.
104
+ - Timeline preview: timeline selection, a read-only track/clip view, playhead, program monitor, frame-by-frame stepping with buttons or arrow keys, one-second jumps with buttons or Shift+arrow keys, audio controls, and diagnostics.
105
+ - A bounded pool of active composition and nested-timeline clips. It loads clips around the playhead instead of preloading a whole long timeline.
76
106
 
77
- It is a developer inspection tool, not a visual composition editor.
107
+ The preview server serves each composition HTML document at its project path and injects the framework import map and host runtime. It serves timeline documents and a timeline preview host. The timeline preview host drives active composition/nested-timeline clips, seeks them, and exposes a timeline API to the preview. Preview audio is mixed by the preview page from the resolved timeline audio clips.
78
108
 
79
- The preview server serves each composition HTML document at its project path and injects the framework import map and a host module into composition entries. The host reads the composition descriptor—metadata, default props, and the controller declaration—from the same browser-safe composition document reader used by static validation, applies bindings, imports the controller from its authoring `src` or from the inline block's virtual module URL, mounts it, and exposes a small `window.__fy` API. Preview loads the document in a same-origin iframe and drives that API. The iframe is not a security sandbox; compositions are trusted local code. The server runs the static composition validation described in §6 before serving a composition to the preview.
109
+ Production composition rendering is unchanged: one Chromium session per composition, explicit frame seeks, PNG capture, and FFmpeg encoding.
80
110
 
81
- Production rendering:
111
+ Production timeline rendering is segmented and parallel:
82
112
 
83
- 1. Uses a pinned, Playwright-managed Chromium through the Chrome DevTools Protocol.
84
- 2. Navigates directly to the composition document, waits for the host, and loads the render's props.
85
- 3. Seeks explicitly to each requested frame.
86
- 4. Waits for composition readiness, embedded media decode, and browser paint.
87
- 5. Captures the frame.
88
- 6. Pipes frames in timeline order to FFmpeg for encoding and audio muxing.
113
+ 1. Run static timeline validation and resolve the render plan.
114
+ 2. Split the timeline into segments at active-layer changes or a maximum segment duration. Each segment has a fixed set of visual layers and a fixed filter graph.
115
+ 3. In parallel workers, capture the active composition and nested-timeline clip frames for each segment with Chromium and encode them to lossless FFV1 intermediates. Transparent clips are captured with an alpha-preserving background.
116
+ 4. Decode media clips, apply trims, fit, static transforms, and opacity, composite the segment with FFmpeg, and encode H.264, HEVC, or VP9.
117
+ 5. Join the ordered segments with a stream copy. Segment boundaries never change frame order or timing.
118
+ 6. Mix the timeline audio once with FFmpeg and mux it into the final output.
119
+ 7. Publish the output atomically; delete scratch intermediates on success and failure.
89
120
 
90
- `check` and `still` use the same host and capture path. `check` evaluates every frame without capturing or encoding. `still` captures one frame through the production path.
121
+ The default timeline worker count is 8, configurable up to the machine's capacity. With `--encoder nvenc` the count is capped at eight concurrent encode sessions, the consumer driver limit; a larger `--workers` value produces an `ENCODER_WARNING` and the lower count. Scratch is a configurable directory with a bounded budget; it must not depend on `/tmp` alone. The reference long-form target is a representative 60-minute 1920×1080 30 fps timeline with a 50/50 composition/media split by layer-seconds, 60% one active layer, 30% two, and 10% three, rendered in 30 minutes or less on the reference machine. The 10-second benchmark measured 10.7 s for two composition captures, 35.2 MB of FFV1, and 6.1 s to composite and encode. Continuous three-layer stress runs measured 52.4 s and 58.5 s for 60 s of timeline, about 1× real time, and are documented as worst-case rather than the target workload. The representative gate measured 29.39 s for 60 s with four workers, exact 1800-frame output, and a 1080p media fast path, extrapolating to about 29.4 minutes for 60 minutes. The full 60-minute composition-only gate rendered 108,000 frames in 35.35 minutes with exact 108,000-frame / 3,600 s output; it is stricter than the 50/50 target mix because every frame is an HTML composition capture. The fast path copies full-frame, untransformed video whose source matches the project format and has keyframes at segment boundaries; media that needs scaling or transforms is re-encoded and adds cost not included in that projection.
91
122
 
92
- The renderer is sequential by default and has no frame cache. Optional `--concurrency <n>` rendering, with `n` from 1 to 16, opens up to `n` independent browser sessions, each loading the same composition document and props from the same preview server. Workers seek and capture assigned frames, while a single ordered writer pipes them to FFmpeg with bounded lookahead. Parallelism therefore changes scheduling only: every frame remains derivable from its integer frame number and immutable props, and output order is unchanged. The default stays at one session for predictable resource use, and the upper bound limits accidental browser-process and memory growth. Simple scenes target real-time preview, but there is no production rendering-speed guarantee; the CLI reports progress, current frame, and elapsed time.
123
+ The renderer uses a bounded segment plan, a worker pool, and a scratch budget. It never publishes a partial output, and it cleans scratch on success and failure. Segment boundaries preserve timeline frame order and timing; the final audio mix is muxed once.
124
+
125
+ `check` and `still` use the same clip capture and timeline composite path for requested frames. `check` evaluates every timeline frame; on long timelines this is a validation cost, not a render. `still` captures one or more requested frames as PNGs without encoding a video.
93
126
 
94
127
  ## 5. Audio and output
95
128
 
96
- Audio is a declarative timeline of `<audio>` elements. Each element uses `data-fy-start` (composition frame), `data-fy-trim` (seconds from the source start), optional `data-fy-duration` (seconds to use), and `data-fy-volume` (0-1). Preview synchronizes this timeline in the browser; production mixes it offline with FFmpeg rather than treating live `<audio>` or Web Audio playback as the production source of truth. Unmuted embedded videos contribute their own audio tracks to the same mix, placed by the video's frame and trim attributes. Muting a video or setting `data-fy-audio="off"` keeps it visual-only.
129
+ A timeline has one audio mix. It combines:
130
+
131
+ - composition audio clips declared inside composition clips, shifted and trimmed by the clip's source range;
132
+ - embedded video audio from composition clips and media video clips, unless muted;
133
+ - timeline audio clips on audio tracks;
134
+ - audio from nested timeline clips, recursively.
135
+
136
+ Per-clip gain and mute, per-audio-track gain and mute, and composition-level volume apply. Audio fades are deferred. The mix is produced once with FFmpeg and muxed into the final output. Preview mixes the same resolved clips in the browser for monitoring; it is not the production source of truth.
137
+
138
+ Audio, video, and transcript `src` values resolve from the project root. The controller `src` and ordinary HTML asset URLs resolve relative to the composition document.
97
139
 
98
- Audio, video, and transcript `src` values resolve from the project root. The controller `src` and ordinary HTML asset URLs (`img`, stylesheets) resolve relative to the composition document.
140
+ Generated speech behaves like any other audio clip. Frameyard does not automatically time-stretch it, change timeline duration, or regenerate it to fit. `check` reports clip durations and overrun warnings so users and agents can revise text, timing, or the timeline deliberately.
99
141
 
100
- Generated speech behaves like any other audio clip. Frameyard does not automatically time-stretch it, change composition duration, or regenerate it to fit. `check` reports clip durations and overrun warnings so users and agents can revise text, timing, or composition deliberately.
142
+ 0.6 produces MP4, MOV, MKV, or WebM files. The default is MP4 with H.264 video and AAC audio; HEVC and VP9 are opt-in, and WebM is VP9 with Opus. HEVC in MP4 and MOV keeps the `hvc1` sample entry and is not playable in browsers without platform HEVC decoding. It exposes only a small set of validated encoding options, not arbitrary FFmpeg arguments.
101
143
 
102
- 0.4 produces portable MP4 files with H.264 video and AAC audio. It exposes only a small set of validated encoding options, not arbitrary FFmpeg arguments. Single-frame PNG output through `still` is a debugging artifact, not general image-sequence support.
144
+ PNG output through `still`, including a bounded set of inspection frames, is a debugging artifact, not general image-sequence support.
103
145
 
104
- A render fails immediately on an uncaught exception, unhandled rejection, required-asset failure, or invalid composition output. Diagnostics identify the composition and, where applicable, the frame. The CLI exits nonzero, cleans temporary files, and never reports a partial video as successful. Existing output files are not overwritten unless explicitly requested.
146
+ A render fails immediately on an uncaught exception, unhandled rejection, required-asset failure, invalid composition output, segment encoding failure, or scratch failure. Diagnostics identify the timeline, clip, segment, and, where applicable, the frame. The CLI exits nonzero, cleans temporary files, and never reports a partial video as successful. Existing output files are not overwritten unless explicitly requested.
105
147
 
106
148
  ## 6. CLI, tooling, and package boundaries
107
149
 
108
150
  Project and composition commands are:
109
151
 
110
- - `init`: scaffold a minimal HTML composition project, including an agent guide (`AGENTS.md`), a `.gitignore` for `out/`, and a private `package.json` with pinned `dev`, `check`, `render`, and `still` scripts
111
- - `dev`: start the preview
112
- - `render <composition-id> [--out video.mp4]`: render a composition, optionally with `--concurrency <n>` from 1 to 16; the default output is `out/<composition-id>.mp4` in the project root
113
- - `check <composition-id>`: run static composition validation, then validate configuration, props, assets, audio, and transcript requirements, load the composition, and evaluate every frame without capturing or encoding
114
- - `still <composition-id> --frame <n>`: capture one requested frame as PNG through the production rendering path
115
- - `doctor`: inspect core requirements and optional integrations without repairing or downloading anything
152
+ - `init`: scaffold a timeline-first project with a composition, a one-clip timeline, a project format, an agent guide, a `.gitignore` for `out/`, and a private `package.json` with pinned `dev`, `check`, `render`, and `still` scripts.
153
+ - `dev`: start the preview with composition and timeline modes.
154
+ - `render <id> [--out video]`: render a timeline or a composition. Timeline renders accept `--workers <n>` and `--scratch <dir>`; composition renders accept `--concurrency <n>` from 1 to 16. `--container <mp4|mov|mkv|webm>` and `--codec <h264|hevc|vp9>` select the delivery pair (MP4/H.264 by default; WebM defaults to VP9), and `--encoder <auto|software|nvenc|qsv>` selects how it is encoded; `auto` is the default and probes NVENC then Quick Sync before falling back to software. The output extension must match the container. The default output is `out/<id>.<ext>`.
155
+ - `check <id>`: run static validation, then validate configuration, props, assets, media, audio, and transcript requirements, load the target, and evaluate every frame without capturing or encoding.
156
+ - `still <id> --frame <spec>`: capture one or more requested frames as PNGs through the clip capture and timeline composite path. `<spec>` is a frame number (`42`), an inclusive range (`2-12`), or a comma-separated list (`0,118-126,240`); duplicates are removed preserving the first occurrence, input order is preserved, and one invocation captures at most 32 unique frames. There is no step or sequence mode. A single frame keeps the existing `--out <file.png>` behavior; multiple frames use `--out <directory>` and write `still-<id>-<frame>.png` files, defaulting to the current working directory. Frames and output conflicts are validated before Chromium, all frames are captured to temporary files before publication, and a failed capture publishes no files. JSON results add `frames` and keep the singular `frame` and `path` only when one frame was captured. See [ADR 0013](./adr/0013-capture-bounded-multi-frame-stills-for-inspection.md).
157
+ - `doctor`: inspect core requirements and optional integrations without repairing or downloading anything.
116
158
 
117
- `check`, `still`, and `render` run a Node-side static composition validation before launching Chromium. It validates metadata, the composition root, the controller declaration, default props, binding attributes and paths, animation attributes, scripts and event handlers, unsupported visual elements, and audio/transcript references. Static validation and the composition host read the composition grammar through one browser-safe composition document reader with parse5 and DOM adapters; the reader returns fixed facts and per-kind declarations, while the validator adds source locations, filesystem checks, and transcript parsing. Controller `src` is project-relative in composition source info and results; the authoring value is retained as `rawSrc` for the host import. Diagnostics carry stable codes and optional source locations; `check` exits nonzero on static errors without starting a browser. `dev` runs the same validation on load and surfaces its diagnostics in the preview.
159
+ `check`, `still`, and `render` run a Node-side static validation before launching Chromium. For a timeline it validates the JSON document, tracks, clips, references, nested timelines, cycles, frame-rate compatibility, overlaps, source bounds, media, transforms, audio settings, props, and every referenced composition. Static validation and the composition host read the composition grammar through one browser-safe document reader. Timeline diagnostics carry stable codes and optional source locations or JSON paths, and composition diagnostics from inside a timeline carry the clip and track context.
118
160
 
119
161
  Local asset-production commands are `tts` and `transcribe`. Every non-interactive command supports `--json` with stable result and error schemas. Structured results go to stdout; human diagnostics and inference progress go to stderr.
120
162
 
121
- The built-in server serves native ES modules and static assets, injects an import map and composition host into composition entries, and serves an inline controller through a virtual module URL under the composition's directory. It performs no dependency pre-bundling. Third-party libraries must initially be browser-ready ES modules referenced explicitly. Running through the CLI server is required; direct `file://` execution is unsupported for deterministic preview or rendering. A composition document can still be opened directly as static HTML/CSS.
163
+ The built-in server serves native ES modules, static assets, composition entries, timeline documents, and the timeline preview host. It injects an import map and composition host into composition entries and an inline controller through a virtual module URL. It performs no dependency pre-bundling. Third-party libraries must initially be browser-ready ES modules referenced explicitly. Running through the CLI server is required; direct `file://` execution is unsupported for deterministic preview or rendering.
122
164
 
123
- Frameyard uses one repository and one publishable npm package with three internal layers:
165
+ Frameyard uses one repository and one publishable npm package with four internal layers:
124
166
 
125
- 1. Shared composition contracts and HTML binding/metadata parsing
126
- 2. Browser host runtime
127
- 3. Node-based CLI and renderer
167
+ 1. Shared composition and timeline contracts and HTML/JSON parsing
168
+ 2. Browser composition host and timeline preview host
169
+ 3. Node-based static validation and segmented timeline renderer
170
+ 4. Node-based CLI, asset production, and doctor
128
171
 
129
- These layers have clean internal boundaries but no public plugin API. FFmpeg is discovered on the user's PATH and checked for required capabilities, with actionable installation and version diagnostics. The project supports the current active Node.js LTS; Linux is the reference and CI platform, while macOS and Windows remain best-effort until covered by real rendering tests.
172
+ These layers have clean internal boundaries but no public plugin API. FFmpeg is required for encoding, probing, media decoding, audio mixing, and timeline compositing. Skia is not required in 0.6. The project supports the current active Node.js LTS; Linux is the reference and CI platform, while macOS and Windows remain best-effort until covered by real rendering tests.
130
173
 
131
174
  ## 7. Official agent skill
132
175
 
133
- Frameyard ships an optional, host-neutral agent skill with a `SKILL.md`, supporting references, and thin scripts. The npm package includes skill files matching its API version, and documentation explains host-specific installation or referencing. 0.4 does not add a skill-installer command.
176
+ Frameyard ships an optional, host-neutral agent skill with a `SKILL.md`, supporting references, and thin scripts. The npm package includes skill files matching its API version, and documentation explains host-specific installation or referencing. 0.6 does not add a skill-installer command.
134
177
 
135
178
  The skill:
136
179
 
137
- - Uses the normal project format, HTML composition contract, optional controller, and CLI available to humans.
180
+ - Uses the normal project format, HTML composition contract, JSON timeline contract, optional controller, and CLI available to humans.
138
181
  - Does not introduce an agent-only runtime, MCP server, hosted control plane, or separate rendering API.
139
182
  - Triggers for Frameyard authoring and operation rather than claiming to be a general-purpose media-generation skill.
140
183
  - Uses Frameyard's audio wrappers instead of invoking `qwen-tts` or `whisper-cli` directly.
141
- - Includes references for deterministic HTML authoring, bindings, animation scheduling, controller rules, static validation, structured diagnostics, common authoring mistakes, audio assets, subtitle and karaoke recipes, JSON command results, and failure recovery.
184
+ - Includes references for deterministic HTML authoring, timeline JSON authoring, tracks, clips, source ranges, props, media, nested timelines, static transforms, audio, validation, diagnostics, common mistakes, subtitle and karaoke recipes, JSON command results, long-form worker/scratch settings, and failure recovery.
142
185
  - Invokes speech synthesis or transcription only when requested or explicitly approved as part of the brief.
143
186
  - Does not overwrite or regenerate existing generated assets without permission.
144
187
  - Runs `check`, inspects representative `still` images when the host supports image viewing, and discloses when visual inspection was impossible.
145
188
  - Does not claim success when checks fail or rendering is partial.
146
189
 
147
- A successful agent workflow preserves and reports the source project, props, generated assets, provenance manifest, validation results, and final video path. The user must be able to inspect, modify, and reproduce the result rather than receiving only an opaque MP4.
190
+ A successful agent workflow preserves and reports the source project, timeline document, props, generated assets, provenance manifest, validation results, and final video path. The user must be able to inspect, modify, and reproduce the result rather than receiving only an opaque MP4.
148
191
 
149
192
  ## 8. Local speech synthesis
150
193
 
151
194
  The `tts` command integrates [`qwentts.cpp`](https://github.com/ServeurpersoCom/qwentts.cpp) through its external `qwen-tts` executable. It does not link the native library or require the upstream server.
152
195
 
153
- 0.4 supports CustomVoice mode only:
196
+ 0.6 supports CustomVoice mode only:
154
197
 
155
198
  - A named speaker is required.
156
199
  - An optional style instruction and language may be supplied.
@@ -176,7 +219,7 @@ The canonical artifact is a versioned Frameyard JSON document containing:
176
219
  - Timed segments
177
220
  - Optional word timings
178
221
 
179
- All timestamps are integer milliseconds relative to the source audio, not composition frame numbers. The controller converts them using the audio clip's start frame, source trim, and composition frame rate.
222
+ All timestamps are integer milliseconds relative to the source audio, not timeline or composition frame numbers. A controller converts them using the audio clip's start frame, source trim, and the composition frame rate.
180
223
 
181
224
  Compositions declare required transcript assets with `<fy-transcript>` elements:
182
225
 
@@ -188,11 +231,11 @@ Compositions declare required transcript assets with `<fy-transcript>` elements:
188
231
 
189
232
  SRT and VTT are derived exports, not the canonical contract. Word-level karaoke requires actual word timings and fails validation when they are unavailable. Frameyard never fabricates word timings by interpolating segment timestamps and never executes Whisper's generated bash/FFmpeg karaoke script.
190
233
 
191
- The official skill documents seek-safe subtitle and karaoke recipes. Compositions retain responsibility for visual presentation; 0.4 has no built-in subtitle renderer and does not automatically burn captions into videos. Karaoke presentation uses the optional controller.
234
+ The official skill documents seek-safe subtitle and karaoke recipes. Compositions retain responsibility for visual presentation; 0.6 has no built-in subtitle renderer and does not automatically burn captions into videos. Karaoke presentation uses the optional controller.
192
235
 
193
236
  ## 10. Generated assets and inference operation
194
237
 
195
- Speech synthesis and transcription are separate asset-production operations that complete before composition loading and rendering. Their outputs are ordinary local project assets. Once those assets are frozen, ordinary authoring, preview, checking, and rendering require neither AI executable nor model.
238
+ Speech synthesis and transcription are separate asset-production operations that complete before a timeline or composition is loaded or rendered. Their outputs are ordinary local project assets. Once those assets are frozen, ordinary authoring, preview, checking, and rendering require neither AI executable nor model.
196
239
 
197
240
  Both integrations are optional and local-only, with no cloud fallback. Network activity needed to acquire tools or models is a separate explicit setup operation, never a hidden side effect of generation or rendering.
198
241
 
@@ -215,19 +258,21 @@ Tests prioritize public behavior and end-to-end evidence over implementation-onl
215
258
 
216
259
  - HTML compositions validate metadata, default props, bindings, conditionals, repeats, and controller declarations, including inline/external parity and the exactly-one-controller rule.
217
260
  - Static composition validation catches metadata, root/controller count, binding, animation, script, element, and asset-reference errors before Chromium.
218
- - Diagnostics carry stable codes and source locations; JSON results expose the composition entry, controller declaration, and top-level diagnostics.
219
- - Sequential, individual, random-order, and parallel frame renders produce equivalent images.
220
- - Determinism checks compare captured pixels or stills; encoded MP4 bytes are not a reproducibility oracle.
221
- - Preview and renderer use the same composition document and host runtime.
222
- - CSS, controller JavaScript, SVG, and Canvas animations seek correctly.
261
+ - Timeline JSON validation catches schema, track, clip, reference, cycle, frame-rate, overlap, bounds, media, transform, audio, and props errors before Chromium.
262
+ - Diagnostics carry stable codes and source locations or JSON paths; JSON results expose the composition or timeline entry, clips, tracks, and top-level diagnostics.
263
+ - Sequential, individual, random-order, and parallel composition renders produce equivalent images; timeline segment workers produce ordered output and identical captured composition frames.
264
+ - Determinism checks compare captured composition pixels or stills. The final encoded bytes and the FFmpeg composite versus Chromium preview are not reproducibility oracles.
265
+ - Preview and renderer use the same composition document and host behavior; timeline preview uses the same active clip hosts.
266
+ - CSS, controller JavaScript, SVG, Canvas, WebGL, embedded video, and audio frames seek correctly.
223
267
  - Frame-scheduled `data-fy-anim-*` animations and frame-state CSS variables produce the expected frames.
224
- - Embedded video and audio frames seek deterministically, including under parallel capture, and their audio appears in the offline mix.
225
- - WebGL canvases produce deterministic stills and rendered frames under the pinned software renderer.
226
- - Executable scripts other than the inline or external controller, inline event handlers, `javascript:` URLs, and unsupported visual elements are rejected.
227
- - Asset and runtime failures produce nonzero CLI exits.
228
- - Rendered MP4 files have the expected dimensions, frame count, duration, and audio stream.
229
- - Golden-frame tests cover representative visual behavior.
230
- - `check` exercises every frame, and `still` follows the production capture path.
268
+ - Timeline capture preserves alpha for transparent compositions; FFV1 intermediates decode pixel-identically to the captured frames.
269
+ - Timeline segment encoding, ordered concatenation, global audio mix, and scratch cleanup produce the expected output dimensions, frame count, duration, and audio stream.
270
+ - Nested timelines preserve group opacity and transform semantics; cycles fail static validation.
271
+ - Media clips trim and fit deterministically; images require durations; source overruns fail.
272
+ - Props overrides for timeline composition clips merge over clip props and are reflected in the render.
273
+ - `check` exercises every frame; `still` follows the clip capture and timeline composite path.
274
+ - The 10-second and 60-second long-form benchmarks record capture, FFV1, composite, encode, disk, worker, and audio numbers. Continuous three-layer runs measured 52.4 s and 58.5 s for 60 s of timeline and are documented as worst-case (~1× real time). The representative 50/50, 60/30/10 occupancy gate measured 29.39 s for 60 s with four workers and a 1080p media fast path, extrapolating to about 29.4 minutes for 60 minutes with exact 1800-frame output. The full 60-minute composition-only gate rendered 108,000 frames in 35.35 minutes with exact output and clean scratch, documenting composition-heavy performance above 30 minutes.
275
+ - Golden-frame tests cover representative composition and timeline visual behavior.
231
276
  - Normal CI uses fake `qwen-tts` and `whisper-cli` executables and small fixtures to verify arguments, artifacts, manifests, cancellation, and errors without model downloads or GPU hardware.
232
277
  - Real-model integration tests are explicitly opt-in and use documented upstream revisions.
233
278
  - Agent-workflow evaluations verify source preservation, provenance, validation, still inspection behavior, failure reporting, and final renders.
@@ -236,7 +281,14 @@ Tests prioritize public behavior and end-to-end evidence over implementation-onl
236
281
 
237
282
  These are scope boundaries, not permanent rejections:
238
283
 
239
- - Visual editing, hosted rendering, collaboration, and asset marketplaces
284
+ - Transitions, effects, keyframes, blend modes, masks, and audio fades
285
+ - Full visual timeline editing, drag-and-drop, clip handles, and undo/redo
286
+ - Timeline-level variables and prop overrides for compositions inside nested timelines
287
+ - Track solo and video-track visibility
288
+ - Per-timeline formats and frame-rate conversion/retiming
289
+ - Persistent clip caches; the segmented renderer cleans its intermediates
290
+ - GPU compositing and codecs beyond H.264, HEVC, and VP9
291
+ - Hosted rendering, collaboration, and asset marketplaces
240
292
  - Cloud generative-AI services, text-to-video generation, and image generation
241
293
  - `qwentts.cpp` reference-audio voice cloning and VoiceDesign
242
294
  - Automatic long-text chunking, streaming speech synthesis, and alternative generated-audio formats
@@ -248,17 +300,16 @@ These are scope boundaries, not permanent rejections:
248
300
  - Agent-only runtimes, MCP servers, hosted agent control planes, and host-specific skill installers
249
301
  - React or other UI-framework adapters in the core
250
302
  - Public plugin APIs and a broad integration ecosystem
251
- - Dynamic composition metadata derived from props
303
+ - Dynamic composition or timeline metadata derived from props
252
304
  - Live streams, remote media, and other visual sources that cannot be deterministically seeked
253
305
  - Dynamic per-frame asset loading or AI inference
254
- - Frame caches
255
- - Batch still capture; agents can run `still` once per representative frame
306
+ - Unbounded batch still capture; `still` accepts at most 32 inspection frames per invocation and is not an image-sequence exporter
256
307
  - Additional final-video formats, general image-sequence output, and arbitrary FFmpeg argument passthrough
257
308
  - Built-in TypeScript transpilation and npm dependency optimization
258
309
  - Rendering untrusted compositions
259
310
  - Bit-identical cross-platform encoded output or AI regeneration
260
- - A minimum production rendering-speed guarantee
311
+ - A minimum production rendering-speed guarantee for compositions or timelines
261
312
 
262
313
  ## 13. Release posture
263
314
 
264
- Frameyard is MIT-licensed. Initial releases use `0.x` versioning, and composition, configuration, CLI, JSON-result, transcript, and skill-contract changes are documented explicitly until those contracts stabilize. 0.4.0 is a breaking authoring-model release: HTML documents replace JavaScript composition modules, and the `fy` namespace replaces `data-frameyard-*` authoring attributes. See [ADR 0007](./adr/0007-make-html-documents-the-composition-unit.md) and the [0.4 migration guide](./migration-0.4.md).
315
+ Frameyard is MIT-licensed. Initial releases use `0.x` versioning, and composition, timeline, configuration, CLI, JSON-result, transcript, and skill-contract changes are documented explicitly until those contracts stabilize. 0.6.0 adds timelines, project formats, segmented timeline rendering, and a timeline-first scaffold without breaking the existing HTML composition authoring contract. See [ADR 0010](./adr/0010-make-timelines-the-render-unit.md), [ADR 0011](./adr/0011-compose-timeline-frames-in-a-live-browser-host.md), and [ADR 0012](./adr/0012-render-timelines-with-segmented-lossless-clip-intermediates.md). 0.7.0 adds delivery container, codec, and encoder selection, bounded multi-frame still inspection, and optional encoder probes in `doctor` without breaking the timeline or composition contracts; bounded still inspection is recorded in [ADR 0013](./adr/0013-capture-bounded-multi-frame-stills-for-inspection.md).