frameyard 0.5.0 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (96) hide show
  1. package/CHANGELOG.md +55 -0
  2. package/GLOSSARY.md +152 -24
  3. package/README.md +94 -29
  4. package/dist/cli.js +111 -14
  5. package/dist/cli.js.map +1 -1
  6. package/dist/contracts.d.ts +12 -0
  7. package/dist/contracts.d.ts.map +1 -1
  8. package/dist/contracts.js.map +1 -1
  9. package/dist/errors.d.ts +1 -1
  10. package/dist/errors.d.ts.map +1 -1
  11. package/dist/errors.js.map +1 -1
  12. package/dist/index.d.ts +2 -0
  13. package/dist/index.d.ts.map +1 -1
  14. package/dist/index.js +1 -0
  15. package/dist/index.js.map +1 -1
  16. package/dist/preview/main.js +89 -11
  17. package/dist/preview/main.js.map +1 -1
  18. package/dist/preview/timeline-data.d.ts +85 -0
  19. package/dist/preview/timeline-data.d.ts.map +1 -0
  20. package/dist/preview/timeline-data.js +2 -0
  21. package/dist/preview/timeline-data.js.map +1 -0
  22. package/dist/preview/timeline-host.d.ts +19 -0
  23. package/dist/preview/timeline-host.d.ts.map +1 -0
  24. package/dist/preview/timeline-host.js +297 -0
  25. package/dist/preview/timeline-host.js.map +1 -0
  26. package/dist/preview/timeline-main.d.ts +17 -0
  27. package/dist/preview/timeline-main.d.ts.map +1 -0
  28. package/dist/preview/timeline-main.js +364 -0
  29. package/dist/preview/timeline-main.js.map +1 -0
  30. package/dist/project.d.ts +16 -1
  31. package/dist/project.d.ts.map +1 -1
  32. package/dist/project.js +81 -26
  33. package/dist/project.js.map +1 -1
  34. package/dist/renderer/browser.d.ts +2 -0
  35. package/dist/renderer/browser.d.ts.map +1 -1
  36. package/dist/renderer/browser.js +6 -1
  37. package/dist/renderer/browser.js.map +1 -1
  38. package/dist/renderer/ffmpeg.d.ts +61 -0
  39. package/dist/renderer/ffmpeg.d.ts.map +1 -1
  40. package/dist/renderer/ffmpeg.js +198 -0
  41. package/dist/renderer/ffmpeg.js.map +1 -1
  42. package/dist/renderer/timeline.d.ts +109 -0
  43. package/dist/renderer/timeline.d.ts.map +1 -0
  44. package/dist/renderer/timeline.js +984 -0
  45. package/dist/renderer/timeline.js.map +1 -0
  46. package/dist/scaffold.d.ts.map +1 -1
  47. package/dist/scaffold.js +89 -24
  48. package/dist/scaffold.js.map +1 -1
  49. package/dist/server/preview-html.d.ts.map +1 -1
  50. package/dist/server/preview-html.js +15 -1
  51. package/dist/server/preview-html.js.map +1 -1
  52. package/dist/server/server.d.ts.map +1 -1
  53. package/dist/server/server.js +39 -1
  54. package/dist/server/server.js.map +1 -1
  55. package/dist/server/timeline-preview-data.d.ts +4 -0
  56. package/dist/server/timeline-preview-data.d.ts.map +1 -0
  57. package/dist/server/timeline-preview-data.js +97 -0
  58. package/dist/server/timeline-preview-data.js.map +1 -0
  59. package/dist/server/timeline-preview-html.d.ts +4 -0
  60. package/dist/server/timeline-preview-html.d.ts.map +1 -0
  61. package/dist/server/timeline-preview-html.js +227 -0
  62. package/dist/server/timeline-preview-html.js.map +1 -0
  63. package/dist/timeline/plan.d.ts +32 -0
  64. package/dist/timeline/plan.d.ts.map +1 -0
  65. package/dist/timeline/plan.js +68 -0
  66. package/dist/timeline/plan.js.map +1 -0
  67. package/dist/timeline/resolve.d.ts +60 -0
  68. package/dist/timeline/resolve.d.ts.map +1 -0
  69. package/dist/timeline/resolve.js +523 -0
  70. package/dist/timeline/resolve.js.map +1 -0
  71. package/dist/timeline/schema.d.ts +71 -0
  72. package/dist/timeline/schema.d.ts.map +1 -0
  73. package/dist/timeline/schema.js +425 -0
  74. package/dist/timeline/schema.js.map +1 -0
  75. package/dist/version.d.ts +1 -1
  76. package/dist/version.js +1 -1
  77. package/docs/adr/0002-make-frame-and-props-the-only-animation-state.md +2 -0
  78. package/docs/adr/0005-keep-parallel-rendering-opt-in-and-frame-ordered.md +2 -0
  79. package/docs/adr/0007-make-html-documents-the-composition-unit.md +1 -1
  80. package/docs/adr/0009-composite-production-frames-in-software.md +1 -1
  81. package/docs/adr/0010-make-timelines-the-render-unit.md +24 -0
  82. package/docs/adr/0011-compose-timeline-frames-in-a-live-browser-host.md +22 -0
  83. package/docs/adr/0012-render-timelines-with-segmented-lossless-clip-intermediates.md +31 -0
  84. package/docs/v0.1-design.md +135 -86
  85. package/examples/demo/README.md +10 -3
  86. package/examples/demo/timelines/demo.timeline.json +12 -0
  87. package/examples/demo/video.config.js +4 -0
  88. package/examples/video/README.md +50 -6
  89. package/examples/video/compositions/video-overlay.html +350 -0
  90. package/examples/video/props/video-timeline.json +10 -0
  91. package/examples/video/timelines/video-timeline-mp4.timeline.json +44 -0
  92. package/examples/video/timelines/video-timeline.timeline.json +44 -0
  93. package/examples/video/video.config.js +6 -0
  94. package/package.json +1 -1
  95. package/skills/frameyard/SKILL.md +17 -13
  96. package/skills/frameyard/references/timelines.md +141 -0
package/CHANGELOG.md CHANGED
@@ -8,6 +8,61 @@ Frameyard uses `0.x` versions until the composition, configuration, CLI,
8
8
  JSON-result, transcript, and skill contracts stabilize. Before `1.0.0`, a
9
9
  contract change may ship in a minor release and is called out explicitly below.
10
10
 
11
+ ## [0.6.0] - 2026-11-01
12
+
13
+ ### Added
14
+
15
+ - JSON timelines with project-fixed format, video/audio tracks, composition,
16
+ media, and nested-timeline clips, source ranges, per-clip props, static
17
+ transforms, audio settings, and derived or overridden duration.
18
+ - Timeline validation: strict schema, references, composition static
19
+ validation, media probing, frame-rate checks, source bounds, overlaps,
20
+ nested cycles, and structured diagnostics.
21
+ - Segmented timeline rendering: Chromium captures composition and
22
+ nested-timeline clips to alpha-preserving FFV1 intermediates; FFmpeg
23
+ composites and encodes segments; segments are concatenated in order; audio
24
+ is mixed once and muxed.
25
+ - Transparent composition capture and nested timeline groups with recursive
26
+ visual, still, check, and audio handling.
27
+ - Timeline preview in `dev`: composition and timeline selectors, program
28
+ monitor, read-only tracks and clips, playhead, playback, frame stepping, and
29
+ one-second jumps with buttons or arrow keys, audio controls, and diagnostics.
30
+ - Timeline visual transforms accept `width`/`height` as an explicit clip box,
31
+ mutually exclusive with `scale`.
32
+ - `render`, `check`, and `still` accept timeline IDs alongside composition
33
+ IDs. Timeline renders add `--workers` and `--scratch`; timeline `--props`
34
+ is a map keyed by composition clip ID.
35
+ - `video.config.js` supports an optional project `format` and explicit
36
+ `timelines` map. `project.json` publishes `format` and `timelines`.
37
+ - Timeline-first `init` scaffold with a composition clip, timeline override
38
+ props, and `main` commands.
39
+ - The video example adds WebM and MP4 timelines that place video clips under a
40
+ transparent HTML overlay.
41
+ - Official skill reference for timelines, clips, tracks, nested timelines,
42
+ props overrides, commands, preview, and determinism.
43
+ - Timeline config, schema, resolution, planning, rendering, preview, and
44
+ runtime preview tests.
45
+
46
+ ### Changed
47
+
48
+ - `render` defaults to `out/<id>.mp4` for timelines and compositions.
49
+ - `check` and `still` use timeline clip capture and composite behavior for
50
+ timeline targets.
51
+ - Timeline audio mixing pads short inputs to the full timeline duration.
52
+ - Timed timeline layers are normalized to the project frame rate before
53
+ compositing.
54
+ - The design baseline, glossary, and architecture decision records describe
55
+ timelines as the render unit and compositions as clip sources.
56
+
57
+ ### Fixed
58
+
59
+ - Composition audio, embedded video audio, and nested timeline audio are
60
+ trimmed and shifted by clip source ranges and groups.
61
+ - Selecting a different composition in the preview picker reloads the iframe
62
+ instead of leaving the previous composition loaded.
63
+
64
+ [0.6.0]: https://github.com/vladiantio/frameyard/compare/v0.5.0...v0.6.0
65
+
11
66
  ## [0.5.0] - 2026-10-31
12
67
 
13
68
  ### Added
package/GLOSSARY.md CHANGED
@@ -1,21 +1,25 @@
1
1
  # Frameyard
2
2
 
3
- Frameyard is a local-first framework for creating short, parameterized videos with HTML, CSS, and JavaScript. Its domain separates reusable compositions from the fixed inputs and deterministic renders that turn them into videos.
3
+ Frameyard is a local-first framework for creating short, parameterized videos with HTML, CSS, and JavaScript. Its domain separates reusable compositions from the timelines that assemble them into a video, and from the fixed inputs and deterministic renders that turn those timelines into output.
4
4
 
5
5
  ## Language
6
6
 
7
7
  ### Authoring
8
8
 
9
9
  **Project**:
10
- A collection of related compositions and the assets and configuration they share.
10
+ A collection of related compositions, timelines, and the assets and configuration they share.
11
11
  _Avoid_: Video, render
12
12
 
13
+ **Project format**:
14
+ The fixed width, height, and frame rate shared by every timeline in a project.
15
+ _Avoid_: Composition metadata, render options
16
+
13
17
  **Composition**:
14
- A uniquely identified, reusable video definition with fixed dimensions, frame rate, duration in frames, and content parameterized by props.
18
+ A uniquely identified, reusable video source with fixed dimensions, frame rate, duration in frames, and content parameterized by props.
15
19
  _Avoid_: Scene, template, video
16
20
 
17
21
  **Composition metadata**:
18
- The fixed dimensions, frame rate, and duration in frames that define a composition's output contract.
22
+ The fixed dimensions, frame rate, and duration in frames that define a composition's source contract.
19
23
  _Avoid_: Render options, props
20
24
 
21
25
  **Composition descriptor**:
@@ -38,6 +42,10 @@ _Avoid_: Build, release
38
42
  A composition whose source is an HTML document.
39
43
  _Avoid_: HTML template, page
40
44
 
45
+ **Transparent composition**:
46
+ A composition that does not paint a background, so a timeline clip can reveal the layers below it.
47
+ _Avoid_: Alpha composition, overlay
48
+
41
49
  **Composition controller**:
42
50
  An optional JavaScript controller, either inline in the composition document or in an external module, that performs per-frame computation an HTML composition's declarative HTML and CSS cannot express.
43
51
  _Avoid_: Composition module, plugin
@@ -55,25 +63,113 @@ An HTML attribute that maps a prop into text, an attribute, a class, or element
55
63
  _Avoid_: Template expression, interpolation
56
64
 
57
65
  **Props**:
58
- The immutable, JSON-serializable inputs supplied to a composition for one render.
66
+ The immutable, JSON-serializable inputs supplied to a composition for one load; a timeline clip can fix the props for its instance of the composition.
59
67
  _Avoid_: Options, settings, mutable state
60
68
 
61
69
  **Asset**:
62
- A local file required by a composition, whether static or generated.
70
+ A local file required by a composition or timeline, whether static or generated.
63
71
  _Avoid_: Resource, media file
64
72
 
73
+ **Timeline**:
74
+ A top-level edit that assembles tracks and clips into one video under the project format, with a duration derived from its clips or explicitly overridden.
75
+ _Avoid_: Sequence, composition, project
76
+
77
+ **Timeline document**:
78
+ The project-local JSON document that declares a timeline's tracks, clips, optional duration, and background.
79
+ _Avoid_: Composition, config
80
+
81
+ **Track**:
82
+ An ordered layer in a timeline. Video tracks composite; audio tracks mix.
83
+ _Avoid_: Channel, lane
84
+
85
+ **Video track**:
86
+ A track whose clips contribute visual layers, ordered from bottom to top.
87
+ _Avoid_: Visual lane
88
+
89
+ **Audio track**:
90
+ A track whose clips are audio-only and contribute additively to the timeline audio mix.
91
+ _Avoid_: Soundtrack
92
+
93
+ **Clip**:
94
+ A placement of a clip source on a track, with a timeline start, source in-point, duration, and presentation settings.
95
+ _Avoid_: Segment, item, block
96
+
97
+ **Clip source**:
98
+ The composition, nested timeline, or local media asset that a clip presents.
99
+ _Avoid_: Media, asset
100
+
101
+ **Composition clip**:
102
+ A clip whose source is a composition, with fixed props for that instance.
103
+ _Avoid_: Composition instance, nested composition
104
+
105
+ **Media clip**:
106
+ A clip whose source is a local video, image, or audio asset.
107
+ _Avoid_: Asset clip, raw clip
108
+
109
+ **Nested timeline clip**:
110
+ A clip whose source is another timeline.
111
+ _Avoid_: Nested sequence, compound clip
112
+
113
+ **Source in-point**:
114
+ The first source frame a clip uses.
115
+ _Avoid_: Trim start, offset
116
+
117
+ **Timeline frame**:
118
+ An integer-addressed position on a timeline's master frame grid.
119
+ _Avoid_: Composition frame, timestamp
120
+
121
+ **Composition frame**:
122
+ An integer-addressed position within a composition; the frame a controller receives.
123
+ _Avoid_: Timeline frame
124
+
125
+ **Timeline duration**:
126
+ The number of timeline frames a timeline produces, derived from its clips or explicitly overridden.
127
+ _Avoid_: Composition duration, length
128
+
129
+ **Timeline background**:
130
+ The optional color behind all layers of a timeline; it is transparent by default and the final video composites it over black.
131
+ _Avoid_: Canvas, backdrop
132
+
133
+ **Fit policy**:
134
+ How a clip source's natural frame is fitted into the project frame or an explicit clip box: contained, covering, or stretched.
135
+ _Avoid_: Resize mode, object-fit
136
+
137
+ **Clip transform**:
138
+ The static size, position, rotation, and opacity applied to a visual clip. Size is either an explicit `width`/`height` box or a `scale` multiplier.
139
+ _Avoid_: Effect, animation
140
+
141
+ **Nested timeline**:
142
+ A timeline used as a clip source inside another timeline.
143
+ _Avoid_: Compound composition
144
+
65
145
  ### Deterministic execution
66
146
 
67
147
  **Frame**:
68
- An integer-addressed position on a composition's timeline.
148
+ An integer-addressed position on a timeline or composition frame range; use timeline frame or composition frame when the distinction matters.
69
149
  _Avoid_: Timestamp, animation tick
70
150
 
151
+ **Active clip**:
152
+ A clip whose timeline range contains the current timeline frame.
153
+ _Avoid_: Visible clip
154
+
155
+ **Clip capture**:
156
+ The pixel frames a composition produces for a timeline clip before the timeline compositor combines layers.
157
+ _Avoid_: Screenshot, render
158
+
159
+ **Timeline composite**:
160
+ The deterministic combination of a timeline's layers into output frames.
161
+ _Avoid_: Browser composite, preview
162
+
163
+ **Render segment**:
164
+ A bounded span of a timeline that is rendered and encoded independently, then joined in timeline order.
165
+ _Avoid_: Chunk, part
166
+
71
167
  **Deterministic execution**:
72
- The rule that a composition's visual state at any frame depends only on that frame, its props, and pre-loaded data and assets.
168
+ The rule that a timeline frame's visual state depends only on that timeline frame, its clip sources and their props, and pre-loaded data and assets; a composition frame depends only on its composition frame, props, and pre-loaded data and assets.
73
169
  _Avoid_: Playback-driven animation
74
170
 
75
171
  **Frame clock**:
76
- The single source of animation time, derived from the current frame and the composition's frame rate.
172
+ The single source of animation time within a composition, derived from its composition frame and frame rate; the preview host or the production segment renderer drives it by seeking each active composition clip.
77
173
  _Avoid_: Wall clock, playback timer
78
174
 
79
175
  **Frame-state variable**:
@@ -81,7 +177,7 @@ A CSS custom property whose value is derived from the current composition frame.
81
177
  _Avoid_: Animation variable, clock variable
82
178
 
83
179
  **Seek safety**:
84
- The property that any frame can be produced correctly without depending on previously produced frames.
180
+ The property that any timeline frame can be produced correctly without depending on previously produced timeline frames, by seeking each active clip to its source frame.
85
181
  _Avoid_: History-dependent updates
86
182
 
87
183
  **Deterministic randomness**:
@@ -89,15 +185,15 @@ Random values derived statelessly from a key so they do not depend on call order
89
185
  _Avoid_: Unseeded randomness
90
186
 
91
187
  **Frame production**:
92
- The phase in which the runtime produces the visual state of a requested frame, without network access or model inference.
188
+ The phase in which the runtime produces the visual state of a requested timeline frame, without network access or model inference.
93
189
  _Avoid_: Real-time playback
94
190
 
95
191
  **Loading phase**:
96
- The phase before frame production when required data and assets become available.
192
+ The phase before frame production when all clip sources, compositions, media, data, and assets become available.
97
193
  _Avoid_: Per-frame fetching
98
194
 
99
195
  **Embedded media**:
100
- A local video file placed in a composition whose presentation time is derived from the composition frame, with optional frame offset and source trim.
196
+ A local video file placed inside a composition whose presentation time is derived from the composition frame, with optional frame offset and source trim.
101
197
  _Avoid_: Live stream, video track
102
198
 
103
199
  **Media settle**:
@@ -110,42 +206,74 @@ _Avoid_: GPU animation, live render
110
206
 
111
207
  ### Audio
112
208
 
113
- **Audio clip**:
114
- An audio source placed on a composition's timeline with a start frame, source trim, and volume.
115
- _Avoid_: Audio track
209
+ **Composition audio clip**:
210
+ An audio source declared inside a composition and placed by composition frames; it joins the timeline mix when the composition is used as a clip.
211
+ _Avoid_: Audio clip, audio track
212
+
213
+ **Timeline audio clip**:
214
+ An audio-only clip placed on a timeline audio track.
215
+ _Avoid_: Audio clip, soundtrack
116
216
 
117
217
  **Audio mix**:
118
- The single offline mix of a composition's audio clips for one render.
218
+ The single offline mix of a timeline's audible clips for one render.
119
219
  _Avoid_: Soundtrack, audio track, final cut
120
220
 
221
+ **Embedded audio**:
222
+ The audio track of an embedded video, contributed to the mix unless the video or clip is muted.
223
+ _Avoid_: Live audio
224
+
121
225
  ### Rendering
122
226
 
123
227
  **Composition host**:
124
228
  The injected runtime that loads an HTML composition, applies its bindings, and exposes its frame clock to preview and render.
125
229
  _Avoid_: Preview runtime, render runtime
126
230
 
231
+ **Timeline host**:
232
+ The preview runtime that loads a timeline, drives a bounded pool of active clip sources, and exposes the timeline frame clock to the preview.
233
+ _Avoid_: Timeline runtime, production renderer, editor
234
+
235
+ **Timeline renderer**:
236
+ The Node-side segmented renderer that captures composition clips, composites and encodes timeline segments, and joins them into one video.
237
+ _Avoid_: Timeline host, browser renderer
238
+
127
239
  **Preview**:
128
- An interactive, frame-accurate inspection of a composition using the same composition behavior as a render.
240
+ An interactive, frame-accurate inspection of a timeline or composition using the same behavior as a render, including a program monitor and a read-only timeline view.
129
241
  _Avoid_: Visual editor
130
242
 
243
+ **Playhead**:
244
+ The current timeline frame in preview.
245
+ _Avoid_: Cursor, scrubber
246
+
247
+ **Program monitor**:
248
+ The preview surface that shows the timeline at the playhead.
249
+ _Avoid_: Stage, canvas
250
+
131
251
  **Check**:
132
- A validation operation that first runs static composition validation, then loads a composition and evaluates every frame without capturing or encoding.
252
+ A validation operation that first runs static validation, then loads a timeline or composition and evaluates every frame without capturing or encoding.
133
253
  _Avoid_: Test render, dry run
134
254
 
135
255
  **Static composition validation**:
136
256
  The Node-side validation of a composition document's metadata, bindings, controller declaration, scripts, and asset references before browser frame production.
137
257
  _Avoid_: Lint, preflight
138
258
 
259
+ **Static timeline validation**:
260
+ The Node-side validation of a timeline document's tracks, clips, references, nested timelines, media, and referenced compositions before browser frame production.
261
+ _Avoid_: Lint, preflight
262
+
139
263
  **Composition diagnostic**:
140
264
  A structured error or warning about a composition's source, metadata, bindings, controller, assets, or frame production, with a stable code and optional source location.
141
265
  _Avoid_: Log message, error string
142
266
 
267
+ **Timeline diagnostic**:
268
+ A structured error or warning about a timeline's source, tracks, clips, references, media, or frame production, with a stable code and optional source location.
269
+ _Avoid_: Log message, error string
270
+
143
271
  **Render**:
144
- The production of one complete video from a composition and a fixed set of props.
272
+ The production of one complete video from a timeline or a standalone composition with fixed inputs.
145
273
  _Avoid_: Recording, export
146
274
 
147
275
  **Render concurrency**:
148
- The optional number of independent browser sessions used to seek and capture frames during a render. Frames are still encoded in timeline order, and concurrency must not change visual state.
276
+ The optional number of independent browser sessions used to seek and capture frames during a composition render; timeline rendering is sequential in 0.6. Frames are still encoded in timeline order, and concurrency must not change visual state.
149
277
  _Avoid_: Parallel composition, out-of-order output
150
278
 
151
279
  **Video**:
@@ -153,11 +281,11 @@ The finished audiovisual output of a successful render.
153
281
  _Avoid_: Composition, project
154
282
 
155
283
  **Still**:
156
- A single frame captured as a PNG through the production rendering path for debugging.
284
+ A single frame captured as a PNG through the production rendering path for a timeline or composition.
157
285
  _Avoid_: Frame export, image sequence
158
286
 
159
287
  **Reproducibility**:
160
- The guarantee that, for the same composition revision, props, assets, and frame under a pinned runtime/browser environment, the visual state is identical. It does not imply identical encoded bytes, even on the same machine; encoded output depends on the encoder and its runtime.
288
+ The guarantee that, for the same timeline or composition revision, props, assets, and frame under a pinned runtime/browser environment, the visual state is identical. It does not imply identical encoded bytes, even on the same machine; encoded output depends on the encoder and its runtime.
161
289
  _Avoid_: Bit-for-bit encoding equality
162
290
 
163
291
  ### Agent operation
@@ -173,7 +301,7 @@ _Avoid_: Agent runtime, MCP server
173
301
  ### Local asset production
174
302
 
175
303
  **Generated asset**:
176
- A frozen local input produced by speech synthesis or transcription before a composition is loaded or rendered.
304
+ A frozen local input produced by speech synthesis or transcription before a timeline or composition is loaded or rendered.
177
305
  _Avoid_: Per-frame generation
178
306
 
179
307
  **Provenance manifest**:
package/README.md CHANGED
@@ -1,8 +1,8 @@
1
1
  # Frameyard
2
2
 
3
- Frameyard is a local-first framework for building short, parameterized videos with HTML, CSS, and an optional JavaScript controller, and rendering them deterministically.
3
+ Frameyard is a local-first framework for building parameterized videos with HTML, CSS, and an optional JavaScript controller, assembling them into timelines, and rendering them deterministically.
4
4
 
5
- The 0.4 design is implemented: HTML composition documents, declarative bindings, CSS animation scheduling, frame-state variables, inline and external controllers, the composition host, static validation with structured diagnostics, iframe preview, direct render/check/still sessions, audio and transcripts, deterministic Chromium capture, FFmpeg output, and the official agent skill.
5
+ The 0.6 design is implemented: HTML composition documents, JSON timelines with tracks and clips, nested timelines, declarative bindings, CSS animation scheduling, frame-state variables, inline and external controllers, composition and timeline hosts, static validation with structured diagnostics, an interactive preview with a read-only timeline, direct composition rendering, segmented lossless timeline rendering, audio and transcripts, deterministic Chromium capture, FFmpeg output, and the official agent skill.
6
6
 
7
7
  ## Requirements
8
8
 
@@ -26,7 +26,7 @@ cd my-video
26
26
  npx frameyard dev
27
27
  ```
28
28
 
29
- `dev` prints a local preview URL. Select a composition, edit its JSON props, scrub frames, and play the composition with its audio timeline.
29
+ `dev` prints a local preview URL. Preview a composition with JSON props editing, or open the timeline for a program monitor, read-only tracks and clips, scrubbing, playback, frame-by-frame stepping, one-second jumps, and diagnostics.
30
30
 
31
31
  The scaffold includes an `AGENTS.md` guide for coding agents working in the project, covering the HTML authoring rules and the `check`, `still`, and `render` workflow. It also includes a `package.json` with `dev`, `check`, `render`, and `still` scripts pinned to the scaffolding CLI version, so `pnpm dev` or `npm run dev` works without a global install, and a `.gitignore` that ignores `out/`.
32
32
 
@@ -36,24 +36,28 @@ Validate and render:
36
36
 
37
37
  ```sh
38
38
  npx frameyard doctor
39
- npx frameyard check hello
40
- npx frameyard render hello --props props/hello.json
41
- npx frameyard still hello --frame 42 --out hello-frame.png
39
+ npx frameyard check main
40
+ npx frameyard render main --props props/main.json --workers 8 --scratch /path/to/scratch
41
+ npx frameyard still main --frame 42 --out main-frame.png
42
42
  ```
43
43
 
44
- `render` writes the video to `out/hello.mp4` in the project by default; pass `--out <file.mp4>` to choose another path. The scaffold's `.gitignore` ignores `out/`.
44
+ The scaffold wraps its composition in `timelines/main.timeline.json`. `render` writes the video to `out/main.mp4` in the project by default; pass `--out <file.mp4>` to choose another path. The composition remains renderable on its own with `npx frameyard render hello --props props/hello.json`. The scaffold's `.gitignore` ignores `out/`.
45
45
 
46
- The 30-second reference composition lives in [`examples/demo`](./examples/demo), a focused WebGL shader example lives in [`examples/webgl`](./examples/webgl), and a two-pass embedded-video example with WebM and MP4 sources lives in [`examples/video`](./examples/video). Each README has the commands for running it from a repository checkout.
46
+ The 30-second reference composition lives in [`examples/demo`](./examples/demo), a focused WebGL shader example lives in [`examples/webgl`](./examples/webgl), and the video example in [`examples/video`](./examples/video) includes two embedded-video compositions plus WebM and MP4 timelines that place the clips on tracks under a transparent HTML overlay. Each README has the commands for running it from a repository checkout.
47
47
 
48
48
  ## Project format
49
49
 
50
- A project is ordinary HTML, CSS, assets, and optional native JavaScript controller modules. `video.config.js` explicitly lists composition entry points; there is no directory scanning:
50
+ A project is ordinary HTML, CSS, JSON timelines, assets, and optional native JavaScript controller modules. `video.config.js` explicitly lists the project format, composition entry points, and timeline entry points; there is no directory scanning:
51
51
 
52
52
  ```js
53
53
  export default {
54
+ format: { width: 1920, height: 1080, fps: 30 },
54
55
  compositions: {
55
56
  hello: "./compositions/hello.html",
56
57
  },
58
+ timelines: {
59
+ main: "./timelines/main.timeline.json",
60
+ },
57
61
  };
58
62
  ```
59
63
 
@@ -89,6 +93,54 @@ A composition is a full HTML document with one `<fy-composition>` root. Metadata
89
93
 
90
94
  `width`, `height`, `fps`, and `duration-in-frames` are fixed. Props are JSON-serializable, deep-frozen, and merged shallowly over the `data-fy="props"` defaults. They cannot change metadata.
91
95
 
96
+ ### Timeline
97
+
98
+ A timeline is a JSON document that places composition and media clips on video and audio tracks. It inherits the project format and derives its duration from the latest clip end unless it declares `durationInFrames`:
99
+
100
+ ```json
101
+ {
102
+ "version": 1,
103
+ "tracks": [
104
+ { "id": "v1", "kind": "video" },
105
+ { "id": "v2", "kind": "video" },
106
+ { "id": "a1", "kind": "audio" }
107
+ ],
108
+ "clips": [
109
+ {
110
+ "id": "hello",
111
+ "track": "v1",
112
+ "source": { "kind": "composition", "id": "hello" },
113
+ "start": 0,
114
+ "duration": 150
115
+ },
116
+ {
117
+ "id": "logo",
118
+ "track": "v2",
119
+ "source": { "kind": "image", "src": "assets/logo.png" },
120
+ "start": 30,
121
+ "duration": 60,
122
+ "transform": { "fit": "contain", "width": 480, "height": 270, "x": 640, "y": -320, "rotation": 0, "opacity": 0.9 }
123
+ },
124
+ {
125
+ "id": "music",
126
+ "track": "a1",
127
+ "source": { "kind": "audio", "src": "assets/music.wav" },
128
+ "start": 0,
129
+ "duration": 150,
130
+ "audio": { "gain": 0.5 }
131
+ }
132
+ ]
133
+ }
134
+ ```
135
+
136
+ - Sources are `composition` + `id`, `timeline` + `id`, or `video`/`image`/`audio` + project-relative `src`.
137
+ - `start`, `sourceIn`, and `duration` use timeline frames. Omitted durations use the rest of the source; image clips require a duration.
138
+ - Composition clips can fix `props`, trim the source range, and repeat the same composition with different settings.
139
+ - Visual clips support static `fit`, `width`/`height` (an explicit box), `x`, `y`, `scale`, `rotation`, and `opacity`. `fit` applies inside the box; `width` and `height` go together and are mutually exclusive with `scale`. No keyframes, transitions, or effects in 0.6.
140
+ - Video, composition, and nested-timeline clips contribute audio automatically unless muted. Timeline audio clips live on audio tracks.
141
+ - A composition used in a timeline must match the project fps; different sizes are fitted by the clip. Clips on one track cannot overlap.
142
+ - Nested timelines are rendered as groups, so the parent clip's opacity and transform apply to the whole nested result.
143
+
92
144
  ### Bindings
93
145
 
94
146
  - `data-fy-text="path"` sets text content.
@@ -216,16 +268,16 @@ Transcript paths resolve from the project root. `words` requires actual measured
216
268
 
217
269
  | Command | Purpose |
218
270
  | --- | --- |
219
- | `frameyard init [directory]` | Scaffold a minimal HTML project, including an `AGENTS.md` agent guide and pinned `package.json` scripts |
220
- | `frameyard dev [--port <port>]` | Start the preview server |
221
- | `frameyard check <id> [--props <file>]` | Run static validation, then evaluate every frame without capture |
222
- | `frameyard still <id> --frame <n>` | Capture one PNG through the production path |
223
- | `frameyard render <id> [--out <video.mp4>]` | Capture frames with Chromium and encode with FFmpeg, optionally in parallel; defaults to `out/<id>.mp4` in the project |
271
+ | `frameyard init [directory]` | Scaffold a timeline-first HTML/JSON project with an `AGENTS.md` guide and pinned scripts |
272
+ | `frameyard dev [--port <port>]` | Start the preview server; preview compositions or open a timeline |
273
+ | `frameyard check <id> [--props <file>]` | Run static validation, then evaluate every frame of a composition or timeline without capture |
274
+ | `frameyard still <id> --frame <n>` | Capture one PNG through the production path for a composition or timeline |
275
+ | `frameyard render <id> [--out <video.mp4>]` | Render a composition directly, or a timeline through the segmented lossless pipeline; defaults to `out/<id>.mp4` |
224
276
  | `frameyard tts ...` | Produce one frozen 24 kHz mono WAV with local qwentts.cpp |
225
277
  | `frameyard transcribe ...` | Produce canonical transcript JSON with optional word timings and SRT/VTT exports |
226
278
  | `frameyard doctor` | Inspect core requirements, optional integrations, and manifest integrity |
227
279
 
228
- Common options: `--project <dir>`, `--json`, `--help`, and `--version`. `still` and `render` accept `--overwrite`; `render` also accepts `--concurrency <n>` (1-16); `tts` and `transcribe` accept `--force`. `init` takes the target directory as a positional argument. Explicit output paths passed with `--out` are relative to the current working directory; `render` defaults to `out/<id>.mp4` inside the project root.
280
+ Common options: `--project <dir>`, `--json`, `--help`, and `--version`. `still` and `render` accept `--overwrite`. Composition renders accept `--concurrency <n>` (1-16); timeline renders accept `--workers <n>` (1-16) and `--scratch <dir>`. For a timeline, `--props` is a JSON object keyed by composition clip id; for a composition, it is the props object. `tts` and `transcribe` accept `--force`. `init` takes the target directory as a positional argument. Explicit output paths passed with `--out` are relative to the current working directory; `render` defaults to `out/<id>.mp4` inside the project root.
229
281
 
230
282
  A render fails immediately on an uncaught error, unhandled rejection, missing required asset, or invalid composition output. It writes to a temporary file and publishes only after FFmpeg succeeds. Existing outputs are not replaced unless `--overwrite` is supplied.
231
283
 
@@ -317,26 +369,39 @@ The optional host-neutral agent skill lives in [`skills/frameyard/`](./skills/fr
317
369
 
318
370
  ## Rendering and reproducibility
319
371
 
320
- Production rendering:
372
+ Production rendering has two targets.
373
+
374
+ A composition renders directly:
375
+
376
+ 1. Static composition validation resolves metadata before launching Chromium.
377
+ 2. Chromium navigates to the composition document with the composition host.
378
+ 3. One session, or several when `--concurrency` is greater than 1, seeks every frame, calls the controller, seeks CSS/WAAPI animations, and waits for embedded media decode and paint.
379
+ 4. The viewport is captured through the Chrome DevTools Protocol, piped to FFmpeg, and encoded as H.264/AAC.
380
+
381
+ A timeline renders through segments:
321
382
 
322
- 1. Runs static composition validation and resolves metadata before launching Chromium.
323
- 2. Navigates directly to the composition document with the composition host.
324
- 3. Launches one pinned Chromium session, or several independent sessions when `--concurrency` is greater than 1.
325
- 4. Seeks explicitly to each frame, calls the controller, seeks CSS/WAAPI animations, and waits for embedded media decode and browser paint.
326
- 5. Captures the viewport through the Chrome DevTools Protocol.
327
- 6. Pipes PNG frames in timeline order and the offline audio mix to FFmpeg.
328
- 7. Encodes H.264 video with `yuv420p` and AAC audio into an MP4.
383
+ 1. Static timeline validation resolves references, media, nested timelines, durations, props, audio, and overlaps.
384
+ 2. The timeline is split at active-layer changes and a maximum segment duration.
385
+ 3. In parallel workers, Chromium captures active composition and nested-timeline clips to alpha-preserving FFV1 intermediates.
386
+ 4. FFmpeg applies fit, static transforms, opacity, video/image decoding, and audio mixing, and encodes each segment as H.264.
387
+ 5. Segments are concatenated in timeline order without re-encoding and the global audio mix is muxed into the final MP4.
329
388
 
330
- Rendering is sequential by default. Pass `--concurrency <n>` (1-16) to seek and capture frames with up to `n` independent browser sessions; the effective value is capped by the composition's frame count. Captured frames are always encoded in timeline order, so parallel and sequential renders of the same inputs produce equivalent visual frames. Concurrency only changes scheduling: it does not change composition behavior or the determinism contract. Parallel rendering uses more CPU and memory, so choose a value appropriate for the machine.
389
+ Composition concurrency changes scheduling only. Timeline workers process independent segments; output order and captured composition frames remain deterministic. `check` and `still` use the same clip capture and composite path as their target.
331
390
 
332
- For the same composition revision, props, assets, frame, and pinned runtime/browser environment, the visual state is identical. Encoded bytes are not a reproducibility guarantee, even on the same machine; they depend on the FFmpeg build, encoder threading, and runtime environment.
391
+ For the same composition or timeline revision, props, assets, segment plan, frame, and pinned runtime/browser/FFmpeg environment, the captured composition frames and deterministic composite are identical. Encoded bytes are not a reproducibility guarantee, even on the same machine; they depend on the FFmpeg build, encoder threading, and runtime environment. Preview composites in Chromium while timeline production composites with FFmpeg, so small differences in scaling, alpha, and video color conversion are expected; parity is guaranteed at clip capture.
333
392
 
334
- For a repeatable local measurement of seek, capture, encode, and full-render time, run `pnpm bench`. See `bench/README.md` for the measurement recipe and browser-flag experiments. The demo and the controlled fixtures under `bench/fixtures/` are the reference workloads.
393
+ For a repeatable local measurement of seek, capture, encode, and full-render time, run `pnpm bench`. See `bench/README.md` for the measurement recipe. Timeline performance is tracked separately with the long-form benchmarks; the representative target and worst-case measurements are in [ADR 0012](./docs/adr/0012-render-timelines-with-segmented-lossless-clip-intermediates.md).
335
394
 
336
395
  ## Current limits
337
396
 
338
397
  Deferred from this implementation slice:
339
398
 
399
+ - Timeline transitions, effects, keyframes, blend modes, masks, and audio fades
400
+ - Full visual timeline editing, drag-and-drop, trim handles, and undo/redo
401
+ - Timeline-level variables and prop overrides inside nested timelines
402
+ - Per-timeline formats and frame-rate conversion/retiming
403
+ - Persistent clip caches; timeline scratch intermediates are cleaned after a render
404
+ - Hardware encoders, GPU compositing, and non-H.264 final output
340
405
  - Built-in subtitle rendering, automatic caption burn-in, and declarative karaoke elements
341
406
  - Nested `data-fy-repeat` scopes and a general template language
342
407
  - Declarative JSON Schema validation
@@ -344,11 +409,11 @@ Deferred from this implementation slice:
344
409
  - `qwentts.cpp` reference-audio voice cloning and VoiceDesign
345
410
  - Automatic long-text chunking, streaming synthesis, and alternative generated-audio formats
346
411
  - Automatic upstream builds, model downloads, environment repair, and hosted rendering
347
- - Frame caches, batch still capture, and general image-sequence output
412
+ - Batch still capture and general image-sequence output
348
413
  - React or other UI-framework adapters, plugin APIs, and agent-only runtimes
349
414
  - TypeScript transpilation for project compositions and npm dependency optimization
350
415
 
351
- See the [design baseline](./docs/v0.1-design.md) for the full product contract, [ADR 0007](./docs/adr/0007-make-html-documents-the-composition-unit.md) for the HTML authoring decision, and the [domain glossary](./GLOSSARY.md) for terminology.
416
+ See the [design baseline](./docs/v0.1-design.md) for the full product contract, [ADR 0007](./docs/adr/0007-make-html-documents-the-composition-unit.md) for the HTML authoring decision, [ADR 0010](./docs/adr/0010-make-timelines-the-render-unit.md) for timelines, [ADR 0012](./docs/adr/0012-render-timelines-with-segmented-lossless-clip-intermediates.md) for segmented rendering, and the [domain glossary](./GLOSSARY.md) for terminology.
352
417
 
353
418
  ## Repository layout
354
419
 
@@ -368,7 +433,7 @@ See the [design baseline](./docs/v0.1-design.md) for the full product contract,
368
433
  - `skills/`: official host-neutral agent skill
369
434
  - `examples/demo/`: the 30-second reference composition
370
435
  - `examples/webgl/`: the focused WebGL shader example
371
- - `examples/video/`: the two-pass embedded-video example with WebM and MP4 sources
436
+ - `examples/video/`: embedded-video compositions and WebM/MP4 timelines with video clips under a transparent HTML overlay
372
437
  - `bench/fixtures/`: controlled HTML compositions for the render benchmark
373
438
  - `scripts/`: benchmark, release checks, versioning, and git-hook installation
374
439
  - `docs/adr/`: architecture decision records