frameyard 0.5.0 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +55 -0
- package/GLOSSARY.md +152 -24
- package/README.md +94 -29
- package/dist/cli.js +111 -14
- package/dist/cli.js.map +1 -1
- package/dist/contracts.d.ts +12 -0
- package/dist/contracts.d.ts.map +1 -1
- package/dist/contracts.js.map +1 -1
- package/dist/errors.d.ts +1 -1
- package/dist/errors.d.ts.map +1 -1
- package/dist/errors.js.map +1 -1
- package/dist/index.d.ts +2 -0
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +1 -0
- package/dist/index.js.map +1 -1
- package/dist/preview/main.js +89 -11
- package/dist/preview/main.js.map +1 -1
- package/dist/preview/timeline-data.d.ts +85 -0
- package/dist/preview/timeline-data.d.ts.map +1 -0
- package/dist/preview/timeline-data.js +2 -0
- package/dist/preview/timeline-data.js.map +1 -0
- package/dist/preview/timeline-host.d.ts +19 -0
- package/dist/preview/timeline-host.d.ts.map +1 -0
- package/dist/preview/timeline-host.js +297 -0
- package/dist/preview/timeline-host.js.map +1 -0
- package/dist/preview/timeline-main.d.ts +17 -0
- package/dist/preview/timeline-main.d.ts.map +1 -0
- package/dist/preview/timeline-main.js +364 -0
- package/dist/preview/timeline-main.js.map +1 -0
- package/dist/project.d.ts +16 -1
- package/dist/project.d.ts.map +1 -1
- package/dist/project.js +81 -26
- package/dist/project.js.map +1 -1
- package/dist/renderer/browser.d.ts +2 -0
- package/dist/renderer/browser.d.ts.map +1 -1
- package/dist/renderer/browser.js +6 -1
- package/dist/renderer/browser.js.map +1 -1
- package/dist/renderer/ffmpeg.d.ts +61 -0
- package/dist/renderer/ffmpeg.d.ts.map +1 -1
- package/dist/renderer/ffmpeg.js +198 -0
- package/dist/renderer/ffmpeg.js.map +1 -1
- package/dist/renderer/timeline.d.ts +109 -0
- package/dist/renderer/timeline.d.ts.map +1 -0
- package/dist/renderer/timeline.js +984 -0
- package/dist/renderer/timeline.js.map +1 -0
- package/dist/scaffold.d.ts.map +1 -1
- package/dist/scaffold.js +89 -24
- package/dist/scaffold.js.map +1 -1
- package/dist/server/preview-html.d.ts.map +1 -1
- package/dist/server/preview-html.js +15 -1
- package/dist/server/preview-html.js.map +1 -1
- package/dist/server/server.d.ts.map +1 -1
- package/dist/server/server.js +39 -1
- package/dist/server/server.js.map +1 -1
- package/dist/server/timeline-preview-data.d.ts +4 -0
- package/dist/server/timeline-preview-data.d.ts.map +1 -0
- package/dist/server/timeline-preview-data.js +97 -0
- package/dist/server/timeline-preview-data.js.map +1 -0
- package/dist/server/timeline-preview-html.d.ts +4 -0
- package/dist/server/timeline-preview-html.d.ts.map +1 -0
- package/dist/server/timeline-preview-html.js +227 -0
- package/dist/server/timeline-preview-html.js.map +1 -0
- package/dist/timeline/plan.d.ts +32 -0
- package/dist/timeline/plan.d.ts.map +1 -0
- package/dist/timeline/plan.js +68 -0
- package/dist/timeline/plan.js.map +1 -0
- package/dist/timeline/resolve.d.ts +60 -0
- package/dist/timeline/resolve.d.ts.map +1 -0
- package/dist/timeline/resolve.js +523 -0
- package/dist/timeline/resolve.js.map +1 -0
- package/dist/timeline/schema.d.ts +71 -0
- package/dist/timeline/schema.d.ts.map +1 -0
- package/dist/timeline/schema.js +425 -0
- package/dist/timeline/schema.js.map +1 -0
- package/dist/version.d.ts +1 -1
- package/dist/version.js +1 -1
- package/docs/adr/0002-make-frame-and-props-the-only-animation-state.md +2 -0
- package/docs/adr/0005-keep-parallel-rendering-opt-in-and-frame-ordered.md +2 -0
- package/docs/adr/0007-make-html-documents-the-composition-unit.md +1 -1
- package/docs/adr/0009-composite-production-frames-in-software.md +1 -1
- package/docs/adr/0010-make-timelines-the-render-unit.md +24 -0
- package/docs/adr/0011-compose-timeline-frames-in-a-live-browser-host.md +22 -0
- package/docs/adr/0012-render-timelines-with-segmented-lossless-clip-intermediates.md +31 -0
- package/docs/v0.1-design.md +135 -86
- package/examples/demo/README.md +10 -3
- package/examples/demo/timelines/demo.timeline.json +12 -0
- package/examples/demo/video.config.js +4 -0
- package/examples/video/README.md +50 -6
- package/examples/video/compositions/video-overlay.html +350 -0
- package/examples/video/props/video-timeline.json +10 -0
- package/examples/video/timelines/video-timeline-mp4.timeline.json +44 -0
- package/examples/video/timelines/video-timeline.timeline.json +44 -0
- package/examples/video/video.config.js +6 -0
- package/package.json +1 -1
- package/skills/frameyard/SKILL.md +17 -13
- package/skills/frameyard/references/timelines.md +141 -0
package/CHANGELOG.md
CHANGED
|
@@ -8,6 +8,61 @@ Frameyard uses `0.x` versions until the composition, configuration, CLI,
|
|
|
8
8
|
JSON-result, transcript, and skill contracts stabilize. Before `1.0.0`, a
|
|
9
9
|
contract change may ship in a minor release and is called out explicitly below.
|
|
10
10
|
|
|
11
|
+
## [0.6.0] - 2026-11-01
|
|
12
|
+
|
|
13
|
+
### Added
|
|
14
|
+
|
|
15
|
+
- JSON timelines with project-fixed format, video/audio tracks, composition,
|
|
16
|
+
media, and nested-timeline clips, source ranges, per-clip props, static
|
|
17
|
+
transforms, audio settings, and derived or overridden duration.
|
|
18
|
+
- Timeline validation: strict schema, references, composition static
|
|
19
|
+
validation, media probing, frame-rate checks, source bounds, overlaps,
|
|
20
|
+
nested cycles, and structured diagnostics.
|
|
21
|
+
- Segmented timeline rendering: Chromium captures composition and
|
|
22
|
+
nested-timeline clips to alpha-preserving FFV1 intermediates; FFmpeg
|
|
23
|
+
composites and encodes segments; segments are concatenated in order; audio
|
|
24
|
+
is mixed once and muxed.
|
|
25
|
+
- Transparent composition capture and nested timeline groups with recursive
|
|
26
|
+
visual, still, check, and audio handling.
|
|
27
|
+
- Timeline preview in `dev`: composition and timeline selectors, program
|
|
28
|
+
monitor, read-only tracks and clips, playhead, playback, frame stepping, and
|
|
29
|
+
one-second jumps with buttons or arrow keys, audio controls, and diagnostics.
|
|
30
|
+
- Timeline visual transforms accept `width`/`height` as an explicit clip box,
|
|
31
|
+
mutually exclusive with `scale`.
|
|
32
|
+
- `render`, `check`, and `still` accept timeline IDs alongside composition
|
|
33
|
+
IDs. Timeline renders add `--workers` and `--scratch`; timeline `--props`
|
|
34
|
+
is a map keyed by composition clip ID.
|
|
35
|
+
- `video.config.js` supports an optional project `format` and explicit
|
|
36
|
+
`timelines` map. `project.json` publishes `format` and `timelines`.
|
|
37
|
+
- Timeline-first `init` scaffold with a composition clip, timeline override
|
|
38
|
+
props, and `main` commands.
|
|
39
|
+
- The video example adds WebM and MP4 timelines that place video clips under a
|
|
40
|
+
transparent HTML overlay.
|
|
41
|
+
- Official skill reference for timelines, clips, tracks, nested timelines,
|
|
42
|
+
props overrides, commands, preview, and determinism.
|
|
43
|
+
- Timeline config, schema, resolution, planning, rendering, preview, and
|
|
44
|
+
runtime preview tests.
|
|
45
|
+
|
|
46
|
+
### Changed
|
|
47
|
+
|
|
48
|
+
- `render` defaults to `out/<id>.mp4` for timelines and compositions.
|
|
49
|
+
- `check` and `still` use timeline clip capture and composite behavior for
|
|
50
|
+
timeline targets.
|
|
51
|
+
- Timeline audio mixing pads short inputs to the full timeline duration.
|
|
52
|
+
- Timed timeline layers are normalized to the project frame rate before
|
|
53
|
+
compositing.
|
|
54
|
+
- The design baseline, glossary, and architecture decision records describe
|
|
55
|
+
timelines as the render unit and compositions as clip sources.
|
|
56
|
+
|
|
57
|
+
### Fixed
|
|
58
|
+
|
|
59
|
+
- Composition audio, embedded video audio, and nested timeline audio are
|
|
60
|
+
trimmed and shifted by clip source ranges and groups.
|
|
61
|
+
- Selecting a different composition in the preview picker reloads the iframe
|
|
62
|
+
instead of leaving the previous composition loaded.
|
|
63
|
+
|
|
64
|
+
[0.6.0]: https://github.com/vladiantio/frameyard/compare/v0.5.0...v0.6.0
|
|
65
|
+
|
|
11
66
|
## [0.5.0] - 2026-10-31
|
|
12
67
|
|
|
13
68
|
### Added
|
package/GLOSSARY.md
CHANGED
|
@@ -1,21 +1,25 @@
|
|
|
1
1
|
# Frameyard
|
|
2
2
|
|
|
3
|
-
Frameyard is a local-first framework for creating short, parameterized videos with HTML, CSS, and JavaScript. Its domain separates reusable compositions from the fixed inputs and deterministic renders that turn
|
|
3
|
+
Frameyard is a local-first framework for creating short, parameterized videos with HTML, CSS, and JavaScript. Its domain separates reusable compositions from the timelines that assemble them into a video, and from the fixed inputs and deterministic renders that turn those timelines into output.
|
|
4
4
|
|
|
5
5
|
## Language
|
|
6
6
|
|
|
7
7
|
### Authoring
|
|
8
8
|
|
|
9
9
|
**Project**:
|
|
10
|
-
A collection of related compositions and the assets and configuration they share.
|
|
10
|
+
A collection of related compositions, timelines, and the assets and configuration they share.
|
|
11
11
|
_Avoid_: Video, render
|
|
12
12
|
|
|
13
|
+
**Project format**:
|
|
14
|
+
The fixed width, height, and frame rate shared by every timeline in a project.
|
|
15
|
+
_Avoid_: Composition metadata, render options
|
|
16
|
+
|
|
13
17
|
**Composition**:
|
|
14
|
-
A uniquely identified, reusable video
|
|
18
|
+
A uniquely identified, reusable video source with fixed dimensions, frame rate, duration in frames, and content parameterized by props.
|
|
15
19
|
_Avoid_: Scene, template, video
|
|
16
20
|
|
|
17
21
|
**Composition metadata**:
|
|
18
|
-
The fixed dimensions, frame rate, and duration in frames that define a composition's
|
|
22
|
+
The fixed dimensions, frame rate, and duration in frames that define a composition's source contract.
|
|
19
23
|
_Avoid_: Render options, props
|
|
20
24
|
|
|
21
25
|
**Composition descriptor**:
|
|
@@ -38,6 +42,10 @@ _Avoid_: Build, release
|
|
|
38
42
|
A composition whose source is an HTML document.
|
|
39
43
|
_Avoid_: HTML template, page
|
|
40
44
|
|
|
45
|
+
**Transparent composition**:
|
|
46
|
+
A composition that does not paint a background, so a timeline clip can reveal the layers below it.
|
|
47
|
+
_Avoid_: Alpha composition, overlay
|
|
48
|
+
|
|
41
49
|
**Composition controller**:
|
|
42
50
|
An optional JavaScript controller, either inline in the composition document or in an external module, that performs per-frame computation an HTML composition's declarative HTML and CSS cannot express.
|
|
43
51
|
_Avoid_: Composition module, plugin
|
|
@@ -55,25 +63,113 @@ An HTML attribute that maps a prop into text, an attribute, a class, or element
|
|
|
55
63
|
_Avoid_: Template expression, interpolation
|
|
56
64
|
|
|
57
65
|
**Props**:
|
|
58
|
-
The immutable, JSON-serializable inputs supplied to a composition for one
|
|
66
|
+
The immutable, JSON-serializable inputs supplied to a composition for one load; a timeline clip can fix the props for its instance of the composition.
|
|
59
67
|
_Avoid_: Options, settings, mutable state
|
|
60
68
|
|
|
61
69
|
**Asset**:
|
|
62
|
-
A local file required by a composition, whether static or generated.
|
|
70
|
+
A local file required by a composition or timeline, whether static or generated.
|
|
63
71
|
_Avoid_: Resource, media file
|
|
64
72
|
|
|
73
|
+
**Timeline**:
|
|
74
|
+
A top-level edit that assembles tracks and clips into one video under the project format, with a duration derived from its clips or explicitly overridden.
|
|
75
|
+
_Avoid_: Sequence, composition, project
|
|
76
|
+
|
|
77
|
+
**Timeline document**:
|
|
78
|
+
The project-local JSON document that declares a timeline's tracks, clips, optional duration, and background.
|
|
79
|
+
_Avoid_: Composition, config
|
|
80
|
+
|
|
81
|
+
**Track**:
|
|
82
|
+
An ordered layer in a timeline. Video tracks composite; audio tracks mix.
|
|
83
|
+
_Avoid_: Channel, lane
|
|
84
|
+
|
|
85
|
+
**Video track**:
|
|
86
|
+
A track whose clips contribute visual layers, ordered from bottom to top.
|
|
87
|
+
_Avoid_: Visual lane
|
|
88
|
+
|
|
89
|
+
**Audio track**:
|
|
90
|
+
A track whose clips are audio-only and contribute additively to the timeline audio mix.
|
|
91
|
+
_Avoid_: Soundtrack
|
|
92
|
+
|
|
93
|
+
**Clip**:
|
|
94
|
+
A placement of a clip source on a track, with a timeline start, source in-point, duration, and presentation settings.
|
|
95
|
+
_Avoid_: Segment, item, block
|
|
96
|
+
|
|
97
|
+
**Clip source**:
|
|
98
|
+
The composition, nested timeline, or local media asset that a clip presents.
|
|
99
|
+
_Avoid_: Media, asset
|
|
100
|
+
|
|
101
|
+
**Composition clip**:
|
|
102
|
+
A clip whose source is a composition, with fixed props for that instance.
|
|
103
|
+
_Avoid_: Composition instance, nested composition
|
|
104
|
+
|
|
105
|
+
**Media clip**:
|
|
106
|
+
A clip whose source is a local video, image, or audio asset.
|
|
107
|
+
_Avoid_: Asset clip, raw clip
|
|
108
|
+
|
|
109
|
+
**Nested timeline clip**:
|
|
110
|
+
A clip whose source is another timeline.
|
|
111
|
+
_Avoid_: Nested sequence, compound clip
|
|
112
|
+
|
|
113
|
+
**Source in-point**:
|
|
114
|
+
The first source frame a clip uses.
|
|
115
|
+
_Avoid_: Trim start, offset
|
|
116
|
+
|
|
117
|
+
**Timeline frame**:
|
|
118
|
+
An integer-addressed position on a timeline's master frame grid.
|
|
119
|
+
_Avoid_: Composition frame, timestamp
|
|
120
|
+
|
|
121
|
+
**Composition frame**:
|
|
122
|
+
An integer-addressed position within a composition; the frame a controller receives.
|
|
123
|
+
_Avoid_: Timeline frame
|
|
124
|
+
|
|
125
|
+
**Timeline duration**:
|
|
126
|
+
The number of timeline frames a timeline produces, derived from its clips or explicitly overridden.
|
|
127
|
+
_Avoid_: Composition duration, length
|
|
128
|
+
|
|
129
|
+
**Timeline background**:
|
|
130
|
+
The optional color behind all layers of a timeline; it is transparent by default and the final video composites it over black.
|
|
131
|
+
_Avoid_: Canvas, backdrop
|
|
132
|
+
|
|
133
|
+
**Fit policy**:
|
|
134
|
+
How a clip source's natural frame is fitted into the project frame or an explicit clip box: contained, covering, or stretched.
|
|
135
|
+
_Avoid_: Resize mode, object-fit
|
|
136
|
+
|
|
137
|
+
**Clip transform**:
|
|
138
|
+
The static size, position, rotation, and opacity applied to a visual clip. Size is either an explicit `width`/`height` box or a `scale` multiplier.
|
|
139
|
+
_Avoid_: Effect, animation
|
|
140
|
+
|
|
141
|
+
**Nested timeline**:
|
|
142
|
+
A timeline used as a clip source inside another timeline.
|
|
143
|
+
_Avoid_: Compound composition
|
|
144
|
+
|
|
65
145
|
### Deterministic execution
|
|
66
146
|
|
|
67
147
|
**Frame**:
|
|
68
|
-
An integer-addressed position on a composition
|
|
148
|
+
An integer-addressed position on a timeline or composition frame range; use timeline frame or composition frame when the distinction matters.
|
|
69
149
|
_Avoid_: Timestamp, animation tick
|
|
70
150
|
|
|
151
|
+
**Active clip**:
|
|
152
|
+
A clip whose timeline range contains the current timeline frame.
|
|
153
|
+
_Avoid_: Visible clip
|
|
154
|
+
|
|
155
|
+
**Clip capture**:
|
|
156
|
+
The pixel frames a composition produces for a timeline clip before the timeline compositor combines layers.
|
|
157
|
+
_Avoid_: Screenshot, render
|
|
158
|
+
|
|
159
|
+
**Timeline composite**:
|
|
160
|
+
The deterministic combination of a timeline's layers into output frames.
|
|
161
|
+
_Avoid_: Browser composite, preview
|
|
162
|
+
|
|
163
|
+
**Render segment**:
|
|
164
|
+
A bounded span of a timeline that is rendered and encoded independently, then joined in timeline order.
|
|
165
|
+
_Avoid_: Chunk, part
|
|
166
|
+
|
|
71
167
|
**Deterministic execution**:
|
|
72
|
-
The rule that a
|
|
168
|
+
The rule that a timeline frame's visual state depends only on that timeline frame, its clip sources and their props, and pre-loaded data and assets; a composition frame depends only on its composition frame, props, and pre-loaded data and assets.
|
|
73
169
|
_Avoid_: Playback-driven animation
|
|
74
170
|
|
|
75
171
|
**Frame clock**:
|
|
76
|
-
The single source of animation time, derived from
|
|
172
|
+
The single source of animation time within a composition, derived from its composition frame and frame rate; the preview host or the production segment renderer drives it by seeking each active composition clip.
|
|
77
173
|
_Avoid_: Wall clock, playback timer
|
|
78
174
|
|
|
79
175
|
**Frame-state variable**:
|
|
@@ -81,7 +177,7 @@ A CSS custom property whose value is derived from the current composition frame.
|
|
|
81
177
|
_Avoid_: Animation variable, clock variable
|
|
82
178
|
|
|
83
179
|
**Seek safety**:
|
|
84
|
-
The property that any frame can be produced correctly without depending on previously produced frames.
|
|
180
|
+
The property that any timeline frame can be produced correctly without depending on previously produced timeline frames, by seeking each active clip to its source frame.
|
|
85
181
|
_Avoid_: History-dependent updates
|
|
86
182
|
|
|
87
183
|
**Deterministic randomness**:
|
|
@@ -89,15 +185,15 @@ Random values derived statelessly from a key so they do not depend on call order
|
|
|
89
185
|
_Avoid_: Unseeded randomness
|
|
90
186
|
|
|
91
187
|
**Frame production**:
|
|
92
|
-
The phase in which the runtime produces the visual state of a requested frame, without network access or model inference.
|
|
188
|
+
The phase in which the runtime produces the visual state of a requested timeline frame, without network access or model inference.
|
|
93
189
|
_Avoid_: Real-time playback
|
|
94
190
|
|
|
95
191
|
**Loading phase**:
|
|
96
|
-
The phase before frame production when
|
|
192
|
+
The phase before frame production when all clip sources, compositions, media, data, and assets become available.
|
|
97
193
|
_Avoid_: Per-frame fetching
|
|
98
194
|
|
|
99
195
|
**Embedded media**:
|
|
100
|
-
A local video file placed
|
|
196
|
+
A local video file placed inside a composition whose presentation time is derived from the composition frame, with optional frame offset and source trim.
|
|
101
197
|
_Avoid_: Live stream, video track
|
|
102
198
|
|
|
103
199
|
**Media settle**:
|
|
@@ -110,42 +206,74 @@ _Avoid_: GPU animation, live render
|
|
|
110
206
|
|
|
111
207
|
### Audio
|
|
112
208
|
|
|
113
|
-
**
|
|
114
|
-
An audio source
|
|
115
|
-
_Avoid_: Audio track
|
|
209
|
+
**Composition audio clip**:
|
|
210
|
+
An audio source declared inside a composition and placed by composition frames; it joins the timeline mix when the composition is used as a clip.
|
|
211
|
+
_Avoid_: Audio clip, audio track
|
|
212
|
+
|
|
213
|
+
**Timeline audio clip**:
|
|
214
|
+
An audio-only clip placed on a timeline audio track.
|
|
215
|
+
_Avoid_: Audio clip, soundtrack
|
|
116
216
|
|
|
117
217
|
**Audio mix**:
|
|
118
|
-
The single offline mix of a
|
|
218
|
+
The single offline mix of a timeline's audible clips for one render.
|
|
119
219
|
_Avoid_: Soundtrack, audio track, final cut
|
|
120
220
|
|
|
221
|
+
**Embedded audio**:
|
|
222
|
+
The audio track of an embedded video, contributed to the mix unless the video or clip is muted.
|
|
223
|
+
_Avoid_: Live audio
|
|
224
|
+
|
|
121
225
|
### Rendering
|
|
122
226
|
|
|
123
227
|
**Composition host**:
|
|
124
228
|
The injected runtime that loads an HTML composition, applies its bindings, and exposes its frame clock to preview and render.
|
|
125
229
|
_Avoid_: Preview runtime, render runtime
|
|
126
230
|
|
|
231
|
+
**Timeline host**:
|
|
232
|
+
The preview runtime that loads a timeline, drives a bounded pool of active clip sources, and exposes the timeline frame clock to the preview.
|
|
233
|
+
_Avoid_: Timeline runtime, production renderer, editor
|
|
234
|
+
|
|
235
|
+
**Timeline renderer**:
|
|
236
|
+
The Node-side segmented renderer that captures composition clips, composites and encodes timeline segments, and joins them into one video.
|
|
237
|
+
_Avoid_: Timeline host, browser renderer
|
|
238
|
+
|
|
127
239
|
**Preview**:
|
|
128
|
-
An interactive, frame-accurate inspection of a composition using the same
|
|
240
|
+
An interactive, frame-accurate inspection of a timeline or composition using the same behavior as a render, including a program monitor and a read-only timeline view.
|
|
129
241
|
_Avoid_: Visual editor
|
|
130
242
|
|
|
243
|
+
**Playhead**:
|
|
244
|
+
The current timeline frame in preview.
|
|
245
|
+
_Avoid_: Cursor, scrubber
|
|
246
|
+
|
|
247
|
+
**Program monitor**:
|
|
248
|
+
The preview surface that shows the timeline at the playhead.
|
|
249
|
+
_Avoid_: Stage, canvas
|
|
250
|
+
|
|
131
251
|
**Check**:
|
|
132
|
-
A validation operation that first runs static
|
|
252
|
+
A validation operation that first runs static validation, then loads a timeline or composition and evaluates every frame without capturing or encoding.
|
|
133
253
|
_Avoid_: Test render, dry run
|
|
134
254
|
|
|
135
255
|
**Static composition validation**:
|
|
136
256
|
The Node-side validation of a composition document's metadata, bindings, controller declaration, scripts, and asset references before browser frame production.
|
|
137
257
|
_Avoid_: Lint, preflight
|
|
138
258
|
|
|
259
|
+
**Static timeline validation**:
|
|
260
|
+
The Node-side validation of a timeline document's tracks, clips, references, nested timelines, media, and referenced compositions before browser frame production.
|
|
261
|
+
_Avoid_: Lint, preflight
|
|
262
|
+
|
|
139
263
|
**Composition diagnostic**:
|
|
140
264
|
A structured error or warning about a composition's source, metadata, bindings, controller, assets, or frame production, with a stable code and optional source location.
|
|
141
265
|
_Avoid_: Log message, error string
|
|
142
266
|
|
|
267
|
+
**Timeline diagnostic**:
|
|
268
|
+
A structured error or warning about a timeline's source, tracks, clips, references, media, or frame production, with a stable code and optional source location.
|
|
269
|
+
_Avoid_: Log message, error string
|
|
270
|
+
|
|
143
271
|
**Render**:
|
|
144
|
-
The production of one complete video from a
|
|
272
|
+
The production of one complete video from a timeline or a standalone composition with fixed inputs.
|
|
145
273
|
_Avoid_: Recording, export
|
|
146
274
|
|
|
147
275
|
**Render concurrency**:
|
|
148
|
-
The optional number of independent browser sessions used to seek and capture frames during a render. Frames are still encoded in timeline order, and concurrency must not change visual state.
|
|
276
|
+
The optional number of independent browser sessions used to seek and capture frames during a composition render; timeline rendering is sequential in 0.6. Frames are still encoded in timeline order, and concurrency must not change visual state.
|
|
149
277
|
_Avoid_: Parallel composition, out-of-order output
|
|
150
278
|
|
|
151
279
|
**Video**:
|
|
@@ -153,11 +281,11 @@ The finished audiovisual output of a successful render.
|
|
|
153
281
|
_Avoid_: Composition, project
|
|
154
282
|
|
|
155
283
|
**Still**:
|
|
156
|
-
A single frame captured as a PNG through the production rendering path for
|
|
284
|
+
A single frame captured as a PNG through the production rendering path for a timeline or composition.
|
|
157
285
|
_Avoid_: Frame export, image sequence
|
|
158
286
|
|
|
159
287
|
**Reproducibility**:
|
|
160
|
-
The guarantee that, for the same composition revision, props, assets, and frame under a pinned runtime/browser environment, the visual state is identical. It does not imply identical encoded bytes, even on the same machine; encoded output depends on the encoder and its runtime.
|
|
288
|
+
The guarantee that, for the same timeline or composition revision, props, assets, and frame under a pinned runtime/browser environment, the visual state is identical. It does not imply identical encoded bytes, even on the same machine; encoded output depends on the encoder and its runtime.
|
|
161
289
|
_Avoid_: Bit-for-bit encoding equality
|
|
162
290
|
|
|
163
291
|
### Agent operation
|
|
@@ -173,7 +301,7 @@ _Avoid_: Agent runtime, MCP server
|
|
|
173
301
|
### Local asset production
|
|
174
302
|
|
|
175
303
|
**Generated asset**:
|
|
176
|
-
A frozen local input produced by speech synthesis or transcription before a composition is loaded or rendered.
|
|
304
|
+
A frozen local input produced by speech synthesis or transcription before a timeline or composition is loaded or rendered.
|
|
177
305
|
_Avoid_: Per-frame generation
|
|
178
306
|
|
|
179
307
|
**Provenance manifest**:
|
package/README.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
# Frameyard
|
|
2
2
|
|
|
3
|
-
Frameyard is a local-first framework for building
|
|
3
|
+
Frameyard is a local-first framework for building parameterized videos with HTML, CSS, and an optional JavaScript controller, assembling them into timelines, and rendering them deterministically.
|
|
4
4
|
|
|
5
|
-
The 0.
|
|
5
|
+
The 0.6 design is implemented: HTML composition documents, JSON timelines with tracks and clips, nested timelines, declarative bindings, CSS animation scheduling, frame-state variables, inline and external controllers, composition and timeline hosts, static validation with structured diagnostics, an interactive preview with a read-only timeline, direct composition rendering, segmented lossless timeline rendering, audio and transcripts, deterministic Chromium capture, FFmpeg output, and the official agent skill.
|
|
6
6
|
|
|
7
7
|
## Requirements
|
|
8
8
|
|
|
@@ -26,7 +26,7 @@ cd my-video
|
|
|
26
26
|
npx frameyard dev
|
|
27
27
|
```
|
|
28
28
|
|
|
29
|
-
`dev` prints a local preview URL.
|
|
29
|
+
`dev` prints a local preview URL. Preview a composition with JSON props editing, or open the timeline for a program monitor, read-only tracks and clips, scrubbing, playback, frame-by-frame stepping, one-second jumps, and diagnostics.
|
|
30
30
|
|
|
31
31
|
The scaffold includes an `AGENTS.md` guide for coding agents working in the project, covering the HTML authoring rules and the `check`, `still`, and `render` workflow. It also includes a `package.json` with `dev`, `check`, `render`, and `still` scripts pinned to the scaffolding CLI version, so `pnpm dev` or `npm run dev` works without a global install, and a `.gitignore` that ignores `out/`.
|
|
32
32
|
|
|
@@ -36,24 +36,28 @@ Validate and render:
|
|
|
36
36
|
|
|
37
37
|
```sh
|
|
38
38
|
npx frameyard doctor
|
|
39
|
-
npx frameyard check
|
|
40
|
-
npx frameyard render
|
|
41
|
-
npx frameyard still
|
|
39
|
+
npx frameyard check main
|
|
40
|
+
npx frameyard render main --props props/main.json --workers 8 --scratch /path/to/scratch
|
|
41
|
+
npx frameyard still main --frame 42 --out main-frame.png
|
|
42
42
|
```
|
|
43
43
|
|
|
44
|
-
`render` writes the video to `out/
|
|
44
|
+
The scaffold wraps its composition in `timelines/main.timeline.json`. `render` writes the video to `out/main.mp4` in the project by default; pass `--out <file.mp4>` to choose another path. The composition remains renderable on its own with `npx frameyard render hello --props props/hello.json`. The scaffold's `.gitignore` ignores `out/`.
|
|
45
45
|
|
|
46
|
-
The 30-second reference composition lives in [`examples/demo`](./examples/demo), a focused WebGL shader example lives in [`examples/webgl`](./examples/webgl), and
|
|
46
|
+
The 30-second reference composition lives in [`examples/demo`](./examples/demo), a focused WebGL shader example lives in [`examples/webgl`](./examples/webgl), and the video example in [`examples/video`](./examples/video) includes two embedded-video compositions plus WebM and MP4 timelines that place the clips on tracks under a transparent HTML overlay. Each README has the commands for running it from a repository checkout.
|
|
47
47
|
|
|
48
48
|
## Project format
|
|
49
49
|
|
|
50
|
-
A project is ordinary HTML, CSS, assets, and optional native JavaScript controller modules. `video.config.js` explicitly lists composition entry points; there is no directory scanning:
|
|
50
|
+
A project is ordinary HTML, CSS, JSON timelines, assets, and optional native JavaScript controller modules. `video.config.js` explicitly lists the project format, composition entry points, and timeline entry points; there is no directory scanning:
|
|
51
51
|
|
|
52
52
|
```js
|
|
53
53
|
export default {
|
|
54
|
+
format: { width: 1920, height: 1080, fps: 30 },
|
|
54
55
|
compositions: {
|
|
55
56
|
hello: "./compositions/hello.html",
|
|
56
57
|
},
|
|
58
|
+
timelines: {
|
|
59
|
+
main: "./timelines/main.timeline.json",
|
|
60
|
+
},
|
|
57
61
|
};
|
|
58
62
|
```
|
|
59
63
|
|
|
@@ -89,6 +93,54 @@ A composition is a full HTML document with one `<fy-composition>` root. Metadata
|
|
|
89
93
|
|
|
90
94
|
`width`, `height`, `fps`, and `duration-in-frames` are fixed. Props are JSON-serializable, deep-frozen, and merged shallowly over the `data-fy="props"` defaults. They cannot change metadata.
|
|
91
95
|
|
|
96
|
+
### Timeline
|
|
97
|
+
|
|
98
|
+
A timeline is a JSON document that places composition and media clips on video and audio tracks. It inherits the project format and derives its duration from the latest clip end unless it declares `durationInFrames`:
|
|
99
|
+
|
|
100
|
+
```json
|
|
101
|
+
{
|
|
102
|
+
"version": 1,
|
|
103
|
+
"tracks": [
|
|
104
|
+
{ "id": "v1", "kind": "video" },
|
|
105
|
+
{ "id": "v2", "kind": "video" },
|
|
106
|
+
{ "id": "a1", "kind": "audio" }
|
|
107
|
+
],
|
|
108
|
+
"clips": [
|
|
109
|
+
{
|
|
110
|
+
"id": "hello",
|
|
111
|
+
"track": "v1",
|
|
112
|
+
"source": { "kind": "composition", "id": "hello" },
|
|
113
|
+
"start": 0,
|
|
114
|
+
"duration": 150
|
|
115
|
+
},
|
|
116
|
+
{
|
|
117
|
+
"id": "logo",
|
|
118
|
+
"track": "v2",
|
|
119
|
+
"source": { "kind": "image", "src": "assets/logo.png" },
|
|
120
|
+
"start": 30,
|
|
121
|
+
"duration": 60,
|
|
122
|
+
"transform": { "fit": "contain", "width": 480, "height": 270, "x": 640, "y": -320, "rotation": 0, "opacity": 0.9 }
|
|
123
|
+
},
|
|
124
|
+
{
|
|
125
|
+
"id": "music",
|
|
126
|
+
"track": "a1",
|
|
127
|
+
"source": { "kind": "audio", "src": "assets/music.wav" },
|
|
128
|
+
"start": 0,
|
|
129
|
+
"duration": 150,
|
|
130
|
+
"audio": { "gain": 0.5 }
|
|
131
|
+
}
|
|
132
|
+
]
|
|
133
|
+
}
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
- Sources are `composition` + `id`, `timeline` + `id`, or `video`/`image`/`audio` + project-relative `src`.
|
|
137
|
+
- `start`, `sourceIn`, and `duration` use timeline frames. Omitted durations use the rest of the source; image clips require a duration.
|
|
138
|
+
- Composition clips can fix `props`, trim the source range, and repeat the same composition with different settings.
|
|
139
|
+
- Visual clips support static `fit`, `width`/`height` (an explicit box), `x`, `y`, `scale`, `rotation`, and `opacity`. `fit` applies inside the box; `width` and `height` go together and are mutually exclusive with `scale`. No keyframes, transitions, or effects in 0.6.
|
|
140
|
+
- Video, composition, and nested-timeline clips contribute audio automatically unless muted. Timeline audio clips live on audio tracks.
|
|
141
|
+
- A composition used in a timeline must match the project fps; different sizes are fitted by the clip. Clips on one track cannot overlap.
|
|
142
|
+
- Nested timelines are rendered as groups, so the parent clip's opacity and transform apply to the whole nested result.
|
|
143
|
+
|
|
92
144
|
### Bindings
|
|
93
145
|
|
|
94
146
|
- `data-fy-text="path"` sets text content.
|
|
@@ -216,16 +268,16 @@ Transcript paths resolve from the project root. `words` requires actual measured
|
|
|
216
268
|
|
|
217
269
|
| Command | Purpose |
|
|
218
270
|
| --- | --- |
|
|
219
|
-
| `frameyard init [directory]` | Scaffold a
|
|
220
|
-
| `frameyard dev [--port <port>]` | Start the preview server |
|
|
221
|
-
| `frameyard check <id> [--props <file>]` | Run static validation, then evaluate every frame without capture |
|
|
222
|
-
| `frameyard still <id> --frame <n>` | Capture one PNG through the production path |
|
|
223
|
-
| `frameyard render <id> [--out <video.mp4>]` |
|
|
271
|
+
| `frameyard init [directory]` | Scaffold a timeline-first HTML/JSON project with an `AGENTS.md` guide and pinned scripts |
|
|
272
|
+
| `frameyard dev [--port <port>]` | Start the preview server; preview compositions or open a timeline |
|
|
273
|
+
| `frameyard check <id> [--props <file>]` | Run static validation, then evaluate every frame of a composition or timeline without capture |
|
|
274
|
+
| `frameyard still <id> --frame <n>` | Capture one PNG through the production path for a composition or timeline |
|
|
275
|
+
| `frameyard render <id> [--out <video.mp4>]` | Render a composition directly, or a timeline through the segmented lossless pipeline; defaults to `out/<id>.mp4` |
|
|
224
276
|
| `frameyard tts ...` | Produce one frozen 24 kHz mono WAV with local qwentts.cpp |
|
|
225
277
|
| `frameyard transcribe ...` | Produce canonical transcript JSON with optional word timings and SRT/VTT exports |
|
|
226
278
|
| `frameyard doctor` | Inspect core requirements, optional integrations, and manifest integrity |
|
|
227
279
|
|
|
228
|
-
Common options: `--project <dir>`, `--json`, `--help`, and `--version`. `still` and `render` accept `--overwrite
|
|
280
|
+
Common options: `--project <dir>`, `--json`, `--help`, and `--version`. `still` and `render` accept `--overwrite`. Composition renders accept `--concurrency <n>` (1-16); timeline renders accept `--workers <n>` (1-16) and `--scratch <dir>`. For a timeline, `--props` is a JSON object keyed by composition clip id; for a composition, it is the props object. `tts` and `transcribe` accept `--force`. `init` takes the target directory as a positional argument. Explicit output paths passed with `--out` are relative to the current working directory; `render` defaults to `out/<id>.mp4` inside the project root.
|
|
229
281
|
|
|
230
282
|
A render fails immediately on an uncaught error, unhandled rejection, missing required asset, or invalid composition output. It writes to a temporary file and publishes only after FFmpeg succeeds. Existing outputs are not replaced unless `--overwrite` is supplied.
|
|
231
283
|
|
|
@@ -317,26 +369,39 @@ The optional host-neutral agent skill lives in [`skills/frameyard/`](./skills/fr
|
|
|
317
369
|
|
|
318
370
|
## Rendering and reproducibility
|
|
319
371
|
|
|
320
|
-
Production rendering
|
|
372
|
+
Production rendering has two targets.
|
|
373
|
+
|
|
374
|
+
A composition renders directly:
|
|
375
|
+
|
|
376
|
+
1. Static composition validation resolves metadata before launching Chromium.
|
|
377
|
+
2. Chromium navigates to the composition document with the composition host.
|
|
378
|
+
3. One session, or several when `--concurrency` is greater than 1, seeks every frame, calls the controller, seeks CSS/WAAPI animations, and waits for embedded media decode and paint.
|
|
379
|
+
4. The viewport is captured through the Chrome DevTools Protocol, piped to FFmpeg, and encoded as H.264/AAC.
|
|
380
|
+
|
|
381
|
+
A timeline renders through segments:
|
|
321
382
|
|
|
322
|
-
1.
|
|
323
|
-
2.
|
|
324
|
-
3.
|
|
325
|
-
4.
|
|
326
|
-
5.
|
|
327
|
-
6. Pipes PNG frames in timeline order and the offline audio mix to FFmpeg.
|
|
328
|
-
7. Encodes H.264 video with `yuv420p` and AAC audio into an MP4.
|
|
383
|
+
1. Static timeline validation resolves references, media, nested timelines, durations, props, audio, and overlaps.
|
|
384
|
+
2. The timeline is split at active-layer changes and a maximum segment duration.
|
|
385
|
+
3. In parallel workers, Chromium captures active composition and nested-timeline clips to alpha-preserving FFV1 intermediates.
|
|
386
|
+
4. FFmpeg applies fit, static transforms, opacity, video/image decoding, and audio mixing, and encodes each segment as H.264.
|
|
387
|
+
5. Segments are concatenated in timeline order without re-encoding and the global audio mix is muxed into the final MP4.
|
|
329
388
|
|
|
330
|
-
|
|
389
|
+
Composition concurrency changes scheduling only. Timeline workers process independent segments; output order and captured composition frames remain deterministic. `check` and `still` use the same clip capture and composite path as their target.
|
|
331
390
|
|
|
332
|
-
For the same composition revision, props, assets, frame, and pinned runtime/browser environment, the
|
|
391
|
+
For the same composition or timeline revision, props, assets, segment plan, frame, and pinned runtime/browser/FFmpeg environment, the captured composition frames and deterministic composite are identical. Encoded bytes are not a reproducibility guarantee, even on the same machine; they depend on the FFmpeg build, encoder threading, and runtime environment. Preview composites in Chromium while timeline production composites with FFmpeg, so small differences in scaling, alpha, and video color conversion are expected; parity is guaranteed at clip capture.
|
|
333
392
|
|
|
334
|
-
For a repeatable local measurement of seek, capture, encode, and full-render time, run `pnpm bench`. See `bench/README.md` for the measurement recipe
|
|
393
|
+
For a repeatable local measurement of seek, capture, encode, and full-render time, run `pnpm bench`. See `bench/README.md` for the measurement recipe. Timeline performance is tracked separately with the long-form benchmarks; the representative target and worst-case measurements are in [ADR 0012](./docs/adr/0012-render-timelines-with-segmented-lossless-clip-intermediates.md).
|
|
335
394
|
|
|
336
395
|
## Current limits
|
|
337
396
|
|
|
338
397
|
Deferred from this implementation slice:
|
|
339
398
|
|
|
399
|
+
- Timeline transitions, effects, keyframes, blend modes, masks, and audio fades
|
|
400
|
+
- Full visual timeline editing, drag-and-drop, trim handles, and undo/redo
|
|
401
|
+
- Timeline-level variables and prop overrides inside nested timelines
|
|
402
|
+
- Per-timeline formats and frame-rate conversion/retiming
|
|
403
|
+
- Persistent clip caches; timeline scratch intermediates are cleaned after a render
|
|
404
|
+
- Hardware encoders, GPU compositing, and non-H.264 final output
|
|
340
405
|
- Built-in subtitle rendering, automatic caption burn-in, and declarative karaoke elements
|
|
341
406
|
- Nested `data-fy-repeat` scopes and a general template language
|
|
342
407
|
- Declarative JSON Schema validation
|
|
@@ -344,11 +409,11 @@ Deferred from this implementation slice:
|
|
|
344
409
|
- `qwentts.cpp` reference-audio voice cloning and VoiceDesign
|
|
345
410
|
- Automatic long-text chunking, streaming synthesis, and alternative generated-audio formats
|
|
346
411
|
- Automatic upstream builds, model downloads, environment repair, and hosted rendering
|
|
347
|
-
-
|
|
412
|
+
- Batch still capture and general image-sequence output
|
|
348
413
|
- React or other UI-framework adapters, plugin APIs, and agent-only runtimes
|
|
349
414
|
- TypeScript transpilation for project compositions and npm dependency optimization
|
|
350
415
|
|
|
351
|
-
See the [design baseline](./docs/v0.1-design.md) for the full product contract, [ADR 0007](./docs/adr/0007-make-html-documents-the-composition-unit.md) for the HTML authoring decision, and the [domain glossary](./GLOSSARY.md) for terminology.
|
|
416
|
+
See the [design baseline](./docs/v0.1-design.md) for the full product contract, [ADR 0007](./docs/adr/0007-make-html-documents-the-composition-unit.md) for the HTML authoring decision, [ADR 0010](./docs/adr/0010-make-timelines-the-render-unit.md) for timelines, [ADR 0012](./docs/adr/0012-render-timelines-with-segmented-lossless-clip-intermediates.md) for segmented rendering, and the [domain glossary](./GLOSSARY.md) for terminology.
|
|
352
417
|
|
|
353
418
|
## Repository layout
|
|
354
419
|
|
|
@@ -368,7 +433,7 @@ See the [design baseline](./docs/v0.1-design.md) for the full product contract,
|
|
|
368
433
|
- `skills/`: official host-neutral agent skill
|
|
369
434
|
- `examples/demo/`: the 30-second reference composition
|
|
370
435
|
- `examples/webgl/`: the focused WebGL shader example
|
|
371
|
-
- `examples/video/`:
|
|
436
|
+
- `examples/video/`: embedded-video compositions and WebM/MP4 timelines with video clips under a transparent HTML overlay
|
|
372
437
|
- `bench/fixtures/`: controlled HTML compositions for the render benchmark
|
|
373
438
|
- `scripts/`: benchmark, release checks, versioning, and git-hook installation
|
|
374
439
|
- `docs/adr/`: architecture decision records
|