frameyard 0.2.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +102 -0
- package/GLOSSARY.md +58 -2
- package/README.md +185 -95
- package/dist/bindings.d.ts +42 -0
- package/dist/bindings.d.ts.map +1 -0
- package/dist/bindings.js +144 -0
- package/dist/bindings.js.map +1 -0
- package/dist/cli.js +37 -14
- package/dist/cli.js.map +1 -1
- package/dist/contracts.d.ts +73 -47
- package/dist/contracts.d.ts.map +1 -1
- package/dist/contracts.js +14 -159
- package/dist/contracts.js.map +1 -1
- package/dist/errors.d.ts +15 -4
- package/dist/errors.d.ts.map +1 -1
- package/dist/errors.js +8 -0
- package/dist/errors.js.map +1 -1
- package/dist/html/index.d.ts +5 -0
- package/dist/html/index.d.ts.map +1 -0
- package/dist/html/index.js +3 -0
- package/dist/html/index.js.map +1 -0
- package/dist/html/parse.d.ts +30 -0
- package/dist/html/parse.d.ts.map +1 -0
- package/dist/html/parse.js +80 -0
- package/dist/html/parse.js.map +1 -0
- package/dist/html/validate.d.ts +14 -0
- package/dist/html/validate.d.ts.map +1 -0
- package/dist/html/validate.js +931 -0
- package/dist/html/validate.js.map +1 -0
- package/dist/index.d.ts +2 -2
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +1 -1
- package/dist/index.js.map +1 -1
- package/dist/preview/main.d.ts +12 -3
- package/dist/preview/main.d.ts.map +1 -1
- package/dist/preview/main.js +208 -85
- package/dist/preview/main.js.map +1 -1
- package/dist/project.d.ts +2 -0
- package/dist/project.d.ts.map +1 -1
- package/dist/project.js +4 -0
- package/dist/project.js.map +1 -1
- package/dist/renderer/audio.d.ts +5 -1
- package/dist/renderer/audio.d.ts.map +1 -1
- package/dist/renderer/audio.js +4 -0
- package/dist/renderer/audio.js.map +1 -1
- package/dist/renderer/browser.d.ts +12 -1
- package/dist/renderer/browser.d.ts.map +1 -1
- package/dist/renderer/browser.js +69 -11
- package/dist/renderer/browser.js.map +1 -1
- package/dist/renderer/index.d.ts +30 -5
- package/dist/renderer/index.d.ts.map +1 -1
- package/dist/renderer/index.js +204 -26
- package/dist/renderer/index.js.map +1 -1
- package/dist/runtime/bindings.d.ts +11 -0
- package/dist/runtime/bindings.d.ts.map +1 -0
- package/dist/runtime/bindings.js +164 -0
- package/dist/runtime/bindings.js.map +1 -0
- package/dist/runtime/diagnostics.d.ts +4 -0
- package/dist/runtime/diagnostics.d.ts.map +1 -0
- package/dist/runtime/diagnostics.js +107 -0
- package/dist/runtime/diagnostics.js.map +1 -0
- package/dist/runtime/host.d.ts +12 -0
- package/dist/runtime/host.d.ts.map +1 -0
- package/dist/runtime/host.js +701 -0
- package/dist/runtime/host.js.map +1 -0
- package/dist/runtime/index.d.ts +0 -26
- package/dist/runtime/index.d.ts.map +1 -1
- package/dist/runtime/index.js +0 -286
- package/dist/runtime/index.js.map +1 -1
- package/dist/runtime/media.d.ts +16 -0
- package/dist/runtime/media.d.ts.map +1 -0
- package/dist/runtime/media.js +307 -0
- package/dist/runtime/media.js.map +1 -0
- package/dist/scaffold.d.ts.map +1 -1
- package/dist/scaffold.js +169 -78
- package/dist/scaffold.js.map +1 -1
- package/dist/server/composition-html.d.ts +20 -0
- package/dist/server/composition-html.d.ts.map +1 -0
- package/dist/server/composition-html.js +45 -0
- package/dist/server/composition-html.js.map +1 -0
- package/dist/server/preview-html.d.ts.map +1 -1
- package/dist/server/preview-html.js +19 -14
- package/dist/server/preview-html.js.map +1 -1
- package/dist/server/server.d.ts.map +1 -1
- package/dist/server/server.js +94 -6
- package/dist/server/server.js.map +1 -1
- package/dist/version.d.ts +1 -1
- package/dist/version.js +1 -1
- package/docs/adr/0001-keep-composition-authoring-web-native.md +2 -0
- package/docs/adr/0003-render-browser-frames-offline.md +2 -0
- package/docs/adr/0005-keep-parallel-rendering-opt-in-and-frame-ordered.md +3 -0
- package/docs/adr/0006-seek-embedded-media-from-the-frame-clock.md +11 -0
- package/docs/adr/0007-make-html-documents-the-composition-unit.md +31 -0
- package/docs/migration-0.4.md +279 -0
- package/docs/v0.1-design.md +82 -57
- package/examples/demo/README.md +7 -3
- package/examples/demo/compositions/demo.controller.js +120 -0
- package/examples/demo/compositions/demo.html +253 -0
- package/examples/demo/video.config.js +1 -1
- package/package.json +9 -3
- package/skills/frameyard/README.md +4 -0
- package/skills/frameyard/SKILL.md +41 -11
- package/skills/frameyard/references/audio-assets.md +25 -14
- package/skills/frameyard/references/bindings.md +73 -0
- package/skills/frameyard/references/common-mistakes.md +59 -0
- package/skills/frameyard/references/controllers.md +126 -0
- package/skills/frameyard/references/deterministic-authoring.md +160 -41
- package/skills/frameyard/references/diagnostics.md +90 -0
- package/skills/frameyard/references/failure-recovery.md +38 -12
- package/skills/frameyard/references/json-results.md +43 -8
- package/skills/frameyard/references/subtitles-karaoke.md +10 -18
- package/examples/demo/compositions/demo.js +0 -206
|
@@ -0,0 +1,279 @@
|
|
|
1
|
+
# Migrating to Frameyard 0.4
|
|
2
|
+
|
|
3
|
+
0.4 replaces JavaScript composition modules with declarative HTML documents. The CLI commands, props files, audio and transcript asset formats, rendering pipeline, and determinism guarantees are unchanged. This guide maps the old authoring surface to the new one.
|
|
4
|
+
|
|
5
|
+
## 1. Change the config entry
|
|
6
|
+
|
|
7
|
+
Before:
|
|
8
|
+
|
|
9
|
+
```js
|
|
10
|
+
export default {
|
|
11
|
+
compositions: {
|
|
12
|
+
hello: "./compositions/hello.js",
|
|
13
|
+
},
|
|
14
|
+
};
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
After:
|
|
18
|
+
|
|
19
|
+
```js
|
|
20
|
+
export default {
|
|
21
|
+
compositions: {
|
|
22
|
+
hello: "./compositions/hello.html",
|
|
23
|
+
},
|
|
24
|
+
};
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
Entries must be `.html`. There is still no directory scanning.
|
|
28
|
+
|
|
29
|
+
## 2. Move metadata and defaults into the document
|
|
30
|
+
|
|
31
|
+
Before:
|
|
32
|
+
|
|
33
|
+
```js
|
|
34
|
+
export default defineComposition({
|
|
35
|
+
width: 1280,
|
|
36
|
+
height: 720,
|
|
37
|
+
fps: 30,
|
|
38
|
+
durationInFrames: 150,
|
|
39
|
+
defaultProps: { title: "Hello", accent: "#b8f36b" },
|
|
40
|
+
// ...
|
|
41
|
+
});
|
|
42
|
+
```
|
|
43
|
+
|
|
44
|
+
After:
|
|
45
|
+
|
|
46
|
+
```html
|
|
47
|
+
<fy-composition width="1280" height="720" fps="30" duration-in-frames="150">
|
|
48
|
+
<!-- composition content -->
|
|
49
|
+
</fy-composition>
|
|
50
|
+
|
|
51
|
+
<script type="application/json" data-fy="props">
|
|
52
|
+
{ "title": "Hello", "accent": "#b8f36b" }
|
|
53
|
+
</script>
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
## 3. Move markup out of `mount()`
|
|
57
|
+
|
|
58
|
+
Before:
|
|
59
|
+
|
|
60
|
+
```js
|
|
61
|
+
mount({ container }) {
|
|
62
|
+
container.innerHTML = "<h1 id='title'></h1>";
|
|
63
|
+
title = container.querySelector("#title");
|
|
64
|
+
}
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
After:
|
|
68
|
+
|
|
69
|
+
```html
|
|
70
|
+
<h1 data-fy-text="title"></h1>
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
The controller remains available for DOM that must be created per frame or from data that CSS cannot express.
|
|
74
|
+
|
|
75
|
+
## 4. Replace `update()` with bindings and CSS
|
|
76
|
+
|
|
77
|
+
Before:
|
|
78
|
+
|
|
79
|
+
```js
|
|
80
|
+
update({ frame, props }) {
|
|
81
|
+
title.textContent = props.title;
|
|
82
|
+
const intro = Math.min(1, frame / 24);
|
|
83
|
+
title.style.opacity = String(intro);
|
|
84
|
+
title.style.transform = `translateY(${(1 - intro) * 28}px)`;
|
|
85
|
+
sweep.style.width = `${(frame / 149) * 100}%`;
|
|
86
|
+
}
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
After:
|
|
90
|
+
|
|
91
|
+
```html
|
|
92
|
+
<h1 class="title"
|
|
93
|
+
data-fy-text="title"
|
|
94
|
+
data-fy-anim-start="0"
|
|
95
|
+
data-fy-anim-duration="24"></h1>
|
|
96
|
+
<div class="sweep" style="width: calc(var(--fy-progress) * 100%)"></div>
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
```css
|
|
100
|
+
.title {
|
|
101
|
+
animation: fade-up 1s ease-out both;
|
|
102
|
+
}
|
|
103
|
+
|
|
104
|
+
@keyframes fade-up {
|
|
105
|
+
from { opacity: 0; transform: translateY(28px); }
|
|
106
|
+
to { opacity: 1; transform: translateY(0); }
|
|
107
|
+
}
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
Use `data-fy-anim-start`, `data-fy-anim-duration`, and `data-fy-anim-easing` for frame-based scheduling. Use `--fy-frame`, `--fy-time`, `--fy-progress`, `--fy-fps`, and `--fy-duration` for computed CSS values. If the motion cannot be expressed in CSS, move it to the controller.
|
|
111
|
+
|
|
112
|
+
## 5. Replace prop-driven DOM with bindings
|
|
113
|
+
|
|
114
|
+
| Old | New |
|
|
115
|
+
| --- | --- |
|
|
116
|
+
| `el.textContent = props.title` | `data-fy-text="title"` |
|
|
117
|
+
| `el.setAttribute("src", props.image)` | `data-fy-attr-src="image"` |
|
|
118
|
+
| `el.classList.toggle("active", props.isActive)` | `data-fy-class-active="isActive"` |
|
|
119
|
+
| `if (props.show) ...` | `data-fy-if="show"` |
|
|
120
|
+
| `props.items.map(...)` | `<template data-fy-repeat="items">...</template>` |
|
|
121
|
+
| `container.style.setProperty("--accent", props.accent)` | `style="color: var(--fy-prop-accent)"` |
|
|
122
|
+
|
|
123
|
+
Bindings resolve dot paths such as `user.name`. They apply once per load; props are immutable. `data-fy-repeat` provides `item` and `index` in the cloned scope and sets `--fy-index`; nested repeats are not supported.
|
|
124
|
+
|
|
125
|
+
## 6. Move audio and transcripts to HTML
|
|
126
|
+
|
|
127
|
+
Before:
|
|
128
|
+
|
|
129
|
+
```js
|
|
130
|
+
audio: [
|
|
131
|
+
{
|
|
132
|
+
src: "assets/voice.wav",
|
|
133
|
+
startFrame: 30,
|
|
134
|
+
trimStart: 0.25,
|
|
135
|
+
trimDuration: 4.2,
|
|
136
|
+
volume: 0.8,
|
|
137
|
+
},
|
|
138
|
+
],
|
|
139
|
+
transcripts: [
|
|
140
|
+
{ src: "assets/voice.transcript.json", words: true },
|
|
141
|
+
],
|
|
142
|
+
```
|
|
143
|
+
|
|
144
|
+
After:
|
|
145
|
+
|
|
146
|
+
```html
|
|
147
|
+
<audio src="assets/voice.wav"
|
|
148
|
+
data-fy-start="30"
|
|
149
|
+
data-fy-trim="0.25"
|
|
150
|
+
data-fy-duration="4.2"
|
|
151
|
+
data-fy-volume="0.8"></audio>
|
|
152
|
+
|
|
153
|
+
<fy-transcript src="assets/voice.transcript.json" words></fy-transcript>
|
|
154
|
+
```
|
|
155
|
+
|
|
156
|
+
Audio, video, and `<fy-transcript>` `src` values resolve from the project root. The controller `src` and ordinary HTML assets (`img`, stylesheets) resolve relative to the composition document.
|
|
157
|
+
|
|
158
|
+
## 7. Rename media attributes
|
|
159
|
+
|
|
160
|
+
All `data-frameyard-*` media attributes become `data-fy-*`:
|
|
161
|
+
|
|
162
|
+
| Old | New |
|
|
163
|
+
| --- | --- |
|
|
164
|
+
| `data-frameyard-start` | `data-fy-start` |
|
|
165
|
+
| `data-frameyard-trim` | `data-fy-trim` |
|
|
166
|
+
| `data-frameyard-duration` | `data-fy-duration` |
|
|
167
|
+
| `data-frameyard-volume` | `data-fy-volume` |
|
|
168
|
+
| `data-frameyard-audio` | `data-fy-audio` |
|
|
169
|
+
|
|
170
|
+
## 8. Move imperative code into a controller
|
|
171
|
+
|
|
172
|
+
Before:
|
|
173
|
+
|
|
174
|
+
```js
|
|
175
|
+
export default defineComposition({
|
|
176
|
+
// metadata, defaults, audio, transcripts
|
|
177
|
+
mount({ container, props }) { /* ... */ },
|
|
178
|
+
update({ frame, props }) { /* ... */ },
|
|
179
|
+
dispose() { /* ... */ },
|
|
180
|
+
validateProps(props) { /* ... */ },
|
|
181
|
+
});
|
|
182
|
+
```
|
|
183
|
+
|
|
184
|
+
After — inline for a small, single-use controller:
|
|
185
|
+
|
|
186
|
+
```html
|
|
187
|
+
<script type="text/fy-controller">
|
|
188
|
+
export default {
|
|
189
|
+
validateProps(props) { /* ... */ },
|
|
190
|
+
mount({ root, props }) { /* ... */ },
|
|
191
|
+
update({ frame, props, root }) { /* ... */ },
|
|
192
|
+
dispose() { /* ... */ },
|
|
193
|
+
};
|
|
194
|
+
</script>
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
After — external for a reusable or complex controller:
|
|
198
|
+
|
|
199
|
+
```html
|
|
200
|
+
<script type="text/fy-controller" src="./hello.controller.js"></script>
|
|
201
|
+
```
|
|
202
|
+
|
|
203
|
+
```js
|
|
204
|
+
export default {
|
|
205
|
+
validateProps(props) { /* ... */ },
|
|
206
|
+
mount({ root, props }) { /* ... */ },
|
|
207
|
+
update({ frame, props, root }) { /* ... */ },
|
|
208
|
+
dispose() { /* ... */ },
|
|
209
|
+
};
|
|
210
|
+
```
|
|
211
|
+
|
|
212
|
+
A composition has exactly one controller. Inline keeps the composition in one file and is the default for small, single-use controllers. Extract to an external module when the controller grows, is reused, needs separate imports or tests, or drives Canvas 2D, WebGL, or karaoke. The host serves inline source through a virtual module URL, so relative imports and `import.meta.url` behave as they do in an external module.
|
|
213
|
+
|
|
214
|
+
Inline source is raw text inside `<script>`; a literal `</script>` in a string must be escaped as `<\/script>`. If the controller contains HTML strings or grows large, prefer an external module.
|
|
215
|
+
|
|
216
|
+
The controller is for Canvas 2D, WebGL, karaoke, arrays that exceed `data-fy-repeat`, and per-frame computation CSS cannot express. It has no metadata, defaults, audio, or transcripts. `mount()` may be asynchronous; `update()` and `validateProps()` are synchronous and seek-safe.
|
|
217
|
+
|
|
218
|
+
## 9. Remove executable scripts and inline handlers
|
|
219
|
+
|
|
220
|
+
Only the controller, inline or external, may execute. Remove `<script type="module">` blocks and `on*` attributes; import libraries from the controller instead. JSON data blocks (`type="application/json"`) and the `text/fy-controller` block are allowed. `javascript:` URLs and unsupported visual elements inside the composition (`iframe`, `object`, `embed`) are rejected. The preview's own host iframe is not a composition element.
|
|
221
|
+
|
|
222
|
+
## 10. Validation and diagnostics
|
|
223
|
+
|
|
224
|
+
`check`, `still`, and `render` run a Node-side static composition validation before Chromium. It catches metadata, root/controller count, default-props JSON, unknown `data-fy-*` attributes, missing binding paths, repeat misuse, animation attributes, executable scripts, inline handlers, `javascript:` URLs, unsupported visual elements, and missing audio/transcript references.
|
|
225
|
+
|
|
226
|
+
Errors and warnings are structured:
|
|
227
|
+
|
|
228
|
+
```json
|
|
229
|
+
{
|
|
230
|
+
"code": "BINDING_PATH_MISSING",
|
|
231
|
+
"message": "data-fy-text references prop \"titel\", which is not in the resolved props.",
|
|
232
|
+
"details": {
|
|
233
|
+
"path": "compositions/hello.html",
|
|
234
|
+
"line": 12,
|
|
235
|
+
"column": 5,
|
|
236
|
+
"element": "h1",
|
|
237
|
+
"attribute": "data-fy-text",
|
|
238
|
+
"value": "titel",
|
|
239
|
+
"suggestion": "title"
|
|
240
|
+
}
|
|
241
|
+
}
|
|
242
|
+
```
|
|
243
|
+
|
|
244
|
+
`check` exits nonzero on static errors without launching Chromium. Warnings allow frame evaluation but should be treated as actionable. `check`, `still`, and `render` JSON results now include `entry`, `controller`, and top-level `warnings`.
|
|
245
|
+
|
|
246
|
+
Props domain validation remains controller-only in 0.4. If an HTML-only composition needs required or typed props, add a tiny inline controller with `validateProps`:
|
|
247
|
+
|
|
248
|
+
```html
|
|
249
|
+
<script type="text/fy-controller">
|
|
250
|
+
export default {
|
|
251
|
+
validateProps(props) {
|
|
252
|
+
return typeof props.title === "string" ? [] : ["title must be a string"];
|
|
253
|
+
},
|
|
254
|
+
};
|
|
255
|
+
</script>
|
|
256
|
+
```
|
|
257
|
+
|
|
258
|
+
## 11. Update preview and render expectations
|
|
259
|
+
|
|
260
|
+
No command changes. Preview loads the composition in a same-origin iframe; `check`, `still`, and `render` navigate directly to the same document. The composition host is injected by the server, so keep composition entries listed in `video.config.js` and run through `frameyard dev` or `frameyard render`, not `file://`.
|
|
261
|
+
|
|
262
|
+
## Migration checklist
|
|
263
|
+
|
|
264
|
+
- [ ] Change `video.config.js` entries to `.html`.
|
|
265
|
+
- [ ] Move metadata to `<fy-composition>`.
|
|
266
|
+
- [ ] Move `defaultProps` to the `data-fy="props"` JSON block.
|
|
267
|
+
- [ ] Move `mount()` markup into the document.
|
|
268
|
+
- [ ] Replace text, attribute, and class assignments with `data-fy-*` bindings.
|
|
269
|
+
- [ ] Replace conditionals and loops with `data-fy-if` and `data-fy-repeat`.
|
|
270
|
+
- [ ] Use the controller for nested data; nested `data-fy-repeat` is rejected.
|
|
271
|
+
- [ ] Replace `update()` motion with CSS animations, `data-fy-anim-*`, and frame-state variables.
|
|
272
|
+
- [ ] Move Canvas 2D, WebGL, karaoke, and complex computation into a controller.
|
|
273
|
+
- [ ] Choose an inline or external controller; keep exactly one per composition.
|
|
274
|
+
- [ ] If props need domain validation, add an inline `validateProps` controller.
|
|
275
|
+
- [ ] Move `audio` and `transcripts` to `<audio>` and `<fy-transcript>`.
|
|
276
|
+
- [ ] Rename `data-frameyard-*` media attributes to `data-fy-*`.
|
|
277
|
+
- [ ] Remove executable scripts, inline handlers, `javascript:` URLs, and unsupported visual elements inside the composition.
|
|
278
|
+
- [ ] Run `check`; treat static errors and warnings as actionable.
|
|
279
|
+
- [ ] Inspect representative stills and render.
|
package/docs/v0.1-design.md
CHANGED
|
@@ -1,10 +1,10 @@
|
|
|
1
|
-
# Frameyard
|
|
1
|
+
# Frameyard Design Baseline
|
|
2
2
|
|
|
3
3
|
**Status:** Accepted
|
|
4
4
|
**Initial acceptance:** 2026-10-27
|
|
5
|
-
**Last design revision:** 2026-10-
|
|
5
|
+
**Last design revision:** 2026-10-30
|
|
6
6
|
|
|
7
|
-
This document records the decisions confirmed during
|
|
7
|
+
This document records the decisions confirmed during design sessions. It is the living product and architecture baseline; later changes revise this document explicitly rather than silently broaden the scope. The original v0.1 baseline has been revised through 0.4. Architecture decisions that changed the baseline are recorded in `docs/adr/`.
|
|
8
8
|
|
|
9
9
|
## 1. Product contract
|
|
10
10
|
|
|
@@ -12,10 +12,10 @@ Frameyard is a local-first, open-source framework for frontend developers and cr
|
|
|
12
12
|
|
|
13
13
|
The initial use cases are product demos, social clips, animated diagrams, data visualizations, karaoke-style subtitles, and personalized videos. Optional local speech synthesis and transcription produce assets for these workflows. Professional long-form editing is outside the target audience.
|
|
14
14
|
|
|
15
|
-
The core
|
|
15
|
+
The core success criterion is an end-to-end example in which a developer can:
|
|
16
16
|
|
|
17
|
-
1. Create a reusable 30-second, 1920×1080 composition.
|
|
18
|
-
2. Animate it with HTML, CSS, SVG,
|
|
17
|
+
1. Create a reusable 30-second, 1920×1080 HTML composition.
|
|
18
|
+
2. Animate it with HTML, CSS, SVG, and, where needed, a seek-safe controller for Canvas 2D and WebGL.
|
|
19
19
|
3. Supply JSON props and synchronized audio.
|
|
20
20
|
4. Preview and scrub it interactively.
|
|
21
21
|
5. Render it reproducibly to MP4 without dropped frames or timing drift.
|
|
@@ -24,31 +24,36 @@ The agent-enabled success criterion additionally requires an agent using the off
|
|
|
24
24
|
|
|
25
25
|
## 2. Project and authoring model
|
|
26
26
|
|
|
27
|
-
- Projects use ordinary HTML, CSS, static assets, and native JavaScript
|
|
28
|
-
- `video.config.js` explicitly lists composition
|
|
27
|
+
- Projects use ordinary HTML, CSS, static assets, and native JavaScript controllers, either inline in a composition document or in external modules.
|
|
28
|
+
- `video.config.js` explicitly lists `.html` composition entry points. There is no directory-scanning discovery convention.
|
|
29
29
|
- A project may contain multiple compositions, each with a unique ID.
|
|
30
|
-
-
|
|
31
|
-
-
|
|
32
|
-
- `
|
|
33
|
-
-
|
|
34
|
-
-
|
|
35
|
-
-
|
|
30
|
+
- A composition is a full HTML document with one `<fy-composition>` root. The root declares fixed `width`, `height`, `fps`, and `duration-in-frames` metadata. Props cannot change this metadata.
|
|
31
|
+
- Default props are a JSON block, `<script type="application/json" data-fy="props">`. A render's JSON props merge shallowly over the defaults, are validated as JSON, deep-frozen, and fixed for the render.
|
|
32
|
+
- Props bind into the document through explicit `data-fy-*` attributes: `data-fy-text`, `data-fy-attr-*`, `data-fy-class-*`, `data-fy-if`, and `data-fy-repeat`. Primitive top-level props are also exposed as `--fy-prop-<name>` CSS custom properties. Bindings apply once per load; props are immutable.
|
|
33
|
+
- CSS and the Web Animations API are the animation language. `data-fy-anim-start`, `data-fy-anim-duration`, and `data-fy-anim-easing` schedule one animation per element in composition frames.
|
|
34
|
+
- An optional composition controller is declared with `<script type="text/fy-controller">`. It is either inline source or an external module referenced by `src`; a composition has exactly one controller. It may export `mount({ root, props })`, `update({ frame, props, root })`, `dispose()`, and `validateProps(props)`; at least one is required, and `mount` and `update` are optional for validation-only controllers. It has no metadata, default props, audio, or transcripts. Inline keeps small compositions in one file; external is for reuse, tests, and complex logic.
|
|
35
|
+
- Inline controllers are the default for small, single-use controllers. Extract to an external module when the controller grows, is reused, needs separate imports or tests, or drives Canvas 2D, WebGL, or karaoke.
|
|
36
|
+
- The controller is the only executable code in a composition document. Other executable `<script>` elements and inline `on*` handlers are rejected. Libraries are imported as ES modules from the controller.
|
|
37
|
+
- `fy` is the authoring namespace: `fy-composition`, `fy-transcript`, `data-fy-*`, `--fy-*`, and `text/fy-controller`. `frameyard` remains the product, package, CLI, config, manifest, server-route, import-specifier, and error-code namespace.
|
|
38
|
+
- User controllers are native JavaScript. The framework itself is authored in TypeScript and ships declarations, but 0.4 does not transpile user TypeScript.
|
|
36
39
|
|
|
37
40
|
## 3. Deterministic execution
|
|
38
41
|
|
|
39
42
|
The only sources of animation state are the integer frame number and immutable props. Composition frame rate establishes the mapping between frames and time.
|
|
40
43
|
|
|
41
|
-
- The runtime owns the frame clock and deterministic randomness.
|
|
42
|
-
- CSS `@keyframes` and Web Animations API animations are paused and positioned from the shared frame clock.
|
|
43
|
-
-
|
|
44
|
+
- The runtime owns the frame clock and deterministic randomness. The controller imports `random("stable:key")` from `frameyard/runtime`.
|
|
45
|
+
- CSS `@keyframes` and Web Animations API animations are paused and positioned from the shared frame clock. `data-fy-anim-*` attributes schedule one animation per element in composition frames; the runtime maps frames to milliseconds and sets the corresponding animation delay and duration.
|
|
46
|
+
- The runtime sets `--fy-frame`, `--fy-time`, `--fy-progress`, `--fy-fps`, `--fy-duration`, and `--fy-duration-frames` on the composition root every seek. Authors use them for simple computed CSS values.
|
|
47
|
+
- JavaScript animation runs in the controller's `update()`, which must be synchronous and seek-safe. It cannot depend on previously visited frames.
|
|
48
|
+
- Declarative bindings apply once per load, before the controller mounts. Controller-created DOM is the controller's responsibility.
|
|
44
49
|
- Independent `requestAnimationFrame` loops, timers, CSS transitions, wall-clock reads, and unseeded randomness are not supported animation drivers.
|
|
45
|
-
-
|
|
46
|
-
-
|
|
47
|
-
-
|
|
48
|
-
-
|
|
49
|
-
-
|
|
50
|
-
-
|
|
51
|
-
-
|
|
50
|
+
- Development mode provides best-effort diagnostics for forbidden clocks and randomness in the controller; it does not claim to police every third-party library perfectly.
|
|
51
|
+
- Assets, remote data, images, fonts, generated audio, and transcripts are resolved during an explicit loading phase before frame production. Network requests are blocked during frame capture.
|
|
52
|
+
- AI model inference is never part of the loading phase, controller lifecycle, or frame production.
|
|
53
|
+
- The baseline supports DOM, CSS, SVG, Canvas 2D, WebGL, and frame-seeked embedded video. Canvas 2D and WebGL are drawn from the current frame by the controller.
|
|
54
|
+
- Embedded video and audio are declared in the composition HTML with `data-fy-*` attributes. The runtime loads local files during the loading phase, pauses them, and seeks video from the frame-derived source time before capture. An unmuted video contributes its audio track to the offline mix unless the composition opts out.
|
|
55
|
+
- WebGL 1 and WebGL 2 canvases are captured through the normal screenshot path. Production sessions pin software WebGL so pixel output does not depend on the host GPU.
|
|
56
|
+
- Compositions are trusted local developer code. Frameyard does not claim to render untrusted compositions safely.
|
|
52
57
|
|
|
53
58
|
### Reproducibility guarantee
|
|
54
59
|
|
|
@@ -58,7 +63,7 @@ Generated assets are covered once frozen. Frameyard does not promise that repeat
|
|
|
58
63
|
|
|
59
64
|
## 4. Preview and production rendering
|
|
60
65
|
|
|
61
|
-
Preview and production execute the same composition
|
|
66
|
+
Preview and production execute the same composition document and browser host runtime. They differ only in which component drives the frame clock and whether frames are captured.
|
|
62
67
|
|
|
63
68
|
The preview provides:
|
|
64
69
|
|
|
@@ -70,23 +75,30 @@ The preview provides:
|
|
|
70
75
|
|
|
71
76
|
It is a developer inspection tool, not a visual composition editor.
|
|
72
77
|
|
|
78
|
+
The preview server serves each composition HTML document at its project path and injects the framework import map and a host module into composition entries. The host reads the composition metadata, applies bindings, imports the controller from its `src` or from the inline block's virtual module URL, mounts it, and exposes a small `window.__fy` API. Preview loads the document in a same-origin iframe and drives that API. The iframe is not a security sandbox; compositions are trusted local code. The server runs the static composition validation described in §6 before serving a composition to the preview.
|
|
79
|
+
|
|
73
80
|
Production rendering:
|
|
74
81
|
|
|
75
82
|
1. Uses a pinned, Playwright-managed Chromium through the Chrome DevTools Protocol.
|
|
76
|
-
2.
|
|
77
|
-
3.
|
|
78
|
-
4.
|
|
79
|
-
5.
|
|
83
|
+
2. Navigates directly to the composition document, waits for the host, and loads the render's props.
|
|
84
|
+
3. Seeks explicitly to each requested frame.
|
|
85
|
+
4. Waits for composition readiness, embedded media decode, and browser paint.
|
|
86
|
+
5. Captures the frame.
|
|
87
|
+
6. Pipes frames in timeline order to FFmpeg for encoding and audio muxing.
|
|
88
|
+
|
|
89
|
+
`check` and `still` use the same host and capture path. `check` evaluates every frame without capturing or encoding. `still` captures one frame through the production path.
|
|
80
90
|
|
|
81
|
-
The
|
|
91
|
+
The renderer is sequential by default and has no frame cache. Optional `--concurrency <n>` rendering, with `n` from 1 to 16, opens up to `n` independent browser sessions, each loading the same composition document and props from the same preview server. Workers seek and capture assigned frames, while a single ordered writer pipes them to FFmpeg with bounded lookahead. Parallelism therefore changes scheduling only: every frame remains derivable from its integer frame number and immutable props, and output order is unchanged. The default stays at one session for predictable resource use, and the upper bound limits accidental browser-process and memory growth. Simple scenes target real-time preview, but there is no production rendering-speed guarantee; the CLI reports progress, current frame, and elapsed time.
|
|
82
92
|
|
|
83
93
|
## 5. Audio and output
|
|
84
94
|
|
|
85
|
-
Audio is a declarative timeline of
|
|
95
|
+
Audio is a declarative timeline of `<audio>` elements. Each element uses `data-fy-start` (composition frame), `data-fy-trim` (seconds from the source start), optional `data-fy-duration` (seconds to use), and `data-fy-volume` (0-1). Preview synchronizes this timeline in the browser; production mixes it offline with FFmpeg rather than treating live `<audio>` or Web Audio playback as the production source of truth. Unmuted embedded videos contribute their own audio tracks to the same mix, placed by the video's frame and trim attributes. Muting a video or setting `data-fy-audio="off"` keeps it visual-only.
|
|
96
|
+
|
|
97
|
+
Audio, video, and transcript `src` values resolve from the project root. The controller `src` and ordinary HTML asset URLs (`img`, stylesheets) resolve relative to the composition document.
|
|
86
98
|
|
|
87
99
|
Generated speech behaves like any other audio clip. Frameyard does not automatically time-stretch it, change composition duration, or regenerate it to fit. `check` reports clip durations and overrun warnings so users and agents can revise text, timing, or composition deliberately.
|
|
88
100
|
|
|
89
|
-
|
|
101
|
+
0.4 produces portable MP4 files with H.264 video and AAC audio. It exposes only a small set of validated encoding options, not arbitrary FFmpeg arguments. Single-frame PNG output through `still` is a debugging artifact, not general image-sequence support.
|
|
90
102
|
|
|
91
103
|
A render fails immediately on an uncaught exception, unhandled rejection, required-asset failure, or invalid composition output. Diagnostics identify the composition and, where applicable, the frame. The CLI exits nonzero, cleans temporary files, and never reports a partial video as successful. Existing output files are not overwritten unless explicitly requested.
|
|
92
104
|
|
|
@@ -94,36 +106,38 @@ A render fails immediately on an uncaught exception, unhandled rejection, requir
|
|
|
94
106
|
|
|
95
107
|
Project and composition commands are:
|
|
96
108
|
|
|
97
|
-
- `init`: scaffold a minimal project, including an agent guide (`AGENTS.md`) and a private `package.json` with pinned `dev`, `check`, `render`, and `still` scripts
|
|
109
|
+
- `init`: scaffold a minimal HTML composition project, including an agent guide (`AGENTS.md`) and a private `package.json` with pinned `dev`, `check`, `render`, and `still` scripts
|
|
98
110
|
- `dev`: start the preview
|
|
99
|
-
- `render <composition-id> --props props.json --out video.mp4`: render a composition
|
|
100
|
-
- `check <composition-id>`: validate configuration, props, assets, audio, and transcript requirements,
|
|
111
|
+
- `render <composition-id> --props props.json --out video.mp4`: render a composition, optionally with `--concurrency <n>` from 1 to 16
|
|
112
|
+
- `check <composition-id>`: run static composition validation, then validate configuration, props, assets, audio, and transcript requirements, load the composition, and evaluate every frame without capturing or encoding
|
|
101
113
|
- `still <composition-id> --frame <n>`: capture one requested frame as PNG through the production rendering path
|
|
102
114
|
- `doctor`: inspect core requirements and optional integrations without repairing or downloading anything
|
|
103
115
|
|
|
116
|
+
`check`, `still`, and `render` run a Node-side static composition validation before launching Chromium. It validates metadata, the composition root, the controller declaration, default props, binding attributes and paths, animation attributes, scripts and event handlers, unsupported visual elements, and audio/transcript references. Diagnostics carry stable codes and optional source locations; `check` exits nonzero on static errors without starting a browser. `dev` runs the same validation on load and surfaces its diagnostics in the preview.
|
|
117
|
+
|
|
104
118
|
Local asset-production commands are `tts` and `transcribe`. Every non-interactive command supports `--json` with stable result and error schemas. Structured results go to stdout; human diagnostics and inference progress go to stderr.
|
|
105
119
|
|
|
106
|
-
The built-in server serves native ES modules and static assets
|
|
120
|
+
The built-in server serves native ES modules and static assets, injects an import map and composition host into composition entries, and serves an inline controller through a virtual module URL under the composition's directory. It performs no dependency pre-bundling. Third-party libraries must initially be browser-ready ES modules referenced explicitly. Running through the CLI server is required; direct `file://` execution is unsupported for deterministic preview or rendering. A composition document can still be opened directly as static HTML/CSS.
|
|
107
121
|
|
|
108
|
-
|
|
122
|
+
Frameyard uses one repository and one publishable npm package with three internal layers:
|
|
109
123
|
|
|
110
|
-
1. Shared composition contracts
|
|
111
|
-
2. Browser runtime
|
|
124
|
+
1. Shared composition contracts and HTML binding/metadata parsing
|
|
125
|
+
2. Browser host runtime
|
|
112
126
|
3. Node-based CLI and renderer
|
|
113
127
|
|
|
114
128
|
These layers have clean internal boundaries but no public plugin API. FFmpeg is discovered on the user's PATH and checked for required capabilities, with actionable installation and version diagnostics. The project supports the current active Node.js LTS; Linux is the reference and CI platform, while macOS and Windows remain best-effort until covered by real rendering tests.
|
|
115
129
|
|
|
116
130
|
## 7. Official agent skill
|
|
117
131
|
|
|
118
|
-
Frameyard ships an optional, host-neutral agent skill with a `SKILL.md`, supporting references, and thin scripts. The npm package includes skill files matching its API version, and documentation explains host-specific installation or referencing.
|
|
132
|
+
Frameyard ships an optional, host-neutral agent skill with a `SKILL.md`, supporting references, and thin scripts. The npm package includes skill files matching its API version, and documentation explains host-specific installation or referencing. 0.4 does not add a skill-installer command.
|
|
119
133
|
|
|
120
134
|
The skill:
|
|
121
135
|
|
|
122
|
-
- Uses the normal project format, composition
|
|
123
|
-
- Does not introduce an agent-only runtime, MCP server, hosted control plane, or
|
|
136
|
+
- Uses the normal project format, HTML composition contract, optional controller, and CLI available to humans.
|
|
137
|
+
- Does not introduce an agent-only runtime, MCP server, hosted control plane, or separate rendering API.
|
|
124
138
|
- Triggers for Frameyard authoring and operation rather than claiming to be a general-purpose media-generation skill.
|
|
125
139
|
- Uses Frameyard's audio wrappers instead of invoking `qwen-tts` or `whisper-cli` directly.
|
|
126
|
-
- Includes references for deterministic authoring, audio assets, subtitle and karaoke recipes, JSON command results, and failure recovery.
|
|
140
|
+
- Includes references for deterministic HTML authoring, bindings, animation scheduling, controller rules, static validation, structured diagnostics, common authoring mistakes, audio assets, subtitle and karaoke recipes, JSON command results, and failure recovery.
|
|
127
141
|
- Invokes speech synthesis or transcription only when requested or explicitly approved as part of the brief.
|
|
128
142
|
- Does not overwrite or regenerate existing generated assets without permission.
|
|
129
143
|
- Runs `check`, inspects representative `still` images when the host supports image viewing, and discloses when visual inspection was impossible.
|
|
@@ -135,7 +149,7 @@ A successful agent workflow preserves and reports the source project, props, gen
|
|
|
135
149
|
|
|
136
150
|
The `tts` command integrates [`qwentts.cpp`](https://github.com/ServeurpersoCom/qwentts.cpp) through its external `qwen-tts` executable. It does not link the native library or require the upstream server.
|
|
137
151
|
|
|
138
|
-
|
|
152
|
+
0.4 supports CustomVoice mode only:
|
|
139
153
|
|
|
140
154
|
- A named speaker is required.
|
|
141
155
|
- An optional style instruction and language may be supplied.
|
|
@@ -161,19 +175,19 @@ The canonical artifact is a versioned Frameyard JSON document containing:
|
|
|
161
175
|
- Timed segments
|
|
162
176
|
- Optional word timings
|
|
163
177
|
|
|
164
|
-
All timestamps are integer milliseconds relative to the source audio, not composition frame numbers.
|
|
178
|
+
All timestamps are integer milliseconds relative to the source audio, not composition frame numbers. The controller converts them using the audio clip's start frame, source trim, and composition frame rate.
|
|
165
179
|
|
|
166
|
-
Compositions
|
|
180
|
+
Compositions declare required transcript assets with `<fy-transcript>` elements:
|
|
167
181
|
|
|
168
|
-
```
|
|
169
|
-
|
|
182
|
+
```html
|
|
183
|
+
<fy-transcript src="assets/voice.transcript.json" words></fy-transcript>
|
|
170
184
|
```
|
|
171
185
|
|
|
172
186
|
`check` and production validate declared transcript files before frame production. `words: true` requires measured word timings and fails when they are unavailable; it never triggers interpolation.
|
|
173
187
|
|
|
174
188
|
SRT and VTT are derived exports, not the canonical contract. Word-level karaoke requires actual word timings and fails validation when they are unavailable. Frameyard never fabricates word timings by interpolating segment timestamps and never executes Whisper's generated bash/FFmpeg karaoke script.
|
|
175
189
|
|
|
176
|
-
The official skill documents seek-safe subtitle and karaoke recipes. Compositions retain responsibility for visual presentation;
|
|
190
|
+
The official skill documents seek-safe subtitle and karaoke recipes. Compositions retain responsibility for visual presentation; 0.4 has no built-in subtitle renderer and does not automatically burn captions into videos. Karaoke presentation uses the optional controller.
|
|
177
191
|
|
|
178
192
|
## 10. Generated assets and inference operation
|
|
179
193
|
|
|
@@ -198,10 +212,17 @@ Release documentation must identify the exact `qwentts.cpp` and `whisper.cpp` re
|
|
|
198
212
|
|
|
199
213
|
Tests prioritize public behavior and end-to-end evidence over implementation-only unit coverage:
|
|
200
214
|
|
|
201
|
-
-
|
|
215
|
+
- HTML compositions validate metadata, default props, bindings, conditionals, repeats, and controller declarations, including inline/external parity and the exactly-one-controller rule.
|
|
216
|
+
- Static composition validation catches metadata, root/controller count, binding, animation, script, element, and asset-reference errors before Chromium.
|
|
217
|
+
- Diagnostics carry stable codes and source locations; JSON results expose the composition entry, controller declaration, and top-level diagnostics.
|
|
218
|
+
- Sequential, individual, random-order, and parallel frame renders produce equivalent images.
|
|
202
219
|
- Determinism checks compare captured pixels or stills; encoded MP4 bytes are not a reproducibility oracle.
|
|
203
|
-
- Preview and renderer use the same runtime.
|
|
204
|
-
- CSS, JavaScript, SVG, and Canvas animations seek correctly.
|
|
220
|
+
- Preview and renderer use the same composition document and host runtime.
|
|
221
|
+
- CSS, controller JavaScript, SVG, and Canvas animations seek correctly.
|
|
222
|
+
- Frame-scheduled `data-fy-anim-*` animations and frame-state CSS variables produce the expected frames.
|
|
223
|
+
- Embedded video and audio frames seek deterministically, including under parallel capture, and their audio appears in the offline mix.
|
|
224
|
+
- WebGL canvases produce deterministic stills and rendered frames under the pinned software renderer.
|
|
225
|
+
- Executable scripts other than the inline or external controller, inline event handlers, `javascript:` URLs, and unsupported visual elements are rejected.
|
|
205
226
|
- Asset and runtime failures produce nonzero CLI exits.
|
|
206
227
|
- Rendered MP4 files have the expected dimensions, frame count, duration, and audio stream.
|
|
207
228
|
- Golden-frame tests cover representative visual behavior.
|
|
@@ -210,7 +231,7 @@ Tests prioritize public behavior and end-to-end evidence over implementation-onl
|
|
|
210
231
|
- Real-model integration tests are explicitly opt-in and use documented upstream revisions.
|
|
211
232
|
- Agent-workflow evaluations verify source preservation, provenance, validation, still inspection behavior, failure reporting, and final renders.
|
|
212
233
|
|
|
213
|
-
## 12. Deferred
|
|
234
|
+
## 12. Deferred
|
|
214
235
|
|
|
215
236
|
These are scope boundaries, not permanent rejections:
|
|
216
237
|
|
|
@@ -218,15 +239,19 @@ These are scope boundaries, not permanent rejections:
|
|
|
218
239
|
- Cloud generative-AI services, text-to-video generation, and image generation
|
|
219
240
|
- `qwentts.cpp` reference-audio voice cloning and VoiceDesign
|
|
220
241
|
- Automatic long-text chunking, streaming speech synthesis, and alternative generated-audio formats
|
|
221
|
-
- Built-in subtitle rendering
|
|
242
|
+
- Built-in subtitle rendering, automatic caption burn-in, and declarative karaoke elements
|
|
243
|
+
- Nested `data-fy-repeat` scopes and a general template language
|
|
244
|
+
- Declarative JSON Schema validation
|
|
245
|
+
- Arbitrary executable scripts in composition HTML
|
|
222
246
|
- Automatic upstream builds, model downloads, weight redistribution, and environment repair
|
|
223
247
|
- Agent-only runtimes, MCP servers, hosted agent control planes, and host-specific skill installers
|
|
224
248
|
- React or other UI-framework adapters in the core
|
|
225
249
|
- Public plugin APIs and a broad integration ecosystem
|
|
226
250
|
- Dynamic composition metadata derived from props
|
|
227
|
-
-
|
|
251
|
+
- Live streams, remote media, and other visual sources that cannot be deterministically seeked
|
|
228
252
|
- Dynamic per-frame asset loading or AI inference
|
|
229
|
-
-
|
|
253
|
+
- Frame caches
|
|
254
|
+
- Batch still capture; agents can run `still` once per representative frame
|
|
230
255
|
- Additional final-video formats, general image-sequence output, and arbitrary FFmpeg argument passthrough
|
|
231
256
|
- Built-in TypeScript transpilation and npm dependency optimization
|
|
232
257
|
- Rendering untrusted compositions
|
|
@@ -235,4 +260,4 @@ These are scope boundaries, not permanent rejections:
|
|
|
235
260
|
|
|
236
261
|
## 13. Release posture
|
|
237
262
|
|
|
238
|
-
Frameyard is MIT-licensed. Initial releases use `0.x` versioning, and composition, configuration, CLI, JSON-result, transcript, and skill-contract changes are documented explicitly until those contracts stabilize.
|
|
263
|
+
Frameyard is MIT-licensed. Initial releases use `0.x` versioning, and composition, configuration, CLI, JSON-result, transcript, and skill-contract changes are documented explicitly until those contracts stabilize. 0.4.0 is a breaking authoring-model release: HTML documents replace JavaScript composition modules, and the `fy` namespace replaces `data-frameyard-*` authoring attributes. See [ADR 0007](./adr/0007-make-html-documents-the-composition-unit.md) and the [0.4 migration guide](./migration-0.4.md).
|
package/examples/demo/README.md
CHANGED
|
@@ -1,12 +1,16 @@
|
|
|
1
1
|
# Frameyard demo
|
|
2
2
|
|
|
3
|
-
A 30-second, 1920×1080 composition that exercises the
|
|
3
|
+
A 30-second, 1920×1080 HTML composition that exercises the current core:
|
|
4
4
|
|
|
5
5
|
- HTML and CSS layout, including seeked CSS keyframe animation
|
|
6
|
-
- deterministic Canvas 2D starfield
|
|
6
|
+
- deterministic Canvas 2D starfield in an external controller
|
|
7
7
|
- SVG path drawing and data bars updated from the frame
|
|
8
8
|
- JSON props for the title, subtitle, and accent color
|
|
9
|
-
-
|
|
9
|
+
- an `<audio>` timeline with repeated local chime clips
|
|
10
|
+
- frame-state CSS variables for the progress bar
|
|
11
|
+
|
|
12
|
+
The composition lives in `compositions/demo.html`; the optional controller is
|
|
13
|
+
`compositions/demo.controller.js`.
|
|
10
14
|
|
|
11
15
|
From the repository root:
|
|
12
16
|
|