frameyard 0.2.0 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +102 -0
- package/GLOSSARY.md +58 -2
- package/README.md +185 -95
- package/dist/bindings.d.ts +42 -0
- package/dist/bindings.d.ts.map +1 -0
- package/dist/bindings.js +144 -0
- package/dist/bindings.js.map +1 -0
- package/dist/cli.js +37 -14
- package/dist/cli.js.map +1 -1
- package/dist/contracts.d.ts +73 -47
- package/dist/contracts.d.ts.map +1 -1
- package/dist/contracts.js +14 -159
- package/dist/contracts.js.map +1 -1
- package/dist/errors.d.ts +15 -4
- package/dist/errors.d.ts.map +1 -1
- package/dist/errors.js +8 -0
- package/dist/errors.js.map +1 -1
- package/dist/html/index.d.ts +5 -0
- package/dist/html/index.d.ts.map +1 -0
- package/dist/html/index.js +3 -0
- package/dist/html/index.js.map +1 -0
- package/dist/html/parse.d.ts +30 -0
- package/dist/html/parse.d.ts.map +1 -0
- package/dist/html/parse.js +80 -0
- package/dist/html/parse.js.map +1 -0
- package/dist/html/validate.d.ts +14 -0
- package/dist/html/validate.d.ts.map +1 -0
- package/dist/html/validate.js +931 -0
- package/dist/html/validate.js.map +1 -0
- package/dist/index.d.ts +2 -2
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +1 -1
- package/dist/index.js.map +1 -1
- package/dist/preview/main.d.ts +12 -3
- package/dist/preview/main.d.ts.map +1 -1
- package/dist/preview/main.js +208 -85
- package/dist/preview/main.js.map +1 -1
- package/dist/project.d.ts +2 -0
- package/dist/project.d.ts.map +1 -1
- package/dist/project.js +4 -0
- package/dist/project.js.map +1 -1
- package/dist/renderer/audio.d.ts +5 -1
- package/dist/renderer/audio.d.ts.map +1 -1
- package/dist/renderer/audio.js +4 -0
- package/dist/renderer/audio.js.map +1 -1
- package/dist/renderer/browser.d.ts +12 -1
- package/dist/renderer/browser.d.ts.map +1 -1
- package/dist/renderer/browser.js +69 -11
- package/dist/renderer/browser.js.map +1 -1
- package/dist/renderer/index.d.ts +30 -5
- package/dist/renderer/index.d.ts.map +1 -1
- package/dist/renderer/index.js +204 -26
- package/dist/renderer/index.js.map +1 -1
- package/dist/runtime/bindings.d.ts +11 -0
- package/dist/runtime/bindings.d.ts.map +1 -0
- package/dist/runtime/bindings.js +164 -0
- package/dist/runtime/bindings.js.map +1 -0
- package/dist/runtime/diagnostics.d.ts +4 -0
- package/dist/runtime/diagnostics.d.ts.map +1 -0
- package/dist/runtime/diagnostics.js +107 -0
- package/dist/runtime/diagnostics.js.map +1 -0
- package/dist/runtime/host.d.ts +12 -0
- package/dist/runtime/host.d.ts.map +1 -0
- package/dist/runtime/host.js +701 -0
- package/dist/runtime/host.js.map +1 -0
- package/dist/runtime/index.d.ts +0 -26
- package/dist/runtime/index.d.ts.map +1 -1
- package/dist/runtime/index.js +0 -286
- package/dist/runtime/index.js.map +1 -1
- package/dist/runtime/media.d.ts +16 -0
- package/dist/runtime/media.d.ts.map +1 -0
- package/dist/runtime/media.js +307 -0
- package/dist/runtime/media.js.map +1 -0
- package/dist/scaffold.d.ts.map +1 -1
- package/dist/scaffold.js +169 -78
- package/dist/scaffold.js.map +1 -1
- package/dist/server/composition-html.d.ts +20 -0
- package/dist/server/composition-html.d.ts.map +1 -0
- package/dist/server/composition-html.js +45 -0
- package/dist/server/composition-html.js.map +1 -0
- package/dist/server/preview-html.d.ts.map +1 -1
- package/dist/server/preview-html.js +19 -14
- package/dist/server/preview-html.js.map +1 -1
- package/dist/server/server.d.ts.map +1 -1
- package/dist/server/server.js +94 -6
- package/dist/server/server.js.map +1 -1
- package/dist/version.d.ts +1 -1
- package/dist/version.js +1 -1
- package/docs/adr/0001-keep-composition-authoring-web-native.md +2 -0
- package/docs/adr/0003-render-browser-frames-offline.md +2 -0
- package/docs/adr/0005-keep-parallel-rendering-opt-in-and-frame-ordered.md +3 -0
- package/docs/adr/0006-seek-embedded-media-from-the-frame-clock.md +11 -0
- package/docs/adr/0007-make-html-documents-the-composition-unit.md +31 -0
- package/docs/migration-0.4.md +279 -0
- package/docs/v0.1-design.md +82 -57
- package/examples/demo/README.md +7 -3
- package/examples/demo/compositions/demo.controller.js +120 -0
- package/examples/demo/compositions/demo.html +253 -0
- package/examples/demo/video.config.js +1 -1
- package/package.json +9 -3
- package/skills/frameyard/README.md +4 -0
- package/skills/frameyard/SKILL.md +41 -11
- package/skills/frameyard/references/audio-assets.md +25 -14
- package/skills/frameyard/references/bindings.md +73 -0
- package/skills/frameyard/references/common-mistakes.md +59 -0
- package/skills/frameyard/references/controllers.md +126 -0
- package/skills/frameyard/references/deterministic-authoring.md +160 -41
- package/skills/frameyard/references/diagnostics.md +90 -0
- package/skills/frameyard/references/failure-recovery.md +38 -12
- package/skills/frameyard/references/json-results.md +43 -8
- package/skills/frameyard/references/subtitles-karaoke.md +10 -18
- package/examples/demo/compositions/demo.js +0 -206
package/CHANGELOG.md
CHANGED
|
@@ -8,6 +8,108 @@ Frameyard uses `0.x` versions until the composition, configuration, CLI,
|
|
|
8
8
|
JSON-result, transcript, and skill contracts stabilize. Before `1.0.0`, a
|
|
9
9
|
contract change may ship in a minor release and is called out explicitly below.
|
|
10
10
|
|
|
11
|
+
## [0.4.0] - 2026-10-30
|
|
12
|
+
|
|
13
|
+
### Breaking
|
|
14
|
+
|
|
15
|
+
- **Composition authoring model:** compositions are HTML documents, not
|
|
16
|
+
JavaScript modules. `video.config.js` entries point to `.html` files, and the
|
|
17
|
+
public `defineComposition` authoring path is removed. The `mount → update →
|
|
18
|
+
dispose` lifecycle remains only as the optional controller contract.
|
|
19
|
+
- **Authoring namespace:** `fy-composition`, `fy-transcript`, `data-fy-*`,
|
|
20
|
+
`--fy-*`, and `text/fy-controller` replace the corresponding
|
|
21
|
+
`frameyard`/`data-frameyard-*` names. Existing embedded-media attributes move
|
|
22
|
+
from `data-frameyard-*` to `data-fy-*`.
|
|
23
|
+
- **Executable code:** only the optional controller, inline or external, may
|
|
24
|
+
execute. Other executable `<script>` elements and inline `on*` handlers are
|
|
25
|
+
rejected.
|
|
26
|
+
- **Validation and karaoke:** domain validation and subtitle/karaoke rendering
|
|
27
|
+
move to the optional controller; no built-in subtitle renderer is added.
|
|
28
|
+
- **Diagnostics:** warnings are structured diagnostic objects with `severity`,
|
|
29
|
+
`code`, `message`, and optional `details`, not plain strings. Error envelopes
|
|
30
|
+
gain optional `error.details` and top-level `diagnostics`. Unsupported visual
|
|
31
|
+
elements inside a composition (`iframe`, `object`, `embed`) and `javascript:`
|
|
32
|
+
URLs are rejected.
|
|
33
|
+
|
|
34
|
+
### Added
|
|
35
|
+
|
|
36
|
+
- Declarative HTML composition contract: `<fy-composition>` metadata, a
|
|
37
|
+
`data-fy="props"` default-props block, `data-fy-text`, `data-fy-attr-*`,
|
|
38
|
+
`data-fy-class-*`, `data-fy-if`, and `data-fy-repeat` bindings.
|
|
39
|
+
- Frame-scheduled CSS animation with `data-fy-anim-start`,
|
|
40
|
+
`data-fy-anim-duration`, and `data-fy-anim-easing`, plus `--fy-frame`,
|
|
41
|
+
`--fy-time`, `--fy-progress`, `--fy-fps`, and `--fy-duration` frame-state
|
|
42
|
+
variables.
|
|
43
|
+
- Optional composition controller, inline or external, with `mount`, `update`,
|
|
44
|
+
`dispose`, and `validateProps`. Inline controllers are extracted and served
|
|
45
|
+
through a virtual module URL so relative imports and `import.meta.url` work.
|
|
46
|
+
- Node-side static composition validation before Chromium in `check`, `still`,
|
|
47
|
+
and `render`, and on load in `dev`, with source-located diagnostics.
|
|
48
|
+
- Binding, animation, script, event-handler, unsupported-element, and
|
|
49
|
+
asset-reference validation.
|
|
50
|
+
- `entry`, `controller`, and top-level `diagnostics` in `check`, `still`, and
|
|
51
|
+
`render` JSON results.
|
|
52
|
+
- Agent-facing minimal skeleton, binding table, controller examples, and
|
|
53
|
+
common-mistakes guidance in the skill and scaffold.
|
|
54
|
+
- Injected composition host exposing `window.__fy` with metadata, resolved
|
|
55
|
+
audio/transcript clips, `load`, `seek`, `settle`, and `dispose`.
|
|
56
|
+
- HTML `<audio>` and `<fy-transcript>` declarations for the audio timeline and
|
|
57
|
+
transcript requirements.
|
|
58
|
+
- Same-origin iframe preview and a direct render/check/still path that share
|
|
59
|
+
the host API.
|
|
60
|
+
|
|
61
|
+
### Changed
|
|
62
|
+
|
|
63
|
+
- Preview loads each composition document in a same-origin iframe; `check`,
|
|
64
|
+
`still`, and `render` navigate directly to the composition URL. The separate
|
|
65
|
+
preview-page render path is removed.
|
|
66
|
+
- `check` fails fast on static errors without launching Chromium; warnings
|
|
67
|
+
still allow frame evaluation. Nested `data-fy-repeat` is rejected with a
|
|
68
|
+
clear diagnostic; nested data uses the controller.
|
|
69
|
+
- `init` scaffolds an HTML composition and no longer generates a JavaScript
|
|
70
|
+
composition module.
|
|
71
|
+
- The agent skill, README, design baseline, and project guides document the
|
|
72
|
+
HTML model. See [the 0.4 migration guide](./docs/migration-0.4.md).
|
|
73
|
+
|
|
74
|
+
### Removed
|
|
75
|
+
|
|
76
|
+
- JavaScript composition modules, `defineComposition`, and the
|
|
77
|
+
`data-frameyard-*` authoring attributes.
|
|
78
|
+
- The preview-page composition loading path used by the renderer.
|
|
79
|
+
|
|
80
|
+
[0.4.0]: https://github.com/vladiantio/frameyard/compare/v0.3.0...v0.4.0
|
|
81
|
+
|
|
82
|
+
## [0.3.0] - 2026-10-30
|
|
83
|
+
|
|
84
|
+
### Added
|
|
85
|
+
|
|
86
|
+
- Frame-accurate embedded video: local `<video>` elements are preloaded, paused,
|
|
87
|
+
and seeked from the composition frame, with `data-frameyard-start`,
|
|
88
|
+
`data-frameyard-trim`, `data-frameyard-duration`, and
|
|
89
|
+
`data-frameyard-volume` attributes.
|
|
90
|
+
- Automatic mixing of unmuted embedded-video audio into the offline FFmpeg
|
|
91
|
+
timeline, mirrored in preview. `muted` or `data-frameyard-audio="off"` keeps a
|
|
92
|
+
video visual-only.
|
|
93
|
+
- Deterministic WebGL capture for WebGL 1 and WebGL 2 canvases. Production
|
|
94
|
+
sessions pin SwiftShader so pixels do not vary with the host GPU.
|
|
95
|
+
- Media-settle step between `seek()` and capture for embedded video decode and
|
|
96
|
+
browser paint.
|
|
97
|
+
- Optional parallel frame capture with `render --concurrency <n>` (1-16).
|
|
98
|
+
Independent browser sessions seek and capture assigned frames while one
|
|
99
|
+
ordered writer feeds a single FFmpeg encoder. Sequential rendering remains the
|
|
100
|
+
default.
|
|
101
|
+
- `render --json` reports the effective `concurrency`.
|
|
102
|
+
- `MEDIA_INVALID` and `MEDIA_SEEK_FAILED` error codes for embedded-media
|
|
103
|
+
validation and seek failures.
|
|
104
|
+
|
|
105
|
+
### Changed
|
|
106
|
+
|
|
107
|
+
- CI and the release workflow share `pnpm check`, and `pnpm verify-release`
|
|
108
|
+
preflights the tag, version locations, changelog, branch, and working tree
|
|
109
|
+
before a release is pushed.
|
|
110
|
+
|
|
111
|
+
[0.3.0]: https://github.com/vladiantio/frameyard/compare/v0.2.0...v0.3.0
|
|
112
|
+
|
|
11
113
|
## [0.2.0] - 2026-10-29
|
|
12
114
|
|
|
13
115
|
### Added
|
package/GLOSSARY.md
CHANGED
|
@@ -19,13 +19,37 @@ The fixed dimensions, frame rate, and duration in frames that define a compositi
|
|
|
19
19
|
_Avoid_: Render options, props
|
|
20
20
|
|
|
21
21
|
**Composition lifecycle**:
|
|
22
|
-
The
|
|
22
|
+
The host-driven sequence that loads an HTML composition, seeks it to frames, and releases it.
|
|
23
23
|
_Avoid_: Render loop, playback lifecycle
|
|
24
24
|
|
|
25
|
+
**Controller lifecycle**:
|
|
26
|
+
The `mount → update → dispose` sequence a composition controller uses for per-frame computation.
|
|
27
|
+
_Avoid_: Composition lifecycle
|
|
28
|
+
|
|
25
29
|
**Composition revision**:
|
|
26
30
|
A specific version of a composition's source and metadata.
|
|
27
31
|
_Avoid_: Build, release
|
|
28
32
|
|
|
33
|
+
**HTML composition**:
|
|
34
|
+
A composition whose source is an HTML document.
|
|
35
|
+
_Avoid_: HTML template, page
|
|
36
|
+
|
|
37
|
+
**Composition controller**:
|
|
38
|
+
An optional JavaScript controller, either inline in the composition document or in an external module, that performs per-frame computation an HTML composition's declarative HTML and CSS cannot express.
|
|
39
|
+
_Avoid_: Composition module, plugin
|
|
40
|
+
|
|
41
|
+
**Inline controller**:
|
|
42
|
+
A composition controller whose source is embedded in the composition document.
|
|
43
|
+
_Avoid_: Embedded script
|
|
44
|
+
|
|
45
|
+
**External controller**:
|
|
46
|
+
A composition controller whose source is a separate JavaScript module referenced by the composition document.
|
|
47
|
+
_Avoid_: Controller file
|
|
48
|
+
|
|
49
|
+
**Declarative binding**:
|
|
50
|
+
An HTML attribute that maps a prop into text, an attribute, a class, or element presence.
|
|
51
|
+
_Avoid_: Template expression, interpolation
|
|
52
|
+
|
|
29
53
|
**Props**:
|
|
30
54
|
The immutable, JSON-serializable inputs supplied to a composition for one render.
|
|
31
55
|
_Avoid_: Options, settings, mutable state
|
|
@@ -48,6 +72,10 @@ _Avoid_: Playback-driven animation
|
|
|
48
72
|
The single source of animation time, derived from the current frame and the composition's frame rate.
|
|
49
73
|
_Avoid_: Wall clock, playback timer
|
|
50
74
|
|
|
75
|
+
**Frame-state variable**:
|
|
76
|
+
A CSS custom property whose value is derived from the current composition frame.
|
|
77
|
+
_Avoid_: Animation variable, clock variable
|
|
78
|
+
|
|
51
79
|
**Seek safety**:
|
|
52
80
|
The property that any frame can be produced correctly without depending on previously produced frames.
|
|
53
81
|
_Avoid_: History-dependent updates
|
|
@@ -64,6 +92,18 @@ _Avoid_: Real-time playback
|
|
|
64
92
|
The phase before frame production when required data and assets become available.
|
|
65
93
|
_Avoid_: Per-frame fetching
|
|
66
94
|
|
|
95
|
+
**Embedded media**:
|
|
96
|
+
A local video file placed in a composition whose presentation time is derived from the composition frame, with optional frame offset and source trim.
|
|
97
|
+
_Avoid_: Live stream, video track
|
|
98
|
+
|
|
99
|
+
**Media settle**:
|
|
100
|
+
The asynchronous step after a seek that waits for embedded video decode and browser paint before a frame is captured or checked.
|
|
101
|
+
_Avoid_: Playback, buffering
|
|
102
|
+
|
|
103
|
+
**WebGL canvas**:
|
|
104
|
+
A canvas redrawn from the current frame through WebGL, captured by the same screenshot path as DOM and Canvas 2D content.
|
|
105
|
+
_Avoid_: GPU animation, live render
|
|
106
|
+
|
|
67
107
|
### Audio
|
|
68
108
|
|
|
69
109
|
**Audio clip**:
|
|
@@ -76,18 +116,34 @@ _Avoid_: Soundtrack, audio track, final cut
|
|
|
76
116
|
|
|
77
117
|
### Rendering
|
|
78
118
|
|
|
119
|
+
**Composition host**:
|
|
120
|
+
The injected runtime that loads an HTML composition, applies its bindings, and exposes its frame clock to preview and render.
|
|
121
|
+
_Avoid_: Preview runtime, render runtime
|
|
122
|
+
|
|
79
123
|
**Preview**:
|
|
80
124
|
An interactive, frame-accurate inspection of a composition using the same composition behavior as a render.
|
|
81
125
|
_Avoid_: Visual editor
|
|
82
126
|
|
|
83
127
|
**Check**:
|
|
84
|
-
A validation operation that
|
|
128
|
+
A validation operation that first runs static composition validation, then loads a composition and evaluates every frame without capturing or encoding.
|
|
85
129
|
_Avoid_: Test render, dry run
|
|
86
130
|
|
|
131
|
+
**Static composition validation**:
|
|
132
|
+
The Node-side validation of a composition document's metadata, bindings, controller declaration, scripts, and asset references before browser frame production.
|
|
133
|
+
_Avoid_: Lint, preflight
|
|
134
|
+
|
|
135
|
+
**Composition diagnostic**:
|
|
136
|
+
A structured error or warning about a composition's source, metadata, bindings, controller, assets, or frame production, with a stable code and optional source location.
|
|
137
|
+
_Avoid_: Log message, error string
|
|
138
|
+
|
|
87
139
|
**Render**:
|
|
88
140
|
The production of one complete video from a composition and a fixed set of props.
|
|
89
141
|
_Avoid_: Recording, export
|
|
90
142
|
|
|
143
|
+
**Render concurrency**:
|
|
144
|
+
The optional number of independent browser sessions used to seek and capture frames during a render. Frames are still encoded in timeline order, and concurrency must not change visual state.
|
|
145
|
+
_Avoid_: Parallel composition, out-of-order output
|
|
146
|
+
|
|
91
147
|
**Video**:
|
|
92
148
|
The finished audiovisual output of a successful render.
|
|
93
149
|
_Avoid_: Composition, project
|
package/README.md
CHANGED
|
@@ -1,8 +1,8 @@
|
|
|
1
1
|
# Frameyard
|
|
2
2
|
|
|
3
|
-
Frameyard is a local-first framework for building short, parameterized videos with HTML, CSS, and JavaScript and rendering them deterministically.
|
|
3
|
+
Frameyard is a local-first framework for building short, parameterized videos with HTML, CSS, and an optional JavaScript controller, and rendering them deterministically.
|
|
4
4
|
|
|
5
|
-
The
|
|
5
|
+
The 0.4 design is implemented: HTML composition documents, declarative bindings, CSS animation scheduling, frame-state variables, inline and external controllers, the composition host, static validation with structured diagnostics, iframe preview, direct render/check/still sessions, audio and transcripts, deterministic Chromium capture, FFmpeg output, and the official agent skill.
|
|
6
6
|
|
|
7
7
|
## Requirements
|
|
8
8
|
|
|
@@ -28,7 +28,7 @@ npx frameyard dev
|
|
|
28
28
|
|
|
29
29
|
`dev` prints a local preview URL. Select a composition, edit its JSON props, scrub frames, and play the composition with its audio timeline.
|
|
30
30
|
|
|
31
|
-
The scaffold includes an `AGENTS.md` guide for coding agents working in the project, covering the
|
|
31
|
+
The scaffold includes an `AGENTS.md` guide for coding agents working in the project, covering the HTML authoring rules and the `check`, `still`, and `render` workflow. It also includes a `package.json` with `dev`, `check`, `render`, and `still` scripts pinned to the scaffolding CLI version, so `pnpm dev` or `npm run dev` works without a global install.
|
|
32
32
|
|
|
33
33
|
`npx frameyard` resolves the latest published version. Pin it with `npx frameyard@<version>`, or install the package locally, when a project needs a fixed CLI version.
|
|
34
34
|
|
|
@@ -45,107 +45,183 @@ The 30-second reference composition lives in [`examples/demo`](./examples/demo).
|
|
|
45
45
|
|
|
46
46
|
## Project format
|
|
47
47
|
|
|
48
|
-
A project is ordinary HTML, CSS, assets, and native JavaScript
|
|
48
|
+
A project is ordinary HTML, CSS, assets, and optional native JavaScript controller modules. `video.config.js` explicitly lists composition entry points; there is no directory scanning:
|
|
49
49
|
|
|
50
50
|
```js
|
|
51
51
|
export default {
|
|
52
52
|
compositions: {
|
|
53
|
-
hello: "./compositions/hello.
|
|
53
|
+
hello: "./compositions/hello.html",
|
|
54
54
|
},
|
|
55
55
|
};
|
|
56
56
|
```
|
|
57
57
|
|
|
58
|
-
A composition
|
|
58
|
+
A composition is a full HTML document with one `<fy-composition>` root. Metadata, default props, markup, CSS, bindings, audio, transcripts, and an optional controller all live in the document:
|
|
59
|
+
|
|
60
|
+
```html
|
|
61
|
+
<!doctype html>
|
|
62
|
+
<html>
|
|
63
|
+
<head>
|
|
64
|
+
<meta charset="utf-8">
|
|
65
|
+
<script type="application/json" data-fy="props">
|
|
66
|
+
{ "title": "Hello", "accent": "#b8f36b" }
|
|
67
|
+
</script>
|
|
68
|
+
<style>
|
|
69
|
+
@keyframes fade-up {
|
|
70
|
+
from { opacity: 0; transform: translateY(24px); }
|
|
71
|
+
to { opacity: 1; transform: translateY(0); }
|
|
72
|
+
}
|
|
73
|
+
#title { animation: fade-up 1s ease-out both; }
|
|
74
|
+
</style>
|
|
75
|
+
</head>
|
|
76
|
+
<body>
|
|
77
|
+
<fy-composition width="1920" height="1080" fps="30" duration-in-frames="900">
|
|
78
|
+
<h1 id="title"
|
|
79
|
+
data-fy-text="title"
|
|
80
|
+
data-fy-anim-start="0"
|
|
81
|
+
data-fy-anim-duration="30"></h1>
|
|
82
|
+
<div id="progress" style="width: calc(var(--fy-progress) * 100%)"></div>
|
|
83
|
+
</fy-composition>
|
|
84
|
+
</body>
|
|
85
|
+
</html>
|
|
86
|
+
```
|
|
59
87
|
|
|
60
|
-
|
|
61
|
-
import { defineComposition } from "frameyard";
|
|
62
|
-
import { random } from "frameyard/runtime";
|
|
63
|
-
|
|
64
|
-
let title;
|
|
65
|
-
|
|
66
|
-
export default defineComposition({
|
|
67
|
-
width: 1920,
|
|
68
|
-
height: 1080,
|
|
69
|
-
fps: 30,
|
|
70
|
-
durationInFrames: 900,
|
|
71
|
-
defaultProps: { title: "Hello" },
|
|
72
|
-
validateProps(props) {
|
|
73
|
-
return typeof props.title === "string" ? [] : ["title must be a string"];
|
|
74
|
-
},
|
|
75
|
-
audio: [{ src: "assets/hit.wav", startFrame: 15, volume: 0.8 }],
|
|
88
|
+
`width`, `height`, `fps`, and `duration-in-frames` are fixed. Props are JSON-serializable, deep-frozen, and merged shallowly over the `data-fy="props"` defaults. They cannot change metadata.
|
|
76
89
|
|
|
77
|
-
|
|
78
|
-
container.innerHTML = "<h1 id='title'></h1>";
|
|
79
|
-
title = container.querySelector("#title");
|
|
80
|
-
},
|
|
90
|
+
### Bindings
|
|
81
91
|
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
92
|
+
- `data-fy-text="path"` sets text content.
|
|
93
|
+
- `data-fy-attr-src="path"` (and other `data-fy-attr-*` names) sets an attribute.
|
|
94
|
+
- `data-fy-class-active="path"` toggles a class.
|
|
95
|
+
- `data-fy-if="path"` removes the element when the value is false.
|
|
96
|
+
- `<template data-fy-repeat="items">` clones its content for each item, with `item` and `index` in scope and `--fy-index` on each clone.
|
|
97
|
+
- Paths are dot-separated (`user.name`, `items.0.label`). Missing paths warn and resolve to an empty value.
|
|
98
|
+
- Primitive top-level props are also exposed as `--fy-prop-<name>` CSS custom properties.
|
|
86
99
|
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
100
|
+
Bindings apply once when a composition loads, before the controller mounts. Controller-created DOM is the controller's responsibility.
|
|
101
|
+
|
|
102
|
+
### Animation
|
|
103
|
+
|
|
104
|
+
CSS `@keyframes` and the Web Animations API are the animation language. The runtime pauses every animation and seeks it from the frame clock.
|
|
105
|
+
|
|
106
|
+
`data-fy-anim-start`, `data-fy-anim-duration`, and `data-fy-anim-easing` schedule one animation per element in composition frames. The CSS `animation` shorthand supplies keyframes and fill mode; the attributes override its delay and duration.
|
|
107
|
+
|
|
108
|
+
The runtime sets frame-state variables on `:root`:
|
|
109
|
+
|
|
110
|
+
| Variable | Meaning |
|
|
111
|
+
| --- | --- |
|
|
112
|
+
| `--fy-frame` | Current frame |
|
|
113
|
+
| `--fy-time` | Current time in seconds |
|
|
114
|
+
| `--fy-progress` | Progress from `0` to `1` |
|
|
115
|
+
| `--fy-fps` | Composition frame rate |
|
|
116
|
+
| `--fy-duration` | Duration in seconds |
|
|
117
|
+
| `--fy-duration-frames` | Duration in frames |
|
|
118
|
+
|
|
119
|
+
Use `--fy-progress` for progress bars and `--fy-frame` or `--fy-time` in `calc()` for simple computed motion. CSS transitions are not deterministic and are diagnosed.
|
|
120
|
+
|
|
121
|
+
### Controller
|
|
122
|
+
|
|
123
|
+
An optional controller handles what HTML and CSS cannot express. It is declared exactly once, either inline or as an external module:
|
|
124
|
+
|
|
125
|
+
```html
|
|
126
|
+
<script type="text/fy-controller">
|
|
127
|
+
export default {
|
|
128
|
+
mount({ root, props }) { /* ... */ },
|
|
129
|
+
update({ frame, props, root }) { /* ... */ },
|
|
130
|
+
dispose() { /* ... */ },
|
|
131
|
+
validateProps(props) { /* ... */ },
|
|
132
|
+
};
|
|
133
|
+
</script>
|
|
134
|
+
```
|
|
135
|
+
|
|
136
|
+
```html
|
|
137
|
+
<script type="text/fy-controller" src="./hello.controller.js"></script>
|
|
91
138
|
```
|
|
92
139
|
|
|
93
|
-
|
|
140
|
+
- `mount()` may be asynchronous; `update()` and `validateProps()` are synchronous and seek-safe; `dispose()` releases resources.
|
|
141
|
+
- Inline controllers are the default for small, single-use controllers. Extract to an external module for Canvas 2D, WebGL, karaoke, reuse, tests, or a large controller.
|
|
142
|
+
- A validation-only controller may export just `validateProps(props)`.
|
|
143
|
+
- The controller is the only executable code in a composition. Other executable scripts and `on*` handlers are rejected.
|
|
144
|
+
- Inline source is served through a virtual module URL, so relative imports and `import.meta.url` behave like an external module. A literal `</script>` inside inline source must be escaped.
|
|
94
145
|
|
|
95
146
|
### Composition lifecycle
|
|
96
147
|
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
148
|
+
The composition host loads the document, applies bindings, imports the controller, calls `mount()`, prepares media, and seeks to frame 0. `check`, `still`, and `render` then seek explicitly to requested frames and wait for embedded media and browser paint before evaluating or capturing. `dispose()` runs before the host releases the document.
|
|
149
|
+
|
|
150
|
+
The controller lifecycle is `mount → update → dispose`. `update({ frame, props })` must be synchronous and seek-safe. Use `random("stable:key")` from `frameyard/runtime` instead of `Math.random()`. Independent `requestAnimationFrame` loops, timers, wall-clock reads, and CSS transitions are not deterministic animation drivers.
|
|
151
|
+
|
|
152
|
+
`check` runs static composition validation before launching Chromium. It validates metadata, the root and controller declarations, default props, bindings, animation attributes, scripts and handlers, unsupported visual elements, and asset references, then reports stable diagnostic codes with source locations.
|
|
153
|
+
|
|
154
|
+
## Embedded video and WebGL
|
|
155
|
+
|
|
156
|
+
Embedded videos are declared in the composition document and frame-seeked by the runtime:
|
|
157
|
+
|
|
158
|
+
```html
|
|
159
|
+
<video
|
|
160
|
+
src="assets/clip.webm"
|
|
161
|
+
data-fy-start="30"
|
|
162
|
+
data-fy-trim="0.5"
|
|
163
|
+
data-fy-duration="4"
|
|
164
|
+
data-fy-volume="0.8"
|
|
165
|
+
></video>
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
| Attribute | Meaning | Default |
|
|
169
|
+
| --- | --- | --- |
|
|
170
|
+
| `data-fy-start` | Composition frame where the source starts | `0` |
|
|
171
|
+
| `data-fy-trim` | Seconds trimmed from the source start | `0` |
|
|
172
|
+
| `data-fy-duration` | Optional seconds of source to use | rest of source |
|
|
173
|
+
| `data-fy-volume` | Embedded audio volume from `0` to `1` | `1` |
|
|
100
174
|
|
|
101
|
-
The
|
|
175
|
+
The runtime pauses every video and seeks it to the frame-derived source time. Do not call `play()`, `pause()`, or set `currentTime` in a controller. Videos must be in the composition document's light DOM; shadow roots are not scanned. Video sources are local project-relative files; live streams, remote URLs, and sources that cannot be seeked are not supported. The runtime loads the complete file during the loading phase, so frame production performs no media network requests.
|
|
102
176
|
|
|
103
|
-
|
|
177
|
+
An unmuted `<video>` also contributes its soundtrack to the same timeline as declarative audio clips. Frameyard extracts that track with FFmpeg for production and mirrors it in preview, while keeping the video element itself silent. Set `muted` or `data-fy-audio="off"` to use a video as a visual-only asset.
|
|
178
|
+
|
|
179
|
+
WebGL 1 and WebGL 2 canvases are captured through the normal frame path. Draw the complete frame from the controller's `update()` and do not depend on the previous frame's drawing buffer; clear or redraw the scene each time. Production sessions pin software WebGL (SwiftShader) so rendered pixels do not vary with the host GPU.
|
|
104
180
|
|
|
105
181
|
## Audio timeline
|
|
106
182
|
|
|
107
|
-
Audio clips are
|
|
183
|
+
Audio clips are `<audio>` elements:
|
|
108
184
|
|
|
109
|
-
```
|
|
110
|
-
audio
|
|
111
|
-
|
|
112
|
-
|
|
113
|
-
|
|
114
|
-
|
|
115
|
-
trimDuration: 4.2, // optional seconds to use
|
|
116
|
-
volume: 0.8, // 0..1
|
|
117
|
-
},
|
|
118
|
-
];
|
|
185
|
+
```html
|
|
186
|
+
<audio src="assets/voice.wav"
|
|
187
|
+
data-fy-start="30"
|
|
188
|
+
data-fy-trim="0.25"
|
|
189
|
+
data-fy-duration="4.2"
|
|
190
|
+
data-fy-volume="0.8"></audio>
|
|
119
191
|
```
|
|
120
192
|
|
|
121
|
-
|
|
193
|
+
- Placement is in composition frames.
|
|
194
|
+
- Trimming is in seconds from the source.
|
|
195
|
+
- Audio, video, and transcript `src` values resolve from the project root.
|
|
196
|
+
- Frameyard never time-stretches audio or changes composition duration to fit it.
|
|
197
|
+
- `check` reports clip durations and overrun diagnostics.
|
|
198
|
+
|
|
199
|
+
Preview synchronizes the timeline in the browser. Production mixes the same clips offline with FFmpeg. Unmuted embedded videos contribute their audio tracks to this mix unless they opt out.
|
|
122
200
|
|
|
123
201
|
## Transcripts
|
|
124
202
|
|
|
125
|
-
A composition
|
|
203
|
+
A composition declares canonical transcript assets with `<fy-transcript>`:
|
|
126
204
|
|
|
127
|
-
```
|
|
128
|
-
|
|
129
|
-
{ src: "assets/voice.transcript.json", words: true },
|
|
130
|
-
],
|
|
205
|
+
```html
|
|
206
|
+
<fy-transcript src="assets/voice.transcript.json" words></fy-transcript>
|
|
131
207
|
```
|
|
132
208
|
|
|
133
|
-
Transcript paths
|
|
209
|
+
Transcript paths resolve from the project root. `words` requires actual measured word timings for karaoke-style presentation; Frameyard fails with `TRANSCRIPT_WORDS_UNAVAILABLE` rather than interpolating them. `check`, `still`, and `render` validate declared transcripts. Visual presentation is the composition's responsibility; a controller can read the transcript JSON and render timed text or karaoke. See [subtitles and karaoke](./skills/frameyard/references/subtitles-karaoke.md) for the canonical JSON shape and the seek-safe frame conversion recipe.
|
|
134
210
|
|
|
135
211
|
## Commands
|
|
136
212
|
|
|
137
213
|
| Command | Purpose |
|
|
138
214
|
| --- | --- |
|
|
139
|
-
| `frameyard init [directory]` | Scaffold a minimal project, including an `AGENTS.md` agent guide and pinned `package.json` scripts |
|
|
215
|
+
| `frameyard init [directory]` | Scaffold a minimal HTML project, including an `AGENTS.md` agent guide and pinned `package.json` scripts |
|
|
140
216
|
| `frameyard dev [--port <port>]` | Start the preview server |
|
|
141
|
-
| `frameyard check <id> [--props <file>]` |
|
|
217
|
+
| `frameyard check <id> [--props <file>]` | Run static validation, then evaluate every frame without capture |
|
|
142
218
|
| `frameyard still <id> --frame <n>` | Capture one PNG through the production path |
|
|
143
|
-
| `frameyard render <id> --out <video.mp4>` | Capture frames with Chromium and encode with FFmpeg |
|
|
219
|
+
| `frameyard render <id> --out <video.mp4>` | Capture frames with Chromium and encode with FFmpeg, optionally in parallel |
|
|
144
220
|
| `frameyard tts ...` | Produce one frozen 24 kHz mono WAV with local qwentts.cpp |
|
|
145
221
|
| `frameyard transcribe ...` | Produce canonical transcript JSON with optional word timings and SRT/VTT exports |
|
|
146
222
|
| `frameyard doctor` | Inspect core requirements, optional integrations, and manifest integrity |
|
|
147
223
|
|
|
148
|
-
Common options: `--project <dir>`, `--json`, `--help`, and `--version`. `still` and `render` accept `--overwrite`; `tts` and `transcribe` accept `--force`. `init` takes the target directory as a positional argument. Output paths are relative to the current working directory.
|
|
224
|
+
Common options: `--project <dir>`, `--json`, `--help`, and `--version`. `still` and `render` accept `--overwrite`; `render` also accepts `--concurrency <n>` (1-16); `tts` and `transcribe` accept `--force`. `init` takes the target directory as a positional argument. Output paths are relative to the current working directory.
|
|
149
225
|
|
|
150
226
|
A render fails immediately on an uncaught error, unhandled rejection, missing required asset, or invalid composition output. It writes to a temporary file and publishes only after FFmpeg succeeds. Existing outputs are not replaced unless `--overwrite` is supplied.
|
|
151
227
|
|
|
@@ -156,26 +232,33 @@ Non-interactive commands support `--json`. Results go to stdout and human progre
|
|
|
156
232
|
```json
|
|
157
233
|
{
|
|
158
234
|
"ok": true,
|
|
159
|
-
"command": "
|
|
235
|
+
"command": "check",
|
|
160
236
|
"composition": "hello",
|
|
161
|
-
"
|
|
162
|
-
"
|
|
163
|
-
"height": 1080,
|
|
164
|
-
"
|
|
165
|
-
"
|
|
166
|
-
"
|
|
167
|
-
"
|
|
168
|
-
"transcripts": { "clips": 1, "warnings": [] }
|
|
237
|
+
"entry": "compositions/hello.html",
|
|
238
|
+
"controller": { "kind": "inline" },
|
|
239
|
+
"metadata": { "width": 1920, "height": 1080, "fps": 30, "durationInFrames": 900 },
|
|
240
|
+
"framesChecked": 900,
|
|
241
|
+
"audio": { "clips": 1 },
|
|
242
|
+
"transcripts": { "clips": 0 },
|
|
243
|
+
"diagnostics": []
|
|
169
244
|
}
|
|
170
245
|
```
|
|
171
246
|
|
|
172
|
-
Errors use the same envelope
|
|
247
|
+
Diagnostics carry `severity`, `code`, `message`, and optional `details` with source locations, element or attribute names, values, and suggestions. Errors use the same envelope and may include `error.details` and top-level `diagnostics`:
|
|
173
248
|
|
|
174
249
|
```json
|
|
175
250
|
{
|
|
176
251
|
"ok": false,
|
|
177
|
-
"command": "
|
|
178
|
-
"error": { "code": "
|
|
252
|
+
"command": "check",
|
|
253
|
+
"error": { "code": "BINDING_INVALID", "message": "Unknown Frameyard attribute ..." },
|
|
254
|
+
"diagnostics": [
|
|
255
|
+
{
|
|
256
|
+
"severity": "error",
|
|
257
|
+
"code": "BINDING_INVALID",
|
|
258
|
+
"message": "Unknown Frameyard attribute \"data-fy-txt\".",
|
|
259
|
+
"details": { "path": "compositions/hello.html", "line": 12, "column": 5, "attribute": "data-fy-txt" }
|
|
260
|
+
}
|
|
261
|
+
]
|
|
179
262
|
}
|
|
180
263
|
```
|
|
181
264
|
|
|
@@ -226,20 +309,23 @@ npx frameyard doctor --talker-model models/talker.gguf --codec-model models/code
|
|
|
226
309
|
|
|
227
310
|
## Agent skill
|
|
228
311
|
|
|
229
|
-
The optional host-neutral agent skill lives in [`skills/frameyard/`](./skills/frameyard). It uses the same project format and CLI as humans, documents
|
|
312
|
+
The optional host-neutral agent skill lives in [`skills/frameyard/`](./skills/frameyard). It uses the same project format and CLI as humans, documents HTML authoring, bindings, controllers, static diagnostics, audio assets, subtitle/karaoke recipes, JSON results, and failure recovery, and ships a thin CLI wrapper. The npm package includes the skill files matching its API version. Install it with `npx skills add vladiantio/frameyard`, or see [`skills/frameyard/README.md`](./skills/frameyard/README.md) for host-specific installation or referencing.
|
|
230
313
|
|
|
231
314
|
## Rendering and reproducibility
|
|
232
315
|
|
|
233
316
|
Production rendering:
|
|
234
317
|
|
|
235
|
-
1.
|
|
236
|
-
2.
|
|
237
|
-
3.
|
|
238
|
-
4.
|
|
239
|
-
5.
|
|
240
|
-
6.
|
|
318
|
+
1. Runs static composition validation and resolves metadata before launching Chromium.
|
|
319
|
+
2. Navigates directly to the composition document with the composition host.
|
|
320
|
+
3. Launches one pinned Chromium session, or several independent sessions when `--concurrency` is greater than 1.
|
|
321
|
+
4. Seeks explicitly to each frame, calls the controller, seeks CSS/WAAPI animations, and waits for embedded media decode and browser paint.
|
|
322
|
+
5. Captures the viewport through the Chrome DevTools Protocol.
|
|
323
|
+
6. Pipes PNG frames in timeline order and the offline audio mix to FFmpeg.
|
|
324
|
+
7. Encodes H.264 video with `yuv420p` and AAC audio into an MP4.
|
|
325
|
+
|
|
326
|
+
Rendering is sequential by default. Pass `--concurrency <n>` (1-16) to seek and capture frames with up to `n` independent browser sessions; the effective value is capped by the composition's frame count. Captured frames are always encoded in timeline order, so parallel and sequential renders of the same inputs produce equivalent visual frames. Concurrency only changes scheduling: it does not change composition behavior or the determinism contract. Parallel rendering uses more CPU and memory, so choose a value appropriate for the machine.
|
|
241
327
|
|
|
242
|
-
|
|
328
|
+
For the same composition revision, props, assets, frame, and pinned runtime/browser environment, the visual state is identical. Encoded bytes are not a reproducibility guarantee, even on the same machine; they depend on the FFmpeg build, encoder threading, and runtime environment.
|
|
243
329
|
|
|
244
330
|
For a repeatable local measurement of seek, capture, encode, and full-render time, run `pnpm bench`. The demo and the controlled fixtures under `bench/fixtures/` are the reference workloads.
|
|
245
331
|
|
|
@@ -247,32 +333,37 @@ For a repeatable local measurement of seek, capture, encode, and full-render tim
|
|
|
247
333
|
|
|
248
334
|
Deferred from this implementation slice:
|
|
249
335
|
|
|
250
|
-
- Built-in subtitle rendering, automatic caption burn-in, and karaoke
|
|
336
|
+
- Built-in subtitle rendering, automatic caption burn-in, and declarative karaoke elements
|
|
337
|
+
- Nested `data-fy-repeat` scopes and a general template language
|
|
338
|
+
- Declarative JSON Schema validation
|
|
339
|
+
- Arbitrary executable scripts in composition HTML
|
|
251
340
|
- `qwentts.cpp` reference-audio voice cloning and VoiceDesign
|
|
252
341
|
- Automatic long-text chunking, streaming synthesis, and alternative generated-audio formats
|
|
253
342
|
- Automatic upstream builds, model downloads, environment repair, and hosted rendering
|
|
254
|
-
-
|
|
343
|
+
- Frame caches, batch still capture, and general image-sequence output
|
|
255
344
|
- React or other UI-framework adapters, plugin APIs, and agent-only runtimes
|
|
256
345
|
- TypeScript transpilation for project compositions and npm dependency optimization
|
|
257
346
|
|
|
258
|
-
See the [
|
|
347
|
+
See the [design baseline](./docs/v0.1-design.md) for the full product contract, [ADR 0007](./docs/adr/0007-make-html-documents-the-composition-unit.md) for the HTML authoring decision, and the [domain glossary](./GLOSSARY.md) for terminology.
|
|
259
348
|
|
|
260
349
|
## Repository layout
|
|
261
350
|
|
|
262
|
-
- `src/contracts.ts`: shared composition, props, audio, and
|
|
351
|
+
- `src/contracts.ts`: shared composition, props, audio, transcript, and diagnostic contracts
|
|
352
|
+
- `src/bindings.ts`: browser-safe binding path and value semantics
|
|
353
|
+
- `src/html/`: Node-side parsing, static composition validation, and host injection
|
|
263
354
|
- `src/transcript.ts`: canonical transcript validation and SRT/VTT formatting
|
|
264
|
-
- `src/runtime/`: browser
|
|
265
|
-
- `src/preview/`: interactive preview client
|
|
266
|
-
- `src/server/`: native ES-module server and
|
|
267
|
-
- `src/renderer/`: Chromium capture, audio mixing, transcript inspection, FFmpeg encoding
|
|
355
|
+
- `src/runtime/`: browser host, frame clock, deterministic randomness, bindings, media, diagnostics
|
|
356
|
+
- `src/preview/`: interactive iframe preview client
|
|
357
|
+
- `src/server/`: native ES-module server, host injection, and inline-controller route
|
|
358
|
+
- `src/renderer/`: Chromium capture, static validation integration, audio mixing, transcript inspection, FFmpeg encoding
|
|
268
359
|
- `src/assets/`: local TTS/transcription wrappers, executable resolution, atomic publishing, provenance
|
|
269
360
|
- `src/doctor.ts`: environment, integration, and manifest inspection
|
|
270
361
|
- `src/scaffold.ts`: `init` scaffold templates, including the generated package manifest, project README, and agent guide
|
|
271
362
|
- `src/cli.ts`: command-line entry point
|
|
272
363
|
- `skills/`: official host-neutral agent skill
|
|
273
364
|
- `examples/demo/`: the 30-second reference composition
|
|
274
|
-
- `bench/fixtures/`: controlled compositions for the render benchmark
|
|
275
|
-
- `scripts/`: benchmark, versioning, and git-hook installation
|
|
365
|
+
- `bench/fixtures/`: controlled HTML compositions for the render benchmark
|
|
366
|
+
- `scripts/`: benchmark, release checks, versioning, and git-hook installation
|
|
276
367
|
- `docs/adr/`: architecture decision records
|
|
277
368
|
|
|
278
369
|
## Development
|
|
@@ -283,14 +374,13 @@ From a repository checkout:
|
|
|
283
374
|
pnpm install
|
|
284
375
|
pnpm exec playwright install chromium
|
|
285
376
|
pnpm build
|
|
286
|
-
pnpm
|
|
287
|
-
pnpm format:check
|
|
377
|
+
pnpm check
|
|
288
378
|
node dist/cli.js doctor
|
|
289
379
|
```
|
|
290
380
|
|
|
291
381
|
These commands use the local build; `npx frameyard` runs the published package.
|
|
292
382
|
|
|
293
|
-
`pnpm install` installs the repository's git hooks
|
|
383
|
+
`pnpm check` runs the script syntax checks, lint, formatting checks, typechecks, and the full suite. `pnpm install` installs the repository's git hooks; pre-commit runs `pnpm check:static` and the unit tests, and pre-push runs the full suite. CI runs the same `pnpm check` on Linux with Node 22 and 24.
|
|
294
384
|
|
|
295
385
|
CI uses fake `qwen-tts` and `whisper-cli` executables. Real-model tests are opt-in:
|
|
296
386
|
|