frameyard 0.2.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (112) hide show
  1. package/CHANGELOG.md +102 -0
  2. package/GLOSSARY.md +58 -2
  3. package/README.md +185 -95
  4. package/dist/bindings.d.ts +42 -0
  5. package/dist/bindings.d.ts.map +1 -0
  6. package/dist/bindings.js +144 -0
  7. package/dist/bindings.js.map +1 -0
  8. package/dist/cli.js +37 -14
  9. package/dist/cli.js.map +1 -1
  10. package/dist/contracts.d.ts +73 -47
  11. package/dist/contracts.d.ts.map +1 -1
  12. package/dist/contracts.js +14 -159
  13. package/dist/contracts.js.map +1 -1
  14. package/dist/errors.d.ts +15 -4
  15. package/dist/errors.d.ts.map +1 -1
  16. package/dist/errors.js +8 -0
  17. package/dist/errors.js.map +1 -1
  18. package/dist/html/index.d.ts +5 -0
  19. package/dist/html/index.d.ts.map +1 -0
  20. package/dist/html/index.js +3 -0
  21. package/dist/html/index.js.map +1 -0
  22. package/dist/html/parse.d.ts +30 -0
  23. package/dist/html/parse.d.ts.map +1 -0
  24. package/dist/html/parse.js +80 -0
  25. package/dist/html/parse.js.map +1 -0
  26. package/dist/html/validate.d.ts +14 -0
  27. package/dist/html/validate.d.ts.map +1 -0
  28. package/dist/html/validate.js +931 -0
  29. package/dist/html/validate.js.map +1 -0
  30. package/dist/index.d.ts +2 -2
  31. package/dist/index.d.ts.map +1 -1
  32. package/dist/index.js +1 -1
  33. package/dist/index.js.map +1 -1
  34. package/dist/preview/main.d.ts +12 -3
  35. package/dist/preview/main.d.ts.map +1 -1
  36. package/dist/preview/main.js +208 -85
  37. package/dist/preview/main.js.map +1 -1
  38. package/dist/project.d.ts +2 -0
  39. package/dist/project.d.ts.map +1 -1
  40. package/dist/project.js +4 -0
  41. package/dist/project.js.map +1 -1
  42. package/dist/renderer/audio.d.ts +5 -1
  43. package/dist/renderer/audio.d.ts.map +1 -1
  44. package/dist/renderer/audio.js +4 -0
  45. package/dist/renderer/audio.js.map +1 -1
  46. package/dist/renderer/browser.d.ts +12 -1
  47. package/dist/renderer/browser.d.ts.map +1 -1
  48. package/dist/renderer/browser.js +69 -11
  49. package/dist/renderer/browser.js.map +1 -1
  50. package/dist/renderer/index.d.ts +30 -5
  51. package/dist/renderer/index.d.ts.map +1 -1
  52. package/dist/renderer/index.js +204 -26
  53. package/dist/renderer/index.js.map +1 -1
  54. package/dist/runtime/bindings.d.ts +11 -0
  55. package/dist/runtime/bindings.d.ts.map +1 -0
  56. package/dist/runtime/bindings.js +164 -0
  57. package/dist/runtime/bindings.js.map +1 -0
  58. package/dist/runtime/diagnostics.d.ts +4 -0
  59. package/dist/runtime/diagnostics.d.ts.map +1 -0
  60. package/dist/runtime/diagnostics.js +107 -0
  61. package/dist/runtime/diagnostics.js.map +1 -0
  62. package/dist/runtime/host.d.ts +12 -0
  63. package/dist/runtime/host.d.ts.map +1 -0
  64. package/dist/runtime/host.js +701 -0
  65. package/dist/runtime/host.js.map +1 -0
  66. package/dist/runtime/index.d.ts +0 -26
  67. package/dist/runtime/index.d.ts.map +1 -1
  68. package/dist/runtime/index.js +0 -286
  69. package/dist/runtime/index.js.map +1 -1
  70. package/dist/runtime/media.d.ts +16 -0
  71. package/dist/runtime/media.d.ts.map +1 -0
  72. package/dist/runtime/media.js +307 -0
  73. package/dist/runtime/media.js.map +1 -0
  74. package/dist/scaffold.d.ts.map +1 -1
  75. package/dist/scaffold.js +169 -78
  76. package/dist/scaffold.js.map +1 -1
  77. package/dist/server/composition-html.d.ts +20 -0
  78. package/dist/server/composition-html.d.ts.map +1 -0
  79. package/dist/server/composition-html.js +45 -0
  80. package/dist/server/composition-html.js.map +1 -0
  81. package/dist/server/preview-html.d.ts.map +1 -1
  82. package/dist/server/preview-html.js +19 -14
  83. package/dist/server/preview-html.js.map +1 -1
  84. package/dist/server/server.d.ts.map +1 -1
  85. package/dist/server/server.js +94 -6
  86. package/dist/server/server.js.map +1 -1
  87. package/dist/version.d.ts +1 -1
  88. package/dist/version.js +1 -1
  89. package/docs/adr/0001-keep-composition-authoring-web-native.md +2 -0
  90. package/docs/adr/0003-render-browser-frames-offline.md +2 -0
  91. package/docs/adr/0005-keep-parallel-rendering-opt-in-and-frame-ordered.md +3 -0
  92. package/docs/adr/0006-seek-embedded-media-from-the-frame-clock.md +11 -0
  93. package/docs/adr/0007-make-html-documents-the-composition-unit.md +31 -0
  94. package/docs/migration-0.4.md +279 -0
  95. package/docs/v0.1-design.md +82 -57
  96. package/examples/demo/README.md +7 -3
  97. package/examples/demo/compositions/demo.controller.js +120 -0
  98. package/examples/demo/compositions/demo.html +253 -0
  99. package/examples/demo/video.config.js +1 -1
  100. package/package.json +9 -3
  101. package/skills/frameyard/README.md +4 -0
  102. package/skills/frameyard/SKILL.md +41 -11
  103. package/skills/frameyard/references/audio-assets.md +25 -14
  104. package/skills/frameyard/references/bindings.md +73 -0
  105. package/skills/frameyard/references/common-mistakes.md +59 -0
  106. package/skills/frameyard/references/controllers.md +126 -0
  107. package/skills/frameyard/references/deterministic-authoring.md +160 -41
  108. package/skills/frameyard/references/diagnostics.md +90 -0
  109. package/skills/frameyard/references/failure-recovery.md +38 -12
  110. package/skills/frameyard/references/json-results.md +43 -8
  111. package/skills/frameyard/references/subtitles-karaoke.md +10 -18
  112. package/examples/demo/compositions/demo.js +0 -206
@@ -0,0 +1,59 @@
1
+ # Common mistakes
2
+
3
+ ## Using the old authoring model
4
+
5
+ - **JavaScript composition modules / `defineComposition`** were removed in 0.4. A composition is a `.html` document listed in `video.config.js`.
6
+ - **`data-frameyard-*`** media attributes were removed in 0.4. Use `data-fy-*`.
7
+
8
+ ## Document structure
9
+
10
+ - Missing `<!doctype html>` or missing/multiple `<fy-composition>` roots.
11
+ - Missing `width`, `height`, `fps`, or `duration-in-frames` on the root.
12
+ - Missing the `data-fy="props"` block when the composition relies on default props.
13
+ - Visual content outside `<fy-composition>`; it is not captured.
14
+ - Using `<base>`, refresh/CSP meta tags, `<iframe>`, `<object>`, or `<embed>`.
15
+
16
+ ## Bindings
17
+
18
+ - Unknown `data-fy-*` attribute, usually a typo such as `data-fy-txt`.
19
+ - A binding path that does not exist in the resolved props. Check the `BINDING_PATH_MISSING` warning and its suggestion.
20
+ - Using `data-fy-repeat` on a normal element instead of `<template>`.
21
+ - Nesting `data-fy-repeat`; use a controller for nested data.
22
+ - Putting `data-fy-if` or `data-fy-text` on a repeat template.
23
+ - Putting `data-fy-text` on an `<img>` or another void element.
24
+ - Expecting bindings to update every frame or to re-bind controller-created DOM.
25
+
26
+ ## Timing
27
+
28
+ - Writing seconds where frames are required, or frames where seconds are required.
29
+ - `data-fy-start` and `data-fy-anim-start`/`duration` are frames.
30
+ - `data-fy-trim`/`duration` are seconds.
31
+ - Using CSS `animation-delay`/`duration` in seconds and expecting the declarative scheduler to ignore its own attributes. `data-fy-anim-*` overrides the shorthand's delay/duration.
32
+ - Adding `data-fy-anim-*` without a CSS animation; this produces `ANIMATION_NOT_FOUND`.
33
+ - Using multiple animations on an element with one declarative schedule.
34
+
35
+ ## Controllers
36
+
37
+ - Declaring both inline source and `src` on the same controller.
38
+ - Declaring more than one controller.
39
+ - Making `update()` async; do asynchronous work in `mount()`.
40
+ - Leaving module-scope state behind in `dispose()`.
41
+ - Writing a literal `</script>` inside inline controller source; escape it.
42
+ - Importing libraries from executable `<script>` tags; import them from the controller instead.
43
+ - Using `Math.random()`, `Date.now()`, `performance.now()`, timers, independent `requestAnimationFrame` loops, or CSS transitions.
44
+
45
+ ## Media and transcripts
46
+
47
+ - Using remote, live, or unseekable media sources.
48
+ - Using `<source>` or `<track>` children; use a local `src` and `<fy-transcript>`.
49
+ - Adding or changing media in `update()`; prepare it in the document or in `mount()`.
50
+ - Calling `play()`, `pause()`, or setting `currentTime` on a video.
51
+ - Assuming media/transcript `src` is document-relative. It is project-root-relative.
52
+ - Assuming the controller `src` is project-root-relative. It is document-relative.
53
+
54
+ ## Validation
55
+
56
+ - Replacing `check` with a single `still`.
57
+ - Ignoring warnings or `diagnostics` in `--json` output.
58
+ - Claiming success after a nonzero exit, a missing still, or skipped visual inspection.
59
+ - Using `--overwrite` or `--force` without permission.
@@ -0,0 +1,126 @@
1
+ # Controllers
2
+
3
+ A composition controller is optional JavaScript for what HTML and CSS cannot express: Canvas 2D, WebGL, karaoke, complex computed motion, or data-driven DOM. It is the only executable code in a composition.
4
+
5
+ ## Inline or external
6
+
7
+ Inline for a small, single-use controller:
8
+
9
+ ```html
10
+ <script type="text/fy-controller">
11
+ export default {
12
+ mount({ root, props }) {
13
+ // create canvases, query elements, load data
14
+ },
15
+ update({ frame, props, root }) {
16
+ // synchronous and seek-safe
17
+ },
18
+ dispose() {
19
+ // release module-scope state and listeners
20
+ },
21
+ };
22
+ </script>
23
+ ```
24
+
25
+ External for reuse, tests, or a large controller:
26
+
27
+ ```html
28
+ <script type="text/fy-controller" src="./hello.controller.js"></script>
29
+ ```
30
+
31
+ ```js
32
+ export default {
33
+ mount({ root, props }) {},
34
+ update({ frame, props, root }) {},
35
+ dispose() {},
36
+ };
37
+ ```
38
+
39
+ Exactly one controller is allowed. Inline source is served through a virtual module URL, so relative imports and `import.meta.url` work like an external module. A literal `</script>` inside inline source must be escaped.
40
+
41
+ ## Lifecycle
42
+
43
+ - `mount({ root, props })` runs once per load and may be asynchronous. Use it to query the root, create canvases, and start loading local data.
44
+ - `update({ frame, props, root })` runs on every seek. It must be synchronous and seek-safe: derive the frame from `frame` and `props` only.
45
+ - `dispose()` releases resources and resets module-scope state. The host calls it before remounting or disposing the document.
46
+ - `validateProps(props)` is optional and synchronous. Return an array of human-readable errors, or `void`.
47
+ - A validation-only controller may export just `validateProps`.
48
+
49
+ Keep mutable state in module scope, never on `this`. The controller module is imported once per document; `load` remounts it without re-importing, so `dispose` must reset module state.
50
+
51
+ ## Props validation
52
+
53
+ ```html
54
+ <script type="text/fy-controller">
55
+ export default {
56
+ validateProps(props) {
57
+ const errors = [];
58
+ if (typeof props.title !== "string") errors.push("title must be a string");
59
+ return errors;
60
+ },
61
+ };
62
+ </script>
63
+ ```
64
+
65
+ Validation runs before the old mount is disposed. Invalid props reject `load` with `PROPS_INVALID` and keep the current composition mounted.
66
+
67
+ ## Canvas and WebGL
68
+
69
+ ```js
70
+ let canvas;
71
+ let context;
72
+
73
+ export default {
74
+ mount({ root }) {
75
+ canvas = root.querySelector("#field");
76
+ context = canvas.getContext("2d");
77
+ },
78
+ update({ frame }) {
79
+ context.clearRect(0, 0, canvas.width, canvas.height);
80
+ context.fillStyle = frame % 2 === 0 ? "#101820" : "#203040";
81
+ context.fillRect(0, 0, canvas.width, canvas.height);
82
+ },
83
+ dispose() {
84
+ canvas = undefined;
85
+ context = undefined;
86
+ },
87
+ };
88
+ ```
89
+
90
+ Redraw WebGL from the current frame; do not depend on the previous drawing buffer. Production pins SwiftShader.
91
+
92
+ ## Randomness and clocks
93
+
94
+ Import `random` from `frameyard/runtime`:
95
+
96
+ ```js
97
+ import { random } from "frameyard/runtime";
98
+
99
+ const x = random("star:17:x") * 1920;
100
+ ```
101
+
102
+ Never use `Math.random()`, `Date.now()`, `performance.now()`, timers, independent `requestAnimationFrame` loops, or CSS transitions as animation inputs. The host diagnoses common cases in `mount` and `update`.
103
+
104
+ ## Media
105
+
106
+ Prefer declarative `<video>`/`<audio>` elements with `data-fy-*` attributes. The host pauses and seeks them; do not call `play()`, `pause()`, or set `currentTime` in `update()`. Media created by the controller is allowed in `mount`; media added or changed in `update` cannot be prepared without network access.
107
+
108
+ Media and transcript `src` values resolve from the project root. The controller `src` resolves relative to the composition document.
109
+
110
+ ## Karaoke
111
+
112
+ Read the declared transcript JSON, convert source milliseconds to frames, and render the active segment or word from `frame`:
113
+
114
+ ```js
115
+ const FPS = 30;
116
+ const clipStartFrame = 30;
117
+ const trimStart = 0.25;
118
+
119
+ function wordProgress(word, frame) {
120
+ const start = clipStartFrame + (word.startMs / 1000 - trimStart) * FPS;
121
+ const end = clipStartFrame + (word.endMs / 1000 - trimStart) * FPS;
122
+ return Math.max(0, Math.min(1, (frame - start) / Math.max(1, end - start)));
123
+ }
124
+ ```
125
+
126
+ Never interpolate segment timings to fake word timings.
@@ -1,6 +1,6 @@
1
- # Deterministic authoring
1
+ # Deterministic HTML authoring
2
2
 
3
- Frameyard compositions are ordinary native JavaScript ES modules. The preview server injects an import map for `frameyard` and `frameyard/runtime`; do not add a bundling step or import from `node_modules` in composition code.
3
+ Frameyard compositions are full HTML documents. The preview server injects the framework import map and composition host; do not add a bundling step. Optional controllers are native JavaScript modules that import bare `frameyard/runtime`.
4
4
 
5
5
  ## Project configuration
6
6
 
@@ -9,50 +9,125 @@ Frameyard compositions are ordinary native JavaScript ES modules. The preview se
9
9
  ```js
10
10
  export default {
11
11
  compositions: {
12
- hello: "./compositions/hello.js",
12
+ hello: "./compositions/hello.html",
13
13
  },
14
14
  };
15
15
  ```
16
16
 
17
- There is no directory scanning. A new composition file is invisible until it is listed.
17
+ There is no directory scanning. A new composition document is invisible until it is listed. Entries must be `.html` files; JavaScript composition modules are rejected.
18
18
 
19
19
  ## Composition shape
20
20
 
21
- ```js
22
- import { defineComposition } from "frameyard";
23
- import { random } from "frameyard/runtime";
21
+ ```html
22
+ <!doctype html>
23
+ <html>
24
+ <head>
25
+ <meta charset="utf-8">
26
+ <script type="application/json" data-fy="props">
27
+ { "title": "Hello", "accent": "#b8f36b" }
28
+ </script>
29
+ <style>
30
+ @keyframes fade-up {
31
+ from { opacity: 0; transform: translateY(24px); }
32
+ to { opacity: 1; transform: translateY(0); }
33
+ }
34
+ #title { animation: fade-up 1s ease-out both; }
35
+ </style>
36
+ </head>
37
+ <body>
38
+ <fy-composition width="1920" height="1080" fps="30" duration-in-frames="900">
39
+ <h1 id="title"
40
+ data-fy-text="title"
41
+ data-fy-anim-start="0"
42
+ data-fy-anim-duration="30"></h1>
43
+ <div style="width: calc(var(--fy-progress) * 100%)"></div>
44
+ </fy-composition>
45
+ </body>
46
+ </html>
47
+ ```
24
48
 
25
- let title;
49
+ - `width`, `height`, `fps`, and `duration-in-frames` are fixed on the root.
50
+ - The `data-fy="props"` block is a JSON object. External props merge shallowly over it and are deep-frozen.
51
+ - There is exactly one `<fy-composition>` root, inside `<body>`.
52
+ - A doctype is required. Unsupported document features such as `<base>`, refresh/CSP meta tags, `<iframe>`, `<object>`, and `<embed>` are rejected.
53
+
54
+ ## Bindings
55
+
56
+ ```html
57
+ <h1 data-fy-text="title"></h1>
58
+ <img data-fy-attr-src="image" data-fy-attr-alt="alt">
59
+ <div data-fy-class-active="isActive"></div>
60
+ <p data-fy-if="showSubtitle" data-fy-text="subtitle"></p>
61
+ <template data-fy-repeat="items">
62
+ <span data-fy-text="item.label" data-fy-attr-data-index="index"></span>
63
+ </template>
64
+ ```
26
65
 
27
- export default defineComposition({
28
- width: 1920,
29
- height: 1080,
30
- fps: 30,
31
- durationInFrames: 900,
32
- defaultProps: { title: "Hello" },
33
- validateProps(props) {
34
- return typeof props.title === "string" ? [] : ["title must be a string"];
35
- },
36
- mount({ container, props }) {
37
- container.innerHTML = "<h1 id='title'></h1>";
38
- title = container.querySelector("#title");
39
- title.textContent = props.title;
66
+ - Paths are dot-separated (`user.name`, `items.0.label`); numeric segments index arrays.
67
+ - `data-fy-text` sets `textContent`.
68
+ - `data-fy-attr-<name>` sets an attribute; `true` writes an empty attribute, `false`/`null`/`undefined` remove it.
69
+ - `data-fy-class-<name>` toggles one class.
70
+ - `data-fy-if` removes the element when the value is false.
71
+ - `data-fy-repeat` must be on a `<template>` and must reference an array. Each clone gets `item`, `index`, and `--fy-index`. Nested repeats are rejected; use a controller.
72
+ - Bindings apply once per load, before the controller mounts. Controller-created DOM is not re-bound.
73
+ - Missing paths warn and resolve to empty text, a removed attribute/class, or false for `if`.
74
+ - Primitive top-level props are also exposed as `--fy-prop-<name>` CSS custom properties. Booleans are `1`/`0`; objects and arrays are omitted.
75
+
76
+ ## Animation
77
+
78
+ CSS `@keyframes` and the Web Animations API are paused and seeked by the runtime inside the composition root.
79
+
80
+ ```css
81
+ @keyframes fade-up {
82
+ from { opacity: 0; transform: translateY(24px); }
83
+ to { opacity: 1; transform: translateY(0); }
84
+ }
85
+ #title { animation: fade-up 1s ease-out both; }
86
+ ```
87
+
88
+ ```html
89
+ <h1 data-fy-anim-start="30" data-fy-anim-duration="60" data-fy-anim-easing="ease-out"></h1>
90
+ ```
91
+
92
+ - `data-fy-anim-start` is a non-negative frame count. `data-fy-anim-duration` is a positive frame count. `data-fy-anim-easing` is a CSS easing.
93
+ - One declarative animation per element. The CSS `animation` shorthand supplies keyframes, fill mode, and iteration count; the attributes override delay/duration/easing.
94
+ - The root exposes `--fy-frame`, `--fy-time`, `--fy-progress`, `--fy-fps`, `--fy-duration`, and `--fy-duration-frames`. Use them in `calc()` for simple computed motion and progress bars.
95
+ - Do not use CSS transitions. They are diagnosed as non-deterministic.
96
+
97
+ ## Controllers
98
+
99
+ An optional controller handles Canvas 2D, WebGL, karaoke, and per-frame computation CSS cannot express.
100
+
101
+ Inline for a small, single-use controller:
102
+
103
+ ```html
104
+ <script type="text/fy-controller">
105
+ export default {
106
+ mount({ root, props }) {
107
+ // query elements, create canvases, load data
40
108
  },
41
- update({ frame, props }) {
42
- title.textContent = props.title;
43
- title.style.transform = `translateX(${random("title:" + frame) * 40}px)`;
109
+ update({ frame, props, root }) {
110
+ // synchronous and seek-safe
44
111
  },
45
112
  dispose() {
46
- title = undefined;
113
+ // release module-scope state and listeners
47
114
  },
48
- });
115
+ };
116
+ </script>
49
117
  ```
50
118
 
51
- - `mount()` runs once and may await loading work.
52
- - `update({ frame, props })` is synchronous and seek-safe.
53
- - `dispose()` releases resources and module-scope references.
54
- - Props are JSON-serializable, deep-frozen, and merged shallowly over `defaultProps`.
55
- - Keep mutable state in module-scope variables, never on `this`.
119
+ External for a reusable or complex controller:
120
+
121
+ ```html
122
+ <script type="text/fy-controller" src="./hello.controller.js"></script>
123
+ ```
124
+
125
+ - Exactly one controller is allowed. It may export `mount`, `update`, `dispose`, and `validateProps`; at least one is required. `mount` may be async; `update` and `validateProps` are synchronous.
126
+ - A validation-only controller may export just `validateProps(props)`.
127
+ - Inline source is served through a virtual module URL, so relative imports and `import.meta.url` work like an external module. A literal `</script>` inside inline source must be escaped.
128
+ - The controller is the only executable code. Other executable `<script>` elements and `on*` handlers are rejected.
129
+ - Controller `src` resolves relative to the composition document. Media and transcript `src` values resolve from the project root.
130
+ - Keep mutable state in module scope, never on `this`. Reset it in `dispose()`.
56
131
 
57
132
  ## Forbidden animation inputs
58
133
 
@@ -65,33 +140,77 @@ Do not use any of these as animation state:
65
140
  - CSS transitions
66
141
  - state left from a previously visited frame
67
142
 
68
- Use `random("stable:key")` for deterministic randomness. The value depends only on the key, not call order.
143
+ Use `random("stable:key")` from `frameyard/runtime` for deterministic randomness. The value depends only on the key, not call order.
69
144
 
70
145
  ```js
71
146
  import { random } from "frameyard/runtime";
72
147
 
73
- const x = random("star:17:x") * width;
148
+ const x = random("star:17:x") * 1920;
74
149
  ```
75
150
 
76
- ## CSS and Canvas
151
+ ## Canvas and WebGL
77
152
 
78
- CSS `@keyframes` and the Web Animations API are paused and seeked by the runtime inside the composition container. They are valid animation drivers. CSS transitions are diagnosed as non-deterministic and should not be used.
79
-
80
- Canvas 2D must be redrawn from the current frame:
153
+ Redraw Canvas 2D and WebGL from the current frame in the controller's `update()`:
81
154
 
82
155
  ```js
83
156
  update({ frame }) {
84
157
  context.clearRect(0, 0, canvas.width, canvas.height);
85
158
  context.fillStyle = frame % 2 === 0 ? "#101820" : "#203040";
86
159
  context.fillRect(0, 0, canvas.width, canvas.height);
160
+
161
+ gl.clearColor(frame % 2 === 0 ? 0.1 : 0.2, 0.1, 0.13, 1);
162
+ gl.clear(gl.COLOR_BUFFER_BIT);
87
163
  }
88
164
  ```
89
165
 
166
+ Do not preserve and accumulate WebGL drawing across frames unless that accumulation is itself deterministic. Production sessions use software WebGL so output does not depend on the host GPU.
167
+
168
+ ## Embedded video and audio
169
+
170
+ Declare local media in the composition document:
171
+
172
+ ```html
173
+ <video
174
+ src="assets/clip.webm"
175
+ data-fy-start="30"
176
+ data-fy-trim="0.5"
177
+ data-fy-duration="4"
178
+ data-fy-volume="0.8"
179
+ ></video>
180
+
181
+ <audio
182
+ src="assets/voice.wav"
183
+ data-fy-start="30"
184
+ data-fy-trim="0.25"
185
+ data-fy-duration="4.2"
186
+ data-fy-volume="0.8"
187
+ ></audio>
188
+ ```
189
+
190
+ - `data-fy-start` is the composition frame where the source starts. Default `0`.
191
+ - `data-fy-trim` is seconds trimmed from the source start. Default `0`.
192
+ - `data-fy-duration` optionally limits how many source seconds are used.
193
+ - `data-fy-volume` sets volume from `0` to `1`. Default `1`.
194
+
195
+ Do not call `play()`, `pause()`, or set `currentTime` in a controller. Media must be in the light DOM; shadow roots are not scanned. Sources must be local project-relative files; live streams, remote URLs, `<source>` children, and `<track>` children are not supported. The runtime loads the complete file during the loading phase and seeks video after each `update()`.
196
+
197
+ An unmuted video's audio is extracted and mixed into the output timeline. Set `muted` or `data-fy-audio="off"` for a silent visual asset.
198
+
199
+ ## Transcripts
200
+
201
+ Declare canonical transcript requirements with:
202
+
203
+ ```html
204
+ <fy-transcript src="assets/voice.transcript.json" words></fy-transcript>
205
+ ```
206
+
207
+ `words` requires measured word timings. `check`, `still`, and `render` validate the transcript file. Visual presentation is the composition's responsibility; a controller can fetch the JSON and render timed text or karaoke. See [subtitles and karaoke](subtitles-karaoke.md).
208
+
90
209
  ## Loading assets
91
210
 
92
- Remote data, images, fonts, and other assets must resolve during `mount()` or before the first frame. The production browser blocks network requests after loading begins. A fetch from `update()` is a bug.
211
+ Remote data, images, fonts, and other assets must resolve during the loading phase before frame production. The production browser blocks network requests after loading begins. A fetch from `update()` is a bug.
93
212
 
94
- Images inserted in `mount()` are awaited by the runtime. Static fonts should be loaded with `document.fonts.ready` or an explicit `FontFace` before frame production.
213
+ Images declared in the document are awaited by the host. Static fonts should be loaded with `document.fonts.ready` or an explicit `FontFace` before frame production.
95
214
 
96
215
  ## Validation and inspection
97
216
 
@@ -100,8 +219,8 @@ npx frameyard check hello --json
100
219
  npx frameyard still hello --frame 42 --out hello-42.png --json
101
220
  ```
102
221
 
103
- `check` mounts the composition, evaluates every frame, and validates audio and declared transcripts without capturing or encoding. Do not replace it with a single still.
222
+ `check` runs static validation before Chromium, then evaluates every frame and validates audio and declared transcripts without capturing or encoding. Read the structured `diagnostics` array. Do not replace `check` with a single still.
104
223
 
105
224
  ## Metadata is fixed
106
225
 
107
- `width`, `height`, `fps`, and `durationInFrames` cannot be derived from props. Choose a reusable resolution, frame rate, and duration; put variation in props. Frameyard does not change duration to fit audio.
226
+ `width`, `height`, `fps`, and `duration-in-frames` cannot be derived from props. Choose a reusable resolution, frame rate, and duration; put variation in props. Frameyard does not change duration to fit audio.
@@ -0,0 +1,90 @@
1
+ # Diagnostics
2
+
3
+ Frameyard reports composition problems as structured diagnostics before and during frame production.
4
+
5
+ ## Static validation
6
+
7
+ `check`, `still`, and `render` run a Node-side static composition validation before launching Chromium. `dev` runs it when a composition is requested. Static errors fail the command without starting a browser; warnings allow frame evaluation.
8
+
9
+ Static validation covers:
10
+
11
+ - doctype, root count, metadata, and default-props JSON
12
+ - controller declaration and file existence
13
+ - executable scripts, `on*` handlers, `javascript:` URLs, and unsupported visual elements
14
+ - unknown `data-fy-*` attributes, binding paths, `if`/`repeat` placement, and animation attributes
15
+ - audio/video/transcript references and ordinary image/stylesheet assets
16
+ - visual content outside `<fy-composition>`
17
+
18
+ ## Diagnostic shape
19
+
20
+ ```json
21
+ {
22
+ "severity": "warning",
23
+ "code": "BINDING_PATH_MISSING",
24
+ "message": "data-fy-text references prop \"titel\", which is not in the resolved props.",
25
+ "details": {
26
+ "path": "compositions/hello.html",
27
+ "line": 12,
28
+ "column": 5,
29
+ "element": "h1",
30
+ "attribute": "data-fy-text",
31
+ "value": "titel",
32
+ "suggestion": "title"
33
+ }
34
+ }
35
+ ```
36
+
37
+ `details` may include `path`, `line`, `column`, `endLine`, `endColumn`, `element`, `attribute`, `value`, `suggestion`, `frame`, and `controller`.
38
+
39
+ ## Common codes
40
+
41
+ | Code | Severity | Meaning |
42
+ | --- | --- | --- |
43
+ | `COMPOSITION_INVALID` | error | Missing/invalid doctype, root, metadata, or document structure |
44
+ | `PROPS_INVALID` | error | Invalid default props JSON, invalid overrides, or `validateProps` failure |
45
+ | `CONTROLLER_INVALID` | error | Missing/duplicate/both-inline-and-external controller, missing file, or invalid exports |
46
+ | `BINDING_INVALID` | error | Unknown `data-fy-*` attribute or malformed binding path |
47
+ | `BINDING_PATH_MISSING` | warning | A binding path is absent from the resolved props |
48
+ | `BINDING_VALUE_INVALID` | warning | A text/attribute/class binding resolved to an object or array |
49
+ | `BINDING_REPEAT_INVALID` | error | Repeat is not on `<template>`, or its path is missing/not an array |
50
+ | `BINDING_REPEAT_NESTED` | error | Nested `data-fy-repeat` |
51
+ | `BINDING_TARGET_INVALID` | error | `data-fy-text` on a void element or the root; invalid binding target |
52
+ | `ANIMATION_INVALID` | error | Invalid `data-fy-anim-*` value |
53
+ | `ANIMATION_NOT_FOUND` | warning | `data-fy-anim-*` is present but the element has no CSS animation |
54
+ | `ANIMATION_MULTIPLE` | warning | Multiple animations on one declaratively scheduled element |
55
+ | `SCRIPT_NOT_ALLOWED` | error | Executable `<script>` other than the controller |
56
+ | `EVENT_HANDLER_NOT_ALLOWED` | error | Inline `on*` handler or `data-fy-attr-on*` |
57
+ | `URL_NOT_ALLOWED` | error | `javascript:` URL |
58
+ | `ELEMENT_NOT_SUPPORTED` | error | `iframe`, `object`, `embed`, `base`, refresh/CSP meta, `source`, `track`, or declarative shadow DOM |
59
+ | `ELEMENT_UNKNOWN` | warning | Unknown `fy-*` element |
60
+ | `CONTENT_OUTSIDE_ROOT` | warning | Visual content outside `<fy-composition>` |
61
+ | `MEDIA_INVALID` | error | Invalid or missing audio/video source or attributes |
62
+ | `TRANSCRIPT_INVALID` | error | Invalid or missing transcript source |
63
+ | `ASSET_NOT_FOUND` | error | Missing image, stylesheet, media, or transcript file |
64
+ | `AUDIO_WARNING` | warning | Audio timeline overrun or silent embedded media |
65
+ | `TRANSCRIPT_WARNING` | warning | Empty transcript text or no timed segments |
66
+ | `DETERMINISM_WARNING` | warning | Forbidden clock/randomness, CSS transition, or runtime diagnostic |
67
+ | `COMPOSITION_NOT_READY` | error | `seek`/`settle` before a successful load |
68
+ | `COMPOSITION_DISPOSED` | error | Operation after the host was disposed |
69
+
70
+ ## Results
71
+
72
+ `check`, `still`, and `render` JSON results expose:
73
+
74
+ - `entry`: project-relative HTML path
75
+ - `controller`: `{ kind: "inline" }`, `{ kind: "external", src }`, or `null`
76
+ - `diagnostics`: all static and runtime diagnostics, deduplicated
77
+
78
+ `check` also exposes `metadata`, `framesChecked`, `audio.details`, and `transcripts.details`. `still` exposes `frame`, `path`, `width`, and `height`. `render` exposes the output path, dimensions, frame count, concurrency, bytes, and audio/transcript clip counts.
79
+
80
+ Failures use the same envelope with `error.code`, `error.message`, optional `error.details`, and top-level `diagnostics`.
81
+
82
+ ## Working with diagnostics
83
+
84
+ 1. Run `npx frameyard check <id> --json`.
85
+ 2. Read every error first; fix the first one and rerun.
86
+ 3. Treat warnings as actionable. `BINDING_PATH_MISSING` is usually a typo; use the `suggestion` field.
87
+ 4. Use `details.path`, `line`, and `column` to edit the exact source.
88
+ 5. Re-run `check`, inspect representative stills, then render.
89
+
90
+ Do not ignore a failing `check`, a missing still, or a nonzero render. Do not describe visual correctness you did not inspect.
@@ -1,6 +1,6 @@
1
1
  # Failure recovery
2
2
 
3
- Do not claim success after a nonzero exit. Fix the cause, rerun `check`, and only then render.
3
+ Do not claim success after a nonzero exit. Fix the cause, rerun `check`, and only then render. `check`, `still`, and `render` run static composition validation before Chromium; read the structured `diagnostics` array for source locations.
4
4
 
5
5
  ## Common codes
6
6
 
@@ -8,28 +8,52 @@ Do not claim success after a nonzero exit. Fix the cause, rerun `check`, and onl
8
8
  | --- | --- | --- |
9
9
  | `USAGE` | Invalid flags or missing required arguments | Read `npx frameyard <command> --help`. |
10
10
  | `CONFIG_NOT_FOUND` | No `video.config.js` found | Run from the project root or pass `--project`. |
11
- | `CONFIG_INVALID` | Invalid config or composition entry | Fix `video.config.js`; entries must be explicit, local, and exist. |
11
+ | `CONFIG_INVALID` | Invalid config or composition entry | Fix `video.config.js`; entries must be explicit, local, `.html`, and exist. |
12
12
  | `COMPOSITION_NOT_FOUND` | ID is not listed | Add the entry to `video.config.js`. |
13
- | `COMPOSITION_INVALID` | Invalid metadata or lifecycle shape | Fix metadata and ensure `update()` is synchronous. |
14
- | `COMPOSITION_ERROR` | Mount, update, or browser page error | Read the frame location in the message; remove forbidden clocks/randomness. |
13
+ | `COMPOSITION_INVALID` | Invalid doctype, root, metadata, or document structure | Fix the composition document; check `details.line` and `details.attribute`. |
14
+ | `COMPOSITION_ERROR` | Mount, update, or browser page error | Read the frame location; remove forbidden clocks/randomness or fix the controller. |
15
15
  | `COMPOSITION_UPDATE_ASYNC` | `update()` returned a promise | Move async work to `mount()`. |
16
- | `PROPS_INVALID` | Props are not JSON or fail `validateProps()` | Fix the props file or validator. |
17
- | `FRAME_OUT_OF_RANGE` | Frame is outside `0..durationInFrames - 1` | Choose a valid frame. |
18
- | `AUDIO_INVALID` | Bad clip or unreadable/unsupported audio | Use a local project-relative file with a decodable audio stream. |
16
+ | `COMPOSITION_NOT_READY` | `seek`/`settle` before a successful load | Await `load`/`ready`. |
17
+ | `COMPOSITION_DISPOSED` | Operation after host disposal | Reload the composition. |
18
+ | `CONTROLLER_INVALID` | Missing, duplicate, or invalid controller declaration/file/exports | Keep exactly one inline or external controller; verify its exports and path. |
19
+ | `BINDING_INVALID` | Unknown `data-fy-*` attribute or malformed path | Fix the attribute name or path. |
20
+ | `BINDING_PATH_MISSING` | Binding path absent from resolved props | Fix the prop or path; use `details.suggestion`. |
21
+ | `BINDING_VALUE_INVALID` | Object/array used where a primitive is required | Use a string/number/boolean or move the logic to a controller. |
22
+ | `BINDING_REPEAT_INVALID` | Repeat not on `<template>`, or path missing/not an array | Use `<template data-fy-repeat="items">` with an array. |
23
+ | `BINDING_REPEAT_NESTED` | Nested repeat | Use a controller for nested data. |
24
+ | `BINDING_TARGET_INVALID` | `data-fy-text` on a void element or the root | Move the binding to a text container. |
25
+ | `ANIMATION_INVALID` | Invalid `data-fy-anim-*` value | Use non-negative integer frames and a positive duration. |
26
+ | `ANIMATION_NOT_FOUND` | `data-fy-anim-*` without a CSS animation | Add an `animation` shorthand or remove the attributes. |
27
+ | `ANIMATION_MULTIPLE` | Multiple animations on one scheduled element | Use one animation or move scheduling to CSS/the controller. |
28
+ | `SCRIPT_NOT_ALLOWED` | Executable script other than the controller | Move code into the controller. |
29
+ | `EVENT_HANDLER_NOT_ALLOWED` | Inline `on*` handler or `data-fy-attr-on*` | Remove the handler; use a controller. |
30
+ | `URL_NOT_ALLOWED` | `javascript:` URL | Use a safe URL. |
31
+ | `ELEMENT_NOT_SUPPORTED` | `iframe`, `object`, `embed`, `base`, refresh/CSP meta, `source`, `track`, or declarative shadow DOM | Use supported DOM/SVG/canvas/video sources. |
32
+ | `ELEMENT_UNKNOWN` | Unknown `fy-*` element | Use `fy-composition`/`fy-transcript` or a normal element. |
33
+ | `CONTENT_OUTSIDE_ROOT` | Visual content outside `<fy-composition>` | Move it inside the root. |
34
+ | `PROPS_INVALID` | Invalid props JSON or `validateProps()` failure | Fix the props file or validator. |
35
+ | `FRAME_OUT_OF_RANGE` | Frame outside `0..durationInFrames - 1` | Choose a valid frame. |
36
+ | `AUDIO_INVALID` | Bad clip or unreadable/unsupported audio | Use a local project-root-relative file with a decodable audio stream. |
37
+ | `AUDIO_WARNING` | Audio timeline overrun or silent embedded media | Adjust timing/trim or remove the silent source. |
38
+ | `MEDIA_INVALID` | Invalid media source or attributes | Use local, seekable files and valid `data-fy-*` values. |
39
+ | `MEDIA_SEEK_FAILED` | Embedded video did not reach the requested time | Re-encode with a regular keyframe interval or replace the source. |
40
+ | `TRANSCRIPT_INVALID` | Invalid/missing transcript declaration or JSON | Regenerate with `npx frameyard transcribe`; do not hand-edit timestamps. |
41
+ | `TRANSCRIPT_WORDS_UNAVAILABLE` | Karaoke requested without measured word timings | Re-run transcription with `--words` or remove `words`. |
42
+ | `TRANSCRIPT_WARNING` | Empty transcript text or no timed segments | Regenerate the transcript. |
19
43
  | `ASSET_NOT_FOUND` | Required file is missing or unreadable | Restore the asset, model, or text file. |
44
+ | `DETERMINISM_WARNING` | Forbidden clock/randomness, CSS transition, or runtime warning | Derive state from `frame`/`props`; use `random("stable:key")`. |
20
45
  | `OUTPUT_EXISTS` | Output exists and replacement was not approved | Reuse it, choose another path, or pass `--overwrite`/`--force` with permission. |
21
46
  | `OUTPUT_FAILED` | An output or manifest could not be written atomically | Check directory permissions and free space; rerun. |
22
47
  | `EXECUTABLE_NOT_FOUND` | `qwen-tts` or `whisper-cli` could not be resolved | Pass `--executable`, set the environment variable, or fix `PATH`. |
23
48
  | `TTS_FAILED` | Speech synthesis failed or produced an invalid WAV | Read stderr; verify the talker/codec models and CLI compatibility. |
24
49
  | `TRANSCRIBE_FAILED` | Normalization or `whisper-cli` failed | Verify FFmpeg, the source media, and the Whisper model. |
25
- | `TRANSCRIPT_INVALID` | Canonical transcript JSON is malformed | Regenerate it with `npx frameyard transcribe`; do not hand-edit timestamps. |
26
- | `TRANSCRIPT_WORDS_UNAVAILABLE` | Karaoke requested without measured word timings | Re-run transcription with `--words` or remove `words: true`. |
27
50
  | `MANIFEST_INVALID` | Provenance manifest is corrupt or unsupported | Restore or regenerate the manifest; do not invent hashes. |
28
51
  | `FFMPEG_FAILED` | FFmpeg is missing, lacks encoders, or failed | Run `npx frameyard doctor --json`; install FFmpeg with libx264 and AAC. |
29
52
  | `FFPROBE_FAILED` | FFprobe is missing or failed | Install FFmpeg/FFprobe together. |
30
53
  | `BROWSER_FAILED` | Chromium could not start | Install the Playwright Chromium build. |
31
54
  | `RENDER_FAILED` | Capture or encoding failed | Read stderr; rerun `check` and a representative `still`. |
32
55
  | `CANCELLED` | The operation was interrupted | Rerun when ready. Partial outputs are removed. |
56
+ | `INTERNAL` | Unexpected internal failure | Rerun with `FRAMEYARD_DEBUG=1` and report the stack. |
33
57
 
34
58
  ## Behavior to rely on
35
59
 
@@ -38,6 +62,8 @@ Do not claim success after a nonzero exit. Fix the cause, rerun `check`, and onl
38
62
  - Existing outputs are replaced only with `--overwrite` (renders/stills) or `--force` (generated assets).
39
63
  - Cancellation terminates the child inference process and cleans temporary work.
40
64
  - `check` validates every frame and declared transcripts without encoding.
65
+ - Static validation failures exit before Chromium starts.
66
+ - `check`, `still`, and `render` seek embedded videos and wait for decode before evaluating or capturing a frame.
41
67
  - `doctor` inspects but never repairs, downloads, or modifies the environment.
42
68
 
43
69
  ## When visual inspection is impossible
@@ -48,10 +74,10 @@ If the host cannot display images, say explicitly that representative stills cou
48
74
 
49
75
  Report all of:
50
76
 
51
- - source project path and composition entry;
77
+ - source project path, composition entry, and controller;
52
78
  - props file or inline props used;
53
79
  - generated audio/transcript paths and provenance manifest path;
54
- - `check` result, including warnings;
80
+ - `check` result, including diagnostics and warnings;
55
81
  - still paths and whether they were viewed;
56
82
  - final video path, frame count, and audio/transcript summary;
57
- - any skipped step or unresolved warning.
83
+ - any skipped step or unresolved diagnostic.