frameyard 0.7.0 → 0.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +164 -7
- package/GLOSSARY.md +152 -18
- package/README.md +328 -285
- package/dist/assets/integrations.d.ts +8 -4
- package/dist/assets/integrations.d.ts.map +1 -1
- package/dist/assets/integrations.js +8 -4
- package/dist/assets/integrations.js.map +1 -1
- package/dist/assets/provenance.d.ts +2 -2
- package/dist/assets/provenance.d.ts.map +1 -1
- package/dist/assets/provenance.js +5 -3
- package/dist/assets/provenance.js.map +1 -1
- package/dist/assets/tts-supertonic.d.ts +28 -0
- package/dist/assets/tts-supertonic.d.ts.map +1 -0
- package/dist/assets/tts-supertonic.js +156 -0
- package/dist/assets/tts-supertonic.js.map +1 -0
- package/dist/assets/tts.d.ts +18 -6
- package/dist/assets/tts.d.ts.map +1 -1
- package/dist/assets/tts.js +219 -97
- package/dist/assets/tts.js.map +1 -1
- package/dist/cli.js +1705 -106
- package/dist/cli.js.map +1 -1
- package/dist/composition/descriptor.d.ts +17 -3
- package/dist/composition/descriptor.d.ts.map +1 -1
- package/dist/composition/descriptor.js +51 -28
- package/dist/composition/descriptor.js.map +1 -1
- package/dist/composition/format.d.ts +12 -0
- package/dist/composition/format.d.ts.map +1 -0
- package/dist/composition/format.js +28 -0
- package/dist/composition/format.js.map +1 -0
- package/dist/composition/index.d.ts +2 -1
- package/dist/composition/index.d.ts.map +1 -1
- package/dist/composition/index.js +1 -0
- package/dist/composition/index.js.map +1 -1
- package/dist/context.d.ts +82 -0
- package/dist/context.d.ts.map +1 -0
- package/dist/context.js +277 -0
- package/dist/context.js.map +1 -0
- package/dist/contracts.d.ts +80 -6
- package/dist/contracts.d.ts.map +1 -1
- package/dist/contracts.js +3 -0
- package/dist/contracts.js.map +1 -1
- package/dist/doctor.d.ts +14 -1
- package/dist/doctor.d.ts.map +1 -1
- package/dist/doctor.js +166 -7
- package/dist/doctor.js.map +1 -1
- package/dist/errors.d.ts +1 -1
- package/dist/errors.d.ts.map +1 -1
- package/dist/errors.js.map +1 -1
- package/dist/fonts/add.d.ts +41 -0
- package/dist/fonts/add.d.ts.map +1 -0
- package/dist/fonts/add.js +259 -0
- package/dist/fonts/add.js.map +1 -0
- package/dist/fonts/binary.d.ts +4 -0
- package/dist/fonts/binary.d.ts.map +1 -0
- package/dist/fonts/binary.js +21 -0
- package/dist/fonts/binary.js.map +1 -0
- package/dist/fonts/css.d.ts +4 -0
- package/dist/fonts/css.d.ts.map +1 -0
- package/dist/fonts/css.js +50 -0
- package/dist/fonts/css.js.map +1 -0
- package/dist/fonts/fontsource.d.ts +107 -0
- package/dist/fonts/fontsource.d.ts.map +1 -0
- package/dist/fonts/fontsource.js +565 -0
- package/dist/fonts/fontsource.js.map +1 -0
- package/dist/fonts/index.d.ts +11 -0
- package/dist/fonts/index.d.ts.map +1 -0
- package/dist/fonts/index.js +11 -0
- package/dist/fonts/index.js.map +1 -0
- package/dist/fonts/install.d.ts +24 -0
- package/dist/fonts/install.d.ts.map +1 -0
- package/dist/fonts/install.js +145 -0
- package/dist/fonts/install.js.map +1 -0
- package/dist/fonts/manifest.d.ts +21 -0
- package/dist/fonts/manifest.d.ts.map +1 -0
- package/dist/fonts/manifest.js +96 -0
- package/dist/fonts/manifest.js.map +1 -0
- package/dist/fonts/options.d.ts +9 -0
- package/dist/fonts/options.d.ts.map +1 -0
- package/dist/fonts/options.js +66 -0
- package/dist/fonts/options.js.map +1 -0
- package/dist/fonts/schema.d.ts +4 -0
- package/dist/fonts/schema.d.ts.map +1 -0
- package/dist/fonts/schema.js +316 -0
- package/dist/fonts/schema.js.map +1 -0
- package/dist/fonts/summary.d.ts +16 -0
- package/dist/fonts/summary.d.ts.map +1 -0
- package/dist/fonts/summary.js +22 -0
- package/dist/fonts/summary.js.map +1 -0
- package/dist/fonts/types.d.ts +56 -0
- package/dist/fonts/types.d.ts.map +1 -0
- package/dist/fonts/types.js +8 -0
- package/dist/fonts/types.js.map +1 -0
- package/dist/fonts/verify.d.ts +33 -0
- package/dist/fonts/verify.d.ts.map +1 -0
- package/dist/fonts/verify.js +200 -0
- package/dist/fonts/verify.js.map +1 -0
- package/dist/html/document-adapter.d.ts +3 -3
- package/dist/html/document-adapter.d.ts.map +1 -1
- package/dist/html/document-adapter.js +1 -1
- package/dist/html/document-adapter.js.map +1 -1
- package/dist/html/index.d.ts +1 -1
- package/dist/html/index.d.ts.map +1 -1
- package/dist/html/parse.d.ts +10 -2
- package/dist/html/parse.d.ts.map +1 -1
- package/dist/html/parse.js +14 -5
- package/dist/html/parse.js.map +1 -1
- package/dist/html/validate.d.ts +12 -1
- package/dist/html/validate.d.ts.map +1 -1
- package/dist/html/validate.js +292 -61
- package/dist/html/validate.js.map +1 -1
- package/dist/index.d.ts +4 -4
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +2 -2
- package/dist/index.js.map +1 -1
- package/dist/inspect.d.ts +61 -0
- package/dist/inspect.d.ts.map +1 -0
- package/dist/inspect.js +325 -0
- package/dist/inspect.js.map +1 -0
- package/dist/lint.d.ts +63 -0
- package/dist/lint.d.ts.map +1 -0
- package/dist/lint.js +265 -0
- package/dist/lint.js.map +1 -0
- package/dist/policy.d.ts +69 -0
- package/dist/policy.d.ts.map +1 -0
- package/dist/policy.js +361 -0
- package/dist/policy.js.map +1 -0
- package/dist/preview/main.d.ts +2 -2
- package/dist/preview/main.d.ts.map +1 -1
- package/dist/preview/main.js +35 -14
- package/dist/preview/main.js.map +1 -1
- package/dist/preview/timeline-data.d.ts +17 -65
- package/dist/preview/timeline-data.d.ts.map +1 -1
- package/dist/preview/timeline-host.d.ts.map +1 -1
- package/dist/preview/timeline-host.js +73 -38
- package/dist/preview/timeline-host.js.map +1 -1
- package/dist/preview/timeline-main.d.ts.map +1 -1
- package/dist/preview/timeline-main.js +47 -3
- package/dist/preview/timeline-main.js.map +1 -1
- package/dist/project.d.ts +13 -8
- package/dist/project.d.ts.map +1 -1
- package/dist/project.js +20 -17
- package/dist/project.js.map +1 -1
- package/dist/renderer/browser.d.ts +2 -2
- package/dist/renderer/browser.d.ts.map +1 -1
- package/dist/renderer/browser.js +10 -4
- package/dist/renderer/browser.js.map +1 -1
- package/dist/renderer/ffmpeg.d.ts +44 -1
- package/dist/renderer/ffmpeg.d.ts.map +1 -1
- package/dist/renderer/ffmpeg.js +371 -73
- package/dist/renderer/ffmpeg.js.map +1 -1
- package/dist/renderer/index.d.ts +40 -4
- package/dist/renderer/index.d.ts.map +1 -1
- package/dist/renderer/index.js +100 -25
- package/dist/renderer/index.js.map +1 -1
- package/dist/renderer/keyframe-expressions.d.ts +39 -0
- package/dist/renderer/keyframe-expressions.d.ts.map +1 -0
- package/dist/renderer/keyframe-expressions.js +180 -0
- package/dist/renderer/keyframe-expressions.js.map +1 -0
- package/dist/renderer/timeline.d.ts +2 -0
- package/dist/renderer/timeline.d.ts.map +1 -1
- package/dist/renderer/timeline.js +168 -106
- package/dist/renderer/timeline.js.map +1 -1
- package/dist/runtime/host.d.ts.map +1 -1
- package/dist/runtime/host.js +159 -31
- package/dist/runtime/host.js.map +1 -1
- package/dist/scaffold.d.ts +3 -0
- package/dist/scaffold.d.ts.map +1 -1
- package/dist/scaffold.js +48 -317
- package/dist/scaffold.js.map +1 -1
- package/dist/server/composition-html.d.ts +15 -7
- package/dist/server/composition-html.d.ts.map +1 -1
- package/dist/server/composition-html.js +72 -16
- package/dist/server/composition-html.js.map +1 -1
- package/dist/server/server.d.ts.map +1 -1
- package/dist/server/server.js +56 -6
- package/dist/server/server.js.map +1 -1
- package/dist/server/timeline-preview-data.d.ts.map +1 -1
- package/dist/server/timeline-preview-data.js +12 -23
- package/dist/server/timeline-preview-data.js.map +1 -1
- package/dist/server/timeline-preview-html.d.ts.map +1 -1
- package/dist/server/timeline-preview-html.js +20 -3
- package/dist/server/timeline-preview-html.js.map +1 -1
- package/dist/templates.d.ts +16 -0
- package/dist/templates.d.ts.map +1 -0
- package/dist/templates.js +186 -0
- package/dist/templates.js.map +1 -0
- package/dist/timeline/easing.d.ts +36 -0
- package/dist/timeline/easing.d.ts.map +1 -0
- package/dist/timeline/easing.js +225 -0
- package/dist/timeline/easing.js.map +1 -0
- package/dist/timeline/keyframes.d.ts +23 -0
- package/dist/timeline/keyframes.d.ts.map +1 -0
- package/dist/timeline/keyframes.js +95 -0
- package/dist/timeline/keyframes.js.map +1 -0
- package/dist/timeline/plan.d.ts +6 -12
- package/dist/timeline/plan.d.ts.map +1 -1
- package/dist/timeline/plan.js +17 -30
- package/dist/timeline/plan.js.map +1 -1
- package/dist/timeline/resolve.d.ts +16 -3
- package/dist/timeline/resolve.d.ts.map +1 -1
- package/dist/timeline/resolve.js +119 -13
- package/dist/timeline/resolve.js.map +1 -1
- package/dist/timeline/scene.d.ts +126 -0
- package/dist/timeline/scene.d.ts.map +1 -0
- package/dist/timeline/scene.js +109 -0
- package/dist/timeline/scene.js.map +1 -0
- package/dist/timeline/schema.d.ts +38 -0
- package/dist/timeline/schema.d.ts.map +1 -1
- package/dist/timeline/schema.js +290 -9
- package/dist/timeline/schema.js.map +1 -1
- package/dist/timeline/transitions.d.ts +21 -0
- package/dist/timeline/transitions.d.ts.map +1 -0
- package/dist/timeline/transitions.js +23 -0
- package/dist/timeline/transitions.js.map +1 -0
- package/dist/version.d.ts +1 -1
- package/dist/version.js +1 -1
- package/docs/adr/0007-make-html-documents-the-composition-unit.md +2 -2
- package/docs/adr/0008-share-the-composition-document-grammar.md +1 -1
- package/docs/adr/0010-make-timelines-the-render-unit.md +3 -2
- package/docs/adr/0011-compose-timeline-frames-in-a-live-browser-host.md +1 -1
- package/docs/adr/0012-render-timelines-with-segmented-lossless-clip-intermediates.md +1 -1
- package/docs/adr/0014-compose-timeline-clips-with-css-blend-modes.md +38 -0
- package/docs/adr/0015-inherit-composition-format-from-the-project.md +28 -0
- package/docs/adr/0016-author-compositions-as-html-fragments.md +32 -0
- package/docs/adr/0017-render-timeless-compositions-as-stills.md +99 -0
- package/docs/adr/0018-animate-timeline-clips-with-keyframes.md +35 -0
- package/docs/adr/0019-make-same-track-overlap-the-transition-window.md +36 -0
- package/docs/adr/0020-crossfade-audio-clips-on-audio-tracks.md +27 -0
- package/docs/adr/0021-project-timeline-scenes-once.md +32 -0
- package/docs/adr/0022-gate-heavy-operations-with-a-workspace-resource-policy.md +35 -0
- package/docs/adr/0023-pin-fonts-as-project-assets.md +98 -0
- package/docs/migration-0.4.md +5 -3
- package/docs/migration-0.8.md +163 -0
- package/docs/preview-testing.md +23 -3
- package/docs/{v0.1-design.md → v0.x-design.md} +86 -53
- package/examples/blend/README.md +106 -0
- package/examples/blend/assets/ink.mp4 +0 -0
- package/examples/blend/compositions/aurora.html +55 -0
- package/examples/blend/compositions/backdrop.html +74 -0
- package/examples/blend/compositions/labels.html +149 -0
- package/examples/blend/fy.config.js +11 -0
- package/examples/blend/scripts/make-ink.sh +26 -0
- package/examples/blend/timelines/blend.timeline.json +155 -0
- package/examples/demo/README.md +2 -2
- package/examples/demo/compositions/demo.html +243 -251
- package/examples/intro/README.md +71 -0
- package/examples/intro/assets/fonts/figtree/LICENSE.txt +93 -0
- package/examples/intro/assets/fonts/figtree/figtree-latin-wght-normal.woff2 +0 -0
- package/examples/intro/assets/fonts/instrument-serif/LICENSE.txt +93 -0
- package/examples/intro/assets/fonts/instrument-serif/instrument-serif-latin-400-italic.woff2 +0 -0
- package/examples/intro/assets/fonts/space-mono/LICENSE.txt +93 -0
- package/examples/intro/assets/fonts/space-mono/space-mono-latin-400-normal.woff2 +0 -0
- package/examples/intro/assets/fonts/space-mono/space-mono-latin-700-normal.woff2 +0 -0
- package/examples/intro/assets/grain.png +0 -0
- package/examples/intro/assets/score.ogg +0 -0
- package/examples/intro/compositions/brand.html +412 -0
- package/examples/intro/compositions/chrome.controller.js +63 -0
- package/examples/intro/compositions/chrome.html +256 -0
- package/examples/intro/compositions/lattice.controller.js +148 -0
- package/examples/intro/compositions/lattice.html +308 -0
- package/examples/intro/compositions/opening.html +504 -0
- package/examples/intro/compositions/studio.html +930 -0
- package/examples/intro/compositions/theme.css +218 -0
- package/examples/intro/fy.config.js +13 -0
- package/examples/intro/fy.fonts.json +100 -0
- package/examples/intro/scripts/make-grain.mjs +79 -0
- package/examples/intro/scripts/make-score.mjs +702 -0
- package/examples/intro/timelines/intro.timeline.json +60 -0
- package/examples/keyframes/README.md +76 -0
- package/examples/keyframes/compositions/backdrop.html +60 -0
- package/examples/keyframes/compositions/card.html +99 -0
- package/examples/keyframes/fy.config.js +10 -0
- package/examples/keyframes/timelines/keyframes.timeline.json +60 -0
- package/examples/nested/README.md +112 -0
- package/examples/nested/assets/pulse.wav +0 -0
- package/examples/nested/compositions/backdrop.html +84 -0
- package/examples/nested/compositions/card.html +275 -0
- package/examples/nested/compositions/overlay.html +307 -0
- package/examples/nested/fy.config.js +13 -0
- package/examples/nested/scripts/make-pulse.sh +20 -0
- package/examples/nested/timelines/chapter-a.timeline.json +39 -0
- package/examples/nested/timelines/chapter-b.timeline.json +39 -0
- package/examples/nested/timelines/nested-reel.timeline.json +52 -0
- package/examples/static/README.md +17 -0
- package/examples/static/compositions/still.html +37 -0
- package/examples/static/fy.config.js +9 -0
- package/examples/static/timelines/still.timeline.json +13 -0
- package/examples/transitions/README.md +128 -0
- package/examples/transitions/assets/alpha.wav +0 -0
- package/examples/transitions/assets/beta.wav +0 -0
- package/examples/transitions/assets/gamma.wav +0 -0
- package/examples/transitions/compositions/alpha.html +56 -0
- package/examples/transitions/compositions/beta.html +56 -0
- package/examples/transitions/compositions/gamma.html +56 -0
- package/examples/transitions/fy.config.js +12 -0
- package/examples/transitions/timelines/audio-crossfade.timeline.json +44 -0
- package/examples/transitions/timelines/transitions.timeline.json +31 -0
- package/examples/video/README.md +4 -4
- package/examples/video/compositions/video-mp4.html +349 -357
- package/examples/video/compositions/video-overlay.html +337 -345
- package/examples/video/compositions/video.html +349 -357
- package/examples/webgl/README.md +1 -1
- package/examples/webgl/compositions/webgl.html +205 -213
- package/package.json +13 -5
- package/scaffolds/init/AGENTS.md +131 -0
- package/scaffolds/init/README.md +56 -0
- package/scaffolds/init/compositions/hello.html +79 -0
- package/scaffolds/init/fy.config.js +9 -0
- package/scaffolds/init/gitignore.template +1 -0
- package/scaffolds/init/package.json +11 -0
- package/scaffolds/init/props/hello.json +4 -0
- package/scaffolds/init/props/main.json +6 -0
- package/scaffolds/init/timelines/main.timeline.json +20 -0
- package/scaffolds/local-models/LOCAL-MODELS.md +77 -0
- package/scaffolds/local-models/fy.policy.json +5 -0
- package/scaffolds/local-models/opencode/opencode.jsonc +17 -0
- package/scaffolds/local-models/pi/APPEND_SYSTEM.md +21 -0
- package/scaffolds/local-models/pi/sandbox.json +15 -0
- package/scaffolds/local-models/pi/settings.json +15 -0
- package/skills/frameyard/README.md +1 -0
- package/skills/frameyard/SKILL.md +59 -59
- package/skills/frameyard/references/audio-assets.md +32 -3
- package/skills/frameyard/references/common-mistakes.md +81 -7
- package/skills/frameyard/references/controllers.md +5 -4
- package/skills/frameyard/references/deterministic-authoring.md +49 -37
- package/skills/frameyard/references/diagnostics.md +42 -4
- package/skills/frameyard/references/failure-recovery.md +56 -10
- package/skills/frameyard/references/fonts.md +132 -0
- package/skills/frameyard/references/json-results.md +165 -10
- package/skills/frameyard/references/quick-reference.md +222 -0
- package/skills/frameyard/references/timelines.md +109 -15
- /package/examples/demo/{video.config.js → fy.config.js} +0 -0
- /package/examples/video/{video.config.js → fy.config.js} +0 -0
- /package/examples/webgl/{video.config.js → fy.config.js} +0 -0
|
@@ -0,0 +1,163 @@
|
|
|
1
|
+
# Migrating to Frameyard 0.8
|
|
2
|
+
|
|
3
|
+
0.8 changes compositions from full HTML documents to partial HTML fragments and adds static (timeless) compositions. The framework generates the document shell, import map, sizing style, format block, and host runtime. Timelines, props files, audio, transcripts, and media keep their existing model; the CLI adds PNG output for static compositions and discriminated result kinds.
|
|
4
|
+
|
|
5
|
+
## Why
|
|
6
|
+
|
|
7
|
+
Every composition used to repeat `<!doctype html>`, `<html>`, `<head>`, `<meta charset>`, and `<body>`, and the author owned a `<head>` that the server already injected into. The fragment model keeps the same URLs, CSS, and relative asset behavior while making the framework responsible for the shell.
|
|
8
|
+
|
|
9
|
+
## Before and after
|
|
10
|
+
|
|
11
|
+
Before:
|
|
12
|
+
|
|
13
|
+
```html
|
|
14
|
+
<!doctype html>
|
|
15
|
+
<html>
|
|
16
|
+
<head>
|
|
17
|
+
<meta charset="utf-8">
|
|
18
|
+
<script type="application/json" data-fy="props">
|
|
19
|
+
{ "title": "Hello" }
|
|
20
|
+
</script>
|
|
21
|
+
<style>
|
|
22
|
+
#title { animation: fade-up 1s ease-out both; }
|
|
23
|
+
</style>
|
|
24
|
+
<script type="importmap">
|
|
25
|
+
{ "imports": { "demo-lib": "./demo-lib.js" } }
|
|
26
|
+
</script>
|
|
27
|
+
</head>
|
|
28
|
+
<body>
|
|
29
|
+
<fy-composition width="1920" height="1080" fps="30" duration-in-frames="900">
|
|
30
|
+
<h1 id="title" data-fy-text="title"></h1>
|
|
31
|
+
</fy-composition>
|
|
32
|
+
<script type="text/fy-controller" src="./hello.controller.js"></script>
|
|
33
|
+
</body>
|
|
34
|
+
</html>
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
After:
|
|
38
|
+
|
|
39
|
+
```html
|
|
40
|
+
<script type="application/json" data-fy="props">
|
|
41
|
+
{ "title": "Hello" }
|
|
42
|
+
</script>
|
|
43
|
+
<style>
|
|
44
|
+
#title { animation: fade-up 1s ease-out both; }
|
|
45
|
+
</style>
|
|
46
|
+
<script type="importmap">
|
|
47
|
+
{ "imports": { "demo-lib": "./demo-lib.js" } }
|
|
48
|
+
</script>
|
|
49
|
+
<fy-composition width="1920" height="1080" fps="30" duration-in-frames="900">
|
|
50
|
+
<h1 id="title" data-fy-text="title"></h1>
|
|
51
|
+
</fy-composition>
|
|
52
|
+
<script type="text/fy-controller" src="./hello.controller.js"></script>
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
The fragment is the content that used to live in `<body>`, plus the head elements that remain author-owned: props, styles, stylesheet links, import maps, and the controller.
|
|
56
|
+
|
|
57
|
+
## Rules
|
|
58
|
+
|
|
59
|
+
- Keep exactly one `<fy-composition>` element. It must be a top-level element, not nested in a `<div>`, `<template>`, or other element.
|
|
60
|
+
- Keep the props block, the controller, and import maps at the top level. They are metadata, not visual content.
|
|
61
|
+
- Keep `<style>` and `<link rel="stylesheet">` at the top level or inside the root. Top-level styles are hoisted into the generated head in source order; styles inside the root stay in the body.
|
|
62
|
+
- Do not add `<!doctype>`, `<html>`, `<head>`, `<body>`, `<title>`, or `<meta>`. A legacy document reports `COMPOSITION_DOCUMENT_SHELL`.
|
|
63
|
+
- Do not use `data-fy-host` or `data-fy="format"`; both are reserved for framework-injected markup.
|
|
64
|
+
- An import map must be a top-level `<script type="importmap">` with a JSON object. `frameyard` and `frameyard/` specifiers are reserved. Invalid maps report `IMPORT_MAP_INVALID`.
|
|
65
|
+
- `.html` is still the composition extension. `project.json` still reports `kind: "html"` and `entry` as the fragment path.
|
|
66
|
+
- Composition URLs, relative controller `src`, relative asset URLs, media paths, and transcript paths keep their existing resolution rules.
|
|
67
|
+
|
|
68
|
+
## If the root was nested
|
|
69
|
+
|
|
70
|
+
Move the root to the top level and move the wrapper's styling into the root or into a top-level `<style>`:
|
|
71
|
+
|
|
72
|
+
Before:
|
|
73
|
+
|
|
74
|
+
```html
|
|
75
|
+
<body>
|
|
76
|
+
<div class="page">
|
|
77
|
+
<fy-composition duration-in-frames="150">...</fy-composition>
|
|
78
|
+
</div>
|
|
79
|
+
</body>
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
After:
|
|
83
|
+
|
|
84
|
+
```html
|
|
85
|
+
<fy-composition duration-in-frames="150">
|
|
86
|
+
<div class="page">...</div>
|
|
87
|
+
</fy-composition>
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
## Static compositions
|
|
91
|
+
|
|
92
|
+
A composition that declares neither `duration-in-frames` nor `fps` is now
|
|
93
|
+
static. It is captured once and `render <id>` writes `out/<id>.png`.
|
|
94
|
+
|
|
95
|
+
If a composition was failing because `duration-in-frames` was missing, decide
|
|
96
|
+
which behavior you intended:
|
|
97
|
+
|
|
98
|
+
- For a still, remove `fps` too and use the composition directly or as a
|
|
99
|
+
timeline image clip.
|
|
100
|
+
- For video, add both `duration-in-frames` and `fps` (or let `fps` inherit from
|
|
101
|
+
the project `format`). Declaring `fps` without `duration-in-frames` remains an
|
|
102
|
+
error.
|
|
103
|
+
|
|
104
|
+
A static composition needs width and height, declared on the root or inherited
|
|
105
|
+
from the project `format`. It is visual-only: `<img>`, CSS, SVG, and canvas are
|
|
106
|
+
allowed; `<video>`, `<audio>`, and `<fy-transcript>` are validation errors. Its
|
|
107
|
+
controller may use `mount()` and `validateProps()`, but `update()` is not called
|
|
108
|
+
and frame-state variables are not set. Put audio, video, and captions on a
|
|
109
|
+
timeline instead.
|
|
110
|
+
|
|
111
|
+
`render` for a static composition rejects `--container`, `--codec`,
|
|
112
|
+
`--encoder`, and `--concurrency`. Use a timeline with an image/static
|
|
113
|
+
composition clip when the deliverable must be a video.
|
|
114
|
+
|
|
115
|
+
## Static timeline clips
|
|
116
|
+
|
|
117
|
+
A clip whose composition source is static behaves like an image clip:
|
|
118
|
+
|
|
119
|
+
- `duration` is required;
|
|
120
|
+
- `sourceIn` must be `0`;
|
|
121
|
+
- `props`, transform, opacity, and blend mode still apply;
|
|
122
|
+
- the clip contributes no audio or transcripts.
|
|
123
|
+
|
|
124
|
+
The renderer captures one transparent PNG per composition/props pair and reuses
|
|
125
|
+
it across segments.
|
|
126
|
+
|
|
127
|
+
## JSON result kinds
|
|
128
|
+
|
|
129
|
+
`render` now returns a discriminated union. Video results keep their fields and
|
|
130
|
+
add `kind: "video"`. Static composition results add `kind: "image"` with
|
|
131
|
+
`path`, `width`, `height`, `bytes`, `diagnostics`, and `elapsedMs`; they have no
|
|
132
|
+
`fps`, `durationInFrames`, `framesRendered`, `codec`, `container`, `encoder`, or
|
|
133
|
+
audio/transcript fields.
|
|
134
|
+
|
|
135
|
+
`check` also returns a union. Animated results add `kind: "animated"`. Static
|
|
136
|
+
results use `kind: "static"`, `metadata: { width, height, static: true }`,
|
|
137
|
+
`framesChecked: 1`, and no audio/transcript fields.
|
|
138
|
+
|
|
139
|
+
`still` on a static composition accepts only frame `0`; other frames fail with
|
|
140
|
+
`FRAME_OUT_OF_RANGE`.
|
|
141
|
+
|
|
142
|
+
Library consumers also see a type change: `FrameyardHost.metadata`,
|
|
143
|
+
`CompositionSourceInfo.metadata`, and browser sessions now expose
|
|
144
|
+
`ResolvedCompositionMetadata`, a union of animated metadata and
|
|
145
|
+
`{ width, height, static: true }`. Narrow with `metadata.static === true`
|
|
146
|
+
before reading `fps` or `durationInFrames`.
|
|
147
|
+
|
|
148
|
+
## Error codes
|
|
149
|
+
|
|
150
|
+
- `COMPOSITION_DOCUMENT_SHELL`: remove the document shell. The diagnostic includes the element and line.
|
|
151
|
+
- `IMPORT_MAP_INVALID`: fix the import map placement, JSON, attributes, or reserved specifier.
|
|
152
|
+
- `PROPS_INVALID` / `CONTROLLER_INVALID`: move the block to the top level and keep exactly one controller.
|
|
153
|
+
- `COMPOSITION_INVALID`: the root is missing, duplicated, or nested, or `fps` is declared without `duration-in-frames`.
|
|
154
|
+
- `STATIC_ANIMATION_UNSUPPORTED`: remove `data-fy-anim-*` or declare `duration-in-frames` and `fps`.
|
|
155
|
+
- `STATIC_MEDIA_UNSUPPORTED`: move video, audio, or transcripts to a timeline, or declare time metadata.
|
|
156
|
+
- `STATIC_UPDATE_IGNORED`: move static initialization from `update()` to `mount()`.
|
|
157
|
+
- `STATIC_ANIMATION_PAUSED`: remove CSS animations/transitions from a static composition; capture uses time 0.
|
|
158
|
+
|
|
159
|
+
`check`, `still`, and `render` run the same validation before Chromium. Run `npx frameyard check <id> --json` after editing each composition.
|
|
160
|
+
|
|
161
|
+
## No migration command
|
|
162
|
+
|
|
163
|
+
0.8 does not ship a migration command. The transformation is mechanical but depends on what the old `<head>` contained and whether the root was nested. Use this guide, migrate one composition at a time, and let `check` report the remaining structural errors.
|
package/docs/preview-testing.md
CHANGED
|
@@ -12,6 +12,10 @@ expose the same seek/settle contract the renderer uses.
|
|
|
12
12
|
`settle()`, and `dispose()`.
|
|
13
13
|
- `seek()` is synchronous. Call `await settle()` before reading pixels; `load()`
|
|
14
14
|
resets the host, so apply props through it.
|
|
15
|
+
- A static composition reports `metadata: { width, height, static: true }`.
|
|
16
|
+
`seek()` is a no-op, `settle()` resolves, and the page disables the transport.
|
|
17
|
+
Wait for the static composition's own marker instead of `waitForFrame()`,
|
|
18
|
+
which expects a controller-updated `#box`.
|
|
15
19
|
|
|
16
20
|
## Timeline preview
|
|
17
21
|
|
|
@@ -21,6 +25,20 @@ expose the same seek/settle contract the renderer uses.
|
|
|
21
25
|
- Order: `await ready`, `await load()`, `await seek(frame)`, `await settle()`.
|
|
22
26
|
- The non-embedded page owns transport and audio; only the embedded host exposes
|
|
23
27
|
`window.__fyTimeline`.
|
|
28
|
+
- A static composition source carries `metadata.static: true` and no
|
|
29
|
+
`metadata.durationInFrames`. The host loads the layer once and skips per-frame
|
|
30
|
+
seek; the layer remains visible while the timeline playhead moves.
|
|
31
|
+
- The embedded host projects the serialized resolved timeline with the shared
|
|
32
|
+
scene module on every seek. Do not add a preview-only clip or transition
|
|
33
|
+
model; give the scene module what it needs instead.
|
|
34
|
+
- A keyframed clip updates the layer wrapper's inline `opacity` and `transform`
|
|
35
|
+
on every seek. Assert those styles or the rendered stage after `settle()`.
|
|
36
|
+
Markers are `.keyframe-marker` elements with a `data-frame` absolute timeline
|
|
37
|
+
frame and a channel/value tooltip; clicking one seeks the playhead.
|
|
38
|
+
- Inside a transition window both clips are active. The incoming layer's inline
|
|
39
|
+
`opacity` is `clipOpacity × eased progress` and it stacks above the outgoing
|
|
40
|
+
layer on the same track. The strip draws a `.transition` band with a
|
|
41
|
+
`from → to` tooltip; it is read-only.
|
|
24
42
|
|
|
25
43
|
## Waiting for a frame
|
|
26
44
|
|
|
@@ -44,9 +62,11 @@ the renderer, or the CLI, and before pushing or publishing.
|
|
|
44
62
|
|
|
45
63
|
## Notes
|
|
46
64
|
|
|
47
|
-
-
|
|
48
|
-
the
|
|
49
|
-
|
|
65
|
+
- Timeline layer iframes pin `color-scheme: light` on both the iframe element
|
|
66
|
+
and the embedded document root to match the production capture context.
|
|
67
|
+
The iframe element's `color-scheme` does not reach the embedded document, so a
|
|
68
|
+
composition that declares a dark scheme would otherwise paint an opaque canvas
|
|
69
|
+
base and hide every layer beneath it.
|
|
50
70
|
- A running `dev` server keeps the `dist/` modules it loaded. Restart it after
|
|
51
71
|
`pnpm build`.
|
|
52
72
|
- The timeline preview host is not the production compositor; compare pixels at
|
|
@@ -2,9 +2,9 @@
|
|
|
2
2
|
|
|
3
3
|
**Status:** Accepted
|
|
4
4
|
**Initial acceptance:** 2026-10-27
|
|
5
|
-
**Last design revision:** 2026-
|
|
5
|
+
**Last design revision:** 2026-11-04
|
|
6
6
|
|
|
7
|
-
This document records the decisions confirmed during design sessions. It is the living product and architecture baseline; later changes revise this document explicitly rather than silently broaden the scope.
|
|
7
|
+
This document records the decisions confirmed during design sessions. It is the living product and architecture baseline; later changes revise this document explicitly rather than silently broaden the scope. Architecture decisions that changed the baseline are recorded in `docs/adr/`.
|
|
8
8
|
|
|
9
9
|
## 1. Product contract
|
|
10
10
|
|
|
@@ -16,7 +16,7 @@ The target use cases are product demos, social clips, animated diagrams, data vi
|
|
|
16
16
|
|
|
17
17
|
The core success criterion is an end-to-end example in which a developer can:
|
|
18
18
|
|
|
19
|
-
1. Create reusable
|
|
19
|
+
1. Create reusable HTML compositions with a fixed duration and a format they either declare or inherit from the project.
|
|
20
20
|
2. Animate them with HTML, CSS, SVG, and, where needed, a seek-safe controller for Canvas 2D and WebGL.
|
|
21
21
|
3. Supply JSON props and synchronized audio.
|
|
22
22
|
4. Assemble compositions and local media into a timeline with tracks, clips, trims, layers, and nested timelines.
|
|
@@ -30,18 +30,20 @@ The agent-enabled success criterion additionally requires an agent using the off
|
|
|
30
30
|
|
|
31
31
|
### Project
|
|
32
32
|
|
|
33
|
-
- Projects use ordinary HTML, CSS, static assets, and native JavaScript controllers, either inline in a composition
|
|
34
|
-
- `
|
|
33
|
+
- Projects use ordinary HTML, CSS, static assets, and native JavaScript controllers, either inline in a composition fragment or in external modules.
|
|
34
|
+
- `fy.config.js` explicitly maps project IDs to source files. There is no directory scanning. It has three optional maps:
|
|
35
35
|
- `format`: the project's fixed `width`, `height`, and `fps`.
|
|
36
|
-
- `compositions`: IDs mapped to `.html` composition
|
|
36
|
+
- `compositions`: IDs mapped to `.html` composition fragments.
|
|
37
37
|
- `timelines`: IDs mapped to JSON timeline documents.
|
|
38
|
-
- `format` is required when a project has timelines. Every timeline uses it. A composition
|
|
38
|
+
- `format` is required when a project has timelines. Every timeline uses it. It also fills `width`, `height`, and `fps` for animated compositions that omit them; a declared value always wins. A composition that declares an `fps` must share the project frame rate exactly; a different size is fitted and transformed by the clip. A composition that declares neither `duration-in-frames` nor `fps` is static; it uses width and height, declared or inherited, and ignores the inherited fps. A composition that declares no dimensions in a project without `format` cannot render and fails with `COMPOSITION_FORMAT_UNRESOLVED`.
|
|
39
39
|
- Composition and timeline IDs share one namespace so `render <id>`, `check <id>`, and `still <id>` are unambiguous. The config rejects duplicates.
|
|
40
40
|
- A project may contain any number of compositions and timelines.
|
|
41
41
|
|
|
42
42
|
### Composition
|
|
43
43
|
|
|
44
|
-
- A composition is a
|
|
44
|
+
- A composition is a partial HTML fragment with one top-level `<fy-composition>` root. The framework wraps it in the document shell, import map, sizing style, and host runtime at serve and render time. `width`, `height`, and `fps` are optional and inherit from the project `format` when absent; declared values win. `width` and `height` are declared together or not at all. Props cannot change metadata.
|
|
45
|
+
- A composition that declares neither `duration-in-frames` nor `fps` is static: it has no frame clock, is captured once in its initial state, and renders to a PNG. Its controller may `mount()` and validate props, but `update()` is not called and frame-state variables are not set. CSS animations and transitions are paused at time 0 with a warning; `data-fy-anim-*`, `<video>`, `<audio>`, and `<fy-transcript>` are errors. `fps` without `duration-in-frames` is an error. See [ADR 0017](./adr/0017-render-timeless-compositions-as-stills.md).
|
|
46
|
+
- Props blocks, the controller, and import maps are top-level siblings of the root. Top-level `<style>` and `<link rel="stylesheet">` elements are hoisted into the generated head; styles inside the root stay in the body. A fragment must not contain `<!doctype>`, `<html>`, `<head>`, `<body>`, `<title>`, `<meta>`, or the reserved `data-fy-host` attribute.
|
|
45
47
|
- Default props are a JSON block, `<script type="application/json" data-fy="props">`. A load's JSON props merge shallowly over the defaults, are validated as JSON, deep-frozen, and fixed for that load.
|
|
46
48
|
- Props bind into the document through explicit `data-fy-*` attributes: `data-fy-text`, `data-fy-attr-*`, `data-fy-class-*`, `data-fy-if`, and `data-fy-repeat`. Primitive top-level props are also exposed as `--fy-prop-<name>` CSS custom properties. Bindings apply once per load; props are immutable.
|
|
47
49
|
- CSS and the Web Animations API are the animation language. `data-fy-anim-start`, `data-fy-anim-duration`, and `data-fy-anim-easing` schedule one animation per element in composition frames.
|
|
@@ -53,20 +55,23 @@ The agent-enabled success criterion additionally requires an agent using the off
|
|
|
53
55
|
|
|
54
56
|
- A timeline is a versioned JSON document. It references compositions and nested timelines by project ID and media clips by project-relative path.
|
|
55
57
|
- A timeline has a project-fixed format, an ordered list of tracks, and a list of clips. It may declare a `background` color and an optional `durationInFrames`.
|
|
56
|
-
- Tracks are `video` or `audio`. Video tracks composite from bottom to top; audio tracks mix. Each clip belongs to exactly one track. Clips on the same track cannot overlap; clips on different tracks can. Touching endpoints are allowed
|
|
58
|
+
- Tracks are `video` or `audio`. Video tracks composite from bottom to top; audio tracks mix. Each clip belongs to exactly one track. Clips on the same track cannot overlap unless a timeline-level `transitions` entry explicitly pairs them; clips on different tracks can. Touching endpoints are allowed but are not a transition. A transition's window is the overlap, and its declared geometry is `from.start < to.start < from.end < to.end`. See [ADR 0019](./adr/0019-make-same-track-overlap-the-transition-window.md).
|
|
57
59
|
- A clip has an ID, a track, a source, a timeline start, a source in-point, a duration, and clip-specific presentation settings. Source in-point, duration, and timeline start are integer project frames.
|
|
58
60
|
- Clip sources are:
|
|
59
61
|
- a composition, identified by its project ID;
|
|
60
62
|
- a nested timeline, identified by its project ID;
|
|
61
63
|
- a local video, image, or audio asset.
|
|
62
|
-
- A composition clip fixes props for that instance and may trim the composition's source range. The same composition may appear multiple times with different props, source ranges, transforms, and audio settings.
|
|
63
|
-
- A nested timeline clip is a visual and audio source. It uses the project format, trims with the same source in-point and duration, contributes its audio, and is rendered as a group so parent opacity and
|
|
64
|
-
- Media clips use local project files only. Video and audio durations, dimensions, and audio presence are probed before frame production. Images
|
|
65
|
-
- Visual clips support a static transform: fit policy (`contain`, `cover`, `stretch`), an optional explicit `width`/`height` box (both fields together, mutually exclusive with `scale`) or a `scale` multiplier, pixel offset, rotation, and opacity. `fit` applies inside the box; without one, the project frame is the box.
|
|
66
|
-
- Visual clips
|
|
64
|
+
- A composition clip fixes props for that instance and may trim the composition's source range. The same composition may appear multiple times with different props, source ranges, transforms, and audio settings. A clip whose composition source is static behaves like an image: it must declare `duration`, must use `sourceIn: 0`, has no intrinsic source duration, and contributes no audio or transcripts. The renderer captures one transparent PNG per composition/props pair and reuses it across segments.
|
|
65
|
+
- A nested timeline clip is a visual and audio source. It uses the project format, trims with the same source in-point and duration, contributes its audio, and is rendered as a group so parent opacity, transform, and blend mode apply to the whole nested result. A blend mode inside a nested timeline sees only that timeline's own backdrop; a transparent nested timeline does not see the parent's layers, and a declared nested background is part of the group. Cycles are rejected statically. Timeline clips do not override props of compositions inside the nested timeline.
|
|
66
|
+
- Media clips use local project files only. Video and audio durations, dimensions, and audio presence are probed before frame production. Images and static composition clips require an explicit duration. A video or audio clip that exceeds its source range is an error; an omitted duration uses the remaining source.
|
|
67
|
+
- Visual clips support a static transform: fit policy (`contain`, `cover`, `stretch`), an optional explicit `width`/`height` box (both fields together, mutually exclusive with `scale`) or a `scale` multiplier, pixel offset, rotation, and opacity. `fit` applies inside the box; without one, the project frame is the box. Rotation expands to the full rotated bounds rather than clipping corners.
|
|
68
|
+
- Visual clips may animate `x`, `y`, `scale`, `rotation`, and `opacity` with a per-channel `keyframes` object. Keys use clip-local integer frames; `0` is the clip start boundary and `duration` the end boundary. Keys hold before the first and after the last key, never extend the clip, and use a CSS timing function on the interval after each key. A keyed channel cannot also be an explicit scalar in `transform`. `opacity` and `scale` are clamped after easing; position and rotation may overshoot. `keyframes.scale` is mutually exclusive with an explicit `width`/`height` box. Timeline document version stays 1.
|
|
69
|
+
- A timeline may declare `transitions`: `{ from, to, kind, easing? }`, where `kind` is `dissolve` on a video track or `crossfade` on an audio track, and `easing` uses the keyframe CSS timing-function vocabulary. Both clips stay on one track and overlap; the window is `to.start .. from.end`. A dissolve composites the incoming clip above the outgoing clip with alpha `clipOpacity × eased progress`; both kinds apply equal-power audio gain (`sqrt(1 - t)` and `sqrt(t)`) to contributed audio or to the two audio clips. An overlap without a declared transition, a non-overlapping pair, reversed or contained geometry, cross-track pairs, a kind that does not match the track, duplicate participation, and unknown kinds are errors. Timeline document version stays 1.
|
|
70
|
+
- Visual clips may declare a static `blend` mode. It uses CSS `mix-blend-mode` semantics against the accumulated backdrop of all lower layers and the timeline background, respects the clip's source alpha, and defaults to `normal`. The supported modes are `normal`, `multiply`, `screen`, `overlay`, `darken`, `lighten`, `color-dodge`, `color-burn`, `hard-light`, `soft-light`, `difference`, and `exclusion`; `hue`, `saturation`, `color`, and `luminosity` are rejected. There are no animated blend modes, masks, filters, or other effects.
|
|
71
|
+
- Visual clips contribute their audio to the timeline mix unless muted: composition audio, embedded video audio, and nested timeline audio. Static compositions contribute no audio; clip audio settings on a static composition are ignored with a warning. Audio-only clips live on audio tracks and may source audio or video files. Per-clip and per-audio-track gain and mute are supported. There are no linked A/V pairs; a J/L cut is expressed by muting a video clip and placing a separate audio clip with a different source in-point.
|
|
67
72
|
- A timeline's duration is derived from the latest clip end. An explicit `durationInFrames` may extend it but not cut clips. A timeline with no clips is invalid; an audio-only timeline is valid and renders black video with audio.
|
|
68
|
-
- A timeline background is transparent by default. The final video composites it over black; a nested timeline remains transparent so lower layers can show through. A composition that paints no background is transparent and may be used as an overlay.
|
|
69
|
-
- The preview provides a read-only timeline view: tracks, clips, playhead, program monitor, and diagnostics. It is not a visual editor.
|
|
73
|
+
- A timeline background is transparent by default. Blend modes see the accumulated backdrop including this background. The final video composites it over black; a nested timeline remains transparent so lower layers can show through. A composition that paints no background is transparent and may be used as an overlay.
|
|
74
|
+
- The preview provides a read-only timeline view: tracks, clips, keyframe markers, playhead, program monitor, and diagnostics. It is not a visual editor.
|
|
70
75
|
|
|
71
76
|
## 3. Deterministic execution
|
|
72
77
|
|
|
@@ -78,9 +83,10 @@ The timeline frame is the master clock. A composition's frame is derived from th
|
|
|
78
83
|
- JavaScript animation runs in the controller's `update()`, which must be synchronous and seek-safe. It cannot depend on previously visited frames.
|
|
79
84
|
- Declarative bindings apply once per load, before the controller mounts. Controller-created DOM is the controller's responsibility.
|
|
80
85
|
- Independent `requestAnimationFrame` loops, timers, CSS transitions, wall-clock reads, and unseeded randomness are not supported animation drivers.
|
|
81
|
-
- Timeline
|
|
86
|
+
- Timeline clip keyframes animate `x`, `y`, `scale`, `rotation`, and `opacity` from the integer clip-local frame. They are evaluated against absolute timeline frames, so segment boundaries do not change values. Preview evaluates the exact CSS timing function; production generates FFmpeg expressions and approximates the curve. Blend modes remain static. Transition progress is likewise evaluated from the absolute timeline frame; only the incoming clip carries transition alpha during a dissolve, and both kinds use the same eased progress for equal-power audio gain.
|
|
82
87
|
- Preview and production seek composition clips explicitly. The timeline compositor is deterministic given the captured clip frames, media frames, clip settings, and segment boundaries.
|
|
83
|
-
-
|
|
88
|
+
- The resolved format—declared width, height, and fps combined with the project format—is part of the immutable frame environment, alongside props and loaded assets. Layouts that use viewport units or percentages therefore depend on the resolved format by design; preview and production always resolve the same values.
|
|
89
|
+
- Assets, remote data, images, fonts, generated audio, transcripts, and media are resolved before frame production. Project fonts are vendored locally, recorded in `fy.fonts.json`, and injected as `@font-face` rules; the host waits for `document.fonts.ready` and fails a used face that reports an error. For a segmented timeline render, each segment's composition and nested-timeline clips are loaded and settled before that segment's frames are captured; no frame production depends on a new network request. Network is blocked during capture.
|
|
84
90
|
- AI model inference is never part of the loading phase, controller lifecycle, or frame production.
|
|
85
91
|
- The baseline supports DOM, CSS, SVG, Canvas 2D, WebGL, and frame-seeked embedded video. Canvas 2D and WebGL are drawn from the current frame by the controller.
|
|
86
92
|
- Embedded video and audio inside compositions are declared with `data-fy-*` attributes. The runtime loads local files, pauses them, and seeks video from the frame-derived source time before capture. An unmuted video contributes its audio track to the timeline mix unless the composition opts out or the clip mutes it.
|
|
@@ -96,33 +102,33 @@ Preview and production share composition behavior and capture composition frames
|
|
|
96
102
|
|
|
97
103
|
## 4. Preview and production rendering
|
|
98
104
|
|
|
99
|
-
Preview and production execute the same composition
|
|
105
|
+
Preview and production execute the same composition sources and host runtime. They differ in which component drives the frame clock, which compositor combines timeline layers, and whether frames are captured. Both also project the same timeline scene from the resolved timeline: the ordered active clips with their transition roles, composite order, frame-to-source mapping, capture modes, and presentation defaults. The scene projection is browser-safe; capture sessions, audio mixing, and both compositors stay separate.
|
|
100
106
|
|
|
101
107
|
The preview provides:
|
|
102
108
|
|
|
103
|
-
- Composition preview: composition selection, JSON props editing, play/pause, frame-accurate scrubbing, frame-by-frame stepping with buttons or arrow keys, one-second jumps with buttons or Shift+arrow keys, current frame and time, audio controls, and diagnostics.
|
|
104
|
-
- Timeline preview: timeline selection, a read-only track/clip view, playhead, program monitor, frame-by-frame stepping with buttons or arrow keys, one-second jumps with buttons or Shift+arrow keys, audio controls, and diagnostics.
|
|
109
|
+
- Composition preview: composition selection, JSON props editing, play/pause, frame-accurate scrubbing, frame-by-frame stepping with buttons or arrow keys, one-second jumps with buttons or Shift+arrow keys, current frame and time, audio controls, and diagnostics. A static composition loads once and disables the transport.
|
|
110
|
+
- Timeline preview: timeline selection, a read-only track/clip view with keyframe markers, playhead, program monitor, frame-by-frame stepping with buttons or arrow keys, one-second jumps with buttons or Shift+arrow keys, audio controls, and diagnostics. A static composition layer stays loaded and is not seeked per frame.
|
|
105
111
|
- A bounded pool of active composition and nested-timeline clips. It loads clips around the playhead instead of preloading a whole long timeline.
|
|
106
112
|
|
|
107
|
-
The preview server serves each composition
|
|
113
|
+
The preview server serves each composition fragment at its project path, wraps it in the fixed document shell, and injects the framework import map and host runtime. It serves timeline documents and a timeline preview host. The timeline preview host drives active composition/nested-timeline clips, seeks them, and exposes a timeline API to the preview. Preview audio is mixed by the preview page from the resolved timeline audio clips.
|
|
108
114
|
|
|
109
|
-
Production composition rendering
|
|
115
|
+
Production composition rendering uses one Chromium session per composition. An animated composition seeks every frame, captures PNGs, and encodes them with FFmpeg. A static composition loads once, captures one PNG, and publishes it without FFmpeg or an encoder.
|
|
110
116
|
|
|
111
117
|
Production timeline rendering is segmented and parallel:
|
|
112
118
|
|
|
113
|
-
1. Run static timeline validation and
|
|
119
|
+
1. Run static timeline validation, resolve the timeline, and project the scene and render plan.
|
|
114
120
|
2. Split the timeline into segments at active-layer changes or a maximum segment duration. Each segment has a fixed set of visual layers and a fixed filter graph.
|
|
115
|
-
3. In parallel workers, capture the active composition and nested-timeline clip frames for each segment with Chromium and encode them to lossless FFV1 intermediates. Transparent clips are captured with an alpha-preserving background.
|
|
116
|
-
4. Decode media clips, apply trims, fit,
|
|
121
|
+
3. In parallel workers, capture the active composition and nested-timeline clip frames for each segment with Chromium and encode them to lossless FFV1 intermediates. Transparent clips are captured with an alpha-preserving background. A static composition is captured once per props variant as a transparent PNG and reused as an image layer across segments and workers.
|
|
122
|
+
4. Decode media clips, apply trims, fit, presentation keyframes, rotation into an expanded bound, and opacity, composite the segment with FFmpeg, and encode H.264, HEVC, or VP9.
|
|
117
123
|
5. Join the ordered segments with a stream copy. Segment boundaries never change frame order or timing.
|
|
118
124
|
6. Mix the timeline audio once with FFmpeg and mux it into the final output.
|
|
119
125
|
7. Publish the output atomically; delete scratch intermediates on success and failure.
|
|
120
126
|
|
|
121
|
-
The default timeline worker count is 8, configurable up to the machine's capacity. With `--encoder nvenc` the count is capped at eight concurrent encode sessions, the consumer driver limit; a larger `--workers` value produces an `ENCODER_WARNING` and the lower count. Scratch is a configurable directory with a bounded budget; it must not depend on `/tmp` alone. The reference long-form target is a representative 60-minute 1920×1080 30 fps timeline with a 50/50 composition/media split by layer-seconds, 60% one active layer, 30% two, and 10% three, rendered in 30 minutes or less on the reference machine. The 10-second benchmark measured 10.7 s for two composition captures, 35.2 MB of FFV1, and 6.1 s to composite and encode. Continuous three-layer stress runs measured 52.4 s and 58.5 s for 60 s of timeline, about 1× real time, and are documented as worst-case rather than the target workload. The representative gate measured 29.39 s for 60 s with four workers, exact 1800-frame output, and a 1080p media fast path, extrapolating to about 29.4 minutes for 60 minutes. The full 60-minute composition-only gate rendered 108,000 frames in 35.35 minutes with exact 108,000-frame / 3,600 s output; it is stricter than the 50/50 target mix because every frame is an HTML composition capture. The fast path copies full-frame, untransformed video whose source matches the project format and
|
|
127
|
+
The default timeline worker count is 8, configurable up to the machine's capacity. With `--encoder nvenc` the count is capped at eight concurrent encode sessions, the consumer driver limit; a larger `--workers` value produces an `ENCODER_WARNING` and the lower count. Scratch is a configurable directory with a bounded budget; it must not depend on `/tmp` alone. The reference long-form target is a representative 60-minute 1920×1080 30 fps timeline with a 50/50 composition/media split by layer-seconds, 60% one active layer, 30% two, and 10% three, rendered in 30 minutes or less on the reference machine. The 10-second benchmark measured 10.7 s for two composition captures, 35.2 MB of FFV1, and 6.1 s to composite and encode. Continuous three-layer stress runs measured 52.4 s and 58.5 s for 60 s of timeline, about 1× real time, and are documented as worst-case rather than the target workload. The representative gate measured 29.39 s for 60 s with four workers, exact 1800-frame output, and a 1080p media fast path, extrapolating to about 29.4 minutes for 60 minutes. The full 60-minute composition-only gate rendered 108,000 frames in 35.35 minutes with exact 108,000-frame / 3,600 s output; it is stricter than the 50/50 target mix because every frame is an HTML composition capture. The fast path copies full-frame, untransformed video whose source matches the project format and whose source keyframes (I-frames) fall on segment boundaries; media that needs scaling or transforms is re-encoded and adds cost not included in that projection. Timeline clip keyframes add per-frame filter expressions and, for animated opacity or scale, an alpha-mask pass per affected layer; the keyframe benchmark reports current timings and is not part of the 60-minute projection.
|
|
122
128
|
|
|
123
129
|
The renderer uses a bounded segment plan, a worker pool, and a scratch budget. It never publishes a partial output, and it cleans scratch on success and failure. Segment boundaries preserve timeline frame order and timing; the final audio mix is muxed once.
|
|
124
130
|
|
|
125
|
-
`check` and `still` use the same clip capture and timeline composite path for requested frames. `check` evaluates every timeline frame; on long timelines this is a validation cost, not a render. `still` captures one or more requested frames as PNGs without encoding a video.
|
|
131
|
+
`check` and `still` use the same clip capture and timeline composite path for requested frames. `check` evaluates every timeline frame; on long timelines this is a validation cost, not a render. A static composition has one state and is inspected once. `still` captures one or more requested frames as PNGs without encoding a video; a static composition exposes only frame 0.
|
|
126
132
|
|
|
127
133
|
## 5. Audio and output
|
|
128
134
|
|
|
@@ -135,13 +141,13 @@ A timeline has one audio mix. It combines:
|
|
|
135
141
|
|
|
136
142
|
Per-clip gain and mute, per-audio-track gain and mute, and composition-level volume apply. Audio fades are deferred. The mix is produced once with FFmpeg and muxed into the final output. Preview mixes the same resolved clips in the browser for monitoring; it is not the production source of truth.
|
|
137
143
|
|
|
138
|
-
Audio, video, and transcript `src` values resolve from the project root. The controller `src` and ordinary HTML asset URLs resolve relative to the composition
|
|
144
|
+
Audio, video, and transcript `src` values resolve from the project root. The controller `src` and ordinary HTML asset URLs resolve relative to the composition URL.
|
|
139
145
|
|
|
140
146
|
Generated speech behaves like any other audio clip. Frameyard does not automatically time-stretch it, change timeline duration, or regenerate it to fit. `check` reports clip durations and overrun warnings so users and agents can revise text, timing, or the timeline deliberately.
|
|
141
147
|
|
|
142
|
-
|
|
148
|
+
Timelines and animated compositions produce MP4, MOV, MKV, or WebM files. The default is MP4 with H.264 video and AAC audio; HEVC and VP9 are opt-in, and WebM is VP9 with Opus. HEVC in MP4 and MOV keeps the `hvc1` sample entry and is not playable in browsers without platform HEVC decoding. Frameyard exposes only a small set of validated encoding options, not arbitrary FFmpeg arguments. A static composition produces one PNG and uses no encoder.
|
|
143
149
|
|
|
144
|
-
PNG output through `still`, including a bounded set of inspection frames, is a debugging artifact, not general image-sequence support.
|
|
150
|
+
PNG output through `still`, including a bounded set of inspection frames, is a debugging artifact, not general image-sequence support. A standalone static composition renders to one PNG; this is the composition's production artifact, not an image sequence.
|
|
145
151
|
|
|
146
152
|
A render fails immediately on an uncaught exception, unhandled rejection, required-asset failure, invalid composition output, segment encoding failure, or scratch failure. Diagnostics identify the timeline, clip, segment, and, where applicable, the frame. The CLI exits nonzero, cleans temporary files, and never reports a partial video as successful. Existing output files are not overwritten unless explicitly requested.
|
|
147
153
|
|
|
@@ -149,18 +155,25 @@ A render fails immediately on an uncaught exception, unhandled rejection, requir
|
|
|
149
155
|
|
|
150
156
|
Project and composition commands are:
|
|
151
157
|
|
|
152
|
-
- `init`: scaffold a timeline-first project with a composition, a one-clip timeline, a project format, an agent guide, a `.gitignore` for `out/`, and a private `package.json` with pinned `dev`, `check`, `render`, and `still` scripts.
|
|
158
|
+
- `init [directory] [--local-models]`: scaffold a timeline-first project with a composition, a one-clip timeline, a project format, an agent guide, a `.gitignore` for `out/`, and a private `package.json` with pinned `dev`, `check`, `render`, and `still` scripts. Without a directory, an interactive terminal prompts for a project name (default `my-video`) and creates that directory; non-interactive and `--json` runs use `my-video`. `--local-models` also installs the optional workspace policy, OpenCode permissions, and Pi settings.
|
|
153
159
|
- `dev`: start the preview with composition and timeline modes.
|
|
154
|
-
- `render <id> [--out video]`: render a timeline or a composition. Timeline renders accept `--workers <n>` and `--scratch <dir>`; composition renders accept `--concurrency <n>` from 1 to 16. `--container <mp4|mov|mkv|webm>` and `--codec <h264|hevc|vp9>` select the delivery pair (MP4/H.264 by default; WebM defaults to VP9), and `--encoder <auto|software|nvenc|qsv>` selects how it is encoded; `auto` is the default and probes NVENC then Quick Sync before falling back to software. The output extension must match the container. The default output is `out/<id>.<ext>`.
|
|
155
|
-
- `
|
|
156
|
-
- `
|
|
157
|
-
- `
|
|
160
|
+
- `render <id> [--out video]`: render a timeline, an animated composition, or a static composition. Timeline renders accept `--workers <n>` and `--scratch <dir>`; composition renders accept `--concurrency <n>` from 1 to 16. `--container <mp4|mov|mkv|webm>` and `--codec <h264|hevc|vp9>` select the delivery pair (MP4/H.264 by default; WebM defaults to VP9), and `--encoder <auto|software|nvenc|qsv>` selects how it is encoded; `auto` is the default and probes NVENC then Quick Sync before falling back to software. The output extension must match the container. The default output is `out/<id>.<ext>`. A static composition writes `out/<id>.png` and rejects `--container`, `--codec`, `--encoder`, and `--concurrency`. The JSON result is a discriminated union with `kind: "video"` or `kind: "image"`.
|
|
161
|
+
- `context`: print a read-only project digest with the format, composition metadata, prop keys and types, controller declarations, timeline clip summaries, unique assets, installed fonts, diagnostics, active policy, and next commands. It never launches Chromium.
|
|
162
|
+
- `lint <id>`: run static validation, timeline resolution, scene and keyframe projection for every timeline frame, a controller determinism scan, and media metadata probing without Chromium. It does not run `validateProps` or composition runtime; `check` remains the full gate.
|
|
163
|
+
- `inspect <image>` and `compare <first> <second>`: decode images through FFmpeg and report blank/alpha/color/content facts or a pixel and 3×3 region diff. These are text-only inspection aids, not a substitute for viewing a still.
|
|
164
|
+
- `add --list`, `add local-models`, and `add <composition|timeline> <id> --template <name>`: list templates and scaffolds, install the optional local-model host scaffold, or write a valid template entry and print the exact `fy.config.js` registration snippet. `add` never edits the config and never overwrites an existing file or registered id.
|
|
165
|
+
- `font search`, `font add <id>`, `font add --file`, `font add --url`, `font list`, `font verify`, and `font remove <id>`: search Fontsource, vendor a Fontsource/local/URL font into `assets/fonts/<id>/`, record it in `fy.fonts.json`, inspect its license/hash status, or remove it. The server serves the generated `@font-face` sheet at `/__frameyard/fonts.css` and injects it into composition documents. Fontsource installs pin `npmVersion` and vendor the package license; URL imports require HTTPS outside loopback tests and record an unknown license only in the `full` tier.
|
|
166
|
+
- `check <id>`: run static validation, then validate configuration, props, assets, media, audio, and transcript requirements, load the target, and evaluate every frame without capturing or encoding. A static composition is loaded and inspected once and returns `kind: "static"` with `framesChecked: 1`.
|
|
167
|
+
- `still <id> --frame <spec>`: capture one or more requested frames as PNGs through the clip capture and timeline composite path. A static composition exposes exactly frame 0. `<spec>` is a frame number (`42`), an inclusive range (`2-12`), or a comma-separated list (`0,118-126,240`); duplicates are removed preserving the first occurrence, input order is preserved, and one invocation captures at most 32 unique frames. There is no step or sequence mode. A single frame keeps the existing `--out <file.png>` behavior; multiple frames use `--out <directory>` and write `still-<id>-<frame>.png` files, defaulting to the current working directory. Frames and output conflicts are validated before Chromium, all frames are captured to temporary files before publication, and a failed capture publishes no files. JSON results add `frames` and keep the singular `frame` and `path` only when one frame was captured. See [ADR 0013](./adr/0013-capture-bounded-multi-frame-stills-for-inspection.md).
|
|
168
|
+
- `doctor`: inspect core requirements, optional integrations, the active resource policy, installed font manifests, and basic CPU/load/memory resources without repairing or downloading anything.
|
|
158
169
|
|
|
159
|
-
`check`, `still`, and `render` run a Node-side static validation
|
|
170
|
+
`lint`, `check`, `still`, and `render` run a Node-side static validation; `check`, `still`, and `render` then launch Chromium. For a timeline it validates the JSON document, tracks, clips, references, nested timelines, cycles, frame-rate compatibility, overlaps and transitions, source bounds, media, transforms, keyframes, audio settings, props, and every referenced composition. Static validation and the composition host read the composition grammar through one browser-safe reader with document and fragment scopes. Timeline diagnostics carry stable codes and optional source locations or JSON paths, and composition diagnostics from inside a timeline carry the clip and track context.
|
|
160
171
|
|
|
161
|
-
Local asset-production commands are `tts` and `transcribe`. Every non-interactive command supports `--json` with stable result and error schemas. Structured results go to stdout; human diagnostics and inference progress go to stderr.
|
|
172
|
+
Local asset-production commands are `tts` and `transcribe`. Every non-interactive command supports `--json` with stable result and error schemas. Structured results go to stdout; human diagnostics and inference progress go to stderr. Project font acquisition is a separate authoring operation: `fy.fonts.json` records vendored files, licenses, and SHA-256 values; there is no shared font cache and no runtime font CDN. Fontsource search and download use the public Fontsource API and pinned CDN versions; local files and explicit HTTPS URLs cover other sources. The generated `@font-face` sheet is served at `/__frameyard/fonts.css` and injected before author styles.
|
|
162
173
|
|
|
163
|
-
The
|
|
174
|
+
A workspace `fy.policy.json` gates expensive operations. The nearest file from the working directory upward applies, independent of `fy.config.js`; `FRAMEYARD_POLICY` replaces it. `light` allows one still frame and blocks `check`, `tts`, `transcribe`, and `render`; `authoring` allows four still frames and `check` at low priority and blocks the three heavy operations; `full` allows everything up to the 32-frame still cap. Font list/verify/search are available in every tier; `font add` requires authoring or full; `font remove`, `font add --force`, and an unlicensed `font add --url` require full. A file can only tighten its tier. `--profile shared-cpu` caps timeline workers at two and composition concurrency at one and lowers process priority. `tts`, `transcribe`, and `render` accept `--dry-run`, which validates inputs and returns `plan`, `commandLine`, `prerequisites`, and `blockers` without writing outputs, loading a model, launching Chromium, or encoding. The policy is a guardrail, not a security boundary; see [ADR 0022](./adr/0022-gate-heavy-operations-with-a-workspace-resource-policy.md).
|
|
175
|
+
|
|
176
|
+
The built-in server serves native ES modules, static assets, composition entries, timeline documents, and the timeline preview host. It wraps each configured composition fragment in the fixed document shell, injects the framework import map and composition host, and serves an inline controller through a virtual module URL. It performs no dependency pre-bundling. Third-party libraries must initially be browser-ready ES modules referenced explicitly. Running through the CLI server is required; direct `file://` execution is unsupported for deterministic preview or rendering.
|
|
164
177
|
|
|
165
178
|
Frameyard uses one repository and one publishable npm package with four internal layers:
|
|
166
179
|
|
|
@@ -180,20 +193,24 @@ The skill:
|
|
|
180
193
|
- Uses the normal project format, HTML composition contract, JSON timeline contract, optional controller, and CLI available to humans.
|
|
181
194
|
- Does not introduce an agent-only runtime, MCP server, hosted control plane, or separate rendering API.
|
|
182
195
|
- Triggers for Frameyard authoring and operation rather than claiming to be a general-purpose media-generation skill.
|
|
183
|
-
- Uses Frameyard's audio wrappers instead of invoking `qwen-tts` or `whisper-cli` directly.
|
|
184
|
-
- Includes references for deterministic HTML authoring, timeline JSON authoring, tracks, clips, source ranges, props, media, nested timelines, static transforms, audio, validation, diagnostics, common mistakes, subtitle and karaoke recipes, JSON command results, long-form worker/scratch settings, and failure recovery.
|
|
196
|
+
- Uses Frameyard's audio wrappers instead of invoking `qwen-tts`, `sherpa-onnx-offline-tts`, or `whisper-cli` directly.
|
|
197
|
+
- Includes references for deterministic HTML authoring, timeline JSON authoring, tracks, clips, source ranges, props, media, nested timelines, static transforms, clip keyframes, clip transitions, audio, validation, diagnostics, common mistakes, subtitle and karaoke recipes, JSON command results, long-form worker/scratch settings, and failure recovery, plus a compact quick reference for limited-context hosts.
|
|
185
198
|
- Invokes speech synthesis or transcription only when requested or explicitly approved as part of the brief.
|
|
186
199
|
- Does not overwrite or regenerate existing generated assets without permission.
|
|
200
|
+
- Uses `lint` for the edit loop, `check` for the full gate, and `--dry-run` to hand off heavy operations when the workspace resource policy blocks them.
|
|
201
|
+
- Respects `fy.policy.json` and does not edit the policy file, host configuration, or skill source.
|
|
187
202
|
- Runs `check`, inspects representative `still` images when the host supports image viewing, and discloses when visual inspection was impossible.
|
|
188
203
|
- Does not claim success when checks fail or rendering is partial.
|
|
189
204
|
|
|
190
205
|
A successful agent workflow preserves and reports the source project, timeline document, props, generated assets, provenance manifest, validation results, and final video path. The user must be able to inspect, modify, and reproduce the result rather than receiving only an opaque MP4.
|
|
191
206
|
|
|
207
|
+
Local-model host setup is optional and separate from a Frameyard project: `scaffolds/local-models/` contains an `authoring` policy, OpenCode permissions, and Pi settings that a workspace can copy when a local model and rendering share the machine. Without the scaffold, the CLI defaults to `full` and no host configuration applies.
|
|
208
|
+
|
|
192
209
|
## 8. Local speech synthesis
|
|
193
210
|
|
|
194
211
|
The `tts` command integrates [`qwentts.cpp`](https://github.com/ServeurpersoCom/qwentts.cpp) through its external `qwen-tts` executable. It does not link the native library or require the upstream server.
|
|
195
212
|
|
|
196
|
-
|
|
213
|
+
The qwen engine supports CustomVoice mode only:
|
|
197
214
|
|
|
198
215
|
- A named speaker is required.
|
|
199
216
|
- An optional style instruction and language may be supplied.
|
|
@@ -203,7 +220,15 @@ The `tts` command integrates [`qwentts.cpp`](https://github.com/ServeurpersoCom/
|
|
|
203
220
|
- One invocation produces one frozen 24 kHz mono WAV from one complete text input.
|
|
204
221
|
- Automatic long-text chunking, streaming, and alternative audio formats are deferred.
|
|
205
222
|
|
|
206
|
-
|
|
223
|
+
`tts --engine supertonic` integrates [`sherpa-onnx`](https://github.com/k2-fsa/sherpa-onnx) through its external `sherpa-onnx-offline-tts` executable and the Supertonic 3 model archive:
|
|
224
|
+
|
|
225
|
+
- The engine covers 31 languages and requires one of their language codes, a numeric speaker id, and a directory containing the seven model files.
|
|
226
|
+
- The engine writes its native WAV; Frameyard normalizes it to the same frozen 24 kHz mono WAV contract with FFmpeg and records `sourceSampleRate` and `sourceChannels` in the manifest.
|
|
227
|
+
- The manifest records the seven model identities, the pinned sherpa-onnx revision, and the model archive name.
|
|
228
|
+
- Supertonic expression tags belong in the text; the qwen-only style and device options are rejected for this engine.
|
|
229
|
+
- The upstream Supertonic repository is archived and its weights use the OpenRAIL-M license. Users acquire the executable and weights explicitly; Frameyard does not redistribute them.
|
|
230
|
+
|
|
231
|
+
Users acquire and provide models explicitly. Frameyard does not build `qwentts.cpp` or `sherpa-onnx`, download or redistribute models, or imply that upstream code licenses settle model-weight licensing. Executable resolution uses an explicit option, then environment configuration, then PATH.
|
|
207
232
|
|
|
208
233
|
## 9. Local transcription and timed text
|
|
209
234
|
|
|
@@ -243,26 +268,26 @@ Generated artifacts follow a shared contract:
|
|
|
243
268
|
|
|
244
269
|
- Outputs are written atomically to explicit paths.
|
|
245
270
|
- Existing outputs are not overwritten unless `--force` is supplied.
|
|
246
|
-
- A versioned project-local provenance manifest records artifact content hashes, generator details, model identities, and parameters. The default manifest is `
|
|
271
|
+
- A versioned project-local provenance manifest records artifact content hashes, generator details, model identities, and parameters. The default manifest is `fy.assets.json` in the project root; `--manifest` selects another path.
|
|
247
272
|
- Transcription provenance also identifies the source asset by content hash.
|
|
248
273
|
- Partial operations leave neither apparently valid artifacts nor manifest entries.
|
|
249
|
-
- `doctor` validates executables, model readability, capabilities, and supported integration versions without modifying the environment.
|
|
274
|
+
- `doctor` validates executables, model readability, capabilities, and supported integration versions when the upstream CLI exposes a version, without modifying the environment.
|
|
250
275
|
|
|
251
276
|
Each operation runs one inference subprocess, and the official agent workflow executes inference sequentially to avoid unexpected CPU/GPU contention. There is no arbitrary fixed timeout. Cancellation terminates the child process and removes partial outputs.
|
|
252
277
|
|
|
253
|
-
Release documentation must identify the exact `qwentts.cpp` and `whisper.cpp` revisions against which the wrappers are tested. Frameyard does not claim compatibility with arbitrary upstream revisions.
|
|
278
|
+
Release documentation must identify the exact `qwentts.cpp`, `sherpa-onnx`, and `whisper.cpp` revisions against which the wrappers are tested. Frameyard does not claim compatibility with arbitrary upstream revisions.
|
|
254
279
|
|
|
255
280
|
## 11. Verification strategy
|
|
256
281
|
|
|
257
282
|
Tests prioritize public behavior and end-to-end evidence over implementation-only unit coverage:
|
|
258
283
|
|
|
259
284
|
- HTML compositions validate metadata, default props, bindings, conditionals, repeats, and controller declarations, including inline/external parity and the exactly-one-controller rule.
|
|
260
|
-
- Static composition validation catches metadata, root/controller count, binding, animation, script, element, and asset-reference errors before Chromium.
|
|
261
|
-
- Timeline JSON validation catches schema, track, clip, reference, cycle, frame-rate, overlap, bounds, media, transform, audio, and props errors before Chromium.
|
|
285
|
+
- Static composition validation catches metadata, root/controller count, binding, animation, script, element, and asset-reference errors before Chromium. It resolves declared format values against the project format and reports an unresolved format to the command rather than the document. A composition with neither time attribute is static; `fps` without `duration-in-frames`, `data-fy-anim-*`, media, and transcripts on a static composition are errors.
|
|
286
|
+
- Timeline JSON validation catches schema, track, clip, reference, cycle, frame-rate, overlap, transition, bounds, media, transform, audio, and props errors before Chromium.
|
|
262
287
|
- Diagnostics carry stable codes and source locations or JSON paths; JSON results expose the composition or timeline entry, clips, tracks, and top-level diagnostics.
|
|
263
288
|
- Sequential, individual, random-order, and parallel composition renders produce equivalent images; timeline segment workers produce ordered output and identical captured composition frames.
|
|
264
289
|
- Determinism checks compare captured composition pixels or stills. The final encoded bytes and the FFmpeg composite versus Chromium preview are not reproducibility oracles.
|
|
265
|
-
- Preview and renderer use the same composition
|
|
290
|
+
- Preview and renderer use the same composition source and host behavior; timeline preview uses the same active clip hosts.
|
|
266
291
|
- CSS, controller JavaScript, SVG, Canvas, WebGL, embedded video, and audio frames seek correctly.
|
|
267
292
|
- Frame-scheduled `data-fy-anim-*` animations and frame-state CSS variables produce the expected frames.
|
|
268
293
|
- Timeline capture preserves alpha for transparent compositions; FFV1 intermediates decode pixel-identically to the captured frames.
|
|
@@ -270,18 +295,19 @@ Tests prioritize public behavior and end-to-end evidence over implementation-onl
|
|
|
270
295
|
- Nested timelines preserve group opacity and transform semantics; cycles fail static validation.
|
|
271
296
|
- Media clips trim and fit deterministically; images require durations; source overruns fail.
|
|
272
297
|
- Props overrides for timeline composition clips merge over clip props and are reflected in the render.
|
|
273
|
-
- `check` exercises every frame; `still` follows the clip capture and timeline composite path.
|
|
298
|
+
- `check` exercises every animated frame and inspects a static composition once; it also evaluates every keyframe channel at every timeline frame. `still` follows the clip capture and timeline composite path. A static composition render captures one PNG; a timeline reuses one static image capture per props variant.
|
|
274
299
|
- The 10-second and 60-second long-form benchmarks record capture, FFV1, composite, encode, disk, worker, and audio numbers. Continuous three-layer runs measured 52.4 s and 58.5 s for 60 s of timeline and are documented as worst-case (~1× real time). The representative 50/50, 60/30/10 occupancy gate measured 29.39 s for 60 s with four workers and a 1080p media fast path, extrapolating to about 29.4 minutes for 60 minutes with exact 1800-frame output. The full 60-minute composition-only gate rendered 108,000 frames in 35.35 minutes with exact output and clean scratch, documenting composition-heavy performance above 30 minutes.
|
|
275
300
|
- Golden-frame tests cover representative composition and timeline visual behavior.
|
|
276
|
-
- Normal CI uses fake `qwen-tts` and `whisper-cli` executables and small fixtures to verify arguments, artifacts, manifests, cancellation, and errors without model downloads or GPU hardware.
|
|
301
|
+
- Normal CI uses fake `qwen-tts`, `sherpa-onnx-offline-tts`, and `whisper-cli` executables and small fixtures to verify arguments, artifacts, manifests, cancellation, and errors without model downloads or GPU hardware.
|
|
277
302
|
- Real-model integration tests are explicitly opt-in and use documented upstream revisions.
|
|
278
303
|
- Agent-workflow evaluations verify source preservation, provenance, validation, still inspection behavior, failure reporting, and final renders.
|
|
304
|
+
- Local-model evals are opt-in and run against a llama.cpp router: eight authoring tasks, first-pass lint, repair turns, token and wall-time counts, and sampled server CPU/RSS/GPU. They exercise the same quick reference and CLI as an agent, never render, and stay outside `pnpm check`.
|
|
279
305
|
|
|
280
306
|
## 12. Deferred
|
|
281
307
|
|
|
282
308
|
These are scope boundaries, not permanent rejections:
|
|
283
309
|
|
|
284
|
-
-
|
|
310
|
+
- Animated blend modes, filters, masks, and audio fades
|
|
285
311
|
- Full visual timeline editing, drag-and-drop, clip handles, and undo/redo
|
|
286
312
|
- Timeline-level variables and prop overrides for compositions inside nested timelines
|
|
287
313
|
- Track solo and video-track visibility
|
|
@@ -295,9 +321,12 @@ These are scope boundaries, not permanent rejections:
|
|
|
295
321
|
- Built-in subtitle rendering, automatic caption burn-in, and declarative karaoke elements
|
|
296
322
|
- Nested `data-fy-repeat` scopes and a general template language
|
|
297
323
|
- Declarative JSON Schema validation
|
|
324
|
+
- A shared font cache, hosted font catalog, or first-class Google Fonts/Fontshare/Bunny providers; Fontsource plus local/HTTPS imports cover acquisition
|
|
325
|
+
- Typography roles, font-pair presets, a preview font picker, and variable axes other than `wght`
|
|
298
326
|
- Arbitrary executable scripts in composition HTML
|
|
299
327
|
- Automatic upstream builds, model downloads, weight redistribution, and environment repair
|
|
300
328
|
- Agent-only runtimes, MCP servers, hosted agent control planes, and host-specific skill installers
|
|
329
|
+
- Automatic idle scheduling, job queues, and OS-level resource enforcement; the workspace policy is a guardrail, not a sandbox
|
|
301
330
|
- React or other UI-framework adapters in the core
|
|
302
331
|
- Public plugin APIs and a broad integration ecosystem
|
|
303
332
|
- Dynamic composition or timeline metadata derived from props
|
|
@@ -305,6 +334,10 @@ These are scope boundaries, not permanent rejections:
|
|
|
305
334
|
- Dynamic per-frame asset loading or AI inference
|
|
306
335
|
- Unbounded batch still capture; `still` accepts at most 32 inspection frames per invocation and is not an image-sequence exporter
|
|
307
336
|
- Additional final-video formats, general image-sequence output, and arbitrary FFmpeg argument passthrough
|
|
337
|
+
- Audio, transcripts, or embedded video in a static composition; use a timeline
|
|
338
|
+
- A `--video` escape hatch for static composition renders; use a timeline image clip
|
|
339
|
+
- Exporting a fully static timeline as a PNG
|
|
340
|
+
- Recursive static optimization of nested timelines
|
|
308
341
|
- Built-in TypeScript transpilation and npm dependency optimization
|
|
309
342
|
- Rendering untrusted compositions
|
|
310
343
|
- Bit-identical cross-platform encoded output or AI regeneration
|
|
@@ -312,4 +345,4 @@ These are scope boundaries, not permanent rejections:
|
|
|
312
345
|
|
|
313
346
|
## 13. Release posture
|
|
314
347
|
|
|
315
|
-
Frameyard is MIT-licensed. Initial releases use `0.x` versioning, and
|
|
348
|
+
Frameyard is MIT-licensed. Initial releases use `0.x` versioning, and features and contract changes are documented explicitly until those contracts stabilize.
|