frameyard 0.7.0 → 0.9.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +164 -7
- package/GLOSSARY.md +152 -18
- package/README.md +328 -285
- package/dist/assets/integrations.d.ts +8 -4
- package/dist/assets/integrations.d.ts.map +1 -1
- package/dist/assets/integrations.js +8 -4
- package/dist/assets/integrations.js.map +1 -1
- package/dist/assets/provenance.d.ts +2 -2
- package/dist/assets/provenance.d.ts.map +1 -1
- package/dist/assets/provenance.js +5 -3
- package/dist/assets/provenance.js.map +1 -1
- package/dist/assets/tts-supertonic.d.ts +28 -0
- package/dist/assets/tts-supertonic.d.ts.map +1 -0
- package/dist/assets/tts-supertonic.js +156 -0
- package/dist/assets/tts-supertonic.js.map +1 -0
- package/dist/assets/tts.d.ts +18 -6
- package/dist/assets/tts.d.ts.map +1 -1
- package/dist/assets/tts.js +219 -97
- package/dist/assets/tts.js.map +1 -1
- package/dist/cli.js +1705 -106
- package/dist/cli.js.map +1 -1
- package/dist/composition/descriptor.d.ts +17 -3
- package/dist/composition/descriptor.d.ts.map +1 -1
- package/dist/composition/descriptor.js +51 -28
- package/dist/composition/descriptor.js.map +1 -1
- package/dist/composition/format.d.ts +12 -0
- package/dist/composition/format.d.ts.map +1 -0
- package/dist/composition/format.js +28 -0
- package/dist/composition/format.js.map +1 -0
- package/dist/composition/index.d.ts +2 -1
- package/dist/composition/index.d.ts.map +1 -1
- package/dist/composition/index.js +1 -0
- package/dist/composition/index.js.map +1 -1
- package/dist/context.d.ts +82 -0
- package/dist/context.d.ts.map +1 -0
- package/dist/context.js +277 -0
- package/dist/context.js.map +1 -0
- package/dist/contracts.d.ts +80 -6
- package/dist/contracts.d.ts.map +1 -1
- package/dist/contracts.js +3 -0
- package/dist/contracts.js.map +1 -1
- package/dist/doctor.d.ts +14 -1
- package/dist/doctor.d.ts.map +1 -1
- package/dist/doctor.js +166 -7
- package/dist/doctor.js.map +1 -1
- package/dist/errors.d.ts +1 -1
- package/dist/errors.d.ts.map +1 -1
- package/dist/errors.js.map +1 -1
- package/dist/fonts/add.d.ts +41 -0
- package/dist/fonts/add.d.ts.map +1 -0
- package/dist/fonts/add.js +259 -0
- package/dist/fonts/add.js.map +1 -0
- package/dist/fonts/binary.d.ts +4 -0
- package/dist/fonts/binary.d.ts.map +1 -0
- package/dist/fonts/binary.js +21 -0
- package/dist/fonts/binary.js.map +1 -0
- package/dist/fonts/css.d.ts +4 -0
- package/dist/fonts/css.d.ts.map +1 -0
- package/dist/fonts/css.js +50 -0
- package/dist/fonts/css.js.map +1 -0
- package/dist/fonts/fontsource.d.ts +107 -0
- package/dist/fonts/fontsource.d.ts.map +1 -0
- package/dist/fonts/fontsource.js +565 -0
- package/dist/fonts/fontsource.js.map +1 -0
- package/dist/fonts/index.d.ts +11 -0
- package/dist/fonts/index.d.ts.map +1 -0
- package/dist/fonts/index.js +11 -0
- package/dist/fonts/index.js.map +1 -0
- package/dist/fonts/install.d.ts +24 -0
- package/dist/fonts/install.d.ts.map +1 -0
- package/dist/fonts/install.js +145 -0
- package/dist/fonts/install.js.map +1 -0
- package/dist/fonts/manifest.d.ts +21 -0
- package/dist/fonts/manifest.d.ts.map +1 -0
- package/dist/fonts/manifest.js +96 -0
- package/dist/fonts/manifest.js.map +1 -0
- package/dist/fonts/options.d.ts +9 -0
- package/dist/fonts/options.d.ts.map +1 -0
- package/dist/fonts/options.js +66 -0
- package/dist/fonts/options.js.map +1 -0
- package/dist/fonts/schema.d.ts +4 -0
- package/dist/fonts/schema.d.ts.map +1 -0
- package/dist/fonts/schema.js +316 -0
- package/dist/fonts/schema.js.map +1 -0
- package/dist/fonts/summary.d.ts +16 -0
- package/dist/fonts/summary.d.ts.map +1 -0
- package/dist/fonts/summary.js +22 -0
- package/dist/fonts/summary.js.map +1 -0
- package/dist/fonts/types.d.ts +56 -0
- package/dist/fonts/types.d.ts.map +1 -0
- package/dist/fonts/types.js +8 -0
- package/dist/fonts/types.js.map +1 -0
- package/dist/fonts/verify.d.ts +33 -0
- package/dist/fonts/verify.d.ts.map +1 -0
- package/dist/fonts/verify.js +200 -0
- package/dist/fonts/verify.js.map +1 -0
- package/dist/html/document-adapter.d.ts +3 -3
- package/dist/html/document-adapter.d.ts.map +1 -1
- package/dist/html/document-adapter.js +1 -1
- package/dist/html/document-adapter.js.map +1 -1
- package/dist/html/index.d.ts +1 -1
- package/dist/html/index.d.ts.map +1 -1
- package/dist/html/parse.d.ts +10 -2
- package/dist/html/parse.d.ts.map +1 -1
- package/dist/html/parse.js +14 -5
- package/dist/html/parse.js.map +1 -1
- package/dist/html/validate.d.ts +12 -1
- package/dist/html/validate.d.ts.map +1 -1
- package/dist/html/validate.js +292 -61
- package/dist/html/validate.js.map +1 -1
- package/dist/index.d.ts +4 -4
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +2 -2
- package/dist/index.js.map +1 -1
- package/dist/inspect.d.ts +61 -0
- package/dist/inspect.d.ts.map +1 -0
- package/dist/inspect.js +325 -0
- package/dist/inspect.js.map +1 -0
- package/dist/lint.d.ts +63 -0
- package/dist/lint.d.ts.map +1 -0
- package/dist/lint.js +265 -0
- package/dist/lint.js.map +1 -0
- package/dist/policy.d.ts +69 -0
- package/dist/policy.d.ts.map +1 -0
- package/dist/policy.js +361 -0
- package/dist/policy.js.map +1 -0
- package/dist/preview/main.d.ts +2 -2
- package/dist/preview/main.d.ts.map +1 -1
- package/dist/preview/main.js +35 -14
- package/dist/preview/main.js.map +1 -1
- package/dist/preview/timeline-data.d.ts +17 -65
- package/dist/preview/timeline-data.d.ts.map +1 -1
- package/dist/preview/timeline-host.d.ts.map +1 -1
- package/dist/preview/timeline-host.js +73 -38
- package/dist/preview/timeline-host.js.map +1 -1
- package/dist/preview/timeline-main.d.ts.map +1 -1
- package/dist/preview/timeline-main.js +47 -3
- package/dist/preview/timeline-main.js.map +1 -1
- package/dist/project.d.ts +13 -8
- package/dist/project.d.ts.map +1 -1
- package/dist/project.js +20 -17
- package/dist/project.js.map +1 -1
- package/dist/renderer/browser.d.ts +2 -2
- package/dist/renderer/browser.d.ts.map +1 -1
- package/dist/renderer/browser.js +10 -4
- package/dist/renderer/browser.js.map +1 -1
- package/dist/renderer/ffmpeg.d.ts +44 -1
- package/dist/renderer/ffmpeg.d.ts.map +1 -1
- package/dist/renderer/ffmpeg.js +371 -73
- package/dist/renderer/ffmpeg.js.map +1 -1
- package/dist/renderer/index.d.ts +40 -4
- package/dist/renderer/index.d.ts.map +1 -1
- package/dist/renderer/index.js +100 -25
- package/dist/renderer/index.js.map +1 -1
- package/dist/renderer/keyframe-expressions.d.ts +39 -0
- package/dist/renderer/keyframe-expressions.d.ts.map +1 -0
- package/dist/renderer/keyframe-expressions.js +180 -0
- package/dist/renderer/keyframe-expressions.js.map +1 -0
- package/dist/renderer/timeline.d.ts +2 -0
- package/dist/renderer/timeline.d.ts.map +1 -1
- package/dist/renderer/timeline.js +168 -106
- package/dist/renderer/timeline.js.map +1 -1
- package/dist/runtime/host.d.ts.map +1 -1
- package/dist/runtime/host.js +159 -31
- package/dist/runtime/host.js.map +1 -1
- package/dist/scaffold.d.ts +3 -0
- package/dist/scaffold.d.ts.map +1 -1
- package/dist/scaffold.js +48 -317
- package/dist/scaffold.js.map +1 -1
- package/dist/server/composition-html.d.ts +15 -7
- package/dist/server/composition-html.d.ts.map +1 -1
- package/dist/server/composition-html.js +72 -16
- package/dist/server/composition-html.js.map +1 -1
- package/dist/server/server.d.ts.map +1 -1
- package/dist/server/server.js +56 -6
- package/dist/server/server.js.map +1 -1
- package/dist/server/timeline-preview-data.d.ts.map +1 -1
- package/dist/server/timeline-preview-data.js +12 -23
- package/dist/server/timeline-preview-data.js.map +1 -1
- package/dist/server/timeline-preview-html.d.ts.map +1 -1
- package/dist/server/timeline-preview-html.js +20 -3
- package/dist/server/timeline-preview-html.js.map +1 -1
- package/dist/templates.d.ts +16 -0
- package/dist/templates.d.ts.map +1 -0
- package/dist/templates.js +186 -0
- package/dist/templates.js.map +1 -0
- package/dist/timeline/easing.d.ts +36 -0
- package/dist/timeline/easing.d.ts.map +1 -0
- package/dist/timeline/easing.js +225 -0
- package/dist/timeline/easing.js.map +1 -0
- package/dist/timeline/keyframes.d.ts +23 -0
- package/dist/timeline/keyframes.d.ts.map +1 -0
- package/dist/timeline/keyframes.js +95 -0
- package/dist/timeline/keyframes.js.map +1 -0
- package/dist/timeline/plan.d.ts +6 -12
- package/dist/timeline/plan.d.ts.map +1 -1
- package/dist/timeline/plan.js +17 -30
- package/dist/timeline/plan.js.map +1 -1
- package/dist/timeline/resolve.d.ts +16 -3
- package/dist/timeline/resolve.d.ts.map +1 -1
- package/dist/timeline/resolve.js +119 -13
- package/dist/timeline/resolve.js.map +1 -1
- package/dist/timeline/scene.d.ts +126 -0
- package/dist/timeline/scene.d.ts.map +1 -0
- package/dist/timeline/scene.js +109 -0
- package/dist/timeline/scene.js.map +1 -0
- package/dist/timeline/schema.d.ts +38 -0
- package/dist/timeline/schema.d.ts.map +1 -1
- package/dist/timeline/schema.js +290 -9
- package/dist/timeline/schema.js.map +1 -1
- package/dist/timeline/transitions.d.ts +21 -0
- package/dist/timeline/transitions.d.ts.map +1 -0
- package/dist/timeline/transitions.js +23 -0
- package/dist/timeline/transitions.js.map +1 -0
- package/dist/version.d.ts +1 -1
- package/dist/version.js +1 -1
- package/docs/adr/0007-make-html-documents-the-composition-unit.md +2 -2
- package/docs/adr/0008-share-the-composition-document-grammar.md +1 -1
- package/docs/adr/0010-make-timelines-the-render-unit.md +3 -2
- package/docs/adr/0011-compose-timeline-frames-in-a-live-browser-host.md +1 -1
- package/docs/adr/0012-render-timelines-with-segmented-lossless-clip-intermediates.md +1 -1
- package/docs/adr/0014-compose-timeline-clips-with-css-blend-modes.md +38 -0
- package/docs/adr/0015-inherit-composition-format-from-the-project.md +28 -0
- package/docs/adr/0016-author-compositions-as-html-fragments.md +32 -0
- package/docs/adr/0017-render-timeless-compositions-as-stills.md +99 -0
- package/docs/adr/0018-animate-timeline-clips-with-keyframes.md +35 -0
- package/docs/adr/0019-make-same-track-overlap-the-transition-window.md +36 -0
- package/docs/adr/0020-crossfade-audio-clips-on-audio-tracks.md +27 -0
- package/docs/adr/0021-project-timeline-scenes-once.md +32 -0
- package/docs/adr/0022-gate-heavy-operations-with-a-workspace-resource-policy.md +35 -0
- package/docs/adr/0023-pin-fonts-as-project-assets.md +98 -0
- package/docs/migration-0.4.md +5 -3
- package/docs/migration-0.8.md +163 -0
- package/docs/preview-testing.md +23 -3
- package/docs/{v0.1-design.md → v0.x-design.md} +86 -53
- package/examples/blend/README.md +106 -0
- package/examples/blend/assets/ink.mp4 +0 -0
- package/examples/blend/compositions/aurora.html +55 -0
- package/examples/blend/compositions/backdrop.html +74 -0
- package/examples/blend/compositions/labels.html +149 -0
- package/examples/blend/fy.config.js +11 -0
- package/examples/blend/scripts/make-ink.sh +26 -0
- package/examples/blend/timelines/blend.timeline.json +155 -0
- package/examples/demo/README.md +2 -2
- package/examples/demo/compositions/demo.html +243 -251
- package/examples/intro/README.md +71 -0
- package/examples/intro/assets/fonts/figtree/LICENSE.txt +93 -0
- package/examples/intro/assets/fonts/figtree/figtree-latin-wght-normal.woff2 +0 -0
- package/examples/intro/assets/fonts/instrument-serif/LICENSE.txt +93 -0
- package/examples/intro/assets/fonts/instrument-serif/instrument-serif-latin-400-italic.woff2 +0 -0
- package/examples/intro/assets/fonts/space-mono/LICENSE.txt +93 -0
- package/examples/intro/assets/fonts/space-mono/space-mono-latin-400-normal.woff2 +0 -0
- package/examples/intro/assets/fonts/space-mono/space-mono-latin-700-normal.woff2 +0 -0
- package/examples/intro/assets/grain.png +0 -0
- package/examples/intro/assets/score.ogg +0 -0
- package/examples/intro/compositions/brand.html +412 -0
- package/examples/intro/compositions/chrome.controller.js +63 -0
- package/examples/intro/compositions/chrome.html +256 -0
- package/examples/intro/compositions/lattice.controller.js +148 -0
- package/examples/intro/compositions/lattice.html +308 -0
- package/examples/intro/compositions/opening.html +504 -0
- package/examples/intro/compositions/studio.html +930 -0
- package/examples/intro/compositions/theme.css +218 -0
- package/examples/intro/fy.config.js +13 -0
- package/examples/intro/fy.fonts.json +100 -0
- package/examples/intro/scripts/make-grain.mjs +79 -0
- package/examples/intro/scripts/make-score.mjs +702 -0
- package/examples/intro/timelines/intro.timeline.json +60 -0
- package/examples/keyframes/README.md +76 -0
- package/examples/keyframes/compositions/backdrop.html +60 -0
- package/examples/keyframes/compositions/card.html +99 -0
- package/examples/keyframes/fy.config.js +10 -0
- package/examples/keyframes/timelines/keyframes.timeline.json +60 -0
- package/examples/nested/README.md +112 -0
- package/examples/nested/assets/pulse.wav +0 -0
- package/examples/nested/compositions/backdrop.html +84 -0
- package/examples/nested/compositions/card.html +275 -0
- package/examples/nested/compositions/overlay.html +307 -0
- package/examples/nested/fy.config.js +13 -0
- package/examples/nested/scripts/make-pulse.sh +20 -0
- package/examples/nested/timelines/chapter-a.timeline.json +39 -0
- package/examples/nested/timelines/chapter-b.timeline.json +39 -0
- package/examples/nested/timelines/nested-reel.timeline.json +52 -0
- package/examples/static/README.md +17 -0
- package/examples/static/compositions/still.html +37 -0
- package/examples/static/fy.config.js +9 -0
- package/examples/static/timelines/still.timeline.json +13 -0
- package/examples/transitions/README.md +128 -0
- package/examples/transitions/assets/alpha.wav +0 -0
- package/examples/transitions/assets/beta.wav +0 -0
- package/examples/transitions/assets/gamma.wav +0 -0
- package/examples/transitions/compositions/alpha.html +56 -0
- package/examples/transitions/compositions/beta.html +56 -0
- package/examples/transitions/compositions/gamma.html +56 -0
- package/examples/transitions/fy.config.js +12 -0
- package/examples/transitions/timelines/audio-crossfade.timeline.json +44 -0
- package/examples/transitions/timelines/transitions.timeline.json +31 -0
- package/examples/video/README.md +4 -4
- package/examples/video/compositions/video-mp4.html +349 -357
- package/examples/video/compositions/video-overlay.html +337 -345
- package/examples/video/compositions/video.html +349 -357
- package/examples/webgl/README.md +1 -1
- package/examples/webgl/compositions/webgl.html +205 -213
- package/package.json +13 -5
- package/scaffolds/init/AGENTS.md +131 -0
- package/scaffolds/init/README.md +56 -0
- package/scaffolds/init/compositions/hello.html +79 -0
- package/scaffolds/init/fy.config.js +9 -0
- package/scaffolds/init/gitignore.template +1 -0
- package/scaffolds/init/package.json +11 -0
- package/scaffolds/init/props/hello.json +4 -0
- package/scaffolds/init/props/main.json +6 -0
- package/scaffolds/init/timelines/main.timeline.json +20 -0
- package/scaffolds/local-models/LOCAL-MODELS.md +77 -0
- package/scaffolds/local-models/fy.policy.json +5 -0
- package/scaffolds/local-models/opencode/opencode.jsonc +17 -0
- package/scaffolds/local-models/pi/APPEND_SYSTEM.md +21 -0
- package/scaffolds/local-models/pi/sandbox.json +15 -0
- package/scaffolds/local-models/pi/settings.json +15 -0
- package/skills/frameyard/README.md +1 -0
- package/skills/frameyard/SKILL.md +59 -59
- package/skills/frameyard/references/audio-assets.md +32 -3
- package/skills/frameyard/references/common-mistakes.md +81 -7
- package/skills/frameyard/references/controllers.md +5 -4
- package/skills/frameyard/references/deterministic-authoring.md +49 -37
- package/skills/frameyard/references/diagnostics.md +42 -4
- package/skills/frameyard/references/failure-recovery.md +56 -10
- package/skills/frameyard/references/fonts.md +132 -0
- package/skills/frameyard/references/json-results.md +165 -10
- package/skills/frameyard/references/quick-reference.md +222 -0
- package/skills/frameyard/references/timelines.md +109 -15
- /package/examples/demo/{video.config.js → fy.config.js} +0 -0
- /package/examples/video/{video.config.js → fy.config.js} +0 -0
- /package/examples/webgl/{video.config.js → fy.config.js} +0 -0
package/README.md
CHANGED
|
@@ -1,24 +1,37 @@
|
|
|
1
|
+
<div align="center">
|
|
2
|
+
|
|
1
3
|
# Frameyard
|
|
2
4
|
|
|
5
|
+
[](https://github.com/vladiantio/frameyard/actions/workflows/ci.yml)
|
|
6
|
+
[](https://www.npmjs.com/package/frameyard)
|
|
7
|
+
[](LICENSE)
|
|
8
|
+
|
|
9
|
+
https://github.com/user-attachments/assets/cade81af-3e87-4577-940d-7d5fee99e357
|
|
10
|
+
|
|
11
|
+
<sup>It is an Frameyard project: <a href="examples/intro">examples/intro</a>.</sup>
|
|
12
|
+
|
|
13
|
+
[Quick start](#quick-start) · [Examples](#examples) · [Project setup](#project-setup) · [Composition authoring](#composition-authoring) · [Timelines](#timelines) · [Media](#media) · [Rendering and reproducibility](#rendering-and-reproducibility) · [Tooling](#tooling) · [Limits and non-goals](#limits-and-non-goals) · [Development](#development)
|
|
14
|
+
|
|
3
15
|
Frameyard is a local-first framework for building parameterized videos with HTML, CSS, and an optional JavaScript controller, assembling them into timelines, and rendering them deterministically.
|
|
4
16
|
|
|
5
|
-
|
|
17
|
+
</div>
|
|
18
|
+
|
|
19
|
+
> [!IMPORTANT]
|
|
20
|
+
> Frameyard is 0.x and evolving. Features may change between releases, and some rough edges remain.
|
|
6
21
|
|
|
7
|
-
##
|
|
22
|
+
## Quick start
|
|
23
|
+
|
|
24
|
+
### Requirements
|
|
8
25
|
|
|
9
26
|
- Node.js 22.12 or newer
|
|
10
27
|
- FFmpeg and FFprobe on `PATH`, built with H.264 (`libx264`), AAC, HEVC (`libx265`), VP9 (`libvpx-vp9`), and Opus (`libopus`) support. The software HEVC and VP9 encoders are only used for software renders; `--encoder nvenc` and `--encoder qsv` need no software encoder for that codec.
|
|
11
28
|
- The Playwright-managed Chromium build
|
|
12
|
-
- Optional: `qwen-tts` from [`qwentts.cpp`](https://github.com/ServeurpersoCom/qwentts.cpp) for local speech synthesis
|
|
13
|
-
- Optional: `whisper-cli` from [`whisper.cpp`](https://github.com/ggml-org/whisper.cpp) for local transcription
|
|
14
29
|
|
|
15
30
|
Install the Playwright Chromium build with `npx playwright install chromium`. Run the CLI with `npx frameyard`; no global install is required.
|
|
16
31
|
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
## Quick start
|
|
32
|
+
Optional local integrations (`qwen-tts`, `sherpa-onnx-offline-tts`, and `whisper-cli`) are documented in [TTS and transcription](#tts-and-transcription).
|
|
20
33
|
|
|
21
|
-
Scaffold a project
|
|
34
|
+
### Scaffold a project
|
|
22
35
|
|
|
23
36
|
```sh
|
|
24
37
|
npx frameyard init my-video
|
|
@@ -26,13 +39,15 @@ cd my-video
|
|
|
26
39
|
npx frameyard dev
|
|
27
40
|
```
|
|
28
41
|
|
|
42
|
+
`init` derives the project name from the positional directory's basename. Run it without a directory to be prompted for a project name (default `my-video`); `init` then creates that directory. Non-interactive shells and `--json` runs use `my-video` without prompting.
|
|
43
|
+
|
|
29
44
|
`dev` prints a local preview URL. Preview a composition with JSON props editing, or open the timeline for a program monitor, read-only tracks and clips, scrubbing, playback, frame-by-frame stepping, one-second jumps, and diagnostics.
|
|
30
45
|
|
|
31
46
|
The scaffold includes an `AGENTS.md` guide for coding agents working in the project, covering the HTML authoring rules and the `check`, `still`, and `render` workflow. It also includes a `package.json` with `dev`, `check`, `render`, and `still` scripts pinned to the scaffolding CLI version, so `pnpm dev` or `npm run dev` works without a global install, and a `.gitignore` that ignores `out/`.
|
|
32
47
|
|
|
33
48
|
`npx frameyard` resolves the latest published version. Pin it with `npx frameyard@<version>`, or install the package locally, when a project needs a fixed CLI version.
|
|
34
49
|
|
|
35
|
-
Validate and render
|
|
50
|
+
### Validate and render
|
|
36
51
|
|
|
37
52
|
```sh
|
|
38
53
|
npx frameyard doctor
|
|
@@ -44,11 +59,101 @@ npx frameyard still main --frame 0,118-126 --out out/stills
|
|
|
44
59
|
|
|
45
60
|
The scaffold wraps its composition in `timelines/main.timeline.json`. `render` writes the video to `out/main.mp4` in the project by default; pass `--out <file.mp4>` to choose another path. The composition remains renderable on its own with `npx frameyard render hello --props props/hello.json`. The scaffold's `.gitignore` ignores `out/`.
|
|
46
61
|
|
|
47
|
-
|
|
62
|
+
Continue with [Examples](#examples) for the bundled projects, or [CLI reference](#cli-reference) for every command and option.
|
|
63
|
+
|
|
64
|
+
## Examples
|
|
65
|
+
|
|
66
|
+
The repository ships runnable example projects. Each example README has the commands for running it from a repository checkout.
|
|
67
|
+
|
|
68
|
+
| Example | What it shows |
|
|
69
|
+
| --- | --- |
|
|
70
|
+
| [`examples/demo`](./examples/demo) | The 30-second reference composition and its timeline. |
|
|
71
|
+
| [`examples/intro`](./examples/intro) | A 20-second intro film with four scenes, a HUD overlay, three dissolves, and an original score. |
|
|
72
|
+
| [`examples/webgl`](./examples/webgl) | A focused WebGL shader example with an external controller. |
|
|
73
|
+
| [`examples/video`](./examples/video) | Embedded-video compositions plus WebM and MP4 timelines that place clips on tracks under a transparent HTML overlay. |
|
|
74
|
+
| [`examples/blend`](./examples/blend) | Static CSS blend modes on composition and video clips. |
|
|
75
|
+
| [`examples/keyframes`](./examples/keyframes) | One timeline clip animated through every presentation channel. |
|
|
76
|
+
| [`examples/transitions`](./examples/transitions) | Three timed chord cards dissolved across one track with an audio crossfade. |
|
|
77
|
+
| [`examples/nested`](./examples/nested) | Two inner timelines repeated inside an outer timeline as full-frame and transformed group clips. |
|
|
78
|
+
| [`examples/static`](./examples/static) | A timeless composition rendered as a PNG and reused as a timeline image. |
|
|
79
|
+
|
|
80
|
+
## CLI reference
|
|
81
|
+
|
|
82
|
+
### Commands
|
|
83
|
+
|
|
84
|
+
| Command | Purpose |
|
|
85
|
+
| --- | --- |
|
|
86
|
+
| `frameyard init [directory] [--local-models]` | Scaffold a timeline-first HTML/JSON project with an `AGENTS.md` guide and pinned scripts; optionally add the local-model host scaffold |
|
|
87
|
+
| `frameyard dev [--port <port>]` | Start the preview server; preview compositions or open a timeline |
|
|
88
|
+
| `frameyard context [--project <dir>]` | Print a compact digest of format, composition metadata, props, timelines, assets, diagnostics, policy, and next commands |
|
|
89
|
+
| `frameyard lint <id> [--props <file>]` | Run static validation, timeline resolution, scene/keyframe projection, and a controller determinism scan without Chromium |
|
|
90
|
+
| `frameyard check <id> [--props <file>]` | Run static validation, then evaluate every animated frame or the single state of a static composition without capture |
|
|
91
|
+
| `frameyard still <id> --frame <spec>` | Capture one or more PNGs through the production path for a composition or timeline; a static composition exposes frame 0 |
|
|
92
|
+
| `frameyard inspect <image>` | Report dimensions, alpha, blank detection, colors, content bbox, luminance grid, and edge density |
|
|
93
|
+
| `frameyard compare <first> <second>` | Compare decoded pixels and 3×3 regions; exits 1 when the images differ |
|
|
94
|
+
| `frameyard render <id> [--out <video.mp4>]` | Render an animated composition directly, a static composition to PNG, or a timeline through the segmented lossless pipeline; defaults to `out/<id>.<ext>` |
|
|
95
|
+
| `frameyard tts ...` | Produce one frozen 24 kHz mono WAV with local qwentts.cpp or sherpa-onnx Supertonic 3 |
|
|
96
|
+
| `frameyard transcribe ...` | Produce canonical transcript JSON with optional word timings and SRT/VTT exports |
|
|
97
|
+
| `frameyard add --list` | List the built-in project templates and scaffolds |
|
|
98
|
+
| `frameyard add local-models` | Install the optional policy, OpenCode, and Pi scaffold into a workspace |
|
|
99
|
+
| `frameyard add <composition\|timeline> <id> --template <name>` | Write a template entry and print the exact `fy.config.js` registration snippet |
|
|
100
|
+
| `frameyard font search [query]` | Search the Fontsource catalog by name, category, subset, style, or variable axis |
|
|
101
|
+
| `frameyard font add <fontsource-id>` | Install a pinned Fontsource family (variable `wght` by default, or static weights with `--static`) |
|
|
102
|
+
| `frameyard font add --file <path> --family <name>` | Vendor a local font into `assets/fonts/<id>/` and update `fy.fonts.json` |
|
|
103
|
+
| `frameyard font add --url <https-url> --family <name>` | Download and vendor a font from HTTPS (loopback HTTP for tests) |
|
|
104
|
+
| `frameyard font list` | List installed families, subsets, licenses, files, and status |
|
|
105
|
+
| `frameyard font verify` | Verify manifest paths, formats, hashes, licenses, and orphaned directories |
|
|
106
|
+
| `frameyard font remove <id>` | Remove a font from the manifest and delete its directory unless `--keep-files` is supplied |
|
|
107
|
+
| `frameyard doctor` | Inspect core requirements, optional integrations, policy, resources, and manifest integrity |
|
|
108
|
+
|
|
109
|
+
### Options
|
|
110
|
+
|
|
111
|
+
Common options: `--project <dir>`, `--json`, `--help`, and `--version`. `still` and `render` accept `--overwrite`; `tts` and `transcribe` accept `--force`; both flags are denied outside `full`.
|
|
112
|
+
|
|
113
|
+
`still --frame <spec>` accepts a frame, an inclusive range (`2-12`), or a comma list. Duplicates are removed, order is preserved, and one invocation captures at most 32 unique frames (1 in `light`, 4 in `authoring`). With one frame `--out` is a `.png`; with multiple it is a directory of `still-<id>-<frame>.png` files. A static composition exposes frame 0.
|
|
114
|
+
|
|
115
|
+
`render` accepts `--concurrency <n>` (1-16) for compositions, `--workers <n>` (1-16) and `--scratch <dir>` for timelines, and `--container`, `--codec`, and `--encoder` (see [Containers, codecs, and encoders](#containers-codecs-and-encoders)). `--out` must match the container; the default is `out/<id>.<ext>`. A static composition writes `out/<id>.png` and rejects the video flags.
|
|
116
|
+
|
|
117
|
+
For a timeline, `--props` is keyed by composition clip id; for a composition, it is the props object. `init` takes the directory positionally, or prompts for a project name when no directory is given; `add --template <name>` selects a template and `add --list` lists them. `--out` paths are relative to the working directory. `tts`, `transcribe`, and `render` accept `--dry-run`, which prints the plan, handoff command, prerequisites, and blockers without writing, loading a model, launching Chromium, or encoding. `--profile shared-cpu` and `--low-priority` adjust caps and priority; see [Resource policy](#resource-policy).
|
|
118
|
+
|
|
119
|
+
`font search` accepts an optional query and `--category`, `--subset`, `--style`, `--variable`, and `--limit` filters. `font add <fontsource-id>` pins the Fontsource `npmVersion`, installs the variable `wght` face by default, and accepts `--static` plus `--weight`/`--style`/`--subset` lists. `font add --file` accepts `--id`, `--weight` (a number or an inclusive range), `--style normal|italic`, `--subset`, `--unicode-range`, `--license`, and `--license-file`. `font add --url` accepts the same metadata plus `--sha256`; HTTPS is required, with loopback HTTP allowed for tests. Font files are copied, hashed, and recorded in `fy.fonts.json`; adding another subset, style, or weight to the same family merges it into the existing entry. `--force` replaces a face or provider and is denied outside `full`.
|
|
120
|
+
|
|
121
|
+
A render fails immediately on an uncaught error, unhandled rejection, missing required asset, or invalid composition output. It writes to a temporary file and publishes only after FFmpeg succeeds; existing outputs are not replaced unless `--overwrite` is supplied.
|
|
122
|
+
|
|
123
|
+
### JSON results
|
|
124
|
+
|
|
125
|
+
Non-interactive commands support `--json`. Results go to stdout and human progress/diagnostics go to stderr:
|
|
126
|
+
|
|
127
|
+
```json
|
|
128
|
+
{
|
|
129
|
+
"ok": true,
|
|
130
|
+
"command": "check",
|
|
131
|
+
"composition": "hello",
|
|
132
|
+
"entry": "compositions/hello.html",
|
|
133
|
+
"metadata": { "width": 1920, "height": 1080, "fps": 30, "durationInFrames": 900 },
|
|
134
|
+
"framesChecked": 900,
|
|
135
|
+
"diagnostics": []
|
|
136
|
+
}
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
Diagnostics carry `severity`, `code`, `message`, and optional `details` with source locations and suggestions. Errors use the same envelope with `error.details` and top-level `diagnostics`:
|
|
140
|
+
|
|
141
|
+
```json
|
|
142
|
+
{
|
|
143
|
+
"ok": false,
|
|
144
|
+
"command": "check",
|
|
145
|
+
"error": { "code": "BINDING_INVALID", "message": "Unknown Frameyard attribute ..." },
|
|
146
|
+
"diagnostics": [
|
|
147
|
+
{ "severity": "error", "code": "BINDING_INVALID", "message": "Unknown Frameyard attribute \"data-fy-txt\".", "details": { "path": "compositions/hello.html", "line": 12, "column": 5, "attribute": "data-fy-txt" } }
|
|
148
|
+
]
|
|
149
|
+
}
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
`doctor --json` reports findings rather than throwing: `ok: true` means inspection completed, while `healthy` reports whether required checks, manifests, models, and integrations passed. See the [JSON result reference](./skills/frameyard/references/json-results.md) for all shapes.
|
|
48
153
|
|
|
49
|
-
## Project
|
|
154
|
+
## Project setup
|
|
50
155
|
|
|
51
|
-
A project is ordinary HTML, CSS, JSON timelines, assets, and optional native JavaScript controller modules. `
|
|
156
|
+
A project is ordinary HTML, CSS, JSON timelines, assets, and optional native JavaScript controller modules. `fy.config.js` explicitly lists the project format, composition entry points, and timeline entry points; there is no directory scanning:
|
|
52
157
|
|
|
53
158
|
```js
|
|
54
159
|
export default {
|
|
@@ -62,39 +167,74 @@ export default {
|
|
|
62
167
|
};
|
|
63
168
|
```
|
|
64
169
|
|
|
65
|
-
A composition
|
|
170
|
+
The project `format` fixes width, height, and fps for timelines and supplies defaults for compositions that omit dimensions. A composition that declares no dimensions in a project without `format` fails with `COMPOSITION_FORMAT_UNRESOLVED`: declare `width`/`height` or add `format { width, height, fps }` to `fy.config.js`. A composition-only project stays valid without `format` as long as its compositions declare their own. Fonts are independent of `fy.config.js`: install them with `npx frameyard font add`, which maintains `fy.fonts.json` and `assets/fonts/`.
|
|
171
|
+
|
|
172
|
+
## Composition authoring
|
|
173
|
+
|
|
174
|
+
A composition is a partial HTML fragment with one top-level `<fy-composition>` root. Metadata, default props, markup, CSS, bindings, audio, transcripts, and an optional controller all live in the fragment; Frameyard generates the document shell at render time:
|
|
175
|
+
|
|
176
|
+
```html
|
|
177
|
+
<script type="application/json" data-fy="props">
|
|
178
|
+
{ "title": "Hello", "accent": "#b8f36b" }
|
|
179
|
+
</script>
|
|
180
|
+
<style>
|
|
181
|
+
@keyframes fade-up {
|
|
182
|
+
from { opacity: 0; transform: translateY(24px); }
|
|
183
|
+
to { opacity: 1; transform: translateY(0); }
|
|
184
|
+
}
|
|
185
|
+
#title { animation: fade-up 1s ease-out both; }
|
|
186
|
+
</style>
|
|
187
|
+
<fy-composition width="1920" height="1080" fps="30" duration-in-frames="900">
|
|
188
|
+
<h1 id="title"
|
|
189
|
+
data-fy-text="title"
|
|
190
|
+
data-fy-anim-start="0"
|
|
191
|
+
data-fy-anim-duration="30"></h1>
|
|
192
|
+
<div id="progress" style="width: calc(var(--fy-progress) * 100%)"></div>
|
|
193
|
+
</fy-composition>
|
|
194
|
+
```
|
|
195
|
+
|
|
196
|
+
Do not add `<!doctype>`, `<html>`, `<head>`, `<body>`, `<title>`, or `<meta>`; the framework owns the document shell, import map, and host runtime. Props, the controller, and import maps are top-level siblings of the root. Top-level `<style>` and `<link rel="stylesheet">` elements are hoisted into the generated head; styles inside the root stay in the body.
|
|
197
|
+
|
|
198
|
+
An optional top-level `<script type="importmap">` can add bare specifiers for controllers. Frameyard hoists it before the host module; `frameyard` and `frameyard/` specifiers are reserved.
|
|
199
|
+
|
|
200
|
+
`width`, `height`, and `fps` are optional: a missing value is inherited from the project `format`, and a declared value always wins. Declare `width` and `height` together or not at all. The same fragment above can therefore be written as `<fy-composition duration-in-frames="900">`, rendering at 1920×1080 and 30 fps because the project fixes that format. Props are JSON-serializable, deep-frozen, and merged shallowly over the `data-fy="props"` defaults. They cannot change metadata.
|
|
201
|
+
|
|
202
|
+
A composition that declares **neither** `duration-in-frames` nor `fps` is static. It has no frame clock, uses width/height only, is captured once, and renders to a PNG; `render hello` writes `out/hello.png`. Its controller may `mount()` and validate props, but `update()` is not called and frame-state variables are not set. CSS animations are paused at time 0 with a warning; `data-fy-anim-*`, `<video>`, `<audio>`, and `<fy-transcript>` are errors. Declaring `fps` without `duration-in-frames` is an error. A static composition used in a timeline behaves as an image clip: declare the clip `duration` and keep `sourceIn: 0`. See [`examples/static`](./examples/static).
|
|
203
|
+
|
|
204
|
+
### Bindings
|
|
205
|
+
|
|
206
|
+
`data-fy-text`, `data-fy-attr-*`, `data-fy-class-*`, `data-fy-if`, and `<template data-fy-repeat>` apply props to the DOM once per load, before the controller mounts. Paths are dot-separated; missing paths warn. Primitive top-level props are also exposed as `--fy-prop-<name>` CSS custom properties. See the [bindings reference](./skills/frameyard/references/bindings.md) for repeat scope, coercion, and ordering.
|
|
207
|
+
|
|
208
|
+
### Animation
|
|
209
|
+
|
|
210
|
+
CSS `@keyframes` and the Web Animations API are the animation language. The runtime pauses and seeks every animation from the frame clock; `data-fy-anim-start`, `data-fy-anim-duration`, and `data-fy-anim-easing` schedule one animation per element in composition frames. Frame-state variables on `:root` are `--fy-frame`, `--fy-time`, `--fy-progress`, `--fy-fps`, `--fy-duration`, and `--fy-duration-frames`. CSS transitions are not deterministic and are diagnosed. See [deterministic authoring](./skills/frameyard/references/deterministic-authoring.md#animation) for the full rules.
|
|
211
|
+
|
|
212
|
+
### Controller
|
|
213
|
+
|
|
214
|
+
An optional controller handles what HTML and CSS cannot express. Declare it exactly once, inline or as an external module:
|
|
66
215
|
|
|
67
216
|
```html
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
@keyframes fade-up {
|
|
77
|
-
from { opacity: 0; transform: translateY(24px); }
|
|
78
|
-
to { opacity: 1; transform: translateY(0); }
|
|
79
|
-
}
|
|
80
|
-
#title { animation: fade-up 1s ease-out both; }
|
|
81
|
-
</style>
|
|
82
|
-
</head>
|
|
83
|
-
<body>
|
|
84
|
-
<fy-composition width="1920" height="1080" fps="30" duration-in-frames="900">
|
|
85
|
-
<h1 id="title"
|
|
86
|
-
data-fy-text="title"
|
|
87
|
-
data-fy-anim-start="0"
|
|
88
|
-
data-fy-anim-duration="30"></h1>
|
|
89
|
-
<div id="progress" style="width: calc(var(--fy-progress) * 100%)"></div>
|
|
90
|
-
</fy-composition>
|
|
91
|
-
</body>
|
|
92
|
-
</html>
|
|
217
|
+
<script type="text/fy-controller">
|
|
218
|
+
export default {
|
|
219
|
+
mount({ root, props }) { /* ... */ },
|
|
220
|
+
update({ frame, props, root }) { /* ... */ },
|
|
221
|
+
dispose() { /* ... */ },
|
|
222
|
+
validateProps(props) { /* ... */ },
|
|
223
|
+
};
|
|
224
|
+
</script>
|
|
93
225
|
```
|
|
94
226
|
|
|
95
|
-
`
|
|
227
|
+
`mount()` may be asynchronous; `update()` and `validateProps()` are synchronous and seek-safe; `dispose()` releases resources. The controller is the only executable code in a composition. Inline source is served as a virtual module, so relative imports and `import.meta.url` work; escape a literal `</script>`. See the [controller reference](./skills/frameyard/references/controllers.md) for external modules, validation-only controllers, and Canvas/WebGL.
|
|
96
228
|
|
|
97
|
-
###
|
|
229
|
+
### Canvas and WebGL
|
|
230
|
+
|
|
231
|
+
WebGL 1 and WebGL 2 canvases are captured through the normal frame path. Draw the complete frame from the controller's `update()` and do not depend on the previous frame's drawing buffer; clear or redraw the scene each time. Production sessions pin software WebGL (SwiftShader) so rendered pixels do not vary with the host GPU. See [`examples/webgl`](./examples/webgl) for a complete shader example.
|
|
232
|
+
|
|
233
|
+
### Composition lifecycle
|
|
234
|
+
|
|
235
|
+
The host loads the document, applies bindings, imports the controller, calls `mount()`, prepares media, and seeks to frame 0. `check`, `still`, and `render` then seek to requested frames and wait for media and paint before evaluating or capturing. `update({ frame, props })` is synchronous and seek-safe; use `random("stable:key")` instead of `Math.random()`, timers, or wall-clock reads. `check` runs static validation before launching Chromium and reports stable diagnostic codes with source locations.
|
|
236
|
+
|
|
237
|
+
## Timelines
|
|
98
238
|
|
|
99
239
|
A timeline is a JSON document that places composition and media clips on video and audio tracks. It inherits the project format and derives its duration from the latest clip end unless it declares `durationInFrames`:
|
|
100
240
|
|
|
@@ -103,7 +243,6 @@ A timeline is a JSON document that places composition and media clips on video a
|
|
|
103
243
|
"version": 1,
|
|
104
244
|
"tracks": [
|
|
105
245
|
{ "id": "v1", "kind": "video" },
|
|
106
|
-
{ "id": "v2", "kind": "video" },
|
|
107
246
|
{ "id": "a1", "kind": "audio" }
|
|
108
247
|
],
|
|
109
248
|
"clips": [
|
|
@@ -114,14 +253,6 @@ A timeline is a JSON document that places composition and media clips on video a
|
|
|
114
253
|
"start": 0,
|
|
115
254
|
"duration": 150
|
|
116
255
|
},
|
|
117
|
-
{
|
|
118
|
-
"id": "logo",
|
|
119
|
-
"track": "v2",
|
|
120
|
-
"source": { "kind": "image", "src": "assets/logo.png" },
|
|
121
|
-
"start": 30,
|
|
122
|
-
"duration": 60,
|
|
123
|
-
"transform": { "fit": "contain", "width": 480, "height": 270, "x": 640, "y": -320, "rotation": 0, "opacity": 0.9 }
|
|
124
|
-
},
|
|
125
256
|
{
|
|
126
257
|
"id": "music",
|
|
127
258
|
"track": "a1",
|
|
@@ -135,80 +266,22 @@ A timeline is a JSON document that places composition and media clips on video a
|
|
|
135
266
|
```
|
|
136
267
|
|
|
137
268
|
- Sources are `composition` + `id`, `timeline` + `id`, or `video`/`image`/`audio` + project-relative `src`.
|
|
138
|
-
- `start`, `sourceIn`, and `duration` use timeline frames.
|
|
139
|
-
- Composition clips can fix `props
|
|
140
|
-
- Visual clips support static `fit`, `width`/`height
|
|
141
|
-
- Video, composition, and nested-timeline clips contribute audio automatically unless muted
|
|
142
|
-
-
|
|
143
|
-
- Nested timelines are
|
|
144
|
-
|
|
145
|
-
### Bindings
|
|
269
|
+
- `start`, `sourceIn`, and `duration` use timeline frames. Image clips and static composition clips require a duration.
|
|
270
|
+
- Composition clips can fix `props` and trim the source range; a static composition clip needs `sourceIn: 0`, an explicit duration, and contributes no audio. An animated composition's `fps` must match the project.
|
|
271
|
+
- Visual clips support static `fit`, `width`/`height`, `x`, `y`, `scale`, `rotation`, `opacity`, and `blend`, and can animate `x`, `y`, `scale`, `rotation`, and `opacity` with clip-local `keyframes`.
|
|
272
|
+
- Video, composition, and nested-timeline clips contribute audio automatically unless muted; timeline audio clips live on audio tracks and support `gain` and `muted`.
|
|
273
|
+
- Two clips on one track may overlap when a timeline-level `transitions` entry pairs them with `dissolve` or `crossfade`; undeclared overlap is an error.
|
|
274
|
+
- Nested timelines are groups: the parent transform, opacity, and blend mode apply to the whole nested result, and nested audio is included.
|
|
146
275
|
|
|
147
|
-
|
|
148
|
-
- `data-fy-attr-src="path"` (and other `data-fy-attr-*` names) sets an attribute.
|
|
149
|
-
- `data-fy-class-active="path"` toggles a class.
|
|
150
|
-
- `data-fy-if="path"` removes the element when the value is false.
|
|
151
|
-
- `<template data-fy-repeat="items">` clones its content for each item, with `item` and `index` in scope and `--fy-index` on each clone.
|
|
152
|
-
- Paths are dot-separated (`user.name`, `items.0.label`). Missing paths warn and resolve to an empty value.
|
|
153
|
-
- Primitive top-level props are also exposed as `--fy-prop-<name>` CSS custom properties.
|
|
276
|
+
The [timeline reference](./skills/frameyard/references/timelines.md) documents every field, constraint, easing, transition geometry, and diagnostic. See the bundled examples for [keyframes](./examples/keyframes), [transitions](./examples/transitions), and [nested timelines](./examples/nested).
|
|
154
277
|
|
|
155
|
-
|
|
278
|
+
## Media
|
|
156
279
|
|
|
157
|
-
|
|
280
|
+
Compositions and timelines can use embedded video, audio, and transcripts.
|
|
158
281
|
|
|
159
|
-
|
|
282
|
+
### Embedded video
|
|
160
283
|
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
The runtime sets frame-state variables on `:root`:
|
|
164
|
-
|
|
165
|
-
| Variable | Meaning |
|
|
166
|
-
| --- | --- |
|
|
167
|
-
| `--fy-frame` | Current frame |
|
|
168
|
-
| `--fy-time` | Current time in seconds |
|
|
169
|
-
| `--fy-progress` | Progress from `0` to `1` |
|
|
170
|
-
| `--fy-fps` | Composition frame rate |
|
|
171
|
-
| `--fy-duration` | Duration in seconds |
|
|
172
|
-
| `--fy-duration-frames` | Duration in frames |
|
|
173
|
-
|
|
174
|
-
Use `--fy-progress` for progress bars and `--fy-frame` or `--fy-time` in `calc()` for simple computed motion. CSS transitions are not deterministic and are diagnosed.
|
|
175
|
-
|
|
176
|
-
### Controller
|
|
177
|
-
|
|
178
|
-
An optional controller handles what HTML and CSS cannot express. It is declared exactly once, either inline or as an external module:
|
|
179
|
-
|
|
180
|
-
```html
|
|
181
|
-
<script type="text/fy-controller">
|
|
182
|
-
export default {
|
|
183
|
-
mount({ root, props }) { /* ... */ },
|
|
184
|
-
update({ frame, props, root }) { /* ... */ },
|
|
185
|
-
dispose() { /* ... */ },
|
|
186
|
-
validateProps(props) { /* ... */ },
|
|
187
|
-
};
|
|
188
|
-
</script>
|
|
189
|
-
```
|
|
190
|
-
|
|
191
|
-
```html
|
|
192
|
-
<script type="text/fy-controller" src="./hello.controller.js"></script>
|
|
193
|
-
```
|
|
194
|
-
|
|
195
|
-
- `mount()` may be asynchronous; `update()` and `validateProps()` are synchronous and seek-safe; `dispose()` releases resources.
|
|
196
|
-
- Inline controllers are the default for small, single-use controllers. Extract to an external module for Canvas 2D, WebGL, karaoke, reuse, tests, or a large controller.
|
|
197
|
-
- A validation-only controller may export just `validateProps(props)`.
|
|
198
|
-
- The controller is the only executable code in a composition. Other executable scripts and `on*` handlers are rejected.
|
|
199
|
-
- Inline source is served through a virtual module URL, so relative imports and `import.meta.url` behave like an external module. A literal `</script>` inside inline source must be escaped.
|
|
200
|
-
|
|
201
|
-
### Composition lifecycle
|
|
202
|
-
|
|
203
|
-
The composition host loads the document, applies bindings, imports the controller, calls `mount()`, prepares media, and seeks to frame 0. `check`, `still`, and `render` then seek explicitly to requested frames and wait for embedded media and browser paint before evaluating or capturing. `dispose()` runs before the host releases the document.
|
|
204
|
-
|
|
205
|
-
The controller lifecycle is `mount → update → dispose`. `update({ frame, props })` must be synchronous and seek-safe. Use `random("stable:key")` from `frameyard/runtime` instead of `Math.random()`. Independent `requestAnimationFrame` loops, timers, wall-clock reads, and CSS transitions are not deterministic animation drivers.
|
|
206
|
-
|
|
207
|
-
`check` runs static composition validation before launching Chromium. It validates metadata, the root and controller declarations, default props, bindings, animation attributes, scripts and handlers, unsupported visual elements, and asset references, then reports stable diagnostic codes with source locations.
|
|
208
|
-
|
|
209
|
-
## Embedded video and WebGL
|
|
210
|
-
|
|
211
|
-
Embedded videos are declared in the composition document and frame-seeked by the runtime:
|
|
284
|
+
Declare videos in the fragment; the runtime pauses and frame-seeks them:
|
|
212
285
|
|
|
213
286
|
```html
|
|
214
287
|
<video
|
|
@@ -227,222 +300,201 @@ Embedded videos are declared in the composition document and frame-seeked by the
|
|
|
227
300
|
| `data-fy-duration` | Optional seconds of source to use | rest of source |
|
|
228
301
|
| `data-fy-volume` | Embedded audio volume from `0` to `1` | `1` |
|
|
229
302
|
|
|
230
|
-
|
|
303
|
+
Do not call `play()`, `pause()`, or set `currentTime`. Videos must be in the light DOM, are local project-relative files, and are fully loaded before frame production; remote URLs and live streams are not supported. An unmuted video contributes its soundtrack to the mix; set `muted` or `data-fy-audio="off"` for a visual-only asset. Fade overlay layers, not a full-scene ancestor of a video. See [`examples/video`](./examples/video) and [deterministic authoring](./skills/frameyard/references/deterministic-authoring.md#embedded-video-and-audio).
|
|
231
304
|
|
|
232
|
-
|
|
305
|
+
### Audio
|
|
233
306
|
|
|
234
|
-
|
|
307
|
+
Audio clips are `<audio>` elements with `data-fy-start` (composition frames), `data-fy-trim` (source seconds), optional `data-fy-duration`, and `data-fy-volume` (0-1). Frameyard never time-stretches audio or changes composition duration to fit it; `check` reports clip durations and overrun diagnostics. Preview synchronizes in the browser and production mixes the same clips offline with FFmpeg. See [audio assets](./skills/frameyard/references/audio-assets.md#audio-timeline).
|
|
235
308
|
|
|
236
|
-
|
|
309
|
+
### Transcripts
|
|
237
310
|
|
|
238
|
-
|
|
311
|
+
Declare canonical transcripts with `<fy-transcript src="assets/voice.transcript.json" words>`. Paths resolve from the project root; `words` requires measured word timings and fails with `TRANSCRIPT_WORDS_UNAVAILABLE` rather than interpolating them. `check`, `still`, and `render` validate declared transcripts; rendering timed text is the composition's responsibility. See [subtitles and karaoke](./skills/frameyard/references/subtitles-karaoke.md).
|
|
239
312
|
|
|
240
|
-
|
|
313
|
+
### Fonts
|
|
241
314
|
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
data-fy-volume="0.8"></audio>
|
|
315
|
+
Frameyard reads installed fonts from `fy.fonts.json` and serves one generated `@font-face` sheet to every composition. Fontsource is the default catalog:
|
|
316
|
+
|
|
317
|
+
```sh
|
|
318
|
+
npx frameyard font search "space grotesk" --subset latin --json
|
|
319
|
+
npx frameyard font add space-grotesk --subset latin
|
|
248
320
|
```
|
|
249
321
|
|
|
250
|
-
-
|
|
251
|
-
|
|
252
|
-
|
|
253
|
-
|
|
254
|
-
|
|
322
|
+
`font add` pins the Fontsource `npmVersion`, downloads only the requested WOFF2 subset/style/weight, vendors the package `LICENSE`, and records the SHA-256 and exact CDN URL. Variable families install the `wght` face by default; pass `--static --weight 400,700` for static files. A local or brand font uses the same manifest:
|
|
323
|
+
|
|
324
|
+
```sh
|
|
325
|
+
npx frameyard font add --file ./fonts/SpaceGrotesk.woff2 \
|
|
326
|
+
--family "Space Grotesk" --weight 300-700 --subset latin \
|
|
327
|
+
--license OFL-1.1 --license-file ./fonts/OFL.txt
|
|
328
|
+
```
|
|
255
329
|
|
|
256
|
-
|
|
330
|
+
An HTTPS URL is also accepted with `--sha256`:
|
|
257
331
|
|
|
258
|
-
|
|
332
|
+
```sh
|
|
333
|
+
npx frameyard font add --url https://example.com/Inter.woff2 \
|
|
334
|
+
--family "Inter" --license OFL-1.1 --sha256 <hash>
|
|
335
|
+
```
|
|
259
336
|
|
|
260
|
-
A
|
|
337
|
+
A URL import without `--license` or `--license-file` is recorded as unknown, warns, and requires the `full` policy. There is no shared font cache. Use the family name in CSS as usual:
|
|
261
338
|
|
|
262
|
-
```
|
|
263
|
-
|
|
339
|
+
```css
|
|
340
|
+
.title {
|
|
341
|
+
font-family: "Space Grotesk", system-ui, sans-serif;
|
|
342
|
+
font-weight: 700;
|
|
343
|
+
}
|
|
264
344
|
```
|
|
265
345
|
|
|
266
|
-
|
|
346
|
+
`font list` and `font verify` report the installed families and integrity. The generated sheet uses `font-display: block`; the host waits for `document.fonts.ready` and fails with `FONT_LOAD_FAILED` when a used face cannot load. Remote font URLs produce a `FONT_REMOTE` warning and are not a reproducibility guarantee. Commit `fy.fonts.json`, `assets/fonts/`, and license files with the project. See [fonts](./skills/frameyard/references/fonts.md) for Fontsource filters, static weights, URL imports, and licensing.
|
|
267
347
|
|
|
268
|
-
##
|
|
348
|
+
## Rendering and reproducibility
|
|
269
349
|
|
|
270
|
-
|
|
271
|
-
| --- | --- |
|
|
272
|
-
| `frameyard init [directory]` | Scaffold a timeline-first HTML/JSON project with an `AGENTS.md` guide and pinned scripts |
|
|
273
|
-
| `frameyard dev [--port <port>]` | Start the preview server; preview compositions or open a timeline |
|
|
274
|
-
| `frameyard check <id> [--props <file>]` | Run static validation, then evaluate every frame of a composition or timeline without capture |
|
|
275
|
-
| `frameyard still <id> --frame <spec>` | Capture one or more PNGs through the production path for a composition or timeline |
|
|
276
|
-
| `frameyard render <id> [--out <video.mp4>]` | Render a composition directly, or a timeline through the segmented lossless pipeline; defaults to `out/<id>.mp4` |
|
|
277
|
-
| `frameyard tts ...` | Produce one frozen 24 kHz mono WAV with local qwentts.cpp |
|
|
278
|
-
| `frameyard transcribe ...` | Produce canonical transcript JSON with optional word timings and SRT/VTT exports |
|
|
279
|
-
| `frameyard doctor` | Inspect core requirements, optional integrations, and manifest integrity |
|
|
350
|
+
Production rendering has two targets.
|
|
280
351
|
|
|
281
|
-
|
|
352
|
+
### Composition pipeline
|
|
282
353
|
|
|
283
|
-
A
|
|
354
|
+
A composition renders directly:
|
|
284
355
|
|
|
285
|
-
|
|
356
|
+
1. Static composition validation resolves metadata before launching Chromium.
|
|
357
|
+
2. Chromium navigates to the served composition document with the composition host.
|
|
358
|
+
3. For an animated composition, one session, or several when `--concurrency` is greater than 1, seeks every frame, calls the controller, seeks CSS/WAAPI animations, and waits for embedded media decode and paint. A static composition loads once and captures one PNG without seeking frames.
|
|
359
|
+
4. The viewport is captured through the Chrome DevTools Protocol, piped to FFmpeg, and encoded as H.264, HEVC, or VP9 with AAC or Opus audio.
|
|
286
360
|
|
|
287
|
-
|
|
361
|
+
### Timeline pipeline
|
|
288
362
|
|
|
289
|
-
|
|
290
|
-
{
|
|
291
|
-
"ok": true,
|
|
292
|
-
"command": "check",
|
|
293
|
-
"composition": "hello",
|
|
294
|
-
"entry": "compositions/hello.html",
|
|
295
|
-
"controller": { "kind": "inline" },
|
|
296
|
-
"metadata": { "width": 1920, "height": 1080, "fps": 30, "durationInFrames": 900 },
|
|
297
|
-
"framesChecked": 900,
|
|
298
|
-
"audio": { "clips": 1 },
|
|
299
|
-
"transcripts": { "clips": 0 },
|
|
300
|
-
"diagnostics": []
|
|
301
|
-
}
|
|
302
|
-
```
|
|
363
|
+
A timeline renders through segments:
|
|
303
364
|
|
|
304
|
-
|
|
365
|
+
1. Static timeline validation resolves references, media, nested timelines, durations, props, audio, overlaps, and transition geometry.
|
|
366
|
+
2. The timeline is split at active-layer changes and a maximum segment duration.
|
|
367
|
+
3. In parallel workers, Chromium captures active composition and nested-timeline clips to alpha-preserving FFV1 intermediates. A static composition is captured once per props variant as a transparent PNG and reused as an image layer across segments.
|
|
368
|
+
4. FFmpeg applies fit, presentation keyframes, transition alpha, opacity, video/image decoding, and audio mixing, and encodes each segment as H.264, HEVC, or VP9.
|
|
369
|
+
5. Segments are concatenated in timeline order without re-encoding and the global audio mix is muxed into the final output.
|
|
305
370
|
|
|
306
|
-
|
|
307
|
-
|
|
308
|
-
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
|
|
313
|
-
|
|
314
|
-
|
|
315
|
-
"message": "Unknown Frameyard attribute \"data-fy-txt\".",
|
|
316
|
-
"details": { "path": "compositions/hello.html", "line": 12, "column": 5, "attribute": "data-fy-txt" }
|
|
317
|
-
}
|
|
318
|
-
]
|
|
319
|
-
}
|
|
320
|
-
```
|
|
371
|
+
Composition concurrency changes scheduling only. Timeline workers process independent segments; output order and captured composition frames remain deterministic. `check` and `still` use the same clip capture and composite path as their target. A multi-frame `still` captures sequentially, reuses one preview server, and its frames are pixel-identical to capturing each frame separately. A static composition render performs one capture and does not seek frames.
|
|
372
|
+
|
|
373
|
+
### Containers, codecs, and encoders
|
|
374
|
+
|
|
375
|
+
Both targets encode H.264 in MP4 by default. `--container` selects `mp4`, `mov`, `mkv`, or `webm`; `--codec` selects `h264`, `hevc`, or `vp9`; `--encoder` selects `software`, `nvenc`, `qsv`, or `auto`. WebM defaults to VP9/Opus; MP4 and MOV hold H.264/HEVC with AAC; MKV holds all three. `auto` probes NVENC, then Quick Sync, then software. Hardware encoders produce different bytes, so use `software` for reproducible A/B measurements. HEVC uses the `hvc1` tag in MP4/MOV; H.264 remains the portable default. NVENC caps timeline workers at eight.
|
|
376
|
+
|
|
377
|
+
### Determinism guarantees
|
|
378
|
+
|
|
379
|
+
For the same composition or timeline revision, props, assets, segment plan, frame, and pinned runtime/browser/FFmpeg environment, the captured composition frames and deterministic composite are identical. Encoded bytes are not a reproducibility guarantee, even on the same machine; they depend on the FFmpeg build, encoder threading, and runtime environment. Preview composites in Chromium while timeline production composites with FFmpeg, so small differences in scaling, alpha, and video color conversion are expected; parity is guaranteed at clip capture.
|
|
321
380
|
|
|
322
|
-
|
|
381
|
+
### Benchmarks and evals
|
|
323
382
|
|
|
324
|
-
|
|
383
|
+
For a repeatable local measurement of seek, capture, encode, and full-render time, run `pnpm bench`; `pnpm bench:static` compares an animated composition with the same visual rendered as a static PNG. See `bench/README.md` for the measurement recipe. For local-model authoring evals against a llama.cpp router, run `pnpm eval:local`; see `evals/README.md`. It is opt-in, never renders, and is not part of `pnpm check`. Timeline performance is tracked separately with the long-form benchmarks; the representative target and worst-case measurements are in [ADR 0012](./docs/adr/0012-render-timelines-with-segmented-lossless-clip-intermediates.md).
|
|
384
|
+
|
|
385
|
+
## Tooling
|
|
386
|
+
|
|
387
|
+
### TTS and transcription
|
|
325
388
|
|
|
326
389
|
`tts` and `transcribe` wrap optional local executables. Frameyard never builds upstream tools, downloads models, or runs inference during composition loading or frame production.
|
|
327
390
|
|
|
391
|
+
Optional executables are `qwen-tts` from [`qwentts.cpp`](https://github.com/ServeurpersoCom/qwentts.cpp), `sherpa-onnx-offline-tts` from [`sherpa-onnx`](https://github.com/k2-fsa/sherpa-onnx) v2.0.1 plus the Supertonic 3 model archive, and `whisper-cli` from [`whisper.cpp`](https://github.com/ggml-org/whisper.cpp).
|
|
392
|
+
|
|
328
393
|
```sh
|
|
329
|
-
|
|
330
|
-
|
|
331
|
-
--
|
|
332
|
-
--talker-model models/qwen-talker-1.7b-customvoice-Q8_0.gguf \
|
|
333
|
-
--codec-model models/qwen-tokenizer-12hz-Q8_0.gguf \
|
|
334
|
-
--language English \
|
|
335
|
-
--text-file script.txt \
|
|
336
|
-
--out assets/voice.wav
|
|
337
|
-
|
|
338
|
-
# Canonical transcript with measured word timings and derived SRT/VTT.
|
|
339
|
-
npx frameyard transcribe assets/voice.wav \
|
|
340
|
-
--model models/ggml-base.en.bin \
|
|
341
|
-
--out assets/voice.transcript.json \
|
|
342
|
-
--words --export srt,vtt
|
|
343
|
-
```
|
|
394
|
+
npx frameyard tts --speaker vivian --talker-model models/qwen-talker-1.7b-customvoice-Q8_0.gguf \
|
|
395
|
+
--codec-model models/qwen-tokenizer-12hz-Q8_0.gguf --language English \
|
|
396
|
+
--text-file script.txt --out assets/voice.wav
|
|
344
397
|
|
|
345
|
-
|
|
346
|
-
|
|
347
|
-
|
|
348
|
-
- `transcribe` normalizes the source with FFmpeg to 16 kHz mono PCM WAV before `whisper-cli`.
|
|
349
|
-
- Outputs are atomic. Existing assets are replaced only with `--force`.
|
|
350
|
-
- A partial operation leaves no apparently valid artifact and no manifest entry.
|
|
351
|
-
- Transcription never fabricates word timings and never executes Whisper's generated karaoke script.
|
|
398
|
+
npx frameyard transcribe assets/voice.wav --model models/ggml-base.en.bin \
|
|
399
|
+
--out assets/voice.transcript.json --words --export srt,vtt
|
|
400
|
+
```
|
|
352
401
|
|
|
353
|
-
|
|
402
|
+
`tts` defaults to qwentts.cpp CustomVoice; `--engine supertonic` requires `--model-dir`, a numeric `--speaker`, and a language code. Executable resolution is `--executable`, then the engine's environment variable, then `PATH`. Outputs are atomic and replaced only with `--force`; successful operations update `fy.assets.json` with hashes, generator details, models, parameters, and transcription source identity. Commit it with the project.
|
|
354
403
|
|
|
355
|
-
|
|
404
|
+
The [audio assets reference](./skills/frameyard/references/audio-assets.md) covers both engines, transcription options, the provenance manifest, regeneration rules, and pinned upstream revisions.
|
|
356
405
|
|
|
357
|
-
|
|
406
|
+
### Doctor
|
|
358
407
|
|
|
359
408
|
```sh
|
|
360
409
|
npx frameyard doctor
|
|
361
410
|
npx frameyard doctor --json
|
|
362
411
|
npx frameyard doctor --talker-model models/talker.gguf --codec-model models/codec.gguf
|
|
412
|
+
npx frameyard doctor --sherpa-onnx-tts /path/to/sherpa-onnx-offline-tts --model-dir models/supertonic
|
|
363
413
|
```
|
|
364
414
|
|
|
365
|
-
`doctor` checks Node, FFmpeg and required encoders, FFprobe, Playwright Chromium, manifest integrity (artifact, source, model, and derived-export hashes), explicitly supplied model readability, and optional `qwen-tts
|
|
415
|
+
`doctor` checks Node, FFmpeg and required encoders, FFprobe, Playwright Chromium, manifest integrity (artifact, source, model, and derived-export hashes), installed fonts, explicitly supplied model readability, and optional `qwen-tts`, `sherpa-onnx-offline-tts`, and `whisper-cli` capabilities. It also reports the active resource policy and basic CPU, load, and memory resources. Pass `--talker-model`, `--codec-model`, `--model-dir`, or `--model` to inspect model readability before generation. It does not repair, download, or modify anything. Missing optional integrations do not fail, but a found executable with incompatible capabilities does.
|
|
366
416
|
|
|
367
|
-
|
|
417
|
+
### Context, inspect, and compare
|
|
368
418
|
|
|
369
|
-
|
|
419
|
+
`frameyard context --json` is a compact, read-only digest for agents and editors. It reports the project root and format, each composition's resolved metadata, prop keys and types, controller, and diagnostics; each timeline's duration, tracks, clip summaries, transitions, audio/transcript counts, and diagnostics; unique assets; installed fonts; the active policy; and a short list of next commands. It never launches Chromium.
|
|
370
420
|
|
|
371
|
-
|
|
421
|
+
`frameyard inspect <image> --json` decodes the first frame and reports dimensions, bytes, alpha range and coverage, blank detection, the dominant background and colors, the content bounding box, a coarse luminance grid, and edge density. `frameyard compare <first> <second> --json` reports differing pixels, percentage, maximum and mean channel delta, a bounding box, and a 3×3 region breakdown. `compare` exits 1 when the images differ; its JSON result still has `ok: true` and `identical: false`. Neither command replaces viewing a still when the host supports images.
|
|
372
422
|
|
|
373
|
-
|
|
423
|
+
### Templates
|
|
374
424
|
|
|
375
|
-
|
|
425
|
+
`frameyard add --list` lists the built-in project templates: `static-poster`, `title-card`, `lower-third`, and `slideshow`. Add writes one valid entry and prints the exact `fy.config.js` registration snippet; it deliberately does not edit the config.
|
|
376
426
|
|
|
377
|
-
|
|
378
|
-
|
|
379
|
-
|
|
380
|
-
|
|
427
|
+
```sh
|
|
428
|
+
npx frameyard add composition intro --template lower-third
|
|
429
|
+
npx frameyard add timeline reel --template slideshow
|
|
430
|
+
npx frameyard add local-models
|
|
431
|
+
```
|
|
381
432
|
|
|
382
|
-
|
|
433
|
+
`init --local-models` installs the same optional host scaffold for a new project:
|
|
383
434
|
|
|
384
|
-
|
|
385
|
-
|
|
386
|
-
|
|
387
|
-
4. FFmpeg applies fit, static transforms, opacity, video/image decoding, and audio mixing, and encodes each segment as H.264, HEVC, or VP9.
|
|
388
|
-
5. Segments are concatenated in timeline order without re-encoding and the global audio mix is muxed into the final output.
|
|
435
|
+
```sh
|
|
436
|
+
npx frameyard init my-project --local-models
|
|
437
|
+
```
|
|
389
438
|
|
|
390
|
-
|
|
439
|
+
The composition and timeline templates inherit the project `format`. The slideshow timeline expects `assets/slide-1.png` through `assets/slide-3.png`; provide them before `check`. An existing file or registered id is never overwritten.
|
|
391
440
|
|
|
392
|
-
|
|
441
|
+
### Resource policy
|
|
393
442
|
|
|
394
|
-
|
|
443
|
+
An optional `fy.policy.json` in the working directory or nearest ancestor gates expensive operations. It is independent of `fy.config.js`, so it also applies with `--project`.
|
|
444
|
+
|
|
445
|
+
| Tier | `still` | `check` | `tts` / `transcribe` / `render` | `--force` / `--overwrite` |
|
|
446
|
+
| --- | --- | --- | --- | --- |
|
|
447
|
+
| `light` | 1 frame | blocked | blocked | denied |
|
|
448
|
+
| `authoring` | 4 frames | allowed at low priority | blocked | denied |
|
|
449
|
+
| `full` | 32 frames | allowed | allowed | allowed |
|
|
395
450
|
|
|
396
|
-
|
|
451
|
+
`lint`, `doctor`, `init`, `dev`, and `--dry-run` are available in every tier. `font search`, `font list`, and `font verify` are available in every tier; `font add` requires `authoring` or `full`; `font remove`, `font add --force`, and `font add --url` without `--license`/`--license-file` require `full`. A policy file can only tighten its tier; unknown fields fail with `POLICY_INVALID`. `FRAMEYARD_POLICY=light|authoring|full` overrides the file; the default is `full`. `--profile shared-cpu` caps timeline workers at 2 and composition concurrency at 1, lowers priority, and leaves the encoder at `auto`; `--low-priority` lowers priority without the caps. Values above the cap fail with `RESOURCE_PROFILE_CONFLICT`. The policy is a guardrail, not a security boundary.
|
|
397
452
|
|
|
398
|
-
|
|
453
|
+
Install the optional local-model scaffold with `npx frameyard add local-models`, or `npx frameyard init my-project --local-models`; see [`LOCAL-MODELS.md`](./scaffolds/local-models/LOCAL-MODELS.md). See [ADR 0022](./docs/adr/0022-gate-heavy-operations-with-a-workspace-resource-policy.md) for the full contract.
|
|
454
|
+
|
|
455
|
+
### Agent skill
|
|
456
|
+
|
|
457
|
+
The optional host-neutral agent skill lives in [`skills/frameyard/`](./skills/frameyard). It uses the same project format and CLI as humans, documents HTML authoring, bindings, controllers, static diagnostics, fonts, audio assets, subtitle/karaoke recipes, JSON results, and failure recovery, and ships a thin CLI wrapper. The npm package includes the skill files matching its API version. Install it with `npx skills add vladiantio/frameyard`, or see [`skills/frameyard/README.md`](./skills/frameyard/README.md) for host-specific installation or referencing.
|
|
458
|
+
|
|
459
|
+
## Limits and non-goals
|
|
399
460
|
|
|
400
461
|
Deferred from this implementation slice:
|
|
401
462
|
|
|
402
|
-
-
|
|
403
|
-
- Full visual timeline editing, drag-and-drop, trim handles, and undo/redo
|
|
404
|
-
-
|
|
405
|
-
- Per-timeline formats and frame-rate conversion/retiming
|
|
406
|
-
- Persistent clip caches; timeline scratch intermediates are cleaned after a render
|
|
407
|
-
- GPU compositing and codecs beyond H.264, HEVC, and VP9
|
|
463
|
+
- Animated blend modes, filters, masks, and audio fades; GPU compositing and codecs beyond H.264, HEVC, and VP9
|
|
464
|
+
- Full visual timeline editing, drag-and-drop, trim handles, and undo/redo; timeline-level variables and prop overrides inside nested timelines
|
|
465
|
+
- Per-timeline formats and frame-rate conversion/retiming; persistent clip caches
|
|
408
466
|
- Built-in subtitle rendering, automatic caption burn-in, and declarative karaoke elements
|
|
409
|
-
- Nested `data-fy-repeat` scopes
|
|
410
|
-
-
|
|
467
|
+
- Nested `data-fy-repeat` scopes, a general template language, and declarative JSON Schema validation
|
|
468
|
+
- Typography roles (`--fy-font-display`, `--fy-font-body`), font-pair presets, a preview font picker, and variable axes other than `wght`
|
|
469
|
+
- `qwentts.cpp` voice cloning and VoiceDesign; Supertonic voice cloning and emotion selection; long-text chunking, streaming synthesis, and alternative generated-audio formats
|
|
470
|
+
- Exporting a fully static timeline as a PNG or recursively optimizing static nested timelines
|
|
471
|
+
|
|
472
|
+
Deliberate non-goals:
|
|
473
|
+
|
|
411
474
|
- Arbitrary executable scripts in composition HTML
|
|
412
|
-
- `qwentts.cpp` reference-audio voice cloning and VoiceDesign
|
|
413
|
-
- Automatic long-text chunking, streaming synthesis, and alternative generated-audio formats
|
|
414
475
|
- Automatic upstream builds, model downloads, environment repair, and hosted rendering
|
|
415
|
-
-
|
|
416
|
-
-
|
|
417
|
-
-
|
|
418
|
-
|
|
419
|
-
|
|
420
|
-
|
|
421
|
-
## Repository layout
|
|
422
|
-
|
|
423
|
-
- `src/contracts.ts`: shared composition, props, audio, transcript, and diagnostic contracts
|
|
424
|
-
- `src/bindings.ts`: browser-safe binding path and value semantics
|
|
425
|
-
- `src/composition/`: browser-safe composition document reader, descriptor, per-kind declarations, and adapter contract
|
|
426
|
-
- `src/html/`: Node-side parse5 adapter and static composition validation
|
|
427
|
-
- `src/transcript.ts`: canonical transcript validation and SRT/VTT formatting
|
|
428
|
-
- `src/runtime/`: browser host, frame clock, deterministic randomness, bindings, media, diagnostics
|
|
429
|
-
- `src/preview/`: interactive iframe preview client
|
|
430
|
-
- `src/server/`: native ES-module server, host injection, and inline-controller route
|
|
431
|
-
- `src/renderer/`: Chromium capture, static validation integration, audio mixing, transcript inspection, FFmpeg encoding
|
|
432
|
-
- `src/assets/`: local TTS/transcription wrappers, executable resolution, atomic publishing, provenance
|
|
433
|
-
- `src/doctor.ts`: environment, integration, and manifest inspection
|
|
434
|
-
- `src/scaffold.ts`: `init` scaffold templates, including the generated package manifest, project README, and agent guide
|
|
435
|
-
- `src/cli.ts`: command-line entry point
|
|
436
|
-
- `skills/`: official host-neutral agent skill
|
|
437
|
-
- `examples/demo/`: the 30-second reference composition
|
|
438
|
-
- `examples/webgl/`: the focused WebGL shader example
|
|
439
|
-
- `examples/video/`: embedded-video compositions and WebM/MP4 timelines with video clips under a transparent HTML overlay
|
|
440
|
-
- `bench/fixtures/`: controlled HTML compositions for the render benchmark
|
|
441
|
-
- `scripts/`: benchmark, release checks, versioning, and git-hook installation
|
|
442
|
-
- `docs/adr/`: architecture decision records
|
|
476
|
+
- A shared font cache or hosted font catalog; Google Fonts, Fontshare, and Bunny Fonts are not first-class providers. Fontsource is the catalog, and local/HTTPS imports cover the rest.
|
|
477
|
+
- Unbounded batch still capture and general image-sequence output; `still` accepts at most 32 inspection frames per invocation, and the active resource policy may cap it lower
|
|
478
|
+
- Audio, transcripts, or embedded video in a static composition; use a timeline. A `--video` escape hatch is not provided; use a timeline image clip.
|
|
479
|
+
- React or other UI-framework adapters, plugin APIs, and agent-only runtimes; TypeScript transpilation for project compositions and npm dependency optimization
|
|
480
|
+
|
|
481
|
+
See the [design baseline](./docs/v0.x-design.md) for the full product contract, [ADR 0007](./docs/adr/0007-make-html-documents-the-composition-unit.md) for the HTML authoring decision, [ADR 0010](./docs/adr/0010-make-timelines-the-render-unit.md) for timelines, [ADR 0012](./docs/adr/0012-render-timelines-with-segmented-lossless-clip-intermediates.md) for segmented rendering, [ADR 0013](./docs/adr/0013-capture-bounded-multi-frame-stills-for-inspection.md) for bounded multi-frame stills, [ADR 0016](./docs/adr/0016-author-compositions-as-html-fragments.md) for fragment composition authoring, [ADR 0017](./docs/adr/0017-render-timeless-compositions-as-stills.md) for static composition rendering, the [0.8 migration guide](./docs/migration-0.8.md), and the [domain glossary](./GLOSSARY.md) for terminology.
|
|
443
482
|
|
|
444
483
|
## Development
|
|
445
484
|
|
|
485
|
+
### Repository layout
|
|
486
|
+
|
|
487
|
+
- `src/`: framework source, split into browser-safe contracts, composition grammar, runtime, preview, server, renderer, assets, and CLI.
|
|
488
|
+
- `scaffolds/`: `init` and optional `local-models` project templates.
|
|
489
|
+
- `skills/frameyard/`: host-neutral agent skill and references.
|
|
490
|
+
- `examples/`: runnable demo, WebGL, video, blend, keyframes, transitions, nested, and static projects.
|
|
491
|
+
- `docs/`: design baseline, ADRs, migration guides, and preview testing.
|
|
492
|
+
- `bench/`, `scripts/`, `test/`: benchmarks, release tooling, and the public-seam test suite.
|
|
493
|
+
|
|
494
|
+
See [AGENTS.md](./AGENTS.md) for the per-file architecture and invariants.
|
|
495
|
+
|
|
496
|
+
### Local development
|
|
497
|
+
|
|
446
498
|
From a repository checkout:
|
|
447
499
|
|
|
448
500
|
```sh
|
|
@@ -455,23 +507,14 @@ node dist/cli.js doctor
|
|
|
455
507
|
|
|
456
508
|
These commands use the local build; `npx frameyard` runs the published package.
|
|
457
509
|
|
|
458
|
-
`pnpm check` runs the script syntax checks, lint, formatting checks, typechecks, and the full suite. `pnpm install` installs the repository's git hooks; pre-commit runs `pnpm check:static` and the unit tests, and pre-push runs the full suite. CI runs the same `pnpm check` on Linux with Node 22 and 24. For preview-only UI work, `pnpm check:preview` runs the static checks and the preview tests without the full suite; it does not replace `pnpm check` in CI or before pushing.
|
|
510
|
+
`pnpm check` runs the script syntax checks, lint, formatting checks, one build with the remaining typechecks, and the full suite against that build. `pnpm test:unit` runs the source-only project in parallel without a build; `pnpm test:integration` builds and runs the browser/FFmpeg project (up to three files at a time locally, scaled to core count, and one on CI). `pnpm install` installs the repository's git hooks; pre-commit runs `pnpm check:static` and the unit tests, and pre-push runs the full suite. CI runs the same `pnpm check` on Linux with Node 22 and 24. For preview-only UI work, `pnpm check:preview` runs the static checks and the preview tests without the full suite; it does not replace `pnpm check` in CI or before pushing.
|
|
459
511
|
|
|
460
|
-
CI uses fake `qwen-tts` and `whisper-cli` executables. Real-model tests are opt-in:
|
|
512
|
+
CI uses fake `qwen-tts`, `sherpa-onnx-offline-tts`, and `whisper-cli` executables. Real-model tests are opt-in: set `FRAMEYARD_REAL_INTEGRATIONS=1` plus the model paths read by `test/real-integrations.test.ts`, then run `pnpm vitest run test/real-integrations.test.ts`. Optional variables include `FRAMEYARD_REAL_TTS_DEVICE=CPU` and `FRAMEYARD_REAL_SUPERTONIC_MODEL_DIR`.
|
|
461
513
|
|
|
462
|
-
|
|
463
|
-
FRAMEYARD_REAL_INTEGRATIONS=1 \
|
|
464
|
-
FRAMEYARD_REAL_TALKER=/path/to/talker.gguf \
|
|
465
|
-
FRAMEYARD_REAL_CODEC=/path/to/codec.gguf \
|
|
466
|
-
FRAMEYARD_REAL_WHISPER_MODEL=/path/to/ggml-model.bin \
|
|
467
|
-
FRAMEYARD_REAL_WHISPER_SOURCE=/path/to/source.wav \
|
|
468
|
-
# optional, for GPUs with too little VRAM:
|
|
469
|
-
FRAMEYARD_REAL_TTS_DEVICE=CPU \
|
|
470
|
-
pnpm vitest run test/real-integrations.test.ts
|
|
471
|
-
```
|
|
514
|
+
### Releases
|
|
472
515
|
|
|
473
516
|
Releases are tag-driven. See [`RELEASING.md`](./RELEASING.md) for version policy, npm trusted publishing, and the release checklist, and [`CHANGELOG.md`](./CHANGELOG.md) for release notes.
|
|
474
517
|
|
|
475
518
|
## License
|
|
476
519
|
|
|
477
|
-
MIT
|
|
520
|
+
[MIT](LICENSE)
|