frameyard 0.8.0 → 0.10.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +177 -14
- package/GLOSSARY.md +112 -2
- package/README.md +302 -265
- package/dist/assets/integrations.d.ts +8 -4
- package/dist/assets/integrations.d.ts.map +1 -1
- package/dist/assets/integrations.js +8 -4
- package/dist/assets/integrations.js.map +1 -1
- package/dist/assets/provenance.d.ts +2 -2
- package/dist/assets/provenance.d.ts.map +1 -1
- package/dist/assets/provenance.js +5 -3
- package/dist/assets/provenance.js.map +1 -1
- package/dist/assets/tts-supertonic.d.ts +28 -0
- package/dist/assets/tts-supertonic.d.ts.map +1 -0
- package/dist/assets/tts-supertonic.js +156 -0
- package/dist/assets/tts-supertonic.js.map +1 -0
- package/dist/assets/tts.d.ts +18 -6
- package/dist/assets/tts.d.ts.map +1 -1
- package/dist/assets/tts.js +219 -97
- package/dist/assets/tts.js.map +1 -1
- package/dist/cli.js +1685 -93
- package/dist/cli.js.map +1 -1
- package/dist/composition/format.d.ts +1 -1
- package/dist/composition/format.d.ts.map +1 -1
- package/dist/composition/format.js +1 -1
- package/dist/composition/format.js.map +1 -1
- package/dist/context.d.ts +82 -0
- package/dist/context.d.ts.map +1 -0
- package/dist/context.js +277 -0
- package/dist/context.js.map +1 -0
- package/dist/contracts.d.ts +25 -0
- package/dist/contracts.d.ts.map +1 -1
- package/dist/contracts.js.map +1 -1
- package/dist/doctor.d.ts +14 -1
- package/dist/doctor.d.ts.map +1 -1
- package/dist/doctor.js +166 -7
- package/dist/doctor.js.map +1 -1
- package/dist/errors.d.ts +1 -1
- package/dist/errors.d.ts.map +1 -1
- package/dist/errors.js.map +1 -1
- package/dist/fonts/add.d.ts +41 -0
- package/dist/fonts/add.d.ts.map +1 -0
- package/dist/fonts/add.js +259 -0
- package/dist/fonts/add.js.map +1 -0
- package/dist/fonts/binary.d.ts +4 -0
- package/dist/fonts/binary.d.ts.map +1 -0
- package/dist/fonts/binary.js +21 -0
- package/dist/fonts/binary.js.map +1 -0
- package/dist/fonts/css.d.ts +4 -0
- package/dist/fonts/css.d.ts.map +1 -0
- package/dist/fonts/css.js +50 -0
- package/dist/fonts/css.js.map +1 -0
- package/dist/fonts/fontsource.d.ts +107 -0
- package/dist/fonts/fontsource.d.ts.map +1 -0
- package/dist/fonts/fontsource.js +565 -0
- package/dist/fonts/fontsource.js.map +1 -0
- package/dist/fonts/index.d.ts +11 -0
- package/dist/fonts/index.d.ts.map +1 -0
- package/dist/fonts/index.js +11 -0
- package/dist/fonts/index.js.map +1 -0
- package/dist/fonts/install.d.ts +24 -0
- package/dist/fonts/install.d.ts.map +1 -0
- package/dist/fonts/install.js +145 -0
- package/dist/fonts/install.js.map +1 -0
- package/dist/fonts/manifest.d.ts +21 -0
- package/dist/fonts/manifest.d.ts.map +1 -0
- package/dist/fonts/manifest.js +96 -0
- package/dist/fonts/manifest.js.map +1 -0
- package/dist/fonts/options.d.ts +9 -0
- package/dist/fonts/options.d.ts.map +1 -0
- package/dist/fonts/options.js +66 -0
- package/dist/fonts/options.js.map +1 -0
- package/dist/fonts/schema.d.ts +4 -0
- package/dist/fonts/schema.d.ts.map +1 -0
- package/dist/fonts/schema.js +316 -0
- package/dist/fonts/schema.js.map +1 -0
- package/dist/fonts/summary.d.ts +16 -0
- package/dist/fonts/summary.d.ts.map +1 -0
- package/dist/fonts/summary.js +22 -0
- package/dist/fonts/summary.js.map +1 -0
- package/dist/fonts/types.d.ts +56 -0
- package/dist/fonts/types.d.ts.map +1 -0
- package/dist/fonts/types.js +8 -0
- package/dist/fonts/types.js.map +1 -0
- package/dist/fonts/verify.d.ts +33 -0
- package/dist/fonts/verify.d.ts.map +1 -0
- package/dist/fonts/verify.js +200 -0
- package/dist/fonts/verify.js.map +1 -0
- package/dist/html/validate.d.ts.map +1 -1
- package/dist/html/validate.js +59 -0
- package/dist/html/validate.js.map +1 -1
- package/dist/index.d.ts +3 -3
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +1 -1
- package/dist/index.js.map +1 -1
- package/dist/inspect.d.ts +61 -0
- package/dist/inspect.d.ts.map +1 -0
- package/dist/inspect.js +325 -0
- package/dist/inspect.js.map +1 -0
- package/dist/lint.d.ts +63 -0
- package/dist/lint.d.ts.map +1 -0
- package/dist/lint.js +265 -0
- package/dist/lint.js.map +1 -0
- package/dist/policy.d.ts +69 -0
- package/dist/policy.d.ts.map +1 -0
- package/dist/policy.js +361 -0
- package/dist/policy.js.map +1 -0
- package/dist/preview/fonts/LICENSE-figtree.txt +93 -0
- package/dist/preview/fonts/LICENSE-space-mono.txt +93 -0
- package/dist/preview/fonts/README.md +23 -0
- package/dist/preview/fonts/figtree-latin-wght-normal.woff2 +0 -0
- package/dist/preview/fonts/space-mono-latin-400-normal.woff2 +0 -0
- package/dist/preview/fonts/space-mono-latin-700-normal.woff2 +0 -0
- package/dist/preview/main.js +5 -2
- package/dist/preview/main.js.map +1 -1
- package/dist/preview/timeline-data.d.ts +17 -69
- package/dist/preview/timeline-data.d.ts.map +1 -1
- package/dist/preview/timeline-host.d.ts.map +1 -1
- package/dist/preview/timeline-host.js +70 -45
- package/dist/preview/timeline-host.js.map +1 -1
- package/dist/preview/timeline-main.d.ts.map +1 -1
- package/dist/preview/timeline-main.js +88 -27
- package/dist/preview/timeline-main.js.map +1 -1
- package/dist/project.d.ts +10 -3
- package/dist/project.d.ts.map +1 -1
- package/dist/project.js +20 -17
- package/dist/project.js.map +1 -1
- package/dist/renderer/audio.d.ts +6 -0
- package/dist/renderer/audio.d.ts.map +1 -1
- package/dist/renderer/audio.js +3 -1
- package/dist/renderer/audio.js.map +1 -1
- package/dist/renderer/browser.d.ts.map +1 -1
- package/dist/renderer/browser.js +13 -0
- package/dist/renderer/browser.js.map +1 -1
- package/dist/renderer/concurrency.d.ts +6 -0
- package/dist/renderer/concurrency.d.ts.map +1 -0
- package/dist/renderer/concurrency.js +27 -0
- package/dist/renderer/concurrency.js.map +1 -0
- package/dist/renderer/ffmpeg.d.ts +104 -5
- package/dist/renderer/ffmpeg.d.ts.map +1 -1
- package/dist/renderer/ffmpeg.js +580 -67
- package/dist/renderer/ffmpeg.js.map +1 -1
- package/dist/renderer/index.js +1 -1
- package/dist/renderer/index.js.map +1 -1
- package/dist/renderer/keyframe-expressions.d.ts +39 -0
- package/dist/renderer/keyframe-expressions.d.ts.map +1 -0
- package/dist/renderer/keyframe-expressions.js +180 -0
- package/dist/renderer/keyframe-expressions.js.map +1 -0
- package/dist/renderer/session-pool.d.ts +28 -0
- package/dist/renderer/session-pool.d.ts.map +1 -0
- package/dist/renderer/session-pool.js +91 -0
- package/dist/renderer/session-pool.js.map +1 -0
- package/dist/renderer/timeline.d.ts +2 -0
- package/dist/renderer/timeline.d.ts.map +1 -1
- package/dist/renderer/timeline.js +600 -147
- package/dist/renderer/timeline.js.map +1 -1
- package/dist/renderer/timings.d.ts +13 -0
- package/dist/renderer/timings.d.ts.map +1 -0
- package/dist/renderer/timings.js +29 -0
- package/dist/renderer/timings.js.map +1 -0
- package/dist/runtime/host.d.ts.map +1 -1
- package/dist/runtime/host.js +4 -0
- package/dist/runtime/host.js.map +1 -1
- package/dist/scaffold.d.ts +3 -0
- package/dist/scaffold.d.ts.map +1 -1
- package/dist/scaffold.js +48 -320
- package/dist/scaffold.js.map +1 -1
- package/dist/server/composition-html.d.ts +2 -0
- package/dist/server/composition-html.d.ts.map +1 -1
- package/dist/server/composition-html.js +7 -0
- package/dist/server/composition-html.js.map +1 -1
- package/dist/server/preview-html.d.ts.map +1 -1
- package/dist/server/preview-html.js +74 -188
- package/dist/server/preview-html.js.map +1 -1
- package/dist/server/preview-style.d.ts +11 -0
- package/dist/server/preview-style.d.ts.map +1 -0
- package/dist/server/preview-style.js +298 -0
- package/dist/server/preview-style.js.map +1 -0
- package/dist/server/server.d.ts.map +1 -1
- package/dist/server/server.js +31 -1
- package/dist/server/server.js.map +1 -1
- package/dist/server/timeline-preview-data.d.ts.map +1 -1
- package/dist/server/timeline-preview-data.js +12 -34
- package/dist/server/timeline-preview-data.js.map +1 -1
- package/dist/server/timeline-preview-html.d.ts.map +1 -1
- package/dist/server/timeline-preview-html.js +140 -148
- package/dist/server/timeline-preview-html.js.map +1 -1
- package/dist/templates.d.ts +16 -0
- package/dist/templates.d.ts.map +1 -0
- package/dist/templates.js +186 -0
- package/dist/templates.js.map +1 -0
- package/dist/timeline/easing.d.ts +36 -0
- package/dist/timeline/easing.d.ts.map +1 -0
- package/dist/timeline/easing.js +225 -0
- package/dist/timeline/easing.js.map +1 -0
- package/dist/timeline/keyframes.d.ts +23 -0
- package/dist/timeline/keyframes.d.ts.map +1 -0
- package/dist/timeline/keyframes.js +95 -0
- package/dist/timeline/keyframes.js.map +1 -0
- package/dist/timeline/plan.d.ts +6 -12
- package/dist/timeline/plan.d.ts.map +1 -1
- package/dist/timeline/plan.js +17 -30
- package/dist/timeline/plan.js.map +1 -1
- package/dist/timeline/resolve.d.ts +12 -0
- package/dist/timeline/resolve.d.ts.map +1 -1
- package/dist/timeline/resolve.js +105 -9
- package/dist/timeline/resolve.js.map +1 -1
- package/dist/timeline/scene.d.ts +126 -0
- package/dist/timeline/scene.d.ts.map +1 -0
- package/dist/timeline/scene.js +109 -0
- package/dist/timeline/scene.js.map +1 -0
- package/dist/timeline/schema.d.ts +31 -0
- package/dist/timeline/schema.d.ts.map +1 -1
- package/dist/timeline/schema.js +240 -9
- package/dist/timeline/schema.js.map +1 -1
- package/dist/timeline/transitions.d.ts +21 -0
- package/dist/timeline/transitions.d.ts.map +1 -0
- package/dist/timeline/transitions.js +23 -0
- package/dist/timeline/transitions.js.map +1 -0
- package/dist/version.d.ts +1 -1
- package/dist/version.d.ts.map +1 -1
- package/dist/version.js +1 -1
- package/dist/version.js.map +1 -1
- package/docs/adr/0007-make-html-documents-the-composition-unit.md +1 -1
- package/docs/adr/0010-make-timelines-the-render-unit.md +3 -2
- package/docs/adr/0012-render-timelines-with-segmented-lossless-clip-intermediates.md +1 -1
- package/docs/adr/0014-compose-timeline-clips-with-css-blend-modes.md +2 -1
- package/docs/adr/0015-inherit-composition-format-from-the-project.md +2 -2
- package/docs/adr/0016-author-compositions-as-html-fragments.md +1 -1
- package/docs/adr/0018-animate-timeline-clips-with-keyframes.md +35 -0
- package/docs/adr/0019-make-same-track-overlap-the-transition-window.md +36 -0
- package/docs/adr/0020-crossfade-audio-clips-on-audio-tracks.md +27 -0
- package/docs/adr/0021-project-timeline-scenes-once.md +32 -0
- package/docs/adr/0022-gate-heavy-operations-with-a-workspace-resource-policy.md +35 -0
- package/docs/adr/0023-pin-fonts-as-project-assets.md +98 -0
- package/docs/adr/0024-stream-composition-captures-and-skip-trivial-composites.md +86 -0
- package/docs/adr/0025-capture-timeline-layers-in-parallel-with-a-global-limit.md +67 -0
- package/docs/adr/0026-reuse-loaded-composition-sessions-across-segments.md +70 -0
- package/docs/adr/0027-encode-two-full-frame-layers-through-dual-pipes.md +76 -0
- package/docs/adr/0028-encode-full-frame-video-segments-directly.md +65 -0
- package/docs/migration-0.4.md +2 -2
- package/docs/preview-testing.md +27 -6
- package/docs/{v0.1-design.md → v0.x-design.md} +57 -32
- package/examples/blend/README.md +3 -3
- package/examples/demo/README.md +2 -2
- package/examples/intro/README.md +71 -0
- package/examples/intro/assets/fonts/figtree/LICENSE.txt +93 -0
- package/examples/intro/assets/fonts/figtree/figtree-latin-wght-normal.woff2 +0 -0
- package/examples/intro/assets/fonts/instrument-serif/LICENSE.txt +93 -0
- package/examples/intro/assets/fonts/instrument-serif/instrument-serif-latin-400-italic.woff2 +0 -0
- package/examples/intro/assets/fonts/space-mono/LICENSE.txt +93 -0
- package/examples/intro/assets/fonts/space-mono/space-mono-latin-400-normal.woff2 +0 -0
- package/examples/intro/assets/fonts/space-mono/space-mono-latin-700-normal.woff2 +0 -0
- package/examples/intro/assets/grain.png +0 -0
- package/examples/intro/assets/score.ogg +0 -0
- package/examples/intro/compositions/brand.html +412 -0
- package/examples/intro/compositions/chrome.controller.js +63 -0
- package/examples/intro/compositions/chrome.html +256 -0
- package/examples/intro/compositions/lattice.controller.js +148 -0
- package/examples/intro/compositions/lattice.html +308 -0
- package/examples/intro/compositions/opening.html +504 -0
- package/examples/intro/compositions/studio.html +930 -0
- package/examples/intro/compositions/theme.css +218 -0
- package/examples/intro/fy.config.js +13 -0
- package/examples/intro/fy.fonts.json +100 -0
- package/examples/intro/scripts/make-grain.mjs +79 -0
- package/examples/intro/scripts/make-score.mjs +702 -0
- package/examples/intro/timelines/intro.timeline.json +60 -0
- package/examples/keyframes/README.md +76 -0
- package/examples/keyframes/compositions/backdrop.html +60 -0
- package/examples/keyframes/compositions/card.html +99 -0
- package/examples/keyframes/fy.config.js +10 -0
- package/examples/keyframes/timelines/keyframes.timeline.json +60 -0
- package/examples/nested/README.md +112 -0
- package/examples/nested/assets/pulse.wav +0 -0
- package/examples/nested/compositions/backdrop.html +84 -0
- package/examples/nested/compositions/card.html +275 -0
- package/examples/nested/compositions/overlay.html +307 -0
- package/examples/nested/fy.config.js +13 -0
- package/examples/nested/scripts/make-pulse.sh +20 -0
- package/examples/nested/timelines/chapter-a.timeline.json +39 -0
- package/examples/nested/timelines/chapter-b.timeline.json +39 -0
- package/examples/nested/timelines/nested-reel.timeline.json +52 -0
- package/examples/static/README.md +2 -2
- package/examples/transitions/README.md +128 -0
- package/examples/transitions/assets/alpha.wav +0 -0
- package/examples/transitions/assets/beta.wav +0 -0
- package/examples/transitions/assets/gamma.wav +0 -0
- package/examples/transitions/compositions/alpha.html +56 -0
- package/examples/transitions/compositions/beta.html +56 -0
- package/examples/transitions/compositions/gamma.html +56 -0
- package/examples/transitions/fy.config.js +12 -0
- package/examples/transitions/timelines/audio-crossfade.timeline.json +44 -0
- package/examples/transitions/timelines/transitions.timeline.json +31 -0
- package/examples/video/README.md +4 -4
- package/examples/webgl/README.md +1 -1
- package/package.json +13 -6
- package/scaffolds/init/AGENTS.md +131 -0
- package/scaffolds/init/README.md +56 -0
- package/scaffolds/init/compositions/hello.html +79 -0
- package/scaffolds/init/fy.config.js +9 -0
- package/scaffolds/init/gitignore.template +1 -0
- package/scaffolds/init/package.json +11 -0
- package/scaffolds/init/props/hello.json +4 -0
- package/scaffolds/init/props/main.json +6 -0
- package/scaffolds/init/timelines/main.timeline.json +20 -0
- package/scaffolds/local-models/LOCAL-MODELS.md +77 -0
- package/scaffolds/local-models/fy.policy.json +5 -0
- package/scaffolds/local-models/opencode/opencode.jsonc +17 -0
- package/scaffolds/local-models/pi/APPEND_SYSTEM.md +21 -0
- package/scaffolds/local-models/pi/sandbox.json +15 -0
- package/scaffolds/local-models/pi/settings.json +15 -0
- package/skills/frameyard/README.md +1 -0
- package/skills/frameyard/SKILL.md +59 -61
- package/skills/frameyard/references/audio-assets.md +32 -3
- package/skills/frameyard/references/common-mistakes.md +53 -2
- package/skills/frameyard/references/deterministic-authoring.md +10 -3
- package/skills/frameyard/references/diagnostics.md +31 -1
- package/skills/frameyard/references/failure-recovery.md +32 -6
- package/skills/frameyard/references/fonts.md +132 -0
- package/skills/frameyard/references/json-results.md +119 -6
- package/skills/frameyard/references/quick-reference.md +222 -0
- package/skills/frameyard/references/timelines.md +97 -7
- /package/examples/blend/{video.config.js → fy.config.js} +0 -0
- /package/examples/demo/{video.config.js → fy.config.js} +0 -0
- /package/examples/static/{video.config.js → fy.config.js} +0 -0
- /package/examples/video/{video.config.js → fy.config.js} +0 -0
- /package/examples/webgl/{video.config.js → fy.config.js} +0 -0
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
/** The overlap window of one transition, in absolute timeline frames. */
|
|
2
|
+
export interface TransitionWindow {
|
|
3
|
+
/** Inclusive first timeline frame. */
|
|
4
|
+
start: number;
|
|
5
|
+
/** Exclusive last timeline frame. */
|
|
6
|
+
end: number;
|
|
7
|
+
/** CSS timing function applied to the progress. Defaults to `linear`. */
|
|
8
|
+
easing?: string;
|
|
9
|
+
}
|
|
10
|
+
export type TransitionRole = "in" | "out";
|
|
11
|
+
/**
|
|
12
|
+
* Eased 0..1 progress of a transition at an absolute timeline frame. Frames
|
|
13
|
+
* outside the window clamp to 0 or 1, so segment boundaries never change values.
|
|
14
|
+
*/
|
|
15
|
+
export declare function transitionProgress(window: TransitionWindow, frame: number): number;
|
|
16
|
+
/**
|
|
17
|
+
* Equal-power gain for one side of a transition at an absolute timeline frame.
|
|
18
|
+
* The incoming clip ramps as `sqrt(t)`; the outgoing as `sqrt(1 - t)`.
|
|
19
|
+
*/
|
|
20
|
+
export declare function transitionGain(window: TransitionWindow, frame: number, role: TransitionRole): number;
|
|
21
|
+
//# sourceMappingURL=transitions.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"transitions.d.ts","sourceRoot":"","sources":["../../src/timeline/transitions.ts"],"names":[],"mappings":"AAEA,yEAAyE;AACzE,MAAM,WAAW,gBAAgB;IAC/B,sCAAsC;IACtC,KAAK,EAAE,MAAM,CAAC;IACd,qCAAqC;IACrC,GAAG,EAAE,MAAM,CAAC;IACZ,yEAAyE;IACzE,MAAM,CAAC,EAAE,MAAM,CAAC;CACjB;AAED,MAAM,MAAM,cAAc,GAAG,IAAI,GAAG,KAAK,CAAC;AAE1C;;;GAGG;AACH,wBAAgB,kBAAkB,CAAC,MAAM,EAAE,gBAAgB,EAAE,KAAK,EAAE,MAAM,GAAG,MAAM,CAOlF;AAED;;;GAGG;AACH,wBAAgB,cAAc,CAAC,MAAM,EAAE,gBAAgB,EAAE,KAAK,EAAE,MAAM,EAAE,IAAI,EAAE,cAAc,GAAG,MAAM,CAIpG"}
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
import { easeProgress, easingFor } from "./easing.js";
|
|
2
|
+
/**
|
|
3
|
+
* Eased 0..1 progress of a transition at an absolute timeline frame. Frames
|
|
4
|
+
* outside the window clamp to 0 or 1, so segment boundaries never change values.
|
|
5
|
+
*/
|
|
6
|
+
export function transitionProgress(window, frame) {
|
|
7
|
+
const length = window.end - window.start;
|
|
8
|
+
if (length <= 0) {
|
|
9
|
+
return 1;
|
|
10
|
+
}
|
|
11
|
+
const raw = (frame - window.start) / length;
|
|
12
|
+
return easeProgress(easingFor(window.easing), Math.max(0, Math.min(1, raw)));
|
|
13
|
+
}
|
|
14
|
+
/**
|
|
15
|
+
* Equal-power gain for one side of a transition at an absolute timeline frame.
|
|
16
|
+
* The incoming clip ramps as `sqrt(t)`; the outgoing as `sqrt(1 - t)`.
|
|
17
|
+
*/
|
|
18
|
+
export function transitionGain(window, frame, role) {
|
|
19
|
+
const progress = transitionProgress(window, frame);
|
|
20
|
+
const value = role === "in" ? progress : 1 - progress;
|
|
21
|
+
return Math.sqrt(Math.max(0, Math.min(1, value)));
|
|
22
|
+
}
|
|
23
|
+
//# sourceMappingURL=transitions.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"transitions.js","sourceRoot":"","sources":["../../src/timeline/transitions.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,SAAS,EAAE,MAAM,aAAa,CAAC;AActD;;;GAGG;AACH,MAAM,UAAU,kBAAkB,CAAC,MAAwB,EAAE,KAAa;IACxE,MAAM,MAAM,GAAG,MAAM,CAAC,GAAG,GAAG,MAAM,CAAC,KAAK,CAAC;IACzC,IAAI,MAAM,IAAI,CAAC,EAAE,CAAC;QAChB,OAAO,CAAC,CAAC;IACX,CAAC;IACD,MAAM,GAAG,GAAG,CAAC,KAAK,GAAG,MAAM,CAAC,KAAK,CAAC,GAAG,MAAM,CAAC;IAC5C,OAAO,YAAY,CAAC,SAAS,CAAC,MAAM,CAAC,MAAM,CAAC,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC,CAAC,CAAC;AAC/E,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,cAAc,CAAC,MAAwB,EAAE,KAAa,EAAE,IAAoB;IAC1F,MAAM,QAAQ,GAAG,kBAAkB,CAAC,MAAM,EAAE,KAAK,CAAC,CAAC;IACnD,MAAM,KAAK,GAAG,IAAI,KAAK,IAAI,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,GAAG,QAAQ,CAAC;IACtD,OAAO,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,KAAK,CAAC,CAAC,CAAC,CAAC;AACpD,CAAC"}
|
package/dist/version.d.ts
CHANGED
|
@@ -1,2 +1,2 @@
|
|
|
1
|
-
export declare const VERSION = "0.
|
|
1
|
+
export declare const VERSION = "0.10.0";
|
|
2
2
|
//# sourceMappingURL=version.d.ts.map
|
package/dist/version.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"version.d.ts","sourceRoot":"","sources":["../src/version.ts"],"names":[],"mappings":"AAAA,eAAO,MAAM,OAAO,
|
|
1
|
+
{"version":3,"file":"version.d.ts","sourceRoot":"","sources":["../src/version.ts"],"names":[],"mappings":"AAAA,eAAO,MAAM,OAAO,WAAW,CAAC"}
|
package/dist/version.js
CHANGED
|
@@ -1,2 +1,2 @@
|
|
|
1
|
-
export const VERSION = "0.
|
|
1
|
+
export const VERSION = "0.10.0";
|
|
2
2
|
//# sourceMappingURL=version.js.map
|
package/dist/version.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"version.js","sourceRoot":"","sources":["../src/version.ts"],"names":[],"mappings":"AAAA,MAAM,CAAC,MAAM,OAAO,GAAG,
|
|
1
|
+
{"version":3,"file":"version.js","sourceRoot":"","sources":["../src/version.ts"],"names":[],"mappings":"AAAA,MAAM,CAAC,MAAM,OAAO,GAAG,QAAQ,CAAC"}
|
|
@@ -11,7 +11,7 @@ Frameyard compositions will be declarative HTML documents rather than JavaScript
|
|
|
11
11
|
- **Pure no-JavaScript HTML/CSS.** Rejected because Canvas 2D and WebGL would become impossible without a new declarative drawing language, contradicting the existing visual-source contract.
|
|
12
12
|
- **HTML fragment injected into the preview container.** Rejected because relative asset URLs, document styles, and `<head>` content would need rewriting or scoping, making preview and render behavior fragile.
|
|
13
13
|
- **Add HTML as an alternative authoring format.** Rejected because two public composition models double the documentation, tests, and agent-skill surface; the repository is pre-1.0 and can make a clean breaking change.
|
|
14
|
-
- **Keep metadata and audio in `
|
|
14
|
+
- **Keep metadata and audio in `fy.config.js`.** Rejected because it splits a composition across files and makes the HTML document less portable.
|
|
15
15
|
- **External-only controllers.** Rejected because it forces a second file for small, single-use compositions, which is worse for agent authoring and review; inline source keeps those compositions self-contained while external modules remain available for reuse, tests, and complex logic.
|
|
16
16
|
- **Browser-only validation.** Rejected because agents would pay a Chromium launch for source-level mistakes and would not get line and column locations; the server already parses composition HTML for host injection, script rejection, and inline-controller extraction.
|
|
17
17
|
|
|
@@ -2,6 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
**Status:** Accepted
|
|
4
4
|
**Date:** 2026-10-31
|
|
5
|
+
**Amended by:** [ADR-0018](0018-animate-timeline-clips-with-keyframes.md) (timeline clip keyframes)
|
|
5
6
|
|
|
6
7
|
Frameyard will render timelines as its primary output unit. A timeline is a project-local JSON document with a project-fixed width, height, and frame rate, ordered video and audio tracks, and clips sourced from HTML compositions, local media, or other timelines. A composition remains the reusable HTML authoring unit and becomes a clip source whose source range and props are fixed by each clip. Standalone composition renders remain supported on the existing direct path.
|
|
7
8
|
|
|
@@ -15,10 +16,10 @@ Frameyard will render timelines as its primary output unit. A timeline is a proj
|
|
|
15
16
|
|
|
16
17
|
## Consequences
|
|
17
18
|
|
|
18
|
-
- `
|
|
19
|
+
- `fy.config.js` gains an optional project `format` and an explicit `timelines` map. IDs are unique across compositions and timelines so `render <id>`, `check <id>`, and `still <id>` stay unambiguous.
|
|
19
20
|
- A composition used in a timeline must share the project frame rate exactly; a different size is fitted and transformed by the clip.
|
|
20
21
|
- Timeline duration is derived from the last clip end, with an optional override. A timeline with no clips is invalid; an audio-only timeline is valid and renders black video with audio.
|
|
21
22
|
- Nested timelines are visual and audio sources, are cycle-checked, and do not accept prop overrides for their internal compositions in 0.6.
|
|
22
|
-
- Timeline clips use static fit, position, scale, rotation, and opacity. Transitions,
|
|
23
|
+
- Timeline clips use static fit, position, scale, rotation, and opacity by default; clip keyframes are added by ADR 0018. Transitions, effects, track solo/hidden, and audio fades are deferred.
|
|
23
24
|
- Composition rendering keeps its existing result schema; timeline rendering returns a new timeline result schema.
|
|
24
25
|
- The design baseline, glossary, agent skill, scaffold, examples, and tests gain timeline language; existing composition projects remain valid without migration.
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Render timelines with segmented lossless clip intermediates
|
|
2
2
|
|
|
3
|
-
**Status:** Accepted
|
|
3
|
+
**Status:** Accepted. Extended by [ADR-0024](./0024-stream-composition-captures-and-skip-trivial-composites.md): captures stream into FFV1, and single full-frame compositions skip the intermediate and composite.
|
|
4
4
|
**Date:** 2026-10-31
|
|
5
5
|
|
|
6
6
|
Timeline rendering will be segmented and parallel. For each segment, Chromium renders the active composition and nested-timeline clips to lossless FFV1 intermediates; FFmpeg decodes media, applies the clip's fit and static transform, composites the segment, encodes H.264, and the ordered segments are concatenated without re-encoding. Audio is mixed once for the whole timeline and muxed at the end. Preview remains a live Chromium timeline host with a bounded pool of active clips; parity is guaranteed at clip capture, not at the final composite.
|
|
@@ -2,10 +2,11 @@
|
|
|
2
2
|
|
|
3
3
|
**Status:** Accepted
|
|
4
4
|
**Date:** 2026-11-03
|
|
5
|
+
**Amended by:** [ADR-0018](0018-animate-timeline-clips-with-keyframes.md) (clip keyframes; blend modes remain static)
|
|
5
6
|
|
|
6
7
|
Visual timeline clips may declare a static `blend` mode. The mode is a CSS `mix-blend-mode` value and composites the clip against the accumulated backdrop of all lower layers and the timeline background. The clip's source alpha gates the blend: transparent pixels leave the backdrop unchanged. The supported modes are `normal`, `multiply`, `screen`, `overlay`, `darken`, `lighten`, `color-dodge`, `color-burn`, `hard-light`, `soft-light`, `difference`, and `exclusion`. `hue`, `saturation`, `color`, and `luminosity` require cross-channel math and are rejected at validation. `blend` is a top-level clip field, omitted means `normal`, and the timeline document stays at version 1.
|
|
7
8
|
|
|
8
|
-
Preview applies `mix-blend-mode` to the clip's DOM wrapper. Production applies FFmpeg's `blend` filter with the source alpha mask, then composites the blended result source-over. Chromium/CSS is the reference; FFmpeg is an approximation. The existing guarantee remains: parity is at clip capture, not at the final encoded pixel. Blend modes are static; there are no
|
|
9
|
+
Preview applies `mix-blend-mode` to the clip's DOM wrapper. Production applies FFmpeg's `blend` filter with the source alpha mask, then composites the blended result source-over. Chromium/CSS is the reference; FFmpeg is an approximation. The existing guarantee remains: parity is at clip capture, not at the final encoded pixel. Blend modes are static; there are no animated blend modes.
|
|
9
10
|
|
|
10
11
|
## Context
|
|
11
12
|
|
|
@@ -3,7 +3,7 @@
|
|
|
3
3
|
**Status:** Accepted. Amends [ADR-0007](./0007-make-html-documents-the-composition-unit.md) and [ADR-0010](./0010-make-timelines-the-render-unit.md).
|
|
4
4
|
**Date:** 2026-11-03
|
|
5
5
|
|
|
6
|
-
`<fy-composition>` will keep `duration-in-frames` required but make `width`, `height`, and `fps` optional. A composition that omits any of them inherits the value from the single project `format` in `
|
|
6
|
+
`<fy-composition>` will keep `duration-in-frames` required but make `width`, `height`, and `fps` optional. A composition that omits any of them inherits the value from the single project `format` in `fy.config.js`; a declared value always wins. Declaring `width` without `height` (or the reverse) remains an error. A composition that declares its full format keeps working unchanged, including in composition-only projects that have no `format`.
|
|
7
7
|
|
|
8
8
|
The unit of reuse moves from "a file with fixed output pixels" to "a source that composes at whatever the project or timeline provides". The same composition can therefore appear in a 1080p timeline and a vertical timeline without editing the file or duplicating attributes that the project already fixes. Timelines do not gain their own format: `ResolvedTimeline.format` is already `project.format`, so timeline and standalone inheritance are the same resolution.
|
|
9
9
|
|
|
@@ -21,7 +21,7 @@ The unit of reuse moves from "a file with fixed output pixels" to "a source that
|
|
|
21
21
|
- Declared metadata is distinct from resolved metadata. `CompositionDescriptor.metadata` exposes `{ format: DeclaredFormat; durationInFrames }`, and `CompositionSourceInfo` carries the resolved `metadata` plus `declaredFormat`. `CompositionMetadata`, `window.__fy.metadata`, and render results keep exposing resolved numbers only.
|
|
22
22
|
- The preview server injects the project format as a `data-fy="format"` JSON block marked `data-fy-host`. The browser host reads it to resolve the same values the Node-side validator used; the existing static-vs-runtime metadata equality check therefore still holds.
|
|
23
23
|
- The composition route and `project.json` resolve metadata in the server. `project.json` keeps exposing resolved metadata, or `null` when the document is invalid or the format cannot be resolved.
|
|
24
|
-
- Rendering a composition that omits format values in a project without `format` fails with `COMPOSITION_FORMAT_UNRESOLVED`, a context error rather than a document diagnostic. The message names both fixes: declare the attributes or add `format` to `
|
|
24
|
+
- Rendering a composition that omits format values in a project without `format` fails with `COMPOSITION_FORMAT_UNRESOLVED`, a context error rather than a document diagnostic. The message names both fixes: declare the attributes or add `format` to `fy.config.js`.
|
|
25
25
|
- A declared `fps` that differs from the project format remains `TIMELINE_FPS_MISMATCH`. The check compares the declared fps, not the resolved one, so inheritance never hides a real mismatch. Width and height keep the existing fit behavior with no mismatch error.
|
|
26
26
|
- The resolved format is part of the immutable frame environment, alongside props and loaded assets. Layouts that use viewport units become format-dependent by design; preview and production use the same resolved values.
|
|
27
27
|
- The scaffold keeps declaring an explicit format as the safe teaching default. `examples/blend/compositions/backdrop.html` demonstrates inheritance.
|
|
@@ -11,7 +11,7 @@ Frameyard 0.8 makes a composition a partial HTML fragment instead of a full HTML
|
|
|
11
11
|
- **Accept both documents and fragments.** Rejected because two authoring shapes double validation, diagnostics, documentation, and agent examples for a pre-1.0 breaking release.
|
|
12
12
|
- **Inject a fragment into the preview container without a document.** Rejected for the reasons in ADR-0007: relative asset URLs, document styles, and head content would need rewriting. Serving a generated document at the real composition URL avoids that problem.
|
|
13
13
|
- **Generate full documents at build time.** Rejected because Frameyard has no build step and the preview server already renders composition entries per request.
|
|
14
|
-
- **Let projects configure the document head.** Rejected because the author goal is to stop managing the head; a configuration surface would recreate the problem in `
|
|
14
|
+
- **Let projects configure the document head.** Rejected because the author goal is to stop managing the head; a configuration surface would recreate the problem in `fy.config.js`.
|
|
15
15
|
|
|
16
16
|
## Decision
|
|
17
17
|
|
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
# Animate timeline clips with keyframes
|
|
2
|
+
|
|
3
|
+
**Status:** Accepted
|
|
4
|
+
**Date:** 2026-11-04
|
|
5
|
+
|
|
6
|
+
Visual timeline clips may animate `x`, `y`, `scale`, `rotation`, and `opacity` through a top-level `keyframes` object. Each channel is an ordered array of `{ frame, value, easing? }` entries in clip-local integer frames. Keys hold before the first and after the last key, never extend the clip, and use a CSS timing function on the interval after each key. A channel cannot also be an explicit scalar in `transform`, and `keyframes.scale` cannot be combined with a `transform.width`/`height` box. The timeline document version remains 1.
|
|
7
|
+
|
|
8
|
+
Preview evaluates the CSS timing function exactly on every seek. Production generates per-frame FFmpeg expressions from the same browser-safe evaluator; position, scale, and rotation use filter expressions, and animated opacity uses an alpha mask applied before blend-mode compositing. Easing curves in the filter graph approximate the CSS curve; parity remains at clip capture, not at the final encoded pixel. `check` evaluates every keyframe channel at every timeline frame. The read-only preview shows merged keyframe markers and seeks from them.
|
|
9
|
+
|
|
10
|
+
## Context
|
|
11
|
+
|
|
12
|
+
Timelines had static transforms and static blend modes; keyframes were explicitly deferred by ADR 0010. The render unit is a segmented FFmpeg composite, and ADR 0012's performance model depends on not capturing every layer frame through Chromium. Animation therefore has to be expressed in the FFmpeg graph, not by adding a second per-frame compositor. At the same time, the preview is the authoring reference, so both paths need one definition of frame sampling and easing.
|
|
13
|
+
|
|
14
|
+
## Considered Options
|
|
15
|
+
|
|
16
|
+
- **Whole-transform keyframe snapshots.** More ergonomic for coordinated moves, but every key needs a rule for omitted channels, and the rule is ambiguous. Per-channel tracks make "keyed or static" explicit and map directly to preview styles and filter parameters.
|
|
17
|
+
- **A separate animation document or named presets.** Adds a reference namespace, resolution, and cross-clip coupling. Inline tracks keep a clip self-contained; presets can be added later without breaking inline tracks.
|
|
18
|
+
- **Source-relative or absolute timeline frames.** Source-relative time preserves animation phase across slips but makes trims confusing. Absolute timeline frames decouple animation from clip placement. Clip-local frames keep the animation with the clip and leave `sourceIn` as a pure media-content selector.
|
|
19
|
+
- **Arbitrary expression values.** A formula language would add determinism, validation, and preview-parity risk. Values stay JSON numbers; complex motion belongs in a composition controller.
|
|
20
|
+
- **Composite keyframed clips in Chromium or software per frame.** Better preview parity, but a second production compositor and a large throughput cost. FFmpeg expressions keep one path and the existing segmentation model.
|
|
21
|
+
- **Allow animated blend modes or audio gain.** Blend modes are discrete and not interpolable; audio fades are a separate contract. Both remain deferred.
|
|
22
|
+
- **Constrain easing to `linear` and `hold` or to named curves only.** The composition runtime already speaks CSS easing, and preview can evaluate it exactly. The filter graph approximates any supported curve with an adaptive Hermite expression within a fixed error budget.
|
|
23
|
+
|
|
24
|
+
## Consequences
|
|
25
|
+
|
|
26
|
+
- Clip keyframes are part of the timeline schema and resolved clips. Structural errors use `TIMELINE_CLIP_INVALID`; keys outside the resolved duration use `TIMELINE_CLIP_OUT_OF_RANGE`, with clip, track, channel, key index, frame, and JSON path details.
|
|
27
|
+
- `opacity` is clamped to `0..1` and `scale` to `>= 0` after easing. Position and rotation may overshoot. A zero-scale frame is invisible; the scale filter uses a zero-safe expression and the opacity mask gates it.
|
|
28
|
+
- Animated opacity and animated scale add a small alpha-mask pass at the fitted box size. The mask is generated from a 1×1 per-frame source and multiplied into the layer alpha, so blend modes see the animated source alpha. No stage-sized mask is added for normal layers.
|
|
29
|
+
- FFmpeg's rotate filter clips to a fixed output size. Static rotation uses the exact rotated bounding box; keyed rotation uses the diagonal of the fitted box, which is segment-stable and large enough for every angle. The filter fills the remaining area with transparent pixels, so rotation no longer cuts off corners.
|
|
30
|
+
- Segments are never split at keyframe boundaries. Values are evaluated against absolute timeline frames, so changing `maxSegmentFrames` or worker count does not change a frame's keyframe values.
|
|
31
|
+
- Production expressions are exact at keyframe frames and within 1/255 opacity, 0.5 px position, 0.1% scale, and 0.1° rotation of the shared evaluator between keys. Encoded output keeps the existing scaling, alpha, and color-conversion differences.
|
|
32
|
+
- `check` adds a per-frame keyframe sweep without capturing or encoding. `still` and `render` share the same composite path and `frameOffset`.
|
|
33
|
+
- The preview draws one marker per distinct keyframe frame across channels, with a channel/value tooltip and click-to-seek. It does not add an editing UI or keyframe manipulation.
|
|
34
|
+
- Timeline documents remain version 1; keyframes are additive. Documentation covers the contract, validation, preview markers, and the production approximation. The reference example is `examples/keyframes`.
|
|
35
|
+
- Performance is measured with a keyframed fixture; no hard CI ratio is enforced.
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# Make same-track overlap the transition window
|
|
2
|
+
|
|
3
|
+
**Status:** Accepted
|
|
4
|
+
**Date:** 2026-11-04
|
|
5
|
+
**Amended by:** [ADR 0020](./0020-crossfade-audio-clips-on-audio-tracks.md) adds the `crossfade` kind for audio tracks; the audio-track prohibition in this record is superseded.
|
|
6
|
+
|
|
7
|
+
A timeline may declare a `transitions` array. Each entry names a `from` clip, a `to` clip, and a `kind`. A dissolve requires the two clips to overlap on one video track; the overlap `[to.start, from.end)` is the transition window. Only the incoming clip carries animated alpha, `opacity × eased progress`, and both clips' contributed audio crossfades with equal power. The annotation is explicit: an overlap without a declared transition is still `TIMELINE_CLIP_OVERLAP`. The timeline document version remains 1.
|
|
8
|
+
|
|
9
|
+
## Context
|
|
10
|
+
|
|
11
|
+
ADR 0010 rejected same-track overlap because transitions were deferred and overlap would make visual and audio precedence ambiguous. Crossfades used separate tracks plus opposing opacity keyframes, which is valid but doubles the track count and leaves the relationship implicit. The roadmap now needs transitions as a first-class authoring feature.
|
|
12
|
+
|
|
13
|
+
The renderer is segmented: every segment has a fixed layer set, and the segment plan builds layers from visual clips that fully cover the range. Preview activates clips pointwise. Both paths already evaluate presentation against absolute timeline frames, which is what keeps segments independent. A transition must not introduce a second timing model, retiming, or media handles.
|
|
14
|
+
|
|
15
|
+
## Considered Options
|
|
16
|
+
|
|
17
|
+
- **Explicit overlap.** The clips visibly overlap by the transition length and the overlap is the window. Every clip's visible range stays exactly its declared range, so the planner, preview active set, `still`, and `check` all agree with the JSON. Authors move the incoming clip earlier by the transition duration. No handles are needed: the outgoing clip's declared duration already covers its side, and the incoming clip starts at its own `start`.
|
|
18
|
+
- **Cut-centered implicit extension.** Clips stay butt-joined and the renderer extends one or both past its declared bounds. This needs media handles that video and nested timelines may not have, and creates frames where a clip is visible outside `[start, end)`. `still`, `check`, and the preview would disagree with the document.
|
|
19
|
+
- **Transition inserts time and retimes downstream clips.** Timeline duration would depend on transition declarations, and every later clip would shift. Declarative JSON should not hide retiming.
|
|
20
|
+
- **Overlap alone implies a dissolve.** An accidental two-frame overlap would silently become a fade instead of failing. Explicit declaration keeps overlap an error unless intended.
|
|
21
|
+
- **FFmpeg `xfade` or a catalog of effects.** `xfade` is relative to a filter offset, not the absolute timeline frame, and most modes have no CSS equivalent. Preview parity would silently break.
|
|
22
|
+
- **Both clips fade (`1 - t` and `t`).** Background leaks through partially transparent clips and the formula depends on both clips' opacity. Overlaying the incoming clip at alpha `t` over a normally rendered outgoing clip is exact when the incoming clip is opaque and is the same alpha product the keyed-opacity path already uses.
|
|
23
|
+
- **Linear audio gain.** Equal-power (`sqrt`) keeps perceived loudness constant through the overlap; linear gain dips for uncorrelated material. `sqrt` is exact in both the browser and FFmpeg expressions.
|
|
24
|
+
- **Audio-track transitions in the same change.** Audio tracks have a separate overlap and gain contract. Deferred.
|
|
25
|
+
|
|
26
|
+
## Consequences
|
|
27
|
+
|
|
28
|
+
- `transitions` entries are `{ from, to, kind, easing? }`. `kind` is required and must be `dissolve` (ADR 0020 extends the accepted kinds with `crossfade` for audio tracks); `easing` defaults to `linear` and uses the keyframe CSS timing-function grammar. There is no `duration`: the overlap is the single source of truth. A clip may be the `to` of one transition and the `from` of another, but not twice on the same side.
|
|
29
|
+
- Valid geometry is `from.start < to.start < from.end < to.end`. Anything else is `TIMELINE_TRANSITION_GEOMETRY` (ordering or containment) or `TIMELINE_TRANSITION_NOT_OVERLAPPING` (touching or later). Reference errors are `TIMELINE_TRANSITION_UNKNOWN_CLIP`, `TIMELINE_TRANSITION_WRONG_TRACK`, and `TIMELINE_TRANSITION_AUDIO_TRACK`; duplicate participation is `TIMELINE_TRANSITION_DUPLICATE`; an unsupported kind is `TIMELINE_TRANSITION_KIND`. Structural entry errors use `TIMELINE_TRANSITION_INVALID`.
|
|
30
|
+
- The pairwise overlap rule now allows only declared pairs on the same track. A third clip that overlaps either side is still an error, so at most two clips are active on a video track at any frame.
|
|
31
|
+
- During the window the incoming clip composites above the outgoing clip on the same track. Its alpha is `clipOpacity × t`, where `t = clamp(ease((frame - to.start) / (from.end - to.start)))`. Clip opacity may be static or keyframed and multiplies rather than being overridden. Blend modes compose with transition alpha because it is the same multiplier the keyed-opacity mask already applies.
|
|
32
|
+
- Production adds the transition factor to the existing `geq` alpha mask (`N + frameOffset` evaluates absolute frames) and does not split segments at transition edges: the edges are already clip boundaries. Segments chopped by `maxSegmentFrames` produce identical values.
|
|
33
|
+
- Contributed audio uses gain `sqrt(1 - t)` for the outgoing clip and `sqrt(t)` for the incoming clip, evaluated after easing. Envelopes are absolute frames and multiply through nested timelines, so a transition on a group clip fades its group audio. `audio.muted` still zeroes its side. Audio-track clips are untouched.
|
|
34
|
+
- Preview multiplies the incoming layer's CSS opacity by the same eased progress, stacks it above the outgoing clip via a per-track z-index pair, ramps `<audio>` volume from wall-clock playback, and draws a read-only hatched band on the strip. Production and preview agree at the composite stage; encoded pixels keep the existing scaling, alpha, and color-conversion differences.
|
|
35
|
+
- `check` sweeps every transition frame and exposes `transitions` in the timeline check result. `still` composites a frame inside the window through the same path as `render`.
|
|
36
|
+
- Timeline documents remain version 1; `transitions` is additive. Documentation and the `examples/transitions` reference project cover the contract. The documented 0.8 non-goal is removed.
|
|
@@ -0,0 +1,27 @@
|
|
|
1
|
+
# Crossfade audio clips on audio tracks
|
|
2
|
+
|
|
3
|
+
**Status:** Accepted
|
|
4
|
+
**Date:** 2026-11-04
|
|
5
|
+
|
|
6
|
+
A timeline may declare a `crossfade` transition between two clips on one audio track. It uses the same overlap window and geometry as a dissolve, but applies only the equal-power audio gain ramp: the outgoing clip at `sqrt(1 - t)` and the incoming clip at `sqrt(t)`, after easing. A `dissolve` remains video-track only; `crossfade` is audio-track only. The timeline document version remains 1.
|
|
7
|
+
|
|
8
|
+
## Context
|
|
9
|
+
|
|
10
|
+
ADR 0019 made same-track overlap the transition window but deferred audio-track transitions. Audio clips could already live directly on audio tracks, and the audio contributed by visual clips already crossfaded inside a dissolve, but two audio clips on one audio track could not overlap. A music or voice edit therefore needed two tracks and hand-tuned gain, and there was no gain envelope contract for arbitrary audio clips. The transition window, geometry, and equal-power curve already exist; only the track-kind gate blocks audio.
|
|
11
|
+
|
|
12
|
+
## Considered Options
|
|
13
|
+
|
|
14
|
+
- **Reuse `dissolve` on audio tracks.** Shorter, but "dissolve" names a visual operation. Allowing it on a track with no picture makes the kind/track contract ambiguous and forecloses future visual-only kinds.
|
|
15
|
+
- **A dedicated `crossfade` kind.** Keeps the matrix explicit: `dissolve` is video, `crossfade` is audio. The schema already carries `kind`, so this is additive and validation is one comparison.
|
|
16
|
+
- **Audio fades inside the composition.** Moves the edit into a reusable asset, so the fade cannot follow clip placement and the same composition cannot be used with a different overlap.
|
|
17
|
+
- **Two audio tracks with opposing gain envelopes.** Needs a new per-clip audio-envelope contract and still leaves the relationship implicit. The transition already supplies the window and curve.
|
|
18
|
+
- **Linear gain.** Dips in perceived loudness for uncorrelated material. Equal power is already the contributed-audio behavior, so using a different curve for audio-track clips would be surprising.
|
|
19
|
+
|
|
20
|
+
## Consequences
|
|
21
|
+
|
|
22
|
+
- `TRANSITION_KINDS` is `dissolve` and `crossfade`. Unknown or missing kinds use `TIMELINE_TRANSITION_KIND`; a dissolve on an audio track uses `TIMELINE_TRANSITION_AUDIO_TRACK`, and a crossfade on a video track uses `TIMELINE_TRANSITION_VIDEO_TRACK`. Both suggest the matching kind.
|
|
23
|
+
- The overlap relaxation covers audio tracks with the same pairwise rule: only a declared pair may overlap, a third clip is still an error, and geometry is `from.start < to.start < from.end < to.end`.
|
|
24
|
+
- `crossfade` changes no pixels and never reaches the segment planner or the composite filter graph. `collectTimelineAudio` already attaches absolute-frame gain envelopes to every clip with a transition, including audio-track clips, so the existing mix path applies the ramp. Clip and track gain still multiply the envelope, and `audio.muted` still zeroes a side.
|
|
25
|
+
- Preview drew the transition band from the resolved timeline already; audio tracks now produce it too. Playback ramps `<audio>` volume from the same envelope.
|
|
26
|
+
- `examples/transitions` gains an `audio-crossfade` timeline: the alpha and beta cards are reused with `audio.muted: true`, two audio clips sit on `a1`, and a `crossfade` shares the 15-frame overlap with the visual dissolve. This shows both the direct-audio placement and the kind/track split.
|
|
27
|
+
- Timeline documents remain version 1; `crossfade` is additive. The audio-track prohibition in ADR 0019 is superseded.
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
# Project timeline scenes once for production and preview
|
|
2
|
+
|
|
3
|
+
**Status:** Accepted
|
|
4
|
+
**Date:** 2026-11-04
|
|
5
|
+
|
|
6
|
+
A timeline frame is produced from a **timeline scene**: the ordered active clips and their presentation at one timeline frame or over one render range, including transition roles and composite order. `src/timeline/scene.ts` is browser-safe and owns active-layer selection, transition pairing, stacking, capture-mode classification, and presentation defaults. `planTimelineRender` keeps segment splitting and projects each segment through the scene module; production capture, `check`, `still`, and preview consume the same projection. Capture sessions, FFmpeg, audio mapping, and both compositors stay outside the seam.
|
|
7
|
+
|
|
8
|
+
## Context
|
|
9
|
+
|
|
10
|
+
The planner, the segment renderer, the still renderer, the audio mapper, and the preview host each derived the active clip set, the frame-to-source mapping, transition roles, stacking, and presentation defaults. Recent timeline features made the fan-out visible: dissolve, crossfade, and keyframes each added another copy. The preview bypassed `planTimelineRender` entirely and re-derived the scene pointwise. The copies already disagreed on static-composition caching and on what `TimelineCompositeLayer.kind` meant.
|
|
11
|
+
|
|
12
|
+
ADR 0018 already requires one keyframe evaluator shared by production and preview, and ADR 0019 requires both paths to agree at the composite stage. This decision applies the same rule to layer selection and presentation, without merging the compositors or the capture lifecycle.
|
|
13
|
+
|
|
14
|
+
## Considered Options
|
|
15
|
+
|
|
16
|
+
- **A browser-safe scene projection (chosen).** One range-based module returns ordered scene layers with the resolved clip, normalized presentation, source start and size, capture mode, and transition window/role. A frame is a one-frame range. Production, `check`, `still`, and preview consume it; capture and audio mapping stay separate.
|
|
17
|
+
- **Projection plus capture/session ownership.** Fewer layers between resolve and pixels, but the module would absorb Chromium sessions, static-image caching, nested capture, and FFmpeg artifacts, making preview depend on production machinery.
|
|
18
|
+
- **Projection plus audio resolution.** One scene would drive both pixels and audio, but audio mapping would pull nested gain, mute, and media probing into the render-facing module before those concerns are ready to move.
|
|
19
|
+
- **Keep the per-consumer models and share only the transition map.** Smaller change, but frame mapping, defaults, capture mode, and stacking would still drift independently.
|
|
20
|
+
- **Precompute the preview scene on the server.** The preview needs arbitrary frames, so it would either ship one scene per frame or keep re-deriving the scene in the browser.
|
|
21
|
+
|
|
22
|
+
## Consequences
|
|
23
|
+
|
|
24
|
+
- `src/timeline/scene.ts` defines `TimelineScene`, `TimelineSceneLayer`, the capture modes `frames | still | video | image | nested`, and one transition pairing used by the scene and by timeline audio envelopes. Layers carry the resolved clip and materialized `transform`, `keyframes`, `blend`, `sourceStart`, source size, and transition role; array order is composite order.
|
|
25
|
+
- `planTimelineRender` keeps segment splitting, removes `TimelineRenderLayer`, and projects each segment through the scene. `TimelineRenderPlan` shrinks to the segment list.
|
|
26
|
+
- `TimelineCompositeLayer` keeps its narrow FFmpeg shape but replaces the overloaded `kind` with an `input` mode (`sequence | image | video`). One mapper builds it from a scene layer plus the captured artifact.
|
|
27
|
+
- The preview payload becomes a serialized resolved timeline: clips keep the resolved shape, sources gain a `url`, and absolute paths are stripped. The browser host calls the scene projection on every seek. `TimelinePreviewClip` stops being a parallel model; the precomputed transitions array disappears.
|
|
28
|
+
- Nested timelines remain groups. The scene projects one timeline level; capture recurses into a nested source's own scene.
|
|
29
|
+
- Static-image caching keeps its current behavior during this change. Cache ownership moves with the capture pipeline, not with the scene projection.
|
|
30
|
+
- Projection errors are `FrameyardError`s carrying the clip or range context; the module does not return diagnostics because resolution already owns them.
|
|
31
|
+
- Tests target `test/timeline-scene.test.ts` for active layers, transition roles, stacking, frame-to-source mapping, capture modes, source sizes, and defaults. Plan tests keep segment splitting; render and preview tests keep pixels.
|
|
32
|
+
- No pixel behavior is intended to change. Preview and production still composite in different engines; parity remains at clip capture.
|
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
# Gate heavy operations with a workspace resource policy
|
|
2
|
+
|
|
3
|
+
**Status:** Accepted
|
|
4
|
+
**Date:** 2026-11-05
|
|
5
|
+
|
|
6
|
+
`fy.policy.json` declares a workspace resource tier (`light`, `authoring`, or `full`), an optional `shared-cpu` profile, and stricter still/overwrite limits. The CLI reads the nearest file from the working directory upward, independent of `fy.config.js`; `FRAMEYARD_POLICY` replaces the file for a human override. `tts`, `transcribe`, and `render` are blocked outside `full`; `lint` and `--dry-run` provide a cheap authoring and handoff path. The policy is a guardrail for cooperative agents, not a security boundary.
|
|
7
|
+
|
|
8
|
+
## Context
|
|
9
|
+
|
|
10
|
+
Local models, browsers, and FFmpeg often share one CPU and GPU. An agent that runs every validation pass through Chromium, or starts a render while the model is loaded, makes the machine unusable for the human. The existing skill says to confirm inference and avoid heavy work, but an instruction is not enforceable: a local model can ignore it or invoke the CLI directly.
|
|
11
|
+
|
|
12
|
+
`check` evaluates every frame through Chromium and is the correctness gate, but it is too expensive for a tight edit loop. `still` is bounded but still launches Chromium. `tts` and `transcribe` load local models, and `render` runs Chromium plus FFmpeg with up to sixteen workers. These operations need different treatment.
|
|
13
|
+
|
|
14
|
+
The Frameyard repository itself has no `fy.config.js`, so a policy tied to the project root would not protect commands that target `examples/*` with `--project`. The policy therefore belongs to the workspace, not the render target.
|
|
15
|
+
|
|
16
|
+
## Considered Options
|
|
17
|
+
|
|
18
|
+
- **A workspace policy file with tiers and a shared profile (chosen).** One small JSON document controls operation gating, still limits, overwrite permission, worker caps, and process priority. It applies to direct CLI invocations, is visible in review, and does not require a host-specific integration.
|
|
19
|
+
- **Skill instructions only.** No new contract, but unenforceable against a model that ignores the skill or runs the binary directly.
|
|
20
|
+
- **A global user configuration.** Stronger against accidental edits, but it breaks the local-first project portability the agent workflows need.
|
|
21
|
+
- **A hosted control plane or MCP server.** Out of scope by design; it adds a second public surface for a problem the CLI can solve.
|
|
22
|
+
- **A daemon or job queue in v1.** It still consumes the same resources and adds state before the simple handoff is proven.
|
|
23
|
+
|
|
24
|
+
## Consequences
|
|
25
|
+
|
|
26
|
+
- The policy is discovered from `cwd` upward. `--project` does not change discovery. `FRAMEYARD_POLICY=light|authoring|full` replaces the file, including its profile and tightened limits.
|
|
27
|
+
- `light` allows `lint`, `context`, `doctor`, `init`, `dev`, and one still frame; it blocks `check`, `tts`, `transcribe`, and `render`. `authoring` allows four still frames and `check` at low priority; it blocks the three heavy operations. `full` allows everything up to the existing 32-frame still cap.
|
|
28
|
+
- A policy file can only tighten the tier: `maxStillFrames` cannot exceed the tier default, and `allowOverwrite` cannot become true outside `full`. Unknown fields, unsupported versions, and invalid values fail with `POLICY_INVALID`.
|
|
29
|
+
- `frameyard lint <id>` runs static validation, timeline resolution, scene and keyframe projection for every timeline frame, controller determinism scanning, and media metadata probing without Chromium. It does not run `validateProps` or evaluate composition runtime; `check` remains the full gate.
|
|
30
|
+
- `tts`, `transcribe`, and `render` accept `--dry-run`. They validate inputs and return `{ dryRun: true, plan, commandLine, prerequisites, blockers }` without creating output directories, writing the manifest, or launching Chromium, FFmpeg, or a model. The handoff command includes `FRAMEYARD_POLICY=full` when the active policy is not `full`.
|
|
31
|
+
- `--profile shared-cpu` caps timeline workers at two and composition concurrency at one, lowers process priority, and leaves the encoder at `auto`. Explicit values above the cap fail with `RESOURCE_PROFILE_CONFLICT`; they are never clamped silently.
|
|
32
|
+
- `authoring` and `light` deny `--overwrite` and `--force`. `still` frames above the limit fail with `RESOURCE_POLICY_LIMIT`.
|
|
33
|
+
- `os.setPriority` is best effort. Linux and macOS use the low nice value; Windows uses below-normal. Unsupported platforms continue without failing.
|
|
34
|
+
- The policy file is editable by the agent, so it is a guardrail. Host permissions and the CLI environment are the second layer; a hostile process with shell access can still bypass it.
|
|
35
|
+
- Tests target `test/policy.test.ts` for parsing, precedence, discovery, tier gates, and profile caps; `test/cli-render.test.ts` for blocked commands, dry-run plans, still limits, and process non-spawn; `test/docs.test.ts`, `test/skill.test.ts`, and `test/scaffold.test.ts` for the contract text.
|
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
# Pin fonts as project assets
|
|
2
|
+
|
|
3
|
+
**Status:** Accepted
|
|
4
|
+
**Date:** 2026-11-05
|
|
5
|
+
|
|
6
|
+
`fy.fonts.json` is a versioned project manifest of vendored font files.
|
|
7
|
+
Font files and their licenses live under `assets/fonts/<font-id>/`; the preview
|
|
8
|
+
server generates the `@font-face` sheet at `/__frameyard/fonts.css` and injects
|
|
9
|
+
it into every composition document before author styles. Fontsource is the
|
|
10
|
+
primary acquisition catalog; local files and pinned HTTPS URLs cover fonts that
|
|
11
|
+
are not in Fontsource. Fontshare, Google Fonts, and Bunny Fonts are not
|
|
12
|
+
providers in the first version.
|
|
13
|
+
|
|
14
|
+
## Context
|
|
15
|
+
|
|
16
|
+
Compositions already load fonts through ordinary local `@font-face` rules or a
|
|
17
|
+
local `<link rel="stylesheet">`. The preview server serves WOFF2/WOFF/TTF/OTF
|
|
18
|
+
with the correct media type, and the host awaits `document.fonts.ready` after
|
|
19
|
+
`mount()`. What is missing is a project-level model: there is no inventory, no
|
|
20
|
+
way for an agent to search or install a font, no license record, no hash, and no
|
|
21
|
+
verification that a captured frame used the intended font. Remote Google Fonts
|
|
22
|
+
links work during the loading phase but make a render depend on the network and
|
|
23
|
+
on a mutable third-party response.
|
|
24
|
+
|
|
25
|
+
Fonts are not an FFmpeg concern: text is rasterized in Chromium during
|
|
26
|
+
composition capture, and FFmpeg only composites the resulting images. The font
|
|
27
|
+
files therefore need to be deterministic inputs to the browser session, not
|
|
28
|
+
dependencies of the encoder.
|
|
29
|
+
|
|
30
|
+
## Considered Options
|
|
31
|
+
|
|
32
|
+
- **A project manifest with vendored files (chosen).** The CLI can update the
|
|
33
|
+
manifest atomically, record file hashes and licenses, and the server can
|
|
34
|
+
generate one stylesheet without scanning directories. The project remains
|
|
35
|
+
portable and renderable offline.
|
|
36
|
+
- **Declare fonts in `fy.config.js`.** Fewer files, but the CLI would need to
|
|
37
|
+
edit JavaScript to add a font, and hashes and licenses still need a sidecar.
|
|
38
|
+
- **Depend on `@fontsource/*` packages in `node_modules`.** Familiar for web
|
|
39
|
+
projects, but a Frameyard project is not necessarily an npm package, packages
|
|
40
|
+
include every subset and weight, and resolution depends on a package manager
|
|
41
|
+
and bundler.
|
|
42
|
+
- **Keep CSS-only fonts with no inventory.** Web-native, but gives the agent no
|
|
43
|
+
discovery, validation, license, or reproducibility surface.
|
|
44
|
+
|
|
45
|
+
## Consequences
|
|
46
|
+
|
|
47
|
+
- `fy.fonts.json` has `version: 1` and a `fonts` array. Each family
|
|
48
|
+
records its CSS family, provider, optional provider version, license, variable
|
|
49
|
+
axes, and a non-empty `files` array. Each file records a project-relative path
|
|
50
|
+
below `assets/fonts/<font-id>/`, subset, style, weight or weight range, format,
|
|
51
|
+
optional unicode range, SHA-256, byte size, and optional upstream source.
|
|
52
|
+
Unknown fields, duplicate ids, duplicate CSS family names, duplicate faces,
|
|
53
|
+
paths outside the font directory, and invalid weights fail with
|
|
54
|
+
`FONT_MANIFEST_INVALID`.
|
|
55
|
+
- Font files and license files are committed with the project. Replacing a font
|
|
56
|
+
face or changing its provider requires `--force`; merging another subset,
|
|
57
|
+
style, or weight into an existing family does not.
|
|
58
|
+
- `frameyard font add --file` vendors a local `.woff2`, `.woff`, `.ttf`, or
|
|
59
|
+
`.otf` file with an explicit family, weight, style, optional subset, optional
|
|
60
|
+
unicode range, and optional license. A missing license produces a warning.
|
|
61
|
+
`font list`, `font verify`, and `font remove` manage the manifest;
|
|
62
|
+
`font remove` deletes the font directory unless `--keep-files` is supplied.
|
|
63
|
+
- `font search` reads the Fontsource catalog and filters by query, category,
|
|
64
|
+
subset, style, variable, and limit. `font add <id>` pins `npmVersion` from the
|
|
65
|
+
Fontsource API, downloads only the requested subset/style/weight, downloads
|
|
66
|
+
the package `LICENSE`, and records the exact CDN URL and SHA-256. The default
|
|
67
|
+
is the variable `wght` face when the family has one; static faces require
|
|
68
|
+
`--static` and a weight or weight list.
|
|
69
|
+
- HTTPS URL imports may include a pinned SHA-256. A URL import without a license
|
|
70
|
+
is recorded as `unknown`, requires the `full` policy, and produces warnings in
|
|
71
|
+
`list`, `verify`, and `doctor`. There is no shared font cache. Fontshare is
|
|
72
|
+
not a provider: its Free Font License prohibits redistribution, subsetting,
|
|
73
|
+
and format conversion for most of its catalog. Only OFL-licensed Fontshare
|
|
74
|
+
files are safe to vendor, and they use the generic URL import.
|
|
75
|
+
- The server generates the font sheet on every request from the manifest and
|
|
76
|
+
injects it after the host style and before author styles. Adding a font is
|
|
77
|
+
visible in `dev` on the next request without restarting the server. The
|
|
78
|
+
generated sheet uses `font-display: block` and one `@font-face` per manifest
|
|
79
|
+
file.
|
|
80
|
+
- The host awaits `document.fonts.ready`, then fails the load with
|
|
81
|
+
`FONT_LOAD_FAILED` if the browser reports a failed `FontFace`. Static
|
|
82
|
+
validation warns with `FONT_REMOTE` once per remote URL found in inline CSS,
|
|
83
|
+
local linked CSS, or a remote stylesheet link. Remote fonts remain compatible
|
|
84
|
+
but are not deterministic.
|
|
85
|
+
- `check`, `still`, and `render` verify manifest file sizes and SHA-256 values
|
|
86
|
+
before launching Chromium. `lint` verifies existence and size. `font verify`
|
|
87
|
+
also checks licenses and orphaned font directories, and `doctor` reports
|
|
88
|
+
manifest integrity. A changed file fails with `FONT_HASH_MISMATCH`.
|
|
89
|
+
- The policy operation is `font`. Search, list, and verify are allowed in every
|
|
90
|
+
tier; `font add` requires `authoring` or `full`; `font remove` and
|
|
91
|
+
`font add --force` require `full`.
|
|
92
|
+
- `context --json` reports installed families, provider, version, variable axes,
|
|
93
|
+
subsets, styles, license, file count, bytes, and status. The agent skill gains
|
|
94
|
+
a font reference; the scaffold continues to use `system-ui` and teaches
|
|
95
|
+
`font add` instead of bundling a default font.
|
|
96
|
+
- Typography roles (`--fy-font-display`, `--fy-font-body`), font pairs, a
|
|
97
|
+
preview font picker, and non-`wght` variable axes are out of scope for the
|
|
98
|
+
first version.
|
|
@@ -0,0 +1,86 @@
|
|
|
1
|
+
# Stream composition captures and skip trivial composites
|
|
2
|
+
|
|
3
|
+
**Status:** Accepted. Extends [ADR-0012](./0012-render-timelines-with-segmented-lossless-clip-intermediates.md). Extended by [ADR-0025](./0025-capture-timeline-layers-in-parallel-with-a-global-limit.md): independent layers are captured concurrently under a global session limit. Extended by [ADR-0026](./0026-reuse-loaded-composition-sessions-across-segments.md): loaded sessions are reused across segments.
|
|
4
|
+
**Date:** 2026-11-06
|
|
5
|
+
|
|
6
|
+
The production timeline renderer will stream captured PNG buffers directly into
|
|
7
|
+
the lossless FFV1 encoder instead of writing a PNG sequence to scratch first. A
|
|
8
|
+
segment whose visual content is a single full-frame composition with no
|
|
9
|
+
transform, opacity, blend, transition, or keyframes will bypass the FFV1
|
|
10
|
+
intermediate and the FFmpeg composite entirely and encode the capture directly
|
|
11
|
+
into the delivery segment over the timeline background. All other segments keep
|
|
12
|
+
the segmented lossless path.
|
|
13
|
+
|
|
14
|
+
## Context
|
|
15
|
+
|
|
16
|
+
The 60-minute composition-only gate rendered 108,000 frames in 37.28 minutes
|
|
17
|
+
with four workers. Per-stage worker time was 48.9%
|
|
18
|
+
capture, 23.2% FFV1, 24.7% composite, and 2.8% session setup. The run wrote
|
|
19
|
+
15.52 GB of PNG intermediates, and the FFV1 stage added 8.64 minutes of wall
|
|
20
|
+
time at four workers. Capture itself was the floor: 92.6% of the capture loop
|
|
21
|
+
was `Page.captureScreenshot` plus PNG encoding.
|
|
22
|
+
|
|
23
|
+
The gate is a one-layer composition workload, so the FFV1 intermediate and the
|
|
24
|
+
composite are identity operations apart from painting the timeline background.
|
|
25
|
+
Streaming alone improved the 60-second gate from 43.27 seconds to 38.76
|
|
26
|
+
seconds, but CPU contention between four concurrent captures and four FFV1
|
|
27
|
+
encoders limited the gain. The direct path removes both stages for qualifying
|
|
28
|
+
segments.
|
|
29
|
+
|
|
30
|
+
## Decision
|
|
31
|
+
|
|
32
|
+
- `captureCompositionLayer` opens an FFV1 encoder before capture and writes
|
|
33
|
+
each PNG buffer to its stdin. The encoder is bounded with
|
|
34
|
+
`FRAMEYARD_FFV1_THREADS` (default 2; `0` uses FFmpeg's default) so concurrent
|
|
35
|
+
captures retain CPU headroom.
|
|
36
|
+
- `processTimelineSegment` detects a single active composition layer whose
|
|
37
|
+
source dimensions equal the stage, whose transform is the identity, and which
|
|
38
|
+
has no keyframes, blend, transition, or opacity. It also requires an opaque
|
|
39
|
+
timeline background. Such a segment uses `startDirectSegmentEncoder`, streams
|
|
40
|
+
the capture into the delivery codec over a solid background, and skips
|
|
41
|
+
`compositeTimelineSegment`.
|
|
42
|
+
- Static compositions, media layers, nested timelines, transformed or animated
|
|
43
|
+
clips, blends, transitions, and transparent nested backgrounds keep the
|
|
44
|
+
segmented lossless path. Nested timelines continue to use FFV1.
|
|
45
|
+
|
|
46
|
+
## Considered Options
|
|
47
|
+
|
|
48
|
+
- **Keep the file-based PNG sequence.** Rejected: it wrote 15.52 GB to scratch,
|
|
49
|
+
serialized capture and FFV1, and spent 8.64 minutes of wall time on FFV1 at
|
|
50
|
+
four workers.
|
|
51
|
+
- **Stream FFV1 only.** Accepted as the general path, but insufficient alone:
|
|
52
|
+
the 60-second gate improved by only 10.4% because capture and FFV1 contend for
|
|
53
|
+
CPU. It remains necessary for multi-layer and transformed segments.
|
|
54
|
+
- **Always encode directly and skip the composite.** Rejected: multi-layer,
|
|
55
|
+
blended, keyframed, transformed, and nested segments need the FFmpeg
|
|
56
|
+
compositor.
|
|
57
|
+
- **Use a GPU/Skia compositor.** Rejected for this change: the gate is capture
|
|
58
|
+
bound, and GPU rasterization would not remove screenshot/PNG encoding. It
|
|
59
|
+
would also complicate the capture-pixel reproducibility guarantee.
|
|
60
|
+
- **Parallelize layer capture first.** Deferred for this change: it helps
|
|
61
|
+
multi-layer timelines but not the one-layer long-form gate. The measured
|
|
62
|
+
60-second two-layer run improved only 5.1% from streaming (62.79 to 59.60
|
|
63
|
+
seconds). ADR 0025 then captured independent layers concurrently and reduced
|
|
64
|
+
that run to 43.78 seconds.
|
|
65
|
+
|
|
66
|
+
## Consequences
|
|
67
|
+
|
|
68
|
+
- A qualifying direct segment emits `timeline.segment.direct` instead of
|
|
69
|
+
`timeline.segment.composite` in the opt-in stage timings.
|
|
70
|
+
- Direct segments are encoded once, without a lossless FFV1 round trip. The
|
|
71
|
+
reproducibility guarantee remains at clip capture. For the 60-minute gate,
|
|
72
|
+
the direct output is byte-identical to the FFV1-and-composite output
|
|
73
|
+
(SHA-256 `02dab384...b6aab`).
|
|
74
|
+
- Scratch no longer contains per-frame PNG sequences. The 60-minute gate wrote
|
|
75
|
+
no PNG files to disk; the 15.52 GB of PNG buffers went through memory and
|
|
76
|
+
pipes. FFV1 files remain for non-direct segments.
|
|
77
|
+
- The 60-minute gate at eight workers completed in 19.83 minutes, 34% inside the
|
|
78
|
+
30-minute target. ADR 0026 later reduced it to 18.78 minutes by reusing loaded
|
|
79
|
+
sessions. At four workers the ten-minute gate projects about 24 minutes.
|
|
80
|
+
- Multi-layer segments keep the lossless path and still pay for FFV1 and the
|
|
81
|
+
composite. ADR 0025 captures their independent layers concurrently; the
|
|
82
|
+
two-layer 60-second run improved from 62.79 to 59.60 seconds with streaming
|
|
83
|
+
and to 43.78 seconds with parallel layer capture.
|
|
84
|
+
- The direct path is covered by the timeline rendering suite, including a
|
|
85
|
+
transparent full-frame composition over a non-black timeline background, and
|
|
86
|
+
by the full 108,000-frame gate with an FFprobe frame-count check.
|
|
@@ -0,0 +1,67 @@
|
|
|
1
|
+
# Capture timeline layers in parallel with a global limit
|
|
2
|
+
|
|
3
|
+
**Status:** Accepted. Extends [ADR-0024](./0024-stream-composition-captures-and-skip-trivial-composites.md). Extended by [ADR-0026](./0026-reuse-loaded-composition-sessions-across-segments.md): loaded sessions are reused across segments.
|
|
4
|
+
**Date:** 2026-11-06
|
|
5
|
+
|
|
6
|
+
Segments with more than one visual layer now capture their composition and
|
|
7
|
+
static-image layers concurrently. A FIFO semaphore bounds active capture
|
|
8
|
+
sessions across the whole render, independent of the segment worker count. The
|
|
9
|
+
layer results are reordered before compositing so track order and output pixels
|
|
10
|
+
are unchanged.
|
|
11
|
+
|
|
12
|
+
## Context
|
|
13
|
+
|
|
14
|
+
The segmented renderer processed `segment.layers` sequentially. A two-layer
|
|
15
|
+
60-second gate with streaming FFV1 took 59.60 s: the segment time was the sum of
|
|
16
|
+
both captures plus the composite. The same one-layer workload took 43.27 s
|
|
17
|
+
before the direct fast path, and 27.64 s after it. The layer captures are
|
|
18
|
+
independent, so the serial loop was leaving the second composition idle while
|
|
19
|
+
the first encoded.
|
|
20
|
+
|
|
21
|
+
An unbounded `Promise.all` is not safe: eight workers with three layers could
|
|
22
|
+
open twenty-four Chromium sessions and run twenty-four concurrent FFV1
|
|
23
|
+
encoders. The segment worker count already bounds the final segment encoders,
|
|
24
|
+
but it does not bound capture sessions.
|
|
25
|
+
|
|
26
|
+
## Decision
|
|
27
|
+
|
|
28
|
+
- `createSemaphore(limit)` provides a FIFO capture limit.
|
|
29
|
+
- The default limit is `min(workers * 2, 12)`. `FRAMEYARD_CAPTURE_CONCURRENCY`
|
|
30
|
+
overrides it for local A/B measurements.
|
|
31
|
+
- Every composition and static-composition capture takes a slot for the whole
|
|
32
|
+
session and encoder lifetime. Nested timelines do not hold an outer slot;
|
|
33
|
+
their inner captures take slots from the same semaphore.
|
|
34
|
+
- A segment starts all layer tasks together and uses `Promise.allSettled`.
|
|
35
|
+
It reorders successful results by layer index before compositing and throws
|
|
36
|
+
the first failure after every task has settled, so no capture is left writing
|
|
37
|
+
into scratch after the segment fails.
|
|
38
|
+
- The direct one-layer segment path is unchanged; it takes one slot.
|
|
39
|
+
|
|
40
|
+
## Considered Options
|
|
41
|
+
|
|
42
|
+
- **Keep the sequential layer loop.** Rejected: it serializes independent
|
|
43
|
+
captures and leaves CPU idle during FFV1.
|
|
44
|
+
- **Use unbounded `Promise.all`.** Rejected: capture sessions multiply with
|
|
45
|
+
workers and layers, causing memory and encoder oversubscription.
|
|
46
|
+
- **Limit only per segment.** Rejected: a per-segment limit still multiplies by
|
|
47
|
+
the worker count, so eight workers with two layers would open sixteen
|
|
48
|
+
sessions.
|
|
49
|
+
- **Reuse sessions instead.** Deferred for this change: a pool removes session
|
|
50
|
+
startup but does not shorten the critical path of a multi-layer segment. ADR
|
|
51
|
+
0026 adds the pool as a separate optimization.
|
|
52
|
+
|
|
53
|
+
## Consequences
|
|
54
|
+
|
|
55
|
+
- The two-layer 60-second gate fell from 59.60 s to 43.78 s (−26.5%) with four
|
|
56
|
+
workers; the output is byte-identical to the sequential run.
|
|
57
|
+
- The one-layer 60-minute gate is unaffected because every segment uses the
|
|
58
|
+
direct path.
|
|
59
|
+
- Peak capture sessions are bounded at 12, including eight-worker renders with
|
|
60
|
+
multi-layer segments.
|
|
61
|
+
- Captures that share a static-composition cache key still wait for one slot
|
|
62
|
+
each; the capture itself is deduplicated by the existing promise cache.
|
|
63
|
+
- A failed layer task settles its siblings before the segment throws; the
|
|
64
|
+
scratch directory is then cleaned by the render's existing failure path.
|
|
65
|
+
- The full timeline rendering suite passes with parallel captures, including
|
|
66
|
+
transparent compositions, blends, keyframes, transitions, nested timelines,
|
|
67
|
+
and static layers.
|