@sequio/skill 0.1.0 → 0.1.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/llms.txt CHANGED
@@ -5,8 +5,10 @@
5
5
  > clock; the SDK owns the low-level runtime — decode, composite, audio, export.
6
6
  > Compositions are ordinary imperative TypeScript (you `new` the engine's own
7
7
  > classes), so the same code previews in a browser, exports in a browser, and
8
- > renders on a server. Persistence, schema, undo, collaboration and UI are out of
9
- > scope they belong to the layer above.
8
+ > renders on a server. sequio does not render HTML/CSS there is no DOM/JSX or
9
+ > declarative-template layer; you build the object graph in code. Persistence,
10
+ > schema, undo, collaboration and UI are out of scope — they belong to the layer
11
+ > above.
10
12
 
11
13
  Key facts an assistant should hold before writing sequio code:
12
14
 
@@ -23,14 +25,16 @@ Key facts an assistant should hold before writing sequio code:
23
25
  - `sequio render <file>` encodes to video (pure-Node WebGPU; needs a GPU or Mesa
24
26
  lavapipe). `sequio frame <file> --time <sec>` exports a single frame as a PNG
25
27
  through the same render core — the fast way to visually check a composition
26
- before a full render. `sequio preview <file> --watch` serves a live preview.
28
+ before a full render. `sequio audio <file>` exports only the audio mix to an
29
+ audio-only file (mp3 default; also m4a/wav/ogg/webm). `sequio preview <file>
30
+ --watch` serves a live preview.
27
31
 
28
32
  ## Docs
29
33
 
30
34
  - [Architecture & the five contracts](https://github.com/slightc/sequio/blob/main/docs/architecture.md): modules, invariants, and the public-surface table.
31
35
  - [Runtime](https://github.com/slightc/sequio/blob/main/docs/runtime.md): compile + link + run multi-file TS/JS into a `Composer`; `defineComposition`, `loadAsset`, externals.
32
36
  - [CLI](https://github.com/slightc/sequio/blob/main/docs/cli.md): `sequio render` / `sequio preview --watch`, `--scale`, fonts, local/network media.
33
- - [Server-side rendering](https://github.com/slightc/sequio/blob/main/docs/server-side-rendering.md): the `TimelineSpec` protocol and both render routes (headless Chrome / pure-Node WebGPU).
37
+ - [Server-side rendering](https://github.com/slightc/sequio/blob/main/docs/server-side-rendering.md): re-run a composition's own code (a `RuntimeBundle`, via `renderBundleToFile`) on a server through either render route (headless Chrome / pure-Node WebGPU) — no spec to serialize or keep in sync.
34
38
  - [Text animation](https://github.com/slightc/sequio/blob/main/docs/text-animation.md): clip/text animators, split motion, the GSAP binding.
35
39
  - [Contributor / agent guide (AGENT.md)](https://github.com/slightc/sequio/blob/main/AGENT.md): conventions, monorepo layout, working agreement.
36
40
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@sequio/skill",
3
- "version": "0.1.0",
3
+ "version": "0.1.2",
4
4
  "description": "An installable AI Agent Skill (SKILL.md) + llms.txt that teach an AI assistant how to use sequio — build a Track/Clip/Effect object graph, preview, and export/render video.",
5
5
  "type": "module",
6
6
  "license": "MIT",
@@ -6,7 +6,9 @@ description: >-
6
6
  video/motion-graphics programmatically — creating a timeline of tracks and
7
7
  clips (text, image, video, shapes), animating them (keyframes or GSAP),
8
8
  applying effects/transitions, mixing audio, and exporting to MP4/WebM or
9
- rendering server-side. Triggers: "sequio", "video editor SDK", "compose a
9
+ rendering server-side. sequio does NOT render HTML/CSS — compositions are
10
+ imperative TypeScript that `new`s engine classes, not a declarative DOM/JSX
11
+ framework. Triggers: "sequio", "video editor SDK", "compose a
10
12
  video", "timeline / track / clip", "render an mp4 from code", "TextClip",
11
13
  "Compositor", "defineComposition".
12
14
  ---
@@ -28,7 +30,7 @@ Pick the package that matches the task before writing code:
28
30
  |---|---|---|
29
31
  | Drive an object graph in an app you control | **engine** directly | `@sequio/engine` |
30
32
  | Author a self-contained composition file that previews **and** renders | **runtime** authoring API | `@sequio/engine` + `@sequio/runtime` |
31
- | Turn a composition file into an `.mp4`, a single-frame PNG, or a live preview from a terminal | **cli** | `sequio render` / `sequio frame` / `sequio preview` |
33
+ | Turn a composition file into an `.mp4`, a single-frame PNG, or a live preview from a terminal | **cli** | `sequio check` / `sequio render` / `sequio frame` / `sequio preview` |
32
34
  | Render a composition to a file on a server (no browser) | **server** Route B | `@sequio/server/route-b` |
33
35
 
34
36
  For most "make me a video from code" tasks, author a **composition file** (runtime
@@ -98,33 +100,53 @@ export default defineComposition(async () => {
98
100
  Then, from the workspace root:
99
101
 
100
102
  ```bash
103
+ sequio check composition.ts # static validation — no GPU, offline (the pre-flight lint)
101
104
  sequio preview composition.ts --watch # live in-browser preview, reloads on edit
102
105
  sequio frame composition.ts --time 2 --out shot.png # export ONE frame at t=2s as a PNG (fast visual check)
106
+ sequio audio composition.ts --out track.mp3 # export ONLY the audio mix (mp3 default; also m4a/wav/ogg/webm)
103
107
  sequio render composition.ts --out out.mp4 # encode the whole thing to video (pure-Node WebGPU)
104
108
  sequio render composition.ts --out out.mp4 --scale 2 # 2× resolution (sharp text/edges)
105
109
  ```
106
110
 
107
- `sequio render` and `sequio frame` need a WebGPU host — a real GPU or the Mesa
111
+ `sequio render`, `sequio frame` and `sequio audio` need a WebGPU host — a real GPU or the Mesa
108
112
  **lavapipe** software driver (`apt install mesa-vulkan-drivers`; then
109
113
  `export VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/lvp_icd.json`). Without one they
110
114
  throw a clear error.
111
115
 
112
- ### Check your work fast: export a single frame
116
+ ### Verify loop: `check` `frame` `render`
113
117
 
114
- When iterating on a composition **without** a live browser (e.g. you're an agent
115
- editing code), don't render the whole video to see if it's right — export one
116
- frame and look at it:
118
+ Tighten the loop from coarsest-and-cheapest to full render. As an agent editing
119
+ code, run them in this order:
117
120
 
118
121
  ```bash
119
- sequio frame composition.ts --time 2.5 --out /tmp/check.png # then open/view the PNG
122
+ sequio check composition.ts # 1. offline lint no GPU, no browser, fast
123
+ sequio frame composition.ts --time 2.5 --out /tmp/check.png # 2. render ONE frame, then open/view the PNG
124
+ sequio render composition.ts --out out.mp4 # 3. encode the whole video
120
125
  ```
121
126
 
122
- `frame` runs the **same render core** as `render` (contract #3), so the PNG is
123
- exactly what the video would contain at that instant the fast way to confirm
124
- layout, position, and color before committing to a full render. `--time` is
125
- clamped to `[0, duration]`; `--scale N` renders at N× like `render`. This is the
126
- recommended verify loop: edit → `sequio frame` at a few representative times
127
- eyeball then `sequio render` once it looks right.
127
+ **1. `check` does the code build a legal object graph?** It compiles + links +
128
+ runs the builder with a **null renderer** (no WebGPU) and walks the graph for the
129
+ mistakes that are cheap to catch offline: illegal clip times (`end ≤ start`), a
130
+ keyframe outside its clip's interval (dead keyframe), a `TextClip.fontFamily` no
131
+ `fonts.load(...)` registered (silent system-font fallback breaks
132
+ preview↔render), a transition whose two clips don't overlap, an `anchor` outside
133
+ `0..1`, a missing local `loadAsset` file. Exits non-zero on any error;
134
+ `--json` emits a machine-readable `Diagnostic[]`. A green `check` doesn't prove
135
+ the picture is right — but a **red `check` means don't bother rendering yet**.
136
+ It's GPU-free and offline, so run it after every edit.
137
+
138
+ **2. `frame` — does the frame *look* right?** `frame` runs the **same render
139
+ core** as `render` (contract #3), so the PNG is exactly what the video would
140
+ contain at that instant — the way to confirm layout, position, and color.
141
+ `--time` is clamped to `[0, duration]`; `--scale N` renders at N× like `render`.
142
+
143
+ **3. `render`** once `check` is green and a couple of `frame`s look right.
144
+
145
+ > `check` deliberately does **not** decode media, hit the network, or compile
146
+ > shaders — those need a browser/GPU and are what `frame`/`render` are for. A
147
+ > composition that fetches a remote image or decodes video *in its builder* gets
148
+ > a `B4` **warning** from `check` (the code compiled fine; verify the picture
149
+ > with `frame`), not a failure.
128
150
 
129
151
  ## Core building blocks
130
152
 
@@ -153,8 +175,13 @@ eyeball → then `sequio render` once it looks right.
153
175
  clip.animator = gsapClipAnimator(gsap, (tl, o) => tl.from(o, { y: -60, alpha: 0, ease: 'back.out(1.7)' }));
154
176
  ```
155
177
  The binding seeks a **paused** timeline, keeping `render(t)` pure.
156
- - **Text motion** — `TextClip.split` + `StaggerTextAnimator` / `gsapTextAnimator`
157
- animate per line/word/char (e.g. a staggered drop-in).
178
+ - **Text motion** — `TextClip.split` + `TextClip.textAnimator` (`StaggerTextAnimator`
179
+ / `gsapTextAnimator`) animate per line/word/char (e.g. a staggered drop-in).
180
+ - **Bring your own** — the four seams are subclass/implement points: subclass
181
+ `Effect` / `Transition` (or an engine effect) and implement `ClipAnimator`
182
+ (`clip.animator`) / `TextAnimator` (`textClip.textAnimator`). Build on the
183
+ engine's own classes and no `pixi.js` is needed — same in preview and render.
184
+ See recipe 7b and `packages/cli/example/custom-fx/`.
158
185
 
159
186
  ## Media, fonts, effects, audio, export — quick pointers
160
187
 
@@ -165,11 +192,16 @@ eyeball → then `sequio render` once it looks right.
165
192
  `TextClip`. Load an explicit web font so preview and Node render match — system
166
193
  defaults differ per platform.
167
194
  - **Effects/Transitions**: `ColorEffect`, `BlurEffect`, `BulgeEffect`,
168
- `CrossfadeTransition`, etc. (some — chroma/LUT/wipe — are still TODO).
195
+ `CrossfadeTransition`, etc. (some — chroma/LUT/wipe — are still TODO). Attach an
196
+ effect with `clip.effects.push(fx)`; bind a transition with
197
+ `track.addTransition(new T(frames).between(a, b))` over the clips' overlap.
198
+ Roll your own by subclassing — see recipe 7b.
169
199
  - **Audio**: `AudioTrack` + `AudioSource`; `AudioEngine` mixes (Web Audio live,
170
200
  OfflineAudioContext for export).
171
- - **Export in an app** (not the CLI): use `Exporter` from `@sequio/engine`, or a
172
- runtime `Composer`'s `composer.export(...)`.
201
+ - **Export in an app** (not the CLI): use `Exporter` from `@sequio/engine`
202
+ (`export(...)` → video, `exportFrame(t, ...)` → one still image, `exportAudio(...)`
203
+ → audio-only file), or a runtime `Composer`'s `composer.export(...)`. Audio-only
204
+ on the CLI is `sequio audio composition.ts` (recipe 11).
173
205
 
174
206
  ## Before you finish
175
207
 
@@ -19,13 +19,16 @@ Times are **seconds** at the API boundary, quantized to frames internally via
19
19
  - `Transform2D` — a clip's spatial state: `position`, `scale`, `rotation`,
20
20
  `anchor`, `alpha` (each an `AnimatableProperty`).
21
21
  - `AnimatableProperty` / `Keyframe` — `.setStatic(value)` or
22
- `.setKeyframes([{ time, value, easing? }, ...])`.
22
+ `.setKeyframes([{ time, value, easing? }, ...])`; `.valueAt(t)`, `.animated`,
23
+ `.keyframeTimes` (sorted keyframe times, for static validation).
23
24
  - Easings: `linear`, `hold`, `cubicBezier`, `easeInQuad`, `easeOutQuad`,
24
25
  `easeInOutQuad`, `easeInCubic`, `easeOutCubic`, `easeInOutCubic`. Type `Easing`.
25
26
  - Clip/text animators: `ClipAnimator`, `TextAnimator`, `StaggerTextAnimator`,
26
27
  `TweenAnimator`, `IDENTITY_SAMPLE`, `lerpSample`, `AnimationSample`, plus
27
28
  `TextPart` / `TextSplit` / `StaggerOrder` / `StaggerTextOptions` /
28
- `TweenAnimatorOptions`.
29
+ `TweenAnimatorOptions`. `ClipAnimator` (`sampleAt`) and `TextAnimator`
30
+ (`sampleForPart`) are **extension seams** — implement your own for custom motion
31
+ (assign to `clip.animator` / `textClip.textAnimator`); see recipe 7b.
29
32
  - GSAP binding (engine has no gsap dep): `gsapClipAnimator`, `gsapTextAnimator`,
30
33
  `identityTarget`; types `GsapLike`, `GsapTimelineLike`, `GsapTarget`.
31
34
  - Text layout: `computeTextParts`, `MeasureWidth`.
@@ -51,6 +54,8 @@ Times are **seconds** at the API boundary, quantized to frames internally via
51
54
  - `FontManager`, `fonts` (singleton), `buildGoogleCss2Url`; types `FontSpec`,
52
55
  `GoogleFontSpec`.
53
56
  - `fonts.load({ family, src })` before rendering a `TextClip` in that family.
57
+ - `fonts.families()` — the family names a load has been requested for (used by
58
+ `sequio check` to catch a `TextClip` referencing an unregistered font).
54
59
 
55
60
  ## Compositor graph
56
61
 
@@ -58,10 +63,28 @@ Times are **seconds** at the API boundary, quantized to frames internally via
58
63
  fps?, background?, timebase?, createRenderer?, resolution? })`, then
59
64
  `await init()`, `addTrack(track)`, `renderPreview(t)`.
60
65
  - Tracks: `Track`, `VisualTrack` (stacks by `.zIndex`), `AudioTrack`.
61
- - Clips: `Clip`, `VisualClip`, `AudioClip`; concrete `VideoClip`, `ImageClip`,
62
- `TextClip`, `ShapeClip`, `GroupClip`. Types: `TextStyleLike`, `ShapeSpec`,
63
- `ShapeKind`.
66
+ `TextClip`, `ShapeClip`, `GroupClip`. Types: `TextStyleLike`, `TextStrokeLike`,
67
+ `ShapeSpec`, `ShapeKind`, `MaskSpec`.
64
68
  - Every clip has `.start` / `.end` (seconds) and `.transform` (`Transform2D`).
69
+ - Time-remap on any clip: `.speed` (default `1`; time-line seconds → `speed`×
70
+ source seconds — variable-speed + pitch shift for audio) and `.reversed`
71
+ (default `false`; 倒放 — plays the same source window backwards. Video feeds a
72
+ decreasing source time to the decoder (decoded in GOP batches so reverse isn't
73
+ O(GOP²)); audio plays a reversed copy of the buffer, since Web Audio has no
74
+ negative playback rate). `clip.sourceTimeAt(t)` is the resulting source time —
75
+ the single mapping both decode-prep and render use, so `speed`/`reversed` set
76
+ on the clip apply everywhere.
77
+ - `TextClip` style (`TextStyleLike`): `text`, `fontFamily`, `fontSize`
78
+ (animatable), `fill`, plus pass-throughs `fontWeight`, `fontStyle`
79
+ (`'italic'`), `letterSpacing`, `align`, `lineHeight` and `stroke`
80
+ (`TextStrokeLike` `{ color, width }` → hollow / outlined type). Weight and
81
+ letter-spacing feed measurement too, so split-text layout stays correct.
82
+ - `VisualClip.maskShape` (`MaskSpec`): clip the content to a rounded-rect
83
+ (`kind:'rect'`, big `radius` → arch/stadium) or `ellipse` (circle crop), sized
84
+ explicitly (`width`/`height`, optional `x`/`y`, `inset`). Apply it to a
85
+ container-backed clip — a `GroupClip` wrapping the image — since a Sprite
86
+ cannot be masked by its own child; lay the content out from `(0,0)` in the
87
+ group.
65
88
 
66
89
  ## Effects & transitions
67
90
 
@@ -72,6 +95,10 @@ Times are **seconds** at the API boundary, quantized to frames internally via
72
95
  - Warp math: `Mat3`, `Quad`, `Vec2`, `UNIT_QUAD`, `squareToQuad`, `invert3x3`,
73
96
  `applyHomography`, `perspectiveSampleMatrix`, `bulgeSourceUv`.
74
97
  - Transitions: `Transition`, `CrossfadeTransition`, `crossfadeAlpha`.
98
+ - Extend: subclass `Effect` (or an engine effect) for a custom filter/param, and
99
+ `Transition` (or `CrossfadeTransition`) for a custom mix — no `pixi.js` needed if
100
+ you build on the engine's own classes. Attach: `clip.effects.push(fx)`,
101
+ `track.addTransition(new T(frames).between(a, b))`. See recipe 7b.
75
102
  - Status: color/blur/warp + crossfade done; chroma/LUT/wipe TODO.
76
103
 
77
104
  ## Audio
@@ -82,9 +109,14 @@ Times are **seconds** at the API boundary, quantized to frames internally via
82
109
 
83
110
  ## Export
84
111
 
85
- - `Exporter` (`ExportOptions`, `ExportFrameOptions`), `ExportCancelledError`.
112
+ - `Exporter` (`ExportOptions`, `ExportFrameOptions`, `AudioExportOptions`),
113
+ `ExportCancelledError`. `Exporter.export` → video, `exportFrame` → one still
114
+ image, `exportAudio` → audio-only file (the `AudioEngine` offline mix).
86
115
  - `ExportSink`, `ResolvedExportOptions`, `exportFrameTimes`,
87
116
  `MediabunnyExportSink`.
117
+ - Audio-only: `AudioExportSink`, `AudioExportFormat`
118
+ (`'m4a' | 'mp3' | 'wav' | 'ogg' | 'webm'`), `ResolvedAudioExportOptions`,
119
+ `MediabunnyAudioExportSink`.
88
120
 
89
121
  ## Re-exported PixiJS types (type-only)
90
122
 
@@ -76,13 +76,16 @@ it directly — the engine declares only the structural `GsapLike` types.
76
76
  import { StaggerTextAnimator } from '@sequio/engine';
77
77
 
78
78
  title.split = 'char'; // split into per-character parts
79
- title.animator = new StaggerTextAnimator({
80
- each: 0.04, // stagger between parts
81
- from: (o) => ({ y: -40, alpha: 0 }), // starting sample per part
79
+ title.textAnimator = new StaggerTextAnimator({
80
+ from: { y: -40, alpha: 0 }, // each part starts 40px up + invisible
81
+ stagger: 0.04, // delay between consecutive parts (s)
82
+ duration: 0.5, // per-part animation length (s)
82
83
  });
83
84
  ```
84
85
 
85
- For gsap-driven per-part motion use `gsapTextAnimator(gsap, split, builder)`.
86
+ Split text uses `textAnimator` (a `TextAnimator`); a whole-clip animator uses
87
+ `animator` (a `ClipAnimator`) — don't mix the two fields. For gsap-driven per-part
88
+ motion use `gsapTextAnimator(gsap, split, builder)`.
86
89
 
87
90
  ## 5. Video and image clips
88
91
 
@@ -127,24 +130,97 @@ const title = new TextClip({ text: 'sequio', fontFamily: 'Poppins', fontSize: 72
127
130
  ```ts
128
131
  import { BlurEffect, CrossfadeTransition } from '@sequio/engine';
129
132
 
130
- clip.effects.add(new BlurEffect({ strength: 8 }));
131
- // A crossfade between two overlapping clips on a track (overlap drives it):
132
- track.transition = new CrossfadeTransition({ duration: 0.5 });
133
+ const blur = new BlurEffect();
134
+ blur.strength.setStatic(8); // strength is an AnimatableProperty (keyframe it too)
135
+ clip.effects.push(blur); // `effects` is an array
136
+
137
+ // A crossfade between two OVERLAPPING clips on one track — their overlap is the
138
+ // transition window (bind order is direction: from → to):
139
+ track.addTransition(new CrossfadeTransition(15).between(clipA, clipB)); // 15 frames
133
140
  ```
134
141
 
135
- (Effect/transition constructor options vary — check `references/api.md` and the
136
- class in `packages/engine/src/effects/`. chroma/LUT/wipe are not yet built.)
142
+ (chroma/LUT/wipe are not yet built — check `references/api.md` and the classes in
143
+ `packages/engine/src/effects/`. To author your own, see the next recipe.)
137
144
 
138
- ## 8. Export from your own app (not the CLI)
145
+ ## 7b. Bring your own custom effect · transition · animation
146
+
147
+ All four extension points are plain subclasses/implementations imported from
148
+ `@sequio/engine` only. Build on the engine's own classes (or keep the animators
149
+ pure math) and you need **no `pixi.js`**, so they render the same in preview
150
+ (WebGL) and Node render (WebGPU). Full worked demo: `packages/cli/example/custom-fx/`.
151
+
152
+ ```ts
153
+ // ── a custom EFFECT — subclass an engine effect, reuse its built-in filter ──
154
+ import { BlurEffect, AnimatableProperty } from '@sequio/engine';
155
+ class FocusPull extends BlurEffect {
156
+ readonly focus = new AnimatableProperty<number>(0); // 0 sharp → 1 blurred
157
+ constructor(readonly maxBlur = 28) { super(); }
158
+ override updateAt(t: number) {
159
+ this.strength.setStatic(this.focus.valueAt(t) * this.maxBlur);
160
+ super.updateAt(t); // let BlurEffect push it into the filter
161
+ }
162
+ }
163
+ const fx = new FocusPull(30);
164
+ fx.focus.setKeyframes([{ time: 0, value: 1 }, { time: 0.8, value: 0 }]);
165
+ clip.effects.push(fx);
166
+
167
+ // ── a custom TRANSITION — subclass CrossfadeTransition, reshape the curve ──
168
+ import { CrossfadeTransition, easeInOutCubic } from '@sequio/engine';
169
+ class EasedCrossfade extends CrossfadeTransition {
170
+ override progressAt(t: number) { return easeInOutCubic(super.progressAt(t)); }
171
+ }
172
+ track.addTransition(new EasedCrossfade(30).between(clipA, clipB));
173
+
174
+ // ── a custom whole-clip ANIMATION — implement ClipAnimator (pure sampleAt) ──
175
+ import type { AnimationSample, ClipAnimator } from '@sequio/engine';
176
+ class OrbitAnimator implements ClipAnimator {
177
+ constructor(private r = 90, private period = 2.6) {}
178
+ sampleAt(localT: number): AnimationSample { // sampled at t - clip.start
179
+ const a = (localT / this.period) * Math.PI * 2;
180
+ return { x: Math.cos(a) * this.r, y: Math.sin(a) * this.r, rotation: a };
181
+ }
182
+ }
183
+ clip.animator = new OrbitAnimator();
184
+
185
+ // ── a custom TEXT ANIMATION — implement TextAnimator (per split part) ──
186
+ import type { AnimationSample, TextAnimator, TextPart } from '@sequio/engine';
187
+ import { easeOutCubic } from '@sequio/engine';
188
+ class DropInText implements TextAnimator {
189
+ constructor(private stagger = 0.06, private dur = 0.5, private drop = 70) {}
190
+ sampleForPart(part: TextPart, localT: number): AnimationSample {
191
+ const k = (localT - part.index * this.stagger) / this.dur;
192
+ const p = k <= 0 ? 0 : k >= 1 ? 1 : easeOutCubic(k);
193
+ return { y: -this.drop * (1 - p), alpha: p };
194
+ }
195
+ }
196
+ title.split = 'char';
197
+ title.textAnimator = new DropInText();
198
+ ```
199
+
200
+ `sampleAt` / `sampleForPart` / `updateAt(t)` / `render(t)` must be **pure functions
201
+ of time** (contract #2) — no wall-clock, no dependence on the previous frame — so
202
+ export stays reproducible. An `AnimationSample` composes over the clip's base
203
+ transform: `x`/`y`/`rotation` add, `scaleX`/`scaleY`/`alpha` multiply.
139
204
 
140
- Two ways:
205
+ ## 8. Export from your own app (not the CLI)
141
206
 
142
207
  - Runtime `Composer`: `const blob = await composer.export({ format: 'mp4' });`
143
- - Engine `Exporter` directly against a `Compositor`:
208
+ - Engine `Exporter` directly against a `Compositor` + `AudioEngine` — one object,
209
+ three exits (all reuse the same render/audio core, contract #3):
144
210
  ```ts
145
211
  import { Exporter } from '@sequio/engine';
146
- const exporter = new Exporter(compositor, { duration, fps: 30, format: 'mp4' });
147
- const blob = await exporter.run(); // FixedStep loop, awaits prepare — no dropped frames
212
+ const exporter = new Exporter(compositor, audioEngine);
213
+
214
+ // Video: FixedStep loop, awaits prepare — no dropped frames.
215
+ const movie = await exporter.export({ fps: 30, container: 'mp4' });
216
+
217
+ // One still image at a time (no fps boundary needed).
218
+ const still = await exporter.exportFrame(2.5, { type: 'image/png' });
219
+
220
+ // Audio only — just the AudioEngine offline mix, no frames rendered, no GPU.
221
+ const track = await exporter.exportAudio({ format: 'mp3' });
222
+ // format: 'mp3' (default) | 'm4a' | 'wav' | 'ogg' | 'webm'
223
+ // codec defaults per format (mp3 / aac / pcm-s16 / opus); bitrate, range, sampleRate optional
148
224
  ```
149
225
 
150
226
  ## 9. Server-side render (no browser)
@@ -181,6 +257,176 @@ few representative times → look at the PNGs → only `sequio render` once it's
181
257
  Programmatically it's `renderBundleFrameToFile(bundle, { out, time, scale })` from
182
258
  `@sequio/server/route-b`, or `runFrame(file, { out, time, scale })` from `@sequio/cli`.
183
259
 
260
+ ## 11. Export just the audio (no video)
261
+
262
+ Encode only the composition's audio track — the same `AudioEngine` offline mix the
263
+ video export muxes — to an audio-only file:
264
+
265
+ ```bash
266
+ sequio audio composition.ts --out track.mp3 # default format: mp3
267
+ sequio audio composition.ts --format m4a --out track.m4a # or m4a / wav / ogg / webm
268
+ sequio audio composition.ts --out track.wav --bitrate 192000
269
+ ```
270
+
271
+ - `--format` is `mp3` (default) `| m4a | wav | ogg | webm`; when omitted it's
272
+ inferred from `--out`'s extension. `--bitrate` is ignored for `wav` (PCM).
273
+ - Same Route B path as `render`, so it needs a WebGPU host (GPU or Mesa lavapipe).
274
+ - Programmatically it's `exportBundleAudioToFile(bundle, { out, format, bitrate })`
275
+ from `@sequio/server/route-b`, or `runAudio(file, { out, format, bitrate })` from
276
+ `@sequio/cli`; in-app it's `Exporter.exportAudio(...)` (recipe 8).
277
+
278
+ ## 12. Bring your own spec (JSON → object graph)
279
+
280
+ sequio ships **no** persistent schema — an imperative `defineComposition` module
281
+ is the only portable format the SDK blesses (the server re-runs that code via a
282
+ `RuntimeBundle`, contract #3). If your app already has a document model — a Y.Doc,
283
+ a database row, an editor's own JSON — **you own the schema**, and mapping it onto
284
+ the engine is a ~30-line builder. This is a **pattern, not an API**: copy it and
285
+ bend it to your data; don't look for an official spec type to import.
286
+
287
+ The mapping has four moving parts, all mechanical:
288
+
289
+ 1. **keyframes** → `prop.setStatic(v)` for a constant, `prop.setKeyframes([...])` to animate.
290
+ 2. **easing** → a name→function lookup table (easings are just exported functions).
291
+ 3. **fonts** → `await fonts.load(...)` up front, before any `TextClip` uses the family.
292
+ 4. **effects** → a `type` switch that `new`s the engine effect and writes its params.
293
+
294
+ ```ts
295
+ import {
296
+ Compositor, VisualTrack, TextClip, ShapeClip, ImageClip, ImageSource,
297
+ BlurEffect, ColorEffect, fonts,
298
+ linear, easeOutCubic, easeInOutCubic,
299
+ type Easing, type Effect,
300
+ } from '@sequio/engine';
301
+ import { defineComposition, loadAsset } from '@sequio/runtime';
302
+
303
+ // ── YOUR schema (you define this — here's a minimal one) ─────────────────────
304
+ type Prop<T> = T | { keyframes: Array<{ time: number; value: T; easing?: string }> };
305
+ interface MySpec {
306
+ width: number; height: number; fps: number; background?: number;
307
+ fonts?: { family: string; src: string }[];
308
+ clips: Array<{
309
+ type: 'text' | 'shape' | 'image';
310
+ start: number; end: number;
311
+ position?: Prop<[number, number]>;
312
+ opacity?: Prop<number>;
313
+ text?: string; fontFamily?: string; fill?: number; // text
314
+ width?: number; height?: number; src?: string; // shape / image
315
+ effects?: Array<{ type: 'blur' | 'color'; strength?: number; brightness?: number }>;
316
+ }>;
317
+ }
318
+
319
+ // ── the fixed lookup tables ──────────────────────────────────────────────────
320
+ const EASINGS: Record<string, Easing> = { linear, easeOutCubic, easeInOutCubic };
321
+
322
+ function applyProp<T>(p: { setStatic(v: T): void; setKeyframes(k: any[]): void }, s: Prop<T> | undefined) {
323
+ if (s === undefined) return;
324
+ if (typeof s === 'object' && s !== null && 'keyframes' in s)
325
+ p.setKeyframes(s.keyframes.map((k) => ({ ...k, easing: k.easing ? EASINGS[k.easing] : undefined })));
326
+ else p.setStatic(s as T);
327
+ }
328
+
329
+ function buildEffect(e: { type: string; strength?: number; brightness?: number }): Effect {
330
+ if (e.type === 'blur') { const fx = new BlurEffect(); if (e.strength != null) fx.strength.setStatic(e.strength); return fx; }
331
+ const fx = new ColorEffect(); if (e.brightness != null) fx.brightness.setStatic(e.brightness); return fx;
332
+ }
333
+
334
+ // ── the builder: MySpec → Composer ───────────────────────────────────────────
335
+ export function fromSpec(spec: MySpec) {
336
+ return defineComposition(async () => {
337
+ const c = new Compositor({ width: spec.width, height: spec.height, fps: spec.fps, background: spec.background });
338
+ await c.init();
339
+ for (const f of spec.fonts ?? []) await fonts.load(f); // fonts BEFORE clips
340
+
341
+ const track = new VisualTrack();
342
+ for (const cs of spec.clips) {
343
+ let clip;
344
+ if (cs.type === 'text') clip = new TextClip({ text: cs.text!, fontFamily: cs.fontFamily, fill: cs.fill });
345
+ else if (cs.type === 'shape') clip = new ShapeClip({ kind: 'rect', width: cs.width!, height: cs.height!, fill: cs.fill ?? 0xffffff });
346
+ else clip = new ImageClip(new ImageSource({ src: cs.src! }));
347
+ clip.start = cs.start; clip.end = cs.end;
348
+ applyProp(clip.transform.position, cs.position);
349
+ applyProp(clip.opacity, cs.opacity);
350
+ for (const e of cs.effects ?? []) clip.effects.push(buildEffect(e));
351
+ track.add(clip);
352
+ }
353
+ c.addTrack(track);
354
+ return { compositor: c };
355
+ });
356
+ }
357
+ ```
358
+
359
+ Feed it your JSON and you get a `Composer` that previews, exports and
360
+ server-renders like any other composition:
361
+
362
+ ```ts
363
+ export default fromSpec(JSON.parse(await loadAsset('./timeline.json')));
364
+ ```
365
+
366
+ **Why the SDK doesn't ship this for you:** any real product already has a document
367
+ model (undo, collaboration, its own field names); an official spec would just be a
368
+ second schema to fight. The fuller reference implementation — video/audio clips,
369
+ `sourceIn`/`sourceOut`, blend modes, a global grade — lives in
370
+ `packages/server/src/timeline.ts` (the `TimelineSpec` type + `buildTimeline`). Read
371
+ it for the complete pattern, then write your own.
372
+
373
+ **Trust boundary:** a `RuntimeBundle` is *code* — re-running it executes arbitrary
374
+ TS/JS on your server. Serve bundles only from trusted (first-party) sources. For a
375
+ multi-tenant or public-facing service, accept *data* (a tenant's JSON + your
376
+ builder above), never their code, and allowlist any media `src` URL your builder
377
+ fetches. See `docs/server-side-rendering.md` § 信任边界与安全模型.
378
+
379
+ ## Outlined / echoed / italic display text
380
+
381
+ `TextStyleLike` passes weight, italic, letter-spacing and a stroke straight into
382
+ Pixi's text style — so one family covers bold / hollow / italic cuts:
383
+
384
+ ```ts
385
+ // Hollow outlined word: pale fill + a coloured stroke.
386
+ new TextClip({ text: 'SUPER', fontFamily: 'Oswald', fontSize: 150, fill: 0xf7cfcf,
387
+ fontWeight: '700', letterSpacing: 2, stroke: { color: 0xa5120f, width: 4 } });
388
+
389
+ // Bold italic call-to-action.
390
+ new TextClip({ text: 'MAKE IT YOURS', fontFamily: 'Oswald', fontSize: 96,
391
+ fill: 0xa5120f, fontWeight: '700', fontStyle: 'italic' });
392
+ ```
393
+
394
+ An "echo" stack (fading after-image) is N copies with decreasing opacity:
395
+
396
+ ```ts
397
+ const g = new GroupClip();
398
+ for (let i = 0; i < 4; i++) {
399
+ const t = new TextClip({ text: 'PROMO', fontFamily: 'Oswald', fontSize: 150, fill: 0xffffff, fontWeight: '700' });
400
+ t.start = 0; t.end = 3;
401
+ t.transform.anchor.setStatic([0, 0]);
402
+ t.transform.position.setStatic([0, i * 205]);
403
+ t.opacity.setStatic(Math.pow(0.7, i)); // each copy fainter
404
+ g.add(t);
405
+ }
406
+ ```
407
+
408
+ ## Arch / circle-cropped photo (`maskShape`)
409
+
410
+ Clip an image to a rounded-rect (arch) or ellipse. Mask a **`GroupClip`** that
411
+ wraps the image — a Sprite can't be masked by its own child — and lay the image
412
+ out from `(0, 0)` in the group, sized to fill the mask:
413
+
414
+ ```ts
415
+ const src = new ImageSource({ src: 'https://…?w=760&h=1500&fit=crop' }); // pre-crop to region aspect
416
+ const meta = await src.load();
417
+ const img = new ImageClip(src);
418
+ img.start = 0; img.end = 3;
419
+ img.transform.anchor.setStatic([0, 0]);
420
+ img.transform.scale.setStatic([760 / meta.width, 1500 / meta.height]);
421
+
422
+ const arch = new GroupClip();
423
+ arch.start = 0; arch.end = 3;
424
+ arch.transform.anchor.setStatic([0, 0]);
425
+ arch.transform.position.setStatic([170, -60]); // where the arch sits
426
+ arch.maskShape = { kind: 'rect', width: 760, height: 1500, radius: 380 }; // big radius → arch
427
+ arch.add(img);
428
+ ```
429
+
184
430
  ## Gotchas
185
431
 
186
432
  - Call `await compositor.init()` before adding tracks.