@sequio/skill 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +68 -0
- package/llms.txt +53 -0
- package/package.json +43 -0
- package/skills/sequio/SKILL.md +181 -0
- package/skills/sequio/references/api.md +105 -0
- package/skills/sequio/references/recipes.md +191 -0
package/README.md
ADDED
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
# `@sequio/skill`
|
|
2
|
+
|
|
3
|
+
An installable **AI Agent Skill** and an **`llms.txt`** that teach an AI assistant
|
|
4
|
+
how to use [sequio](https://github.com/slightc/sequio) — how to build a
|
|
5
|
+
`Track / Clip / Effect` object graph, animate it, and preview / export / render
|
|
6
|
+
video. Drop it into any project that consumes `@sequio/engine` so the assistant
|
|
7
|
+
working in that repo already knows the API and the mental model.
|
|
8
|
+
|
|
9
|
+
This package ships **no runtime code** — just docs an agent reads.
|
|
10
|
+
|
|
11
|
+
```
|
|
12
|
+
skills/sequio/
|
|
13
|
+
SKILL.md the skill (YAML frontmatter: name + description, then the guide)
|
|
14
|
+
references/
|
|
15
|
+
api.md the full public surface, grouped by module
|
|
16
|
+
recipes.md copy-paste composition patterns
|
|
17
|
+
llms.txt the llms.txt index (llmstxt.org) — links to the canonical docs
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
## What's inside
|
|
21
|
+
|
|
22
|
+
- **`SKILL.md`** — an [Agent Skill](https://docs.claude.com/en/docs/claude-code/skills):
|
|
23
|
+
a Markdown file with `name` / `description` frontmatter that an AI agent
|
|
24
|
+
auto-loads when a task involves sequio (composing video, timelines, `TextClip`,
|
|
25
|
+
`Compositor`, rendering an MP4 from code, …). It links to the two reference
|
|
26
|
+
files for the exact API and fuller examples.
|
|
27
|
+
- **`llms.txt`** — a root-level [`/llms.txt`](https://llmstxt.org/) index: a short
|
|
28
|
+
project summary plus curated links to sequio's docs, reference, and examples, so
|
|
29
|
+
any LLM tool (not just Claude) can discover the canonical material.
|
|
30
|
+
|
|
31
|
+
## Install the skill into your project
|
|
32
|
+
|
|
33
|
+
Agent Skills live under `.claude/skills/<name>/`. Copy this package's `skills/`
|
|
34
|
+
folder there:
|
|
35
|
+
|
|
36
|
+
```bash
|
|
37
|
+
# from your project root, after `npm install @sequio/skill` (or pnpm/yarn)
|
|
38
|
+
mkdir -p .claude/skills
|
|
39
|
+
cp -R node_modules/@sequio/skill/skills/sequio .claude/skills/sequio
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
Or symlink it so it tracks the installed version:
|
|
43
|
+
|
|
44
|
+
```bash
|
|
45
|
+
ln -s ../../node_modules/@sequio/skill/skills/sequio .claude/skills/sequio
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
Commit `.claude/skills/sequio/` and every agent working in the repo picks it up.
|
|
49
|
+
The skill is host-agnostic Markdown — any tool that reads Agent Skills can use it.
|
|
50
|
+
|
|
51
|
+
## Serve the `llms.txt`
|
|
52
|
+
|
|
53
|
+
If your project (or sequio's) has a website, publish `llms.txt` at the site root
|
|
54
|
+
so it is reachable at `https://your-site/llms.txt` — the convention LLM tools look
|
|
55
|
+
for. The file also stands alone as a compact, link-first briefing you can paste
|
|
56
|
+
into any assistant.
|
|
57
|
+
|
|
58
|
+
```bash
|
|
59
|
+
cp node_modules/@sequio/skill/llms.txt public/llms.txt # e.g. for a Vite/Next site
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
## Keeping it accurate
|
|
63
|
+
|
|
64
|
+
The skill's source of truth is the engine barrel
|
|
65
|
+
(`packages/engine/src/index.ts`). When the public API changes, update
|
|
66
|
+
`references/api.md` (and `SKILL.md` if the mental model shifts) in the same
|
|
67
|
+
change — the package's tests assert the frontmatter is well-formed and that every
|
|
68
|
+
intra-skill and `llms.txt` on-disk link still resolves.
|
package/llms.txt
ADDED
|
@@ -0,0 +1,53 @@
|
|
|
1
|
+
# sequio
|
|
2
|
+
|
|
3
|
+
> sequio is a command-style object-graph engine for building video editors on top
|
|
4
|
+
> of PixiJS v8. You construct a tree of Track / Clip / Effect objects and drive a
|
|
5
|
+
> clock; the SDK owns the low-level runtime — decode, composite, audio, export.
|
|
6
|
+
> Compositions are ordinary imperative TypeScript (you `new` the engine's own
|
|
7
|
+
> classes), so the same code previews in a browser, exports in a browser, and
|
|
8
|
+
> renders on a server. Persistence, schema, undo, collaboration and UI are out of
|
|
9
|
+
> scope — they belong to the layer above.
|
|
10
|
+
|
|
11
|
+
Key facts an assistant should hold before writing sequio code:
|
|
12
|
+
|
|
13
|
+
- Packages: `@sequio/engine` (the SDK), `@sequio/runtime` (compile+run TS/JS into
|
|
14
|
+
a `Composer`), `@sequio/server` (server-side render), `@sequio/cli` (the
|
|
15
|
+
`sequio render` / `sequio preview` commands), `@sequio/studio` (reference editor).
|
|
16
|
+
- Author a composition as a module whose default export is
|
|
17
|
+
`defineComposition(builder)`; inside, `new Compositor(...)`, add `VisualTrack`s,
|
|
18
|
+
`track.add(new TextClip(...))`, return `{ compositor, duration? }`.
|
|
19
|
+
- Times are seconds at the API boundary. Clips carry `.start` / `.end` and a
|
|
20
|
+
`.transform` (position/scale/rotation/anchor/alpha, each keyframable).
|
|
21
|
+
- Invariant: `render(t)` is a pure function of (object graph, t) — reproducible
|
|
22
|
+
export, golden-frame tests. Preview and export share one render core.
|
|
23
|
+
- `sequio render <file>` encodes to video (pure-Node WebGPU; needs a GPU or Mesa
|
|
24
|
+
lavapipe). `sequio frame <file> --time <sec>` exports a single frame as a PNG
|
|
25
|
+
through the same render core — the fast way to visually check a composition
|
|
26
|
+
before a full render. `sequio preview <file> --watch` serves a live preview.
|
|
27
|
+
|
|
28
|
+
## Docs
|
|
29
|
+
|
|
30
|
+
- [Architecture & the five contracts](https://github.com/slightc/sequio/blob/main/docs/architecture.md): modules, invariants, and the public-surface table.
|
|
31
|
+
- [Runtime](https://github.com/slightc/sequio/blob/main/docs/runtime.md): compile + link + run multi-file TS/JS into a `Composer`; `defineComposition`, `loadAsset`, externals.
|
|
32
|
+
- [CLI](https://github.com/slightc/sequio/blob/main/docs/cli.md): `sequio render` / `sequio preview --watch`, `--scale`, fonts, local/network media.
|
|
33
|
+
- [Server-side rendering](https://github.com/slightc/sequio/blob/main/docs/server-side-rendering.md): the `TimelineSpec` protocol and both render routes (headless Chrome / pure-Node WebGPU).
|
|
34
|
+
- [Text animation](https://github.com/slightc/sequio/blob/main/docs/text-animation.md): clip/text animators, split motion, the GSAP binding.
|
|
35
|
+
- [Contributor / agent guide (AGENT.md)](https://github.com/slightc/sequio/blob/main/AGENT.md): conventions, monorepo layout, working agreement.
|
|
36
|
+
|
|
37
|
+
## Reference
|
|
38
|
+
|
|
39
|
+
- [Public API surface](https://github.com/slightc/sequio/blob/main/packages/engine/src/index.ts): the engine barrel — the single source of truth for what is stable API.
|
|
40
|
+
- [Agent Skill: SKILL.md](https://github.com/slightc/sequio/blob/main/packages/skill/skills/sequio/SKILL.md): the installable skill that teaches how to use sequio.
|
|
41
|
+
- [Skill API cheatsheet](https://github.com/slightc/sequio/blob/main/packages/skill/skills/sequio/references/api.md): every exported class/type, grouped by module.
|
|
42
|
+
- [Skill recipes](https://github.com/slightc/sequio/blob/main/packages/skill/skills/sequio/references/recipes.md): copy-paste composition patterns.
|
|
43
|
+
|
|
44
|
+
## Examples
|
|
45
|
+
|
|
46
|
+
- [CLI sample composition](https://github.com/slightc/sequio/blob/main/packages/cli/example/index.ts): a title + shapes, GSAP-driven entrance, embedded font.
|
|
47
|
+
- [YC-style 15s motion poster](https://github.com/slightc/sequio/blob/main/packages/cli/example/yc-spot/index.ts): a multi-scene showcase built entirely from ShapeClip/TextClip/GroupClip.
|
|
48
|
+
|
|
49
|
+
## Optional
|
|
50
|
+
|
|
51
|
+
- [Project website](https://sequio-demo.vercel.app/): home, demo gallery (live sequio renders), in-browser Code Mode, API reference.
|
|
52
|
+
- [README](https://github.com/slightc/sequio/blob/main/README.md): package table, quick start, install.
|
|
53
|
+
- [Roadmap & milestones](https://github.com/slightc/sequio/blob/main/todo/README.md): what's implemented vs. in progress.
|
package/package.json
ADDED
|
@@ -0,0 +1,43 @@
|
|
|
1
|
+
{
|
|
2
|
+
"name": "@sequio/skill",
|
|
3
|
+
"version": "0.1.0",
|
|
4
|
+
"description": "An installable AI Agent Skill (SKILL.md) + llms.txt that teach an AI assistant how to use sequio — build a Track/Clip/Effect object graph, preview, and export/render video.",
|
|
5
|
+
"type": "module",
|
|
6
|
+
"license": "MIT",
|
|
7
|
+
"author": "slightc",
|
|
8
|
+
"homepage": "https://sequio-demo.vercel.app/",
|
|
9
|
+
"repository": {
|
|
10
|
+
"type": "git",
|
|
11
|
+
"url": "git+https://github.com/slightc/sequio.git",
|
|
12
|
+
"directory": "packages/skill"
|
|
13
|
+
},
|
|
14
|
+
"bugs": {
|
|
15
|
+
"url": "https://github.com/slightc/sequio/issues"
|
|
16
|
+
},
|
|
17
|
+
"files": [
|
|
18
|
+
"skills",
|
|
19
|
+
"llms.txt",
|
|
20
|
+
"README.md"
|
|
21
|
+
],
|
|
22
|
+
"exports": {
|
|
23
|
+
"./llms.txt": "./llms.txt",
|
|
24
|
+
"./skill": "./skills/sequio/SKILL.md"
|
|
25
|
+
},
|
|
26
|
+
"publishConfig": {
|
|
27
|
+
"access": "public"
|
|
28
|
+
},
|
|
29
|
+
"keywords": [
|
|
30
|
+
"sequio",
|
|
31
|
+
"agent-skill",
|
|
32
|
+
"claude",
|
|
33
|
+
"llms-txt",
|
|
34
|
+
"ai",
|
|
35
|
+
"video-editor",
|
|
36
|
+
"pixijs"
|
|
37
|
+
],
|
|
38
|
+
"scripts": {
|
|
39
|
+
"typecheck": "tsc --noEmit",
|
|
40
|
+
"test": "vitest run",
|
|
41
|
+
"test:watch": "vitest"
|
|
42
|
+
}
|
|
43
|
+
}
|
|
@@ -0,0 +1,181 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: sequio
|
|
3
|
+
description: >-
|
|
4
|
+
Build, preview, and export/render video with sequio (@sequio/engine +
|
|
5
|
+
@sequio/runtime + @sequio/cli). Use this whenever a task involves composing
|
|
6
|
+
video/motion-graphics programmatically — creating a timeline of tracks and
|
|
7
|
+
clips (text, image, video, shapes), animating them (keyframes or GSAP),
|
|
8
|
+
applying effects/transitions, mixing audio, and exporting to MP4/WebM or
|
|
9
|
+
rendering server-side. Triggers: "sequio", "video editor SDK", "compose a
|
|
10
|
+
video", "timeline / track / clip", "render an mp4 from code", "TextClip",
|
|
11
|
+
"Compositor", "defineComposition".
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
# Using sequio
|
|
15
|
+
|
|
16
|
+
sequio is a **command-style object-graph engine** for building video on top of
|
|
17
|
+
**PixiJS v8**. You construct a tree of `Track / Clip / Effect` objects and drive
|
|
18
|
+
a clock; the SDK owns decode, composite, audio, and export. It is *not*
|
|
19
|
+
declarative JSON — you write ordinary imperative TypeScript with the engine's own
|
|
20
|
+
classes, so anything that runs in a demo runs everywhere (browser preview,
|
|
21
|
+
browser export, server render).
|
|
22
|
+
|
|
23
|
+
## Decide the entry point first
|
|
24
|
+
|
|
25
|
+
Pick the package that matches the task before writing code:
|
|
26
|
+
|
|
27
|
+
| Goal | Use | Import from |
|
|
28
|
+
|---|---|---|
|
|
29
|
+
| Drive an object graph in an app you control | **engine** directly | `@sequio/engine` |
|
|
30
|
+
| Author a self-contained composition file that previews **and** renders | **runtime** authoring API | `@sequio/engine` + `@sequio/runtime` |
|
|
31
|
+
| Turn a composition file into an `.mp4`, a single-frame PNG, or a live preview from a terminal | **cli** | `sequio render` / `sequio frame` / `sequio preview` |
|
|
32
|
+
| Render a composition to a file on a server (no browser) | **server** Route B | `@sequio/server/route-b` |
|
|
33
|
+
|
|
34
|
+
For most "make me a video from code" tasks, author a **composition file** (runtime
|
|
35
|
+
API) and drive it with the **CLI** — that is the path the examples use.
|
|
36
|
+
|
|
37
|
+
## The mental model (do not fight these)
|
|
38
|
+
|
|
39
|
+
sequio rests on five invariants. Respect them and everything composes; break them
|
|
40
|
+
and preview/export diverge:
|
|
41
|
+
|
|
42
|
+
1. **Async `prepare` / sync `render`.** Decoding is async; a frame is a pure sync
|
|
43
|
+
render once inputs are ready. You rarely call these directly — the Compositor,
|
|
44
|
+
preview clock, and Exporter do.
|
|
45
|
+
2. **`render(t)` is a pure function of (object graph, `t`).** No dependence on the
|
|
46
|
+
previous frame or wall-clock. This is what makes export reproducible. Never
|
|
47
|
+
store per-frame mutable state that a later `render(t)` reads back.
|
|
48
|
+
3. **Preview and export share one render core** — same resolution and color
|
|
49
|
+
pipeline. A composition that looks right in preview renders identically.
|
|
50
|
+
4. **Explicit resource ownership.** Every SDK object has `dispose()`. Long-lived
|
|
51
|
+
apps must dispose what they create; a one-shot render is torn down for you.
|
|
52
|
+
5. **Invalidate / dirty-flag.** The engine never repaints on its own. Mutating a
|
|
53
|
+
property marks the graph dirty; the host schedules a repaint.
|
|
54
|
+
|
|
55
|
+
## Author a composition (the common path)
|
|
56
|
+
|
|
57
|
+
A composition is a TS/JS module whose default export is `defineComposition(builder)`.
|
|
58
|
+
Inside the builder you `new` engine classes exactly like a demo — no `env`
|
|
59
|
+
plumbing. Times are **seconds** at the API boundary.
|
|
60
|
+
|
|
61
|
+
```ts
|
|
62
|
+
import { Compositor, VisualTrack, TextClip, ShapeClip, easeInOutCubic } from '@sequio/engine';
|
|
63
|
+
import { defineComposition } from '@sequio/runtime';
|
|
64
|
+
|
|
65
|
+
export default defineComposition(async () => {
|
|
66
|
+
const compositor = new Compositor({ width: 1280, height: 720, fps: 30, background: 0x0b0b0e });
|
|
67
|
+
await compositor.init();
|
|
68
|
+
|
|
69
|
+
// A background rectangle for the whole timeline.
|
|
70
|
+
const bg = new VisualTrack();
|
|
71
|
+
const backdrop = new ShapeClip({ kind: 'rect', width: 1280, height: 720, fill: 0x0f172a });
|
|
72
|
+
backdrop.start = 0;
|
|
73
|
+
backdrop.end = 4;
|
|
74
|
+
backdrop.transform.anchor.setStatic([0, 0]);
|
|
75
|
+
backdrop.transform.position.setStatic([0, 0]);
|
|
76
|
+
bg.add(backdrop);
|
|
77
|
+
compositor.addTrack(bg);
|
|
78
|
+
|
|
79
|
+
// A title on a higher track (z-index stacks tracks).
|
|
80
|
+
const text = new VisualTrack();
|
|
81
|
+
text.zIndex = 1;
|
|
82
|
+
const title = new TextClip({ text: 'sequio', fontSize: 72, fill: 0xffffff });
|
|
83
|
+
title.start = 0;
|
|
84
|
+
title.end = 4;
|
|
85
|
+
title.transform.anchor.setStatic([0.5, 0.5]);
|
|
86
|
+
// Keyframe the position (values interpolate with the given easing).
|
|
87
|
+
title.transform.position.setKeyframes([
|
|
88
|
+
{ time: 0, value: [640, 300] },
|
|
89
|
+
{ time: 4, value: [640, 420], easing: easeInOutCubic },
|
|
90
|
+
]);
|
|
91
|
+
text.add(title);
|
|
92
|
+
compositor.addTrack(text);
|
|
93
|
+
|
|
94
|
+
return { compositor, duration: 4 }; // `duration` optional — derived from clip ends
|
|
95
|
+
});
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
Then, from the workspace root:
|
|
99
|
+
|
|
100
|
+
```bash
|
|
101
|
+
sequio preview composition.ts --watch # live in-browser preview, reloads on edit
|
|
102
|
+
sequio frame composition.ts --time 2 --out shot.png # export ONE frame at t=2s as a PNG (fast visual check)
|
|
103
|
+
sequio render composition.ts --out out.mp4 # encode the whole thing to video (pure-Node WebGPU)
|
|
104
|
+
sequio render composition.ts --out out.mp4 --scale 2 # 2× resolution (sharp text/edges)
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
`sequio render` and `sequio frame` need a WebGPU host — a real GPU or the Mesa
|
|
108
|
+
**lavapipe** software driver (`apt install mesa-vulkan-drivers`; then
|
|
109
|
+
`export VK_ICD_FILENAMES=/usr/share/vulkan/icd.d/lvp_icd.json`). Without one they
|
|
110
|
+
throw a clear error.
|
|
111
|
+
|
|
112
|
+
### Check your work fast: export a single frame
|
|
113
|
+
|
|
114
|
+
When iterating on a composition **without** a live browser (e.g. you're an agent
|
|
115
|
+
editing code), don't render the whole video to see if it's right — export one
|
|
116
|
+
frame and look at it:
|
|
117
|
+
|
|
118
|
+
```bash
|
|
119
|
+
sequio frame composition.ts --time 2.5 --out /tmp/check.png # then open/view the PNG
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
`frame` runs the **same render core** as `render` (contract #3), so the PNG is
|
|
123
|
+
exactly what the video would contain at that instant — the fast way to confirm
|
|
124
|
+
layout, position, and color before committing to a full render. `--time` is
|
|
125
|
+
clamped to `[0, duration]`; `--scale N` renders at N× like `render`. This is the
|
|
126
|
+
recommended verify loop: edit → `sequio frame` at a few representative times →
|
|
127
|
+
eyeball → then `sequio render` once it looks right.
|
|
128
|
+
|
|
129
|
+
## Core building blocks
|
|
130
|
+
|
|
131
|
+
- **`Compositor({ width, height, fps?, background?, timebase? })`** — the root. Call
|
|
132
|
+
`await compositor.init()` before adding tracks; `compositor.addTrack(track)`.
|
|
133
|
+
- **Tracks**: `VisualTrack` (stacked by `.zIndex`) and `AudioTrack`. A track holds
|
|
134
|
+
clips: `track.add(clip)`.
|
|
135
|
+
- **Clips** carry `.start` / `.end` (seconds) and a `.transform` (a `Transform2D`
|
|
136
|
+
with `position`, `scale`, `rotation`, `anchor`, `alpha` — each an
|
|
137
|
+
`AnimatableProperty` you drive with `.setStatic(v)` or `.setKeyframes([...])`):
|
|
138
|
+
- `TextClip({ text, fontFamily?, fontSize?, fill? })`
|
|
139
|
+
- `ImageClip` / `VideoClip` (backed by an `ImageSource` / `VideoSource`)
|
|
140
|
+
- `ShapeClip({ kind: 'rect' | 'ellipse' | ..., width, height, fill })`
|
|
141
|
+
- `GroupClip` — nest clips and transform them together.
|
|
142
|
+
- **`anchor`** is normalized (`[0.5, 0.5]` = center); `position` is in pixels.
|
|
143
|
+
|
|
144
|
+
## Animation
|
|
145
|
+
|
|
146
|
+
- **Keyframes** — `prop.setKeyframes([{ time, value, easing? }, ...])`. Easings are
|
|
147
|
+
exported: `linear`, `easeInOutCubic`, `cubicBezier(...)`, etc.
|
|
148
|
+
- **GSAP** — the engine ships *no* gsap dependency but has a binding. The host
|
|
149
|
+
injects gsap; the CLI already does, so a composition can just:
|
|
150
|
+
```ts
|
|
151
|
+
import gsap from 'gsap';
|
|
152
|
+
import { gsapClipAnimator } from '@sequio/engine';
|
|
153
|
+
clip.animator = gsapClipAnimator(gsap, (tl, o) => tl.from(o, { y: -60, alpha: 0, ease: 'back.out(1.7)' }));
|
|
154
|
+
```
|
|
155
|
+
The binding seeks a **paused** timeline, keeping `render(t)` pure.
|
|
156
|
+
- **Text motion** — `TextClip.split` + `StaggerTextAnimator` / `gsapTextAnimator`
|
|
157
|
+
animate per line/word/char (e.g. a staggered drop-in).
|
|
158
|
+
|
|
159
|
+
## Media, fonts, effects, audio, export — quick pointers
|
|
160
|
+
|
|
161
|
+
- **Media**: `new VideoSource({ src })` / `new ImageSource({ src })` accept a URL or
|
|
162
|
+
a `Blob`. For a file next to the composition, use runtime's
|
|
163
|
+
`await loadAsset('./clip.mp4')` (host provides the bytes; never bundled).
|
|
164
|
+
- **Fonts**: `await fonts.load({ family, src })` before using the family in a
|
|
165
|
+
`TextClip`. Load an explicit web font so preview and Node render match — system
|
|
166
|
+
defaults differ per platform.
|
|
167
|
+
- **Effects/Transitions**: `ColorEffect`, `BlurEffect`, `BulgeEffect`,
|
|
168
|
+
`CrossfadeTransition`, etc. (some — chroma/LUT/wipe — are still TODO).
|
|
169
|
+
- **Audio**: `AudioTrack` + `AudioSource`; `AudioEngine` mixes (Web Audio live,
|
|
170
|
+
OfflineAudioContext for export).
|
|
171
|
+
- **Export in an app** (not the CLI): use `Exporter` from `@sequio/engine`, or a
|
|
172
|
+
runtime `Composer`'s `composer.export(...)`.
|
|
173
|
+
|
|
174
|
+
## Before you finish
|
|
175
|
+
|
|
176
|
+
- Load `references/api.md` for the exact public surface (every exported class/type)
|
|
177
|
+
and `references/recipes.md` for fuller copy-paste patterns.
|
|
178
|
+
- Keep the public API surface honest: it is whatever
|
|
179
|
+
`packages/engine/src/index.ts` exports.
|
|
180
|
+
- If a code path throws "not implemented", that milestone is unbuilt — see the
|
|
181
|
+
pointer in the error and `todo/`. Do not paper over it.
|
|
@@ -0,0 +1,105 @@
|
|
|
1
|
+
# sequio public API reference
|
|
2
|
+
|
|
3
|
+
The stable public surface is exactly what `@sequio/engine`'s barrel
|
|
4
|
+
(`packages/engine/src/index.ts`) exports. This mirrors it, grouped by module.
|
|
5
|
+
Internal helpers (`Reconciler`, `FrameCache`, `TextureManager`, demux/mux
|
|
6
|
+
adapters) are exported for advanced extension but are **not** stable API.
|
|
7
|
+
|
|
8
|
+
Times are **seconds** at the API boundary, quantized to frames internally via
|
|
9
|
+
`Timebase`.
|
|
10
|
+
|
|
11
|
+
## Time & clock
|
|
12
|
+
|
|
13
|
+
- `Timebase` — frame quantization (`new Timebase(fps)`).
|
|
14
|
+
- `Clock`, `RealtimeClock`, `FixedStepClock` — preview uses `RealtimeClock`;
|
|
15
|
+
export uses a fixed step so no frame is dropped.
|
|
16
|
+
|
|
17
|
+
## Animation
|
|
18
|
+
|
|
19
|
+
- `Transform2D` — a clip's spatial state: `position`, `scale`, `rotation`,
|
|
20
|
+
`anchor`, `alpha` (each an `AnimatableProperty`).
|
|
21
|
+
- `AnimatableProperty` / `Keyframe` — `.setStatic(value)` or
|
|
22
|
+
`.setKeyframes([{ time, value, easing? }, ...])`.
|
|
23
|
+
- Easings: `linear`, `hold`, `cubicBezier`, `easeInQuad`, `easeOutQuad`,
|
|
24
|
+
`easeInOutQuad`, `easeInCubic`, `easeOutCubic`, `easeInOutCubic`. Type `Easing`.
|
|
25
|
+
- Clip/text animators: `ClipAnimator`, `TextAnimator`, `StaggerTextAnimator`,
|
|
26
|
+
`TweenAnimator`, `IDENTITY_SAMPLE`, `lerpSample`, `AnimationSample`, plus
|
|
27
|
+
`TextPart` / `TextSplit` / `StaggerOrder` / `StaggerTextOptions` /
|
|
28
|
+
`TweenAnimatorOptions`.
|
|
29
|
+
- GSAP binding (engine has no gsap dep): `gsapClipAnimator`, `gsapTextAnimator`,
|
|
30
|
+
`identityTarget`; types `GsapLike`, `GsapTimelineLike`, `GsapTarget`.
|
|
31
|
+
- Text layout: `computeTextParts`, `MeasureWidth`.
|
|
32
|
+
|
|
33
|
+
## Media
|
|
34
|
+
|
|
35
|
+
- `MediaSource`, `VisualSource`, `SourceMetadata`.
|
|
36
|
+
- `VideoSource` (`VideoSourceOptions`) — decode via Mediabunny; `src` = URL | Blob.
|
|
37
|
+
- `ImageSource` (`ImageSourceOptions`).
|
|
38
|
+
- `AudioSource` (`AudioSourceOptions`).
|
|
39
|
+
- Decoder seam: `VideoDecoderBackend`, `DecodedFrame`, `MediabunnyVideoDecoder`,
|
|
40
|
+
`setFrameImageExtractor` / `FrameImageExtractor`, `MediabunnyDemux`, `VideoInput`.
|
|
41
|
+
- `FrameCache`, `Closable`.
|
|
42
|
+
- Mediabunny module control (dual-package hazard): `loadMediabunny`,
|
|
43
|
+
`setMediabunnyModule`, `MediabunnyModule`.
|
|
44
|
+
|
|
45
|
+
## Texture
|
|
46
|
+
|
|
47
|
+
- `TextureManager` — GPU byte budget + LRU eviction.
|
|
48
|
+
|
|
49
|
+
## Text / fonts
|
|
50
|
+
|
|
51
|
+
- `FontManager`, `fonts` (singleton), `buildGoogleCss2Url`; types `FontSpec`,
|
|
52
|
+
`GoogleFontSpec`.
|
|
53
|
+
- `fonts.load({ family, src })` before rendering a `TextClip` in that family.
|
|
54
|
+
|
|
55
|
+
## Compositor graph
|
|
56
|
+
|
|
57
|
+
- `Compositor` (`CompositorOptions`) — root. `new Compositor({ width, height,
|
|
58
|
+
fps?, background?, timebase?, createRenderer?, resolution? })`, then
|
|
59
|
+
`await init()`, `addTrack(track)`, `renderPreview(t)`.
|
|
60
|
+
- Tracks: `Track`, `VisualTrack` (stacks by `.zIndex`), `AudioTrack`.
|
|
61
|
+
- Clips: `Clip`, `VisualClip`, `AudioClip`; concrete `VideoClip`, `ImageClip`,
|
|
62
|
+
`TextClip`, `ShapeClip`, `GroupClip`. Types: `TextStyleLike`, `ShapeSpec`,
|
|
63
|
+
`ShapeKind`.
|
|
64
|
+
- Every clip has `.start` / `.end` (seconds) and `.transform` (`Transform2D`).
|
|
65
|
+
|
|
66
|
+
## Effects & transitions
|
|
67
|
+
|
|
68
|
+
- `Effect`, `EffectRegistry` (`EffectFactory`), `registerBuiltins`,
|
|
69
|
+
`BUILTIN_EFFECTS`.
|
|
70
|
+
- Effects: `ColorEffect`, `BlurEffect`, `BulgeEffect`, `PerspectiveEffect`,
|
|
71
|
+
`DisplacementEffect` (`DisplacementEffectOptions`).
|
|
72
|
+
- Warp math: `Mat3`, `Quad`, `Vec2`, `UNIT_QUAD`, `squareToQuad`, `invert3x3`,
|
|
73
|
+
`applyHomography`, `perspectiveSampleMatrix`, `bulgeSourceUv`.
|
|
74
|
+
- Transitions: `Transition`, `CrossfadeTransition`, `crossfadeAlpha`.
|
|
75
|
+
- Status: color/blur/warp + crossfade done; chroma/LUT/wipe TODO.
|
|
76
|
+
|
|
77
|
+
## Audio
|
|
78
|
+
|
|
79
|
+
- `AudioEngine` — Web Audio live + OfflineAudioContext for export.
|
|
80
|
+
- Scheduling helpers: `clipPlaybackAt`, `effectiveGain`, `fadeFactor`,
|
|
81
|
+
`gainEventsAt`; types `ClipPlayback`, `GainEvent`.
|
|
82
|
+
|
|
83
|
+
## Export
|
|
84
|
+
|
|
85
|
+
- `Exporter` (`ExportOptions`, `ExportFrameOptions`), `ExportCancelledError`.
|
|
86
|
+
- `ExportSink`, `ResolvedExportOptions`, `exportFrameTimes`,
|
|
87
|
+
`MediabunnyExportSink`.
|
|
88
|
+
|
|
89
|
+
## Re-exported PixiJS types (type-only)
|
|
90
|
+
|
|
91
|
+
- `Renderer`, `AutoDetectOptions`, `BLEND_MODES` — so a consumer types against the
|
|
92
|
+
injected renderer / clip blend mode without importing `pixi.js` directly.
|
|
93
|
+
|
|
94
|
+
## Core primitives
|
|
95
|
+
|
|
96
|
+
- `Disposable`, `Subscription`, `createSubscription`.
|
|
97
|
+
|
|
98
|
+
## Runtime authoring API (`@sequio/runtime`)
|
|
99
|
+
|
|
100
|
+
- `defineComposition(builder)` — default-export a builder that returns
|
|
101
|
+
`{ compositor, audioEngine?, duration? }`.
|
|
102
|
+
- `loadAsset('./path')` — fetch a local media file as a `Blob` (host-provided).
|
|
103
|
+
- `Runtime`, `Composer` — compile+link+run files → a `Composer` that previews,
|
|
104
|
+
exports (client), or `toBundle()`s source for server render.
|
|
105
|
+
- `@sequio/runtime/node-fs` (subpath) — `NodeFileSystem` for real-disk files.
|
|
@@ -0,0 +1,191 @@
|
|
|
1
|
+
# sequio recipes
|
|
2
|
+
|
|
3
|
+
Copy-paste patterns. Each is an imperative composition — the same code runs in
|
|
4
|
+
browser preview, browser export, and server render (contract #3). Author them as
|
|
5
|
+
a `defineComposition(builder)` module and drive with `sequio preview` / `sequio
|
|
6
|
+
render`, or lift the body into an app that owns its own clock.
|
|
7
|
+
|
|
8
|
+
## 1. Title over a background
|
|
9
|
+
|
|
10
|
+
```ts
|
|
11
|
+
import { Compositor, VisualTrack, TextClip, ShapeClip } from '@sequio/engine';
|
|
12
|
+
import { defineComposition } from '@sequio/runtime';
|
|
13
|
+
|
|
14
|
+
export default defineComposition(async () => {
|
|
15
|
+
const c = new Compositor({ width: 1280, height: 720, fps: 30 });
|
|
16
|
+
await c.init();
|
|
17
|
+
|
|
18
|
+
const bg = new VisualTrack();
|
|
19
|
+
const rect = new ShapeClip({ kind: 'rect', width: 1280, height: 720, fill: 0x101018 });
|
|
20
|
+
rect.start = 0; rect.end = 5;
|
|
21
|
+
rect.transform.anchor.setStatic([0, 0]);
|
|
22
|
+
rect.transform.position.setStatic([0, 0]);
|
|
23
|
+
bg.add(rect);
|
|
24
|
+
c.addTrack(bg);
|
|
25
|
+
|
|
26
|
+
const layer = new VisualTrack();
|
|
27
|
+
layer.zIndex = 1;
|
|
28
|
+
const title = new TextClip({ text: 'Hello', fontSize: 96, fill: 0xffffff });
|
|
29
|
+
title.start = 0; title.end = 5;
|
|
30
|
+
title.transform.anchor.setStatic([0.5, 0.5]);
|
|
31
|
+
title.transform.position.setStatic([640, 360]);
|
|
32
|
+
layer.add(title);
|
|
33
|
+
c.addTrack(layer);
|
|
34
|
+
|
|
35
|
+
return { compositor: c, duration: 5 };
|
|
36
|
+
});
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
## 2. Keyframed motion + fade
|
|
40
|
+
|
|
41
|
+
```ts
|
|
42
|
+
import { easeOutCubic } from '@sequio/engine';
|
|
43
|
+
// clip.transform properties are AnimatableProperty — keyframe any of them.
|
|
44
|
+
clip.transform.position.setKeyframes([
|
|
45
|
+
{ time: 0, value: [200, 360] },
|
|
46
|
+
{ time: 1.2, value: [640, 360], easing: easeOutCubic },
|
|
47
|
+
]);
|
|
48
|
+
clip.transform.scale.setKeyframes([
|
|
49
|
+
{ time: 0, value: [0.6, 0.6] },
|
|
50
|
+
{ time: 1.2, value: [1, 1], easing: easeOutCubic },
|
|
51
|
+
]);
|
|
52
|
+
clip.transform.alpha.setKeyframes([
|
|
53
|
+
{ time: 0, value: 0 },
|
|
54
|
+
{ time: 0.6, value: 1 },
|
|
55
|
+
]);
|
|
56
|
+
```
|
|
57
|
+
|
|
58
|
+
## 3. GSAP-driven entrance (CLI injects gsap)
|
|
59
|
+
|
|
60
|
+
```ts
|
|
61
|
+
import gsap from 'gsap';
|
|
62
|
+
import { gsapClipAnimator } from '@sequio/engine';
|
|
63
|
+
|
|
64
|
+
title.animator = gsapClipAnimator(gsap, (tl, o) => {
|
|
65
|
+
tl.from(o, { y: -80, alpha: 0, duration: 0.8, ease: 'back.out(1.7)' });
|
|
66
|
+
});
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
The binding seeks a **paused** gsap timeline so `render(t)` stays pure. In your
|
|
70
|
+
own app (not the CLI) you must inject gsap via `RuntimeOptions.externals` or use
|
|
71
|
+
it directly — the engine declares only the structural `GsapLike` types.
|
|
72
|
+
|
|
73
|
+
## 4. Per-character text drop-in
|
|
74
|
+
|
|
75
|
+
```ts
|
|
76
|
+
import { StaggerTextAnimator } from '@sequio/engine';
|
|
77
|
+
|
|
78
|
+
title.split = 'char'; // split into per-character parts
|
|
79
|
+
title.animator = new StaggerTextAnimator({
|
|
80
|
+
each: 0.04, // stagger between parts
|
|
81
|
+
from: (o) => ({ y: -40, alpha: 0 }), // starting sample per part
|
|
82
|
+
});
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
For gsap-driven per-part motion use `gsapTextAnimator(gsap, split, builder)`.
|
|
86
|
+
|
|
87
|
+
## 5. Video and image clips
|
|
88
|
+
|
|
89
|
+
```ts
|
|
90
|
+
import { Compositor, VisualTrack, VideoClip, ImageClip, VideoSource, ImageSource } from '@sequio/engine';
|
|
91
|
+
import { defineComposition, loadAsset } from '@sequio/runtime';
|
|
92
|
+
|
|
93
|
+
export default defineComposition(async () => {
|
|
94
|
+
const c = new Compositor({ width: 1920, height: 1080, fps: 30 });
|
|
95
|
+
await c.init();
|
|
96
|
+
|
|
97
|
+
// URL sources (browser fetch / Node UrlSource) — nothing committed to the repo:
|
|
98
|
+
const web = new ImageSource({ src: 'https://picsum.photos/id/1015/1280/720' });
|
|
99
|
+
// Local file next to the composition — host provides the bytes, never bundled:
|
|
100
|
+
const local = new VideoSource({ src: await loadAsset('./clip.mp4') });
|
|
101
|
+
|
|
102
|
+
const track = new VisualTrack();
|
|
103
|
+
const img = new ImageClip({ source: web });
|
|
104
|
+
img.start = 0; img.end = 3;
|
|
105
|
+
const vid = new VideoClip({ source: local });
|
|
106
|
+
vid.start = 3; vid.end = 8;
|
|
107
|
+
track.add(img); track.add(vid);
|
|
108
|
+
c.addTrack(track);
|
|
109
|
+
|
|
110
|
+
return { compositor: c };
|
|
111
|
+
});
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
## 6. Custom font (preview == render)
|
|
115
|
+
|
|
116
|
+
```ts
|
|
117
|
+
import { fonts, TextClip } from '@sequio/engine';
|
|
118
|
+
|
|
119
|
+
// Load an explicit web font so Node render matches browser preview
|
|
120
|
+
// (system defaults differ per platform). `src` may be a URL or a data: URL.
|
|
121
|
+
await fonts.load({ family: 'Poppins', src: POPPINS_DATA_URL });
|
|
122
|
+
const title = new TextClip({ text: 'sequio', fontFamily: 'Poppins', fontSize: 72, fill: 0xffffff });
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
## 7. An effect on a clip + a crossfade
|
|
126
|
+
|
|
127
|
+
```ts
|
|
128
|
+
import { BlurEffect, CrossfadeTransition } from '@sequio/engine';
|
|
129
|
+
|
|
130
|
+
clip.effects.add(new BlurEffect({ strength: 8 }));
|
|
131
|
+
// A crossfade between two overlapping clips on a track (overlap drives it):
|
|
132
|
+
track.transition = new CrossfadeTransition({ duration: 0.5 });
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
(Effect/transition constructor options vary — check `references/api.md` and the
|
|
136
|
+
class in `packages/engine/src/effects/`. chroma/LUT/wipe are not yet built.)
|
|
137
|
+
|
|
138
|
+
## 8. Export from your own app (not the CLI)
|
|
139
|
+
|
|
140
|
+
Two ways:
|
|
141
|
+
|
|
142
|
+
- Runtime `Composer`: `const blob = await composer.export({ format: 'mp4' });`
|
|
143
|
+
- Engine `Exporter` directly against a `Compositor`:
|
|
144
|
+
```ts
|
|
145
|
+
import { Exporter } from '@sequio/engine';
|
|
146
|
+
const exporter = new Exporter(compositor, { duration, fps: 30, format: 'mp4' });
|
|
147
|
+
const blob = await exporter.run(); // FixedStep loop, awaits prepare — no dropped frames
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
## 9. Server-side render (no browser)
|
|
151
|
+
|
|
152
|
+
Snapshot a composition's files into a bundle and hand it to Route B:
|
|
153
|
+
|
|
154
|
+
```bash
|
|
155
|
+
pnpm sequio render composition.ts --out out.mp4 --scale 2
|
|
156
|
+
# or, from a RuntimeBundle programmatically:
|
|
157
|
+
# import { renderBundleToFile } from '@sequio/server/route-b';
|
|
158
|
+
# await renderBundleToFile(bundle, { out: 'out.mp4', scale: 2 });
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
Route B is pure Node + PixiJS WebGPU (Dawn); it needs a GPU or the Mesa lavapipe
|
|
162
|
+
software driver.
|
|
163
|
+
|
|
164
|
+
## 10. Quick visual check — export a single frame (no full render)
|
|
165
|
+
|
|
166
|
+
The fast way to confirm a composition looks right without rendering the whole
|
|
167
|
+
video (and without a browser). `sequio frame` seeks to one time and writes a PNG
|
|
168
|
+
through the same render core as `render`:
|
|
169
|
+
|
|
170
|
+
```bash
|
|
171
|
+
sequio frame composition.ts --time 2.5 --out /tmp/check.png # then open/view the PNG
|
|
172
|
+
sequio frame composition.ts --time 0 --out /tmp/start.png --scale 2
|
|
173
|
+
```
|
|
174
|
+
|
|
175
|
+
- `--time <sec>` is clamped to `[0, duration]` (so `--time 999` gives the last frame).
|
|
176
|
+
- `--scale N` renders at N× like `render`.
|
|
177
|
+
- Needs a WebGPU host (GPU or Mesa lavapipe), same as `render`.
|
|
178
|
+
|
|
179
|
+
Recommended iterate loop for an agent: edit the composition → `sequio frame` at a
|
|
180
|
+
few representative times → look at the PNGs → only `sequio render` once it's right.
|
|
181
|
+
Programmatically it's `renderBundleFrameToFile(bundle, { out, time, scale })` from
|
|
182
|
+
`@sequio/server/route-b`, or `runFrame(file, { out, time, scale })` from `@sequio/cli`.
|
|
183
|
+
|
|
184
|
+
## Gotchas
|
|
185
|
+
|
|
186
|
+
- Call `await compositor.init()` before adding tracks.
|
|
187
|
+
- `anchor` is normalized (`[0.5,0.5]` = center); `position` is pixels.
|
|
188
|
+
- Set both `.start` and `.end` on every clip, in seconds.
|
|
189
|
+
- Don't read the previous frame's state inside a render — `render(t)` must be pure.
|
|
190
|
+
- In a long-lived app, `dispose()` what you create; a one-shot render tears down
|
|
191
|
+
for you.
|