storyshot 0.3.1 → 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +40 -10
- package/bin/storyshot.mjs +10 -2
- package/package.json +1 -1
- package/skills/storyshot/SKILL.md +12 -3
- package/src/audio.mjs +7 -1
- package/src/build.mjs +2 -2
- package/src/check.mjs +12 -6
- package/src/compose.mjs +95 -41
- package/src/config.mjs +14 -4
- package/src/inspect.mjs +4 -2
- package/src/outputs.mjs +11 -6
- package/src/record.mjs +135 -16
- package/src/setup.mjs +81 -2
package/README.md
CHANGED
|
@@ -6,7 +6,7 @@ docs.
|
|
|
6
6
|
|
|
7
7
|
> *"make a 15 s promo video of https://myapp.com"*
|
|
8
8
|
|
|
9
|
-
Storyshot gives Cursor, Claude Code
|
|
9
|
+
Storyshot gives coding agents (Cursor, Claude Code, Codex, Gemini CLI, Copilot in VS Code, Windsurf, Cline) what they need to
|
|
10
10
|
do this well. A real browser clicks through your site, and the recording gets
|
|
11
11
|
a smooth cursor, automatic zoom, captions, an optional AI voiceover, music
|
|
12
12
|
and an end card. Loading time is squeezed out, so a 30-second AI job takes a
|
|
@@ -15,6 +15,13 @@ second or two on screen.
|
|
|
15
15
|
The agent plans, writes and fixes the storyboard. Storyshot does the
|
|
16
16
|
recording and editing, and the same storyboard always renders the same video.
|
|
17
17
|
|
|
18
|
+
## Gallery
|
|
19
|
+
|
|
20
|
+
**[boheling.github.io/storyshot-gallery](https://boheling.github.io/storyshot-gallery/)**: promos,
|
|
21
|
+
tutorials, lessons and help-center docs of real web apps (Squoosh, Excalidraw, NCBI BLAST, Desmos,
|
|
22
|
+
regex101, RCSB PDB, Datasette and more, two of them in Chinese). Each one was made by an agent from a
|
|
23
|
+
one-paragraph request, and comes with its storyboard and notes on how it went.
|
|
24
|
+
|
|
18
25
|
## Next to similar tools
|
|
19
26
|
|
|
20
27
|
Storyshot, [promo-video-recorder](https://github.com/palamut62/promo-video-recorder)
|
|
@@ -46,11 +53,28 @@ How each one was made, what we saw, and the caveats:
|
|
|
46
53
|
|
|
47
54
|
## Quick start
|
|
48
55
|
|
|
49
|
-
Open your project in
|
|
56
|
+
Open your project in your coding agent and paste this into its chat, with your
|
|
57
|
+
agent's name (`cursor`, `claude`, `codex`, `gemini`, `vscode`, `windsurf` or `cline`):
|
|
58
|
+
|
|
59
|
+
> Set up Storyshot in this project: run `npx -y storyshot setup cursor` and do
|
|
60
|
+
> everything it says. Then make a short promo video of https://myapp.com
|
|
61
|
+
|
|
62
|
+
| agent | what `setup` writes |
|
|
63
|
+
|---|---|
|
|
64
|
+
| Cursor | `.cursor/mcp.json` and the skill in `.cursor/skills/storyshot/` |
|
|
65
|
+
| Claude Code | `.mcp.json` and the skill in `.claude/skills/storyshot/` |
|
|
66
|
+
| Codex | the skill in `.agents/skills/storyshot/`, a short section in `AGENTS.md` pointing to it, and the MCP server through `codex mcp add` (in `~/.codex/config.toml`, for your user). Storyshot launches a browser, which Codex's sandbox blocks: approve it when Codex asks, or run Codex with `--sandbox danger-full-access` |
|
|
67
|
+
| Gemini CLI | `.gemini/settings.json`, a section in `GEMINI.md`, the skill in `.agents/skills/storyshot/` (trust the folder when Gemini asks, or it ignores project settings) |
|
|
68
|
+
| Copilot in VS Code (`vscode`) | `.vscode/mcp.json`, a section in `.github/copilot-instructions.md`, the skill in `.agents/skills/storyshot/` |
|
|
69
|
+
| Windsurf | `.windsurf/rules/storyshot.md`, the skill in `.agents/skills/storyshot/`, and the MCP server in `~/.codeium/windsurf/mcp_config.json` (for your user) |
|
|
70
|
+
| Cline | `.clinerules/storyshot.md`, the skill in `.agents/skills/storyshot/`, and the MCP server in Cline's own settings in each editor it is installed in (for your user) |
|
|
71
|
+
|
|
72
|
+
Existing files are merged, not replaced: other MCP servers stay, and instruction files get one marked
|
|
73
|
+
Storyshot section that a second `setup` updates instead of duplicating.
|
|
50
74
|
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
75
|
+
**Tested end to end** (setup, then an agent making a real video from a plain request): **Claude Code,
|
|
76
|
+
Cursor and Codex**. Gemini CLI, Copilot in VS Code, Windsurf and Cline get the configuration their docs
|
|
77
|
+
describe, but haven't been tested with a real session yet; reports are welcome.
|
|
54
78
|
|
|
55
79
|
The agent does the rest:
|
|
56
80
|
- `setup` writes the MCP server config and the Storyshot skill, adds
|
|
@@ -61,7 +85,7 @@ The agent does the rest:
|
|
|
61
85
|
- The agent reads the skill and starts on your video in the same chat, using
|
|
62
86
|
the CLI.
|
|
63
87
|
|
|
64
|
-
|
|
88
|
+
Restart or reload your agent when convenient (`setup` says how for each one). From the next chat on,
|
|
65
89
|
the agent uses the Storyshot MCP tools automatically, and you can just ask:
|
|
66
90
|
|
|
67
91
|
- *"make a 15 s promo video of https://myapp.com"*, or type `/storyshot`
|
|
@@ -169,7 +193,7 @@ from `storyshot guide`.
|
|
|
169
193
|
| `check_storyboard` | Validates in about a second: typos with "did you mean", action and target shapes, `TODO`s, keys, the session file. |
|
|
170
194
|
| `render_video` / `render_status` | Renders in the background and returns the MP4 path, warnings, a per-step speed log and the preview image. If the render isn't done within 45 s, it returns a `job_id`, so client tool timeouts never kill a render. |
|
|
171
195
|
|
|
172
|
-
|
|
196
|
+
Any other MCP client: run `npx -y storyshot mcp` (stdio), and point the agent at the playbook (`storyshot guide`). When run from a git
|
|
173
197
|
checkout, `setup` points to `node /path/to/checkout/bin/storyshot.mjs mcp` instead.
|
|
174
198
|
|
|
175
199
|
When a step fails, the error names the step and action, and says whether the
|
|
@@ -192,7 +216,7 @@ Everything the agent does is also a command:
|
|
|
192
216
|
| `storyshot compose videos/acme-promo.yaml [--draft]` | Rebuild from the last recording, in seconds. |
|
|
193
217
|
| `storyshot record videos/acme-promo.yaml` | Only record. |
|
|
194
218
|
| `storyshot guide` | Print the playbook and the reference. |
|
|
195
|
-
| `storyshot setup cursor
|
|
219
|
+
| `storyshot setup <agent>` (`cursor`, `claude`, `codex`, `gemini`, `vscode`, `windsurf`, `cline`) | Set up the MCP server and the skill in this project, and install or check what Storyshot needs. |
|
|
196
220
|
| `storyshot doctor` | Check Node, ffmpeg, Chromium and voices; install Chromium if it's missing. |
|
|
197
221
|
| `storyshot mcp` | Run the MCP server. |
|
|
198
222
|
|
|
@@ -234,7 +258,9 @@ stepNumbers: false # numbered caption badges
|
|
|
234
258
|
subtitles: false # write <out>.srt (narration, else captions); `burn` also burns the narration into the video, each word lighting up as it is spoken (captions move to the top)
|
|
235
259
|
pace: slow # fast (pitch) | normal (docs) | slow (tutorial, course): cursor and typing speed, a beat after each click, hold after narration, voice speed, how much of a wait is shown
|
|
236
260
|
follow: true # tutorial/course default: the camera zooms in on what each step touches and pans between steps (a step's own zoom wins; follow: false on a step skips it); or { scale: 1.8 }
|
|
237
|
-
|
|
261
|
+
article: { title: Export a drawing, intro: … } # docs: heading and first paragraph of <out>-steps/README.md
|
|
262
|
+
cards: auto # intro/section/end cards: auto follows the frame (dark frames get dark cards), or light | dark
|
|
263
|
+
frame: aurora # the recording sits in a browser window on a backdrop: aurora (dark, grain; pitch default), studio (light, grain; tutorial and course default), dark, pastel, blur (the page itself, blurred); false = full frame; or { style, background: <css> }
|
|
238
264
|
chapters: false # write <out>.chapters.txt for YouTube
|
|
239
265
|
clips: null # gif | webm | mp4 | png, or a list (docs default: [gif, png]): one file per step in <out>-steps/, cropped to what the step touches, plus README.md, a ready-to-paste article (step captions, `doc:` text, images)
|
|
240
266
|
maxWait: 1.0 # with duration: auto, the longest a wait is shown
|
|
@@ -293,8 +319,9 @@ You can add `exact: true`, `last: true` or `nth: 2` to any of them.
|
|
|
293
319
|
| action | example |
|
|
294
320
|
|---|---|
|
|
295
321
|
| `click` | `click: { role: button, name: Convert, ms: 800, offset: { x: 0.3, y: 0.5 } }` |
|
|
322
|
+
| `dblclick` | `dblclick: { text: Start }` double-clicks (e.g. to edit a label in place) |
|
|
296
323
|
| `move` / `hover` | `move: { text: "$0" }` or `move: { x: 0.5, y: 0.4, ms: 1200 }` |
|
|
297
|
-
| `type` | `type: { target: { label: Email }, text: hi@acme.com, delay: 60 }`, or `type: hello` to type into whatever has focus |
|
|
324
|
+
| `type` | `type: { target: { label: Email }, text: hi@acme.com, delay: 60 }`, or `type: hello` to type into whatever has focus. `clear: true` empties a pre-filled field first |
|
|
298
325
|
| `drag` | `drag: { from: { css: ".slider" }, by: [{ x: -300 }, { x: 450 }] }` (several moves while held), or `to: <target or point>` |
|
|
299
326
|
| `press` | `press: Enter`, `press: "^"`, `press: { key: Meta+K, show: true }`. When highlighting, shortcuts and special keys show an on-screen badge; plain characters don't. `show: false` hides it. |
|
|
300
327
|
| `pause` | `pause: 700` (ms) |
|
|
@@ -305,9 +332,12 @@ You can add `exact: true`, `last: true` or `nth: 2` to any of them.
|
|
|
305
332
|
| `scrollTo` | `scrollTo: { css: "#pricing", ms: 1200 }` |
|
|
306
333
|
| `scroll` | `scroll: 600` |
|
|
307
334
|
| `dismiss` | `dismiss: { role: button, name: Close }` (clicks only if visible) |
|
|
335
|
+
| `hide` | `hide: '[id*="QSI"], .chat-widget'` hides pop-ups, survey prompts, feedback tabs and chat widgets for the whole video, including ones that appear late (use it in `setup`) |
|
|
308
336
|
| `mark` | `mark: "automatic zoom, captions"` sweeps a highlighter over that phrase, even inside a paragraph (scrolling to it if needed). `mark: { css: "table tr:nth-child(3)" }` marks any target with a box. Options: `ms` (sweep time, default 700), `color` (default: theme accent), `box: true/false`. Marks stay until the step ends. |
|
|
309
337
|
| `unmark` | `unmark: true` clears the marks (and notes) earlier |
|
|
310
338
|
| `note` | `note: { target: { label: Database }, text: Curated proteins only }` puts a short label with an arrow next to the element, on the side with the most room (or `side: left/right/top/bottom`). It stays until the step ends and gets reading time (about 3 words a second, or `ms`). |
|
|
339
|
+
| `paste` | `paste: { target: { label: JSON }, text: '{"a": 1}' }` pastes text in one go through the real clipboard (with a ⌘V badge), so editors react as to a real paste; `paste: "..."` pastes into whatever has focus |
|
|
340
|
+
| `slide` | `slide: { target: { label: Quality }, to: 0.2 }` drags a slider to a position along its track (0 = start, 1 = end), found from the element, not pixels |
|
|
311
341
|
| `select` | `select: { target: { label: Database }, option: "UniProtKB/Swiss-Prot" }` picks an option in a dropdown (by its visible text, or part of it). `inspect` lists each dropdown's options. |
|
|
312
342
|
|
|
313
343
|
`ms` sets how long the cursor takes to travel to the target (`dragMs` for each
|
package/bin/storyshot.mjs
CHANGED
|
@@ -24,7 +24,8 @@ Usage:
|
|
|
24
24
|
(after changing captions, durations, zoom, music, voice, end card)
|
|
25
25
|
storyshot record <storyboard.yaml> only (re-)record the browser session
|
|
26
26
|
storyshot guide print the agent playbook + storyboard reference
|
|
27
|
-
storyshot setup
|
|
27
|
+
storyshot setup <agent> add the MCP server + playbook to this project for your coding agent:
|
|
28
|
+
cursor, claude, codex, gemini, vscode (Copilot), windsurf, cline —
|
|
28
29
|
and install / check everything Storyshot needs
|
|
29
30
|
storyshot doctor check (and install what it can of) Chromium, ffmpeg, voices
|
|
30
31
|
storyshot mcp run as an MCP server (stdio)
|
|
@@ -77,7 +78,14 @@ try {
|
|
|
77
78
|
` show up in this session, use the CLI instead: ${cliCommand()} <command>`,
|
|
78
79
|
arg === 'cursor'
|
|
79
80
|
? ' User: reload Cursor (Cmd/Ctrl+Shift+P → "Reload Window") so the storyshot MCP tools load; future chats use them automatically.'
|
|
80
|
-
:
|
|
81
|
+
: ({
|
|
82
|
+
claude: ' User: restart Claude Code in this project and approve the storyshot MCP server; future sessions use it automatically.',
|
|
83
|
+
codex: ' User: start a new Codex session in this project; it reads AGENTS.md and loads the storyshot MCP server. Storyshot needs to launch a browser, which Codex\'s sandbox blocks: approve it when Codex asks, or start Codex with --sandbox danger-full-access.',
|
|
84
|
+
gemini: ' User: restart Gemini CLI in this project and trust the folder when asked (Gemini ignores project settings in untrusted folders); /mcp then lists the storyshot tools.',
|
|
85
|
+
vscode: ' User: in VS Code, open .vscode/mcp.json and click Start on "storyshot", then use Copilot Chat in Agent mode.',
|
|
86
|
+
windsurf: ' User: in Windsurf, open Cascade → MCP servers and refresh; the storyshot tools then appear.',
|
|
87
|
+
cline: ' User: in Cline, open MCP Servers and make sure "storyshot" is enabled (restart the editor if it is not listed).',
|
|
88
|
+
})[arg],
|
|
81
89
|
].join('\n'));
|
|
82
90
|
} else if (cmd === 'login') {
|
|
83
91
|
const { login, authFileFor } = await import('../src/auth.mjs');
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "storyshot",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.4.0",
|
|
4
4
|
"description": "Turn a short YAML storyboard into a polished product demo video of your web app — smooth cursor, auto zoom, captions, music.",
|
|
5
5
|
"author": "boheling <boheling@gmail.com>",
|
|
6
6
|
"repository": {
|
|
@@ -38,6 +38,12 @@ is missing, a tool says so and gives the install command. Run that command
|
|
|
38
38
|
yourself (with the user's approval), then retry. `storyshot doctor` checks
|
|
39
39
|
everything.
|
|
40
40
|
|
|
41
|
+
**Never fake the video.** Storyshot records the real site in a real browser. If
|
|
42
|
+
the browser can't start (an agent sandbox that blocks it, a missing Chromium),
|
|
43
|
+
or the tools keep failing, stop and tell the user what blocked it and how to
|
|
44
|
+
allow it. Never produce a substitute: no mock-ups, redrawn screens, scripted
|
|
45
|
+
frames or stock footage presented as a recording of their app.
|
|
46
|
+
|
|
41
47
|
## 1. Look, then confirm the brief
|
|
42
48
|
|
|
43
49
|
1. **Look first.** `inspect_page` the start URL (about 5 s). Note what the site
|
|
@@ -166,7 +172,9 @@ small controls are readable. Your job is the structure:
|
|
|
166
172
|
cropped to what it touches and holds the result before looping; a PNG of
|
|
167
173
|
each result and `<out>-steps/README.md` are written too. Give every step a
|
|
168
174
|
short `caption` (the heading) and a `doc:` line (the article text), and
|
|
169
|
-
`note` the field the reader must not miss.
|
|
175
|
+
`note` the field the reader must not miss. Set the article's heading and
|
|
176
|
+
first paragraph with `article: { title: …, intro: … }`. Downloads show a
|
|
177
|
+
"Downloaded" bubble; `paste` and copy-to-clipboard work.
|
|
170
178
|
- **End with a recap**: `end_card` with `bullets` listing the steps or rules
|
|
171
179
|
to remember, and a `say` that repeats them.
|
|
172
180
|
- **Narrating real results.** You can't know the numbers (scores, counts,
|
|
@@ -219,8 +227,9 @@ Rules of thumb from real renders (promos and tutorials):
|
|
|
219
227
|
`mark` the claims as they are spoken, zoom on 2–3 of them, and end on a
|
|
220
228
|
command or URL. If the product has a real flow, show that instead.
|
|
221
229
|
- **Cut boring navigation** with a `hidden: true` step (e.g. `goto: /pricing`).
|
|
222
|
-
- **Clean the frame** in `setup`: dismiss cookie banners
|
|
223
|
-
|
|
230
|
+
- **Clean the frame** in `setup`: dismiss cookie banners and announcement bars
|
|
231
|
+
(`dismiss` is a no-op when the element isn't there), and `hide` survey
|
|
232
|
+
pop-ups, feedback tabs and chat widgets, which often appear late.
|
|
224
233
|
- **Keep the caption off the subject.** Captions sit bottom-centre. When one
|
|
225
234
|
would cover a target the step clicks, hovers, types into or drags, or the
|
|
226
235
|
zoom subject, it moves to the top automatically (with a warning). Other things
|
package/src/audio.mjs
CHANGED
|
@@ -92,8 +92,14 @@ export async function soundtrack(cfg, events, silentVideo, { log = console.log }
|
|
|
92
92
|
|
|
93
93
|
// --- sound effects ------------------------------------------------------------------------
|
|
94
94
|
if (cfg.sfx) {
|
|
95
|
+
// a mouse click: the press (band-passed noise with a small body thump), then a quieter release
|
|
95
96
|
for (const c of events.clicks) {
|
|
96
|
-
|
|
97
|
+
const fa = svf(), fb = svf();
|
|
98
|
+
add(sfx, c, 0.1, (t) => {
|
|
99
|
+
const press = t < 0.02 ? (fa(rand(), 3200, 2.5).bp * 1.4 + Math.sin(2 * Math.PI * 190 * t) * 0.5) * Math.exp(-t * 320) : 0;
|
|
100
|
+
const r = t - 0.075, release = r > 0 && r < 0.015 ? fb(rand(), 4200, 3).bp * 0.8 * Math.exp(-r * 420) : 0;
|
|
101
|
+
return press + release;
|
|
102
|
+
}, 0.55);
|
|
97
103
|
}
|
|
98
104
|
for (const c of events.whooshes) {
|
|
99
105
|
const filt = svf(), len = 0.45;
|
package/src/build.mjs
CHANGED
|
@@ -10,9 +10,9 @@ export async function build(cfg, mode = 'render', { log = console.log } = {}) {
|
|
|
10
10
|
if (mode !== 'compose') await record(cfg, { log });
|
|
11
11
|
if (mode === 'record') return { warnings: [] };
|
|
12
12
|
const voiceLens = await measureVoice(cfg, { log });
|
|
13
|
-
const { silent, events, preview, warnings, captions, cards, segs } = await compose(cfg, { log, voiceLens });
|
|
13
|
+
const { silent, clean, events, preview, warnings, captions, cards, segs } = await compose(cfg, { log, voiceLens });
|
|
14
14
|
const audio = await soundtrack(cfg, events, silent, { log });
|
|
15
15
|
if (cfg.subtitles === 'burn') burnSubtitles(cfg, audio.narration ?? [], { log });
|
|
16
|
-
const extras = await writeOutputs(cfg, { captions, cards, segs, narration: audio.narration, total: events.total }, { log });
|
|
16
|
+
const extras = await writeOutputs(cfg, { captions, cards, segs, clean, narration: audio.narration, total: events.total }, { log });
|
|
17
17
|
return { out: audio.out, extras, preview, warnings: [...warnings, ...audio.warnings], duration: events.total, seconds: Math.round((Date.now() - t0) / 1000) };
|
|
18
18
|
}
|
package/src/check.mjs
CHANGED
|
@@ -4,7 +4,7 @@ import YAML from 'yaml';
|
|
|
4
4
|
import { loadConfig } from './config.mjs';
|
|
5
5
|
import { gitIgnored } from './auth.mjs';
|
|
6
6
|
|
|
7
|
-
const TOP = ['name', 'url', 'out', 'purpose', 'viewport', 'output', 'theme', 'music', 'voice', 'sfx', 'storageState', 'ignoreHTTPSErrors', 'intro', 'frame', 'pace', 'follow', 'colorScheme',
|
|
7
|
+
const TOP = ['name', 'url', 'out', 'purpose', 'viewport', 'output', 'theme', 'music', 'voice', 'sfx', 'storageState', 'ignoreHTTPSErrors', 'intro', 'frame', 'pace', 'follow', 'article', 'cards', 'lang', 'colorScheme',
|
|
8
8
|
'scrollOffset', 'captionPosition', 'highlight', 'stepNumbers', 'subtitles', 'chapters', 'clips', 'maxWait', 'setup', 'steps', 'end_card'];
|
|
9
9
|
const STEP = ['caption', 'say', 'sayAt', 'duration', 'zoom', 'actions', 'hidden', 'sfx', 'highlight', 'captionPosition', 'card', 'follow', 'doc'];
|
|
10
10
|
const ZOOM = ['target', 'scale', 'start', 'end', 'ramp', 'offset'];
|
|
@@ -12,9 +12,9 @@ const END = ['logo', 'title', 'subtitle', 'cta', 'duration', 'say', 'kicker', 'b
|
|
|
12
12
|
const TARGET = ['role', 'name', 'text', 'label', 'placeholder', 'alt', 'testId', 'css', 'exact', 'last', 'nth'];
|
|
13
13
|
const OPTS = ['ms', 'offset', 'timeout', 'highlight', 'color', 'box'];
|
|
14
14
|
const ACTIONS = {
|
|
15
|
-
pause: 'number', wait: 'number', move: 'target', hover: 'target', click: 'target', dismiss: 'target', waitFor: 'target',
|
|
15
|
+
pause: 'number', wait: 'number', move: 'target', hover: 'target', click: 'target', dblclick: 'target', dismiss: 'target', waitFor: 'target',
|
|
16
16
|
waitForUrl: 'string', networkIdle: 'any', goto: 'string', scrollTo: 'target', scroll: 'any', type: 'type', drag: 'drag', press: 'any',
|
|
17
|
-
mark: 'mark', unmark: 'any', note: 'note', select: 'select',
|
|
17
|
+
mark: 'mark', unmark: 'any', note: 'note', select: 'select', hide: 'any', slide: 'slide', paste: 'type',
|
|
18
18
|
};
|
|
19
19
|
|
|
20
20
|
const dist = (a, b) => {
|
|
@@ -73,7 +73,7 @@ export function check(file, { draft = false } = {}) {
|
|
|
73
73
|
// tutorials: one thing per step, so the viewer can follow (and the narration can explain it)
|
|
74
74
|
const purpose = raw.purpose ?? 'pitch';
|
|
75
75
|
if ((purpose === 'tutorial' || purpose === 'course') && !s.hidden) {
|
|
76
|
-
const doing = (s.actions ?? []).filter((a) => a && ['click', 'type', 'drag', 'press', 'goto', 'select'].includes(Object.keys(a)[0]) && !(Object.keys(a)[0] === 'press' && /^(Enter|Tab)$/.test(String(Object.values(a)[0]))));
|
|
76
|
+
const doing = (s.actions ?? []).filter((a) => a && ['click', 'dblclick', 'type', 'drag', 'press', 'goto', 'select', 'slide', 'paste'].includes(Object.keys(a)[0]) && !(Object.keys(a)[0] === 'press' && /^(Enter|Tab)$/.test(String(Object.values(a)[0]))));
|
|
77
77
|
if (doing.length > 2) warnings.push(`${where}: does ${doing.length} things (${doing.map((a) => Object.keys(a)[0]).join(', ')}) in one step — in a ${purpose}, split it so each action gets its own explanation and the viewer can keep up`);
|
|
78
78
|
if (doing.length && !s.say) warnings.push(`${where}: does something but has no say: line — in a ${purpose}, tell the viewer what and why`);
|
|
79
79
|
const kinds = (s.actions ?? []).map((a) => a && Object.keys(a)[0]);
|
|
@@ -84,7 +84,8 @@ export function check(file, { draft = false } = {}) {
|
|
|
84
84
|
}
|
|
85
85
|
if (raw.end_card) unknown(raw.end_card, END, 'end_card');
|
|
86
86
|
if (raw.intro) unknown(raw.intro, END, 'intro');
|
|
87
|
-
if (raw.
|
|
87
|
+
if (raw.article) unknown(raw.article, ['title', 'intro'], 'article');
|
|
88
|
+
if (raw.frame && typeof raw.frame === 'object') unknown(raw.frame, ['background', 'style'], 'frame');
|
|
88
89
|
const todo = (fs.readFileSync(file, 'utf8').split('\n').map((l, i) => [l.replace(/#.*$/, ''), i + 1]).filter(([l]) => /\bTODO\b/.test(l)));
|
|
89
90
|
for (const [, n] of todo) errors.push(`line ${n}: replace the TODO placeholder`);
|
|
90
91
|
|
|
@@ -101,18 +102,23 @@ export function check(file, { draft = false } = {}) {
|
|
|
101
102
|
if (type === 'string' && typeof v !== 'string') errors.push(`${where}: ${kind} takes a string`);
|
|
102
103
|
if (type === 'target') target(v, `${where} (${kind})`);
|
|
103
104
|
if (type === 'mark' && typeof v !== 'string') target(v, `${where} (mark)`); // a string is a phrase to highlight
|
|
105
|
+
if (type === 'slide') {
|
|
106
|
+
if (!v?.target || typeof v?.to !== 'number' || v.to < 0 || v.to > 1) errors.push(`${where}: slide needs { target: <slider>, to: 0..1 } (0 = start of the track, 1 = end)`);
|
|
107
|
+
else target(v.target, `${where} (slide.target)`);
|
|
108
|
+
}
|
|
104
109
|
if (type === 'select') {
|
|
105
110
|
if (!v?.target || v?.option == null) errors.push(`${where}: select needs { target: <dropdown>, option: "visible text of the option" }`);
|
|
106
111
|
else target(v.target, `${where} (select.target)`);
|
|
107
112
|
}
|
|
108
113
|
if (type === 'note') {
|
|
109
114
|
if (!v?.target || !v?.text) errors.push(`${where}: note needs { target: <target>, text: "short label" }`);
|
|
115
|
+
else if (isPoint(v.target)) errors.push(`${where}: a note points at an element, not a screen position — target the element (a label, button, cell…) next to that spot`);
|
|
110
116
|
else { target(v.target, `${where} (note.target)`); unknown(v, ['target', 'text', 'side', 'ms', 'timeout'], `${where} (note)`); }
|
|
111
117
|
if (v?.side && !['auto', 'left', 'right', 'top', 'bottom'].includes(v.side)) errors.push(`${where}: note side must be auto, left, right, top or bottom`);
|
|
112
118
|
}
|
|
113
119
|
if (type === 'type') {
|
|
114
120
|
if (typeof v === 'string' || typeof v === 'number') return;
|
|
115
|
-
if (v?.text == null) errors.push(`${where}:
|
|
121
|
+
if (v?.text == null) errors.push(`${where}: ${kind} needs text — "type: hello" types into the focused element, "type: { target: <target>, text: hello }" clicks the target first`);
|
|
116
122
|
else if (v.target) target(v.target, `${where} (type.target)`);
|
|
117
123
|
}
|
|
118
124
|
if (type === 'drag') {
|
package/src/compose.mjs
CHANGED
|
@@ -116,17 +116,30 @@ export async function compose(cfg, { log = console.log, voiceLens = { steps: {},
|
|
|
116
116
|
const tw = 160, th = 90, rate = 10;
|
|
117
117
|
const buf = execFileSync('ffmpeg', ['-loglevel', 'error', '-framerate', `${FPS}`, '-i', path.join(SEQ, '%05d.jpg'),
|
|
118
118
|
'-vf', `fps=${rate},scale=${tw}:${th},format=gray`, '-f', 'rawvideo', 'pipe:1'], { maxBuffer: 1 << 28 });
|
|
119
|
-
|
|
119
|
+
// plus the share of 8x8 blocks with no detail at all: an empty canvas or image area that is still
|
|
120
|
+
// loading (dark or light) next to toolbars that are already there
|
|
121
|
+
const n = Math.floor(buf.length / (tw * th)), score = [], flat = [];
|
|
120
122
|
for (let f = 0; f < n; f++) {
|
|
121
123
|
let e = 0;
|
|
124
|
+
const blocks = new Float64Array(Math.ceil(tw / 8) * Math.ceil(th / 8));
|
|
122
125
|
for (let y = 0, o = f * tw * th; y < th - 1; y++) for (let x = 0; x < tw - 1; x++) {
|
|
123
|
-
const k = o + y * tw + x;
|
|
124
|
-
e += Math.
|
|
126
|
+
const k = o + y * tw + x, d = Math.abs(buf[k] - buf[k + 1]) + Math.abs(buf[k] - buf[k + tw]);
|
|
127
|
+
e += d; blocks[Math.floor(y / 8) * Math.ceil(tw / 8) + Math.floor(x / 8)] += d;
|
|
125
128
|
}
|
|
126
129
|
score.push(e / (tw * th));
|
|
130
|
+
flat.push(blocks.filter((b) => b / 64 < 0.6).length / blocks.length);
|
|
127
131
|
}
|
|
128
132
|
const median = [...score].sort((a, b) => a - b)[Math.floor(n / 2)] ?? 0;
|
|
129
133
|
const blank = score.map((e) => e < 3 && e < median * 0.4);
|
|
134
|
+
for (const g of segs) {
|
|
135
|
+
// compared with how the step ends (its content loaded): much emptier = still loading. Only right
|
|
136
|
+
// after the step waited for something (a load), so drawing on an empty canvas isn't flagged.
|
|
137
|
+
const last = Math.min(n - 1, Math.max(0, Math.floor(g.o1 * rate) - 1));
|
|
138
|
+
const loading = (tl.steps[g.i].waits ?? []).map(([a, b]) => [toOut(a), toOut(b)]).filter(([a]) => a != null).map(([a, b]) => [a, (b ?? a) + 1.5]);
|
|
139
|
+
for (let f = Math.ceil(g.o0 * rate); f <= last; f++) {
|
|
140
|
+
if (flat[f] > 0.55 && flat[f] - flat[last] > 0.25 && loading.some(([a, b]) => f / rate >= a - 0.1 && f / rate <= b)) blank[f] = true;
|
|
141
|
+
}
|
|
142
|
+
}
|
|
130
143
|
for (const g of segs) {
|
|
131
144
|
let run = 0, from = 0;
|
|
132
145
|
const report = (to) => { if (run / rate >= 0.2) warn(`step ${g.i + 1}: ${(run / rate).toFixed(1)}s of near-blank frames at ${from.toFixed(1)}–${to.toFixed(1)}s (the page is still loading) — move that load into a hidden step, or wait for the loaded content before the step starts`); };
|
|
@@ -172,7 +185,9 @@ export async function compose(cfg, { log = console.log, voiceLens = { steps: {},
|
|
|
172
185
|
if (t && typeof t === 'object' && typeof t.x === 'number' && typeof t.y === 'number' && !t.role && !t.css && !t.text) {
|
|
173
186
|
// a point needs no measuring, so it works even if the zoom was added after recording
|
|
174
187
|
const { width: VW, height: VH } = cfg.viewport;
|
|
175
|
-
|
|
188
|
+
// a point names the middle of the subject: give it some area, so captions keep off it too
|
|
189
|
+
const px = t.x <= 1 ? t.x * VW : t.x, py = t.y <= 1 ? t.y * VH : t.y, bw = VW * 0.12, bh = VH * 0.1;
|
|
190
|
+
g.zoomBox = { x: px - bw / 2, y: py - bh / 2, width: bw, height: bh };
|
|
176
191
|
} else if (t && 'zoomTarget' in tl.steps[g.i] && JSON.stringify(tl.steps[g.i].zoomTarget) !== JSON.stringify(t)) {
|
|
177
192
|
warn(`step ${g.i + 1}: zoom target was added or changed after recording, so its position is unknown — re-render with mode "render" (or use a point target)`);
|
|
178
193
|
g.zoomBox = null;
|
|
@@ -213,7 +228,7 @@ export async function compose(cfg, { log = console.log, voiceLens = { steps: {},
|
|
|
213
228
|
};
|
|
214
229
|
// browser-window frame: the page is drawn at (FX, FY), FW x FH, inside a window on a backdrop
|
|
215
230
|
const even = (x) => 2 * Math.round(x / 2);
|
|
216
|
-
const FW = cfg.frame ? even(W * 0.
|
|
231
|
+
const FW = cfg.frame ? even(W * 0.82) : W, FH = cfg.frame ? even(H * 0.82) : H, BAR = cfg.frame ? Math.round(H * 0.034) : 0;
|
|
217
232
|
const FX = even((W - FW) / 2), FY = cfg.frame ? even((H - FH - BAR) / 2 + BAR) : 0, F = FW / W;
|
|
218
233
|
const onScreen = (b, t) => {
|
|
219
234
|
const v = view(t);
|
|
@@ -280,51 +295,61 @@ export async function compose(cfg, { log = console.log, voiceLens = { steps: {},
|
|
|
280
295
|
warn(`step ${g.i + 1}: the ${want} caption would cover ${hit.what}, so it was moved to the ${other}` + (explicit ? ` (despite captionPosition: ${explicit})` : '') + ` — set captionPosition: ${other} to make this explicit`);
|
|
281
296
|
} else if (hit) warn(`step ${g.i + 1}: the caption covers ${hit.what} at both top and bottom — shorten the caption, or change the zoom`);
|
|
282
297
|
if (prev && n === prev.n && prev.pos === pos) prev.t[1] = g.o1;
|
|
283
|
-
else captions.push({ text: g.step.caption, t: [g.o0, g.o1], pos, n });
|
|
298
|
+
else captions.push({ text: g.step.caption, t: [g.o0, g.o1], pos, n, h });
|
|
284
299
|
}
|
|
285
300
|
for (const [i, c] of captions.entries()) await shot(bubble(c.text, c.n, c.pos), path.join(cfg.work, `cap${i}.png`), true);
|
|
286
301
|
// Intro and end cards are short animations (logo pops, title rises, the highlight sweeps, a command
|
|
287
302
|
// CTA types itself out), rendered frame by frame by seeking the page's CSS animations.
|
|
288
303
|
const u = H / 1080;
|
|
304
|
+
// cards follow the frame: dark backdrops (aurora, dark, blur) get dark cards, so a dark app doesn't
|
|
305
|
+
// cut to a white card; `cards: light | dark` overrides
|
|
306
|
+
const darkCards = cfg.cards === 'dark' || (cfg.cards !== 'light' && ['aurora', 'dark', 'blur'].includes(cfg.frame?.style));
|
|
307
|
+
const CK = darkCards
|
|
308
|
+
? { bg: `radial-gradient(${W * 0.62}px ${H * 0.65}px at 50% 40%,#1e2a44 0%,#111830 60%,#0b1020 100%)`, title: '#f1f5f9', sub: '#cbd5e1', muted: '#94a3b8', item: '#e2e8f0', track: '#26314d',
|
|
309
|
+
pillBg: '#1a2238', pillFg: '#e2e8f0', pillLine: '1px solid #33405f', ctaBg: accent, ctaFg: ink, shadow: 'rgba(0,0,0,.45)' }
|
|
310
|
+
: { bg: `radial-gradient(${W * 0.62}px ${H * 0.65}px at 50% 40%,#ffffff 0%,#eef2f7 60%,#e2e8f0 100%)`, title: ink, sub: '#334155', muted: '#64748b', item: '#1e293b', track: '#e2e8f0',
|
|
311
|
+
pillBg: ink, pillFg: '#e2e8f0', pillLine: 'none', ctaBg: ink, ctaFg: '#fff', shadow: 'rgba(15,23,42,.25)' };
|
|
289
312
|
const card = (c, logoPath, { thinkAt = 0 } = {}) => {
|
|
290
313
|
const image = c.imageFile && fs.existsSync(c.imageFile)
|
|
291
314
|
? `<img class="a rise" src="data:image/${path.extname(c.imageFile).slice(1).replace('jpg', 'jpeg').replace('svg', 'svg+xml')};base64,${fs.readFileSync(c.imageFile).toString('base64')}" style="animation-delay:.4s;max-width:${W * 0.6}px;max-height:${H * 0.42}px;border-radius:${12 * H / 1080}px;box-shadow:0 ${12 * H / 1080}px ${36 * H / 1080}px rgba(15,23,42,.18)">` : '';
|
|
292
|
-
const think = c.think ? `<div class="a rise" style="animation-delay:${thinkAt.toFixed(2)}s;position:absolute;left:50%;bottom:${H * 0.1}px;transform:translateX(-50%);width:${W * 0.3}px;text-align:center;font-size:${26 * H / 1080}px;color
|
|
293
|
-
<div style="margin-bottom:${12 * H / 1080}px"
|
|
294
|
-
<div style="height:${8 * H / 1080}px;border-radius:99px;background
|
|
315
|
+
const think = c.think ? `<div class="a rise" style="animation-delay:${thinkAt.toFixed(2)}s;position:absolute;left:50%;bottom:${H * 0.1}px;transform:translateX(-50%);width:${W * 0.3}px;text-align:center;font-size:${26 * H / 1080}px;color:${CK.muted}">
|
|
316
|
+
<div style="margin-bottom:${12 * H / 1080}px">${cfg.lang === 'zh' ? '先想一想' : 'Pause and think'}</div>
|
|
317
|
+
<div style="height:${8 * H / 1080}px;border-radius:99px;background:${CK.track};overflow:hidden"><div style="height:100%;background:${accent};transform-origin:left;animation:drain ${c.think}s linear ${thinkAt.toFixed(2)}s both"></div></div></div>
|
|
295
318
|
<style>@keyframes drain{from{transform:scaleX(1)}to{transform:scaleX(0)}}</style>` : '';
|
|
296
319
|
const logo = fs.existsSync(logoPath)
|
|
297
|
-
? `<img class="a pop" src="data:image/${path.extname(logoPath).slice(1).replace('jpg', 'jpeg').replace('svg', 'svg+xml')};base64,${fs.readFileSync(logoPath).toString('base64')}" style="height:${Math.round(H * 0.09)}px">` : '';
|
|
320
|
+
? `<img class="a pop" src="data:image/${path.extname(logoPath).slice(1).replace('jpg', 'jpeg').replace('svg', 'svg+xml')};base64,${fs.readFileSync(logoPath).toString('base64')}" style="height:${Math.round(H * 0.09)}px;${darkCards ? `background:#fff;padding:${Math.round(10 * u)}px ${Math.round(16 * u)}px;border-radius:${Math.round(14 * u)}px;box-shadow:0 ${8 * u}px ${24 * u}px rgba(0,0,0,.35)` : ''}">` : ''; // logos captured from a light page often carry its white: give them a plate on dark cards
|
|
298
321
|
const cmd = c.cta && /^\$?\s*(npx|npm|pnpm|yarn|bun|pip|pipx|uv|brew|curl|git|docker|cargo|go)\b/.test(c.cta);
|
|
299
322
|
const ctaText = String(c.cta ?? '').replace(/^\$\s*/, '');
|
|
300
323
|
return `<body style="margin:0;width:${W}px;height:${H}px;${fontCss}
|
|
301
|
-
background
|
|
324
|
+
background:${CK.bg};
|
|
302
325
|
display:flex;flex-direction:column;align-items:center;justify-content:center;gap:${28 * u}px;text-align:center">
|
|
303
326
|
<style>
|
|
304
327
|
.a{animation-fill-mode:both;animation-timing-function:cubic-bezier(.2,.8,.2,1)}
|
|
305
328
|
.pop{animation-name:pop;animation-duration:.55s;animation-delay:.05s}
|
|
306
329
|
.rise{animation-name:rise;animation-duration:.6s}
|
|
307
|
-
|
|
330
|
+
${darkCards
|
|
331
|
+
? `.hl{color:${accent};background:linear-gradient(${accent},${accent}) no-repeat 0 100%/0% ${Math.max(2, Math.round(3 * u))}px;padding-bottom:${Math.round(4 * u)}px;animation:sweep .6s cubic-bezier(.4,0,.2,1) .75s both}`
|
|
332
|
+
: `.hl{background:linear-gradient(transparent 60%,${accent} 60%) no-repeat 0 0/0% 100%;animation:sweep .6s cubic-bezier(.4,0,.2,1) .75s both}`}
|
|
308
333
|
.type{display:inline-block;overflow:hidden;white-space:nowrap;vertical-align:bottom;width:0;animation:type ${(ctaText.length * 0.045).toFixed(2)}s steps(${ctaText.length || 1}) .8s both}
|
|
309
334
|
.caret{display:inline-block;width:.55em;height:1.1em;margin-left:.1em;vertical-align:-.15em;background:${accent};animation:blink 1s steps(1) .8s infinite}
|
|
310
335
|
@keyframes pop{from{opacity:0;transform:scale(.7)}to{opacity:1;transform:none}}
|
|
311
336
|
@keyframes rise{from{opacity:0;transform:translateY(${24 * u}px)}to{opacity:1;transform:none}}
|
|
312
|
-
@keyframes sweep{to{background-size:100% 100%}}
|
|
337
|
+
@keyframes sweep{to{background-size:100% ${darkCards ? `${Math.max(2, Math.round(3 * u))}px` : '100%'}}}
|
|
313
338
|
@keyframes type{to{width:${ctaText.length}ch}}
|
|
314
339
|
@keyframes blink{50%{opacity:0}}
|
|
315
340
|
</style>
|
|
316
|
-
${c.kicker ? `<div class="a rise" style="font-size:${30 * u}px;font-weight:600;letter-spacing:.08em;text-transform:uppercase;color
|
|
341
|
+
${c.kicker ? `<div class="a rise" style="font-size:${30 * u}px;font-weight:600;letter-spacing:.08em;text-transform:uppercase;color:${CK.muted}">${esc(c.kicker)}</div>` : ''}
|
|
317
342
|
<div style="display:flex;align-items:center;gap:${22 * u}px">${logo}
|
|
318
|
-
${c.title ? `<span class="a rise" style="animation-delay:.12s;font-size:${92 * u}px;font-weight:600;letter-spacing:-.03em;color:${
|
|
319
|
-
${c.subtitle ? `<div class="a rise" style="animation-delay:.3s;font-size:${44 * u}px;color
|
|
343
|
+
${c.title ? `<span class="a rise" style="animation-delay:.12s;display:block;max-width:${W * 0.8}px;text-wrap:balance;line-height:1.15;font-size:${(c.title.length > 40 ? 68 : 92) * u}px;font-weight:600;letter-spacing:-.03em;color:${CK.title}">${esc(c.title).replace(/\*\*(.+?)\*\*/g, '<span class="hl">$1</span>')}</span>` : ''}</div>
|
|
344
|
+
${c.subtitle ? `<div class="a rise" style="animation-delay:.3s;font-size:${44 * u}px;color:${CK.sub};letter-spacing:-.01em">${esc(c.subtitle).replace(/\*\*(.+?)\*\*/g, '<span class="hl">$1</span>')}</div>` : ''}
|
|
320
345
|
${image}${think}
|
|
321
346
|
${c.bullets?.length ? `<ol style="margin:${10 * u}px 0 0;padding:0;list-style:none;text-align:left;display:flex;flex-direction:column;gap:${18 * u}px">${c.bullets.map((b, k) =>
|
|
322
|
-
`<li class="a rise" style="animation-delay:${(0.6 + 0.4 * k).toFixed(2)}s;display:flex;align-items:center;gap:${20 * u}px;font-size:${38 * u}px;color
|
|
347
|
+
`<li class="a rise" style="animation-delay:${(0.6 + 0.4 * k).toFixed(2)}s;display:flex;align-items:center;gap:${20 * u}px;font-size:${38 * u}px;color:${CK.item}">
|
|
323
348
|
<span style="flex:none;display:inline-flex;align-items:center;justify-content:center;width:${52 * u}px;height:${52 * u}px;border-radius:50%;background:${accent};color:${ink};font-weight:700;font-size:${28 * u}px">${k + 1}</span>
|
|
324
|
-
<span>${md(b, ink)}</span></li>`).join('')}</ol>` : ''}
|
|
349
|
+
<span>${md(b, darkCards ? accent : ink)}</span></li>`).join('')}</ol>` : ''}
|
|
325
350
|
${!c.cta ? '' : cmd
|
|
326
|
-
? `<div class="a rise" style="animation-delay:.45s;margin-top:${18 * u}px;font:500 ${36 * u}px ui-monospace,SFMono-Regular,Menlo,monospace;color
|
|
327
|
-
: `<div class="a rise" style="animation-delay:.45s;margin-top:${18 * u}px;font-size:${38 * u}px;font-weight:600;color
|
|
351
|
+
? `<div class="a rise" style="animation-delay:.45s;margin-top:${18 * u}px;font:500 ${36 * u}px ui-monospace,SFMono-Regular,Menlo,monospace;color:${CK.pillFg};background:${CK.pillBg};border:${CK.pillLine};padding:${20 * u}px ${36 * u}px;border-radius:${16 * u}px;text-align:left;box-shadow:0 ${16 * u}px ${40 * u}px ${CK.shadow}"><span style="color:${accent}">$</span> <span class="type">${esc(ctaText)}</span><span class="caret"></span></div>`
|
|
352
|
+
: `<div class="a rise" style="animation-delay:.45s;margin-top:${18 * u}px;font-size:${38 * u}px;font-weight:600;color:${CK.ctaFg};background:${CK.ctaBg};padding:${20 * u}px ${46 * u}px;border-radius:${16 * u}px">${esc(c.cta)}</div>`}
|
|
328
353
|
</body>`;
|
|
329
354
|
};
|
|
330
355
|
const animate = async (html, seconds, dir) => {
|
|
@@ -343,20 +368,32 @@ export async function compose(cfg, { log = console.log, voiceLens = { steps: {},
|
|
|
343
368
|
for (const [k, c] of cards.entries()) await animate(card(c.card, c.card.logoFile ?? '', { thinkAt: c.thinkAt }), c.t1 - c.t0, path.join(cfg.work, `card${k}`));
|
|
344
369
|
if (in0) await animate(card(in0, in0.logoFile ?? path.join(cfg.work, 'intro-logo.png')), INTRO, path.join(cfg.work, 'intro'));
|
|
345
370
|
if (cfg.frame) {
|
|
346
|
-
//
|
|
347
|
-
|
|
348
|
-
const
|
|
349
|
-
const
|
|
350
|
-
|
|
351
|
-
|
|
352
|
-
|
|
371
|
+
// three layers: the backdrop (a picture, or the page itself blurred), the window chrome (shadow and
|
|
372
|
+
// title bar), and the page with rounded bottom corners
|
|
373
|
+
const grain = `url('data:image/svg+xml,${encodeURIComponent('<svg xmlns="http://www.w3.org/2000/svg" width="200" height="200"><filter id="n"><feTurbulence type="fractalNoise" baseFrequency=".9" numOctaves="2" stitchTiles="stitch"/><feColorMatrix values="0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 .09 0"/></filter><rect width="200" height="200" filter="url(#n)"/></svg>')}')`;
|
|
374
|
+
const STYLES = {
|
|
375
|
+
aurora: { dark: true, bg: `${grain}, radial-gradient(${W * 0.47}px ${H * 0.65}px at 20% 20%, #22d3ee99, transparent 60%), radial-gradient(${W * 0.47}px ${H * 0.65}px at 80% 30%, #a78bfa99, transparent 60%), radial-gradient(${W * 0.52}px ${H * 0.74}px at 50% 100%, #34d39988, transparent 60%), #0b1020` },
|
|
376
|
+
studio: { dark: false, bg: `${grain}, radial-gradient(${W * 0.73}px ${H * 0.83}px at 50% 0%, #ffffff, #e9edf3 60%, #d6dce6)` },
|
|
377
|
+
dark: { dark: true, bg: `radial-gradient(${W * 0.7}px ${H * 0.8}px at 15% 0%, ${accent}66, transparent 70%), radial-gradient(${W * 0.7}px ${H * 0.8}px at 100% 100%, #6366f166, transparent 70%), linear-gradient(135deg, #1e293b, ${ink})` },
|
|
378
|
+
pastel: { dark: false, bg: 'linear-gradient(135deg, #fde2e4 0%, #e2ece9 35%, #dfe7fd 70%, #f0e6ff 100%)' },
|
|
379
|
+
blur: { dark: true, bg: null },
|
|
380
|
+
};
|
|
381
|
+
const st = STYLES[cfg.frame.style], bg = cfg.frame.background ?? st.bg;
|
|
382
|
+
const R = Math.round(14 * u), host = new URL(cfg.url).host.replace(/^www\./, '') + new URL(cfg.url).pathname.replace(/\/$/, '');
|
|
383
|
+
const dot = (col) => `<span style="width:${13 * u}px;height:${13 * u}px;border-radius:50%;background:${col};display:inline-block"></span>`;
|
|
384
|
+
const [barBg, barFg, pill, line] = st.dark ? ['#1f2430', '#9aa4b2', '#2b3140', '#11151d'] : ['#e8ebef', '#4b5563', '#fff', '#d5d9df'];
|
|
385
|
+
if (bg) await shot(`<body style="margin:0;width:${W}px;height:${H}px;background:${bg}"></body>`, path.join(cfg.work, 'frame-bg.png'), false);
|
|
386
|
+
await shot(`<body style="margin:0;width:${W}px;height:${H}px;background:transparent;${fontCss}">
|
|
387
|
+
<div style="position:absolute;left:${FX}px;top:${FY - BAR}px;width:${FW}px;height:${FH + BAR}px;border-radius:${R}px;background:#fff;box-shadow:0 ${30 * u}px ${80 * u}px rgba(0,0,0,${st.dark ? '.5' : '.25'}),0 0 0 1px rgba(255,255,255,.08)"></div>
|
|
388
|
+
<div style="position:absolute;left:${FX}px;top:${FY - BAR}px;width:${FW}px;height:${BAR}px;border-radius:${R}px ${R}px 0 0;background:${barBg};display:flex;align-items:center;border-bottom:1px solid ${line}">
|
|
353
389
|
<div style="display:flex;gap:${8 * u}px;margin-left:${16 * u}px">${dot('#ff5f57')}${dot('#febc2e')}${dot('#28c840')}</div>
|
|
354
|
-
<div style="position:absolute;left:50%;transform:translateX(-50%);font-size:${15 * u}px;color
|
|
355
|
-
</div></body>`, path.join(cfg.work, 'frame-
|
|
356
|
-
|
|
357
|
-
await
|
|
358
|
-
|
|
359
|
-
|
|
390
|
+
<div style="position:absolute;left:50%;transform:translateX(-50%);font-size:${15 * u}px;color:${barFg};background:${pill};border-radius:${8 * u}px;padding:${4 * u}px ${60 * u}px">${esc(host)}</div>
|
|
391
|
+
</div></body>`, path.join(cfg.work, 'frame-chrome.png'), true);
|
|
392
|
+
// alpha mask for the page: rounded bottom corners (the title bar covers the top ones)
|
|
393
|
+
await pg.setViewportSize({ width: FW, height: FH });
|
|
394
|
+
await pg.setContent(`<body style="margin:0;width:${FW}px;height:${FH}px;background:#000"><div style="width:100%;height:100%;background:#fff;border-radius:0 0 ${R}px ${R}px"></div></body>`);
|
|
395
|
+
await pg.screenshot({ path: path.join(cfg.work, 'frame-mask.png') });
|
|
396
|
+
await pg.setViewportSize({ width: W, height: H });
|
|
360
397
|
}
|
|
361
398
|
await browser.close();
|
|
362
399
|
|
|
@@ -367,11 +404,23 @@ export async function compose(cfg, { log = console.log, voiceLens = { steps: {},
|
|
|
367
404
|
`:d=1:fps=${FPS}:s=${W}x${H},format=yuv420p${END ? `,tpad=stop_mode=clone:stop_duration=${END}` : ''}[v0];\n`;
|
|
368
405
|
let last = 'v0', idx = 1;
|
|
369
406
|
if (cfg.frame) {
|
|
370
|
-
inputs.push('-loop', '1', '-t', `${TOTAL}`, '-framerate', `${FPS}`, '-i', path.join(cfg.work,
|
|
371
|
-
|
|
372
|
-
|
|
373
|
-
|
|
407
|
+
const still = (f) => inputs.push('-loop', '1', '-t', `${TOTAL}`, '-framerate', `${FPS}`, '-i', path.join(cfg.work, f));
|
|
408
|
+
const blur = !cfg.frame.background && cfg.frame.style === 'blur';
|
|
409
|
+
let base;
|
|
410
|
+
if (blur) {
|
|
411
|
+
graph += `[v0]split[v0a][v0b];[v0b]scale=${even(W / 12)}:${even(H / 12)},gblur=sigma=4,scale=${W}:${H},eq=brightness=-0.12:saturation=1.3[fbg];\n`;
|
|
412
|
+
base = 'fbg';
|
|
413
|
+
} else { still('frame-bg.png'); base = `${idx++}:v`; }
|
|
414
|
+
still('frame-chrome.png'); const chrome = idx++;
|
|
415
|
+
still('frame-mask.png'); const mask = idx++;
|
|
416
|
+
graph += `[${blur ? 'v0a' : 'v0'}]scale=${FW}:${FH}:flags=lanczos,format=rgba[pg0];[${mask}:v]format=gray,scale=${FW}:${FH}[pm];[pg0][pm]alphamerge[pg];\n` +
|
|
417
|
+
`[${base}][${chrome}:v]overlay=0:0[fc];[fc][pg]overlay=${FX}:${FY},format=yuv420p[vf];\n`;
|
|
418
|
+
last = 'vf';
|
|
374
419
|
}
|
|
420
|
+
// docs: a second, caption-free copy for the per-step clips and stills (the article carries the
|
|
421
|
+
// text), so crops never have to dodge a caption
|
|
422
|
+
const clean = cfg.clips.length ? path.join(cfg.work, 'video-clean.mp4') : null;
|
|
423
|
+
if (clean) { graph += `[${last}]split[vmain][vclean0];\n`; last = 'vmain'; }
|
|
375
424
|
for (const [i, c] of captions.entries()) {
|
|
376
425
|
const a = c.t[0] + 0.1, b = c.t[1] - 0.1;
|
|
377
426
|
// each caption is an input only as long as it is on screen (decoding a looped PNG for the whole
|
|
@@ -401,6 +450,7 @@ export async function compose(cfg, { log = console.log, voiceLens = { steps: {},
|
|
|
401
450
|
last = `v${idx}`;
|
|
402
451
|
}
|
|
403
452
|
graph += `[${last}]trim=duration=${TOTAL},format=yuv420p[out]`;
|
|
453
|
+
if (clean) graph += `;\n[vclean0]trim=duration=${TOTAL},format=yuv420p[clean]`;
|
|
404
454
|
const graphFile = path.join(cfg.work, 'graph.txt');
|
|
405
455
|
fs.writeFileSync(graphFile, graph);
|
|
406
456
|
const silent = path.join(cfg.work, 'video-silent.mp4');
|
|
@@ -409,7 +459,8 @@ export async function compose(cfg, { log = console.log, voiceLens = { steps: {},
|
|
|
409
459
|
const major = Number((execFileSync('ffmpeg', ['-version']).toString().match(/ffmpeg version n?(\d+)/) ?? [])[1] ?? 7);
|
|
410
460
|
const graphArgs = major >= 7 ? ['-/filter_complex', graphFile] : ['-filter_complex_script', graphFile];
|
|
411
461
|
execFileSync('ffmpeg', ['-y', '-loglevel', 'error', ...inputs, ...graphArgs, '-map', '[out]',
|
|
412
|
-
'-c:v', 'libx264', '-preset', 'slow', '-crf', '18', '-pix_fmt', 'yuv420p', '-r', `${FPS}`, silent
|
|
462
|
+
'-c:v', 'libx264', '-preset', 'slow', '-crf', '18', '-pix_fmt', 'yuv420p', '-r', `${FPS}`, silent,
|
|
463
|
+
...(clean ? ['-map', '[clean]', '-c:v', 'libx264', '-preset', 'veryfast', '-crf', '18', '-pix_fmt', 'yuv420p', '-r', `${FPS}`, clean] : [])]);
|
|
413
464
|
|
|
414
465
|
// --- 6. preview contact sheet: one frame per step (at its zoom peak) + end card ------------
|
|
415
466
|
// zoomed steps: at the zoom peak; others: late in the step, where its actions' result is visible
|
|
@@ -441,7 +492,10 @@ export async function compose(cfg, { log = console.log, voiceLens = { steps: {},
|
|
|
441
492
|
endCard: ec ? VIDEO : null,
|
|
442
493
|
clicks: tl.clicks.map(toOut).filter((t) => t != null),
|
|
443
494
|
whooshes: dedupe([
|
|
444
|
-
|
|
495
|
+
// whoosh on deliberate scene changes (goto) only: a page the app opens after a click, or a
|
|
496
|
+
// results page reloading itself while you wait, is not a scene change
|
|
497
|
+
...(tl.gotos ?? tl.navs.filter((n) => !tl.clicks.some((c) => n - c >= 0 && n - c < 2) && !tl.steps.some((s) => (s.waits ?? []).some(([a, b]) => n >= a - 0.5 && n <= b + 0.5))))
|
|
498
|
+
.map(toOut).filter((t) => t != null && t > 0.2),
|
|
445
499
|
...segs.filter((g) => g.afterHidden).map((g) => g.o0),
|
|
446
500
|
...(in0 ? [INTRO - 0.3] : []),
|
|
447
501
|
...cards.flatMap((c) => [c.t0, c.t1 - 0.3]),
|
|
@@ -472,8 +526,8 @@ export async function compose(cfg, { log = console.log, voiceLens = { steps: {},
|
|
|
472
526
|
let w = Math.max(x1 - x0 + 2 * pad, W * 0.5), h = Math.max(y1 - y0 + 2 * pad, H * 0.5);
|
|
473
527
|
if (w / h > W / H) h = w * H / W; else w = h * W / H;
|
|
474
528
|
w = Math.min(W, w); h = Math.min(H, h);
|
|
475
|
-
|
|
529
|
+
let cx = (x0 + x1) / 2, cy = (y0 + y1) / 2;
|
|
476
530
|
g.crop = { x: even(Math.max(0, Math.min(W - w, cx - w / 2))), y: even(Math.max(0, Math.min(H - h, cy - h / 2))), w: even(w), h: even(h) };
|
|
477
531
|
}
|
|
478
|
-
return { silent, events, preview, warnings, captions, cards: cards.map((c) => ({ t: c.t0, title: [c.card.kicker, c.card.title].filter(Boolean).join(' · ') })), segs: segs.map((g) => ({ i: g.i, o0: g.o0, o1: g.o1, caption: g.step.caption, doc: g.step.doc ?? g.step.say, crop: g.crop })) };
|
|
532
|
+
return { silent, clean, events, preview, warnings, captions, cards: cards.map((c) => ({ t: c.t0, title: [c.card.kicker, c.card.title].filter(Boolean).join(' · ') })), segs: segs.map((g) => ({ i: g.i, o0: g.o0, o1: g.o1, caption: g.step.caption, doc: g.step.doc ?? g.step.say, crop: g.crop })) };
|
|
479
533
|
}
|
package/src/config.mjs
CHANGED
|
@@ -19,9 +19,9 @@ export function loadEnv(dir) {
|
|
|
19
19
|
|
|
20
20
|
// What the video is for sets sensible defaults; anything in the YAML overrides them.
|
|
21
21
|
export const PURPOSES = {
|
|
22
|
-
pitch: { duration: null, music: { bpm: 120, volume: 1 }, sfx: true, highlight: false, stepNumbers: false, subtitles: false, chapters: false, clips: null, frame:
|
|
23
|
-
tutorial: { duration: 'auto', music: { bpm: 96, volume: 0.35 }, sfx: true, highlight: true, stepNumbers: true, subtitles: true, chapters: true, clips: null, frame:
|
|
24
|
-
course: { duration: 'auto', music: false, sfx: true, highlight: true, stepNumbers: false, subtitles: true, chapters: true, clips: null, frame:
|
|
22
|
+
pitch: { duration: null, music: { bpm: 120, volume: 1 }, sfx: true, highlight: false, stepNumbers: false, subtitles: false, chapters: false, clips: null, frame: 'aurora', pace: 'fast', follow: false },
|
|
23
|
+
tutorial: { duration: 'auto', music: { bpm: 96, volume: 0.35 }, sfx: true, highlight: true, stepNumbers: true, subtitles: true, chapters: true, clips: null, frame: 'studio', pace: 'slow', follow: true },
|
|
24
|
+
course: { duration: 'auto', music: false, sfx: true, highlight: true, stepNumbers: false, subtitles: true, chapters: true, clips: null, frame: 'studio', pace: 'slow', follow: true },
|
|
25
25
|
docs: { duration: 'auto', music: false, sfx: false, highlight: true, stepNumbers: true, subtitles: false, chapters: false, clips: ['gif', 'png'], frame: false, pace: 'normal', follow: false },
|
|
26
26
|
};
|
|
27
27
|
|
|
@@ -36,6 +36,8 @@ export const PACES = {
|
|
|
36
36
|
slow: { cursor: 1200, type: 95, beat: 800, tail: 1.6, maxWait: 3.0, voiceSpeed: 0.92 },
|
|
37
37
|
};
|
|
38
38
|
|
|
39
|
+
export const FRAME_STYLES = ['aurora', 'studio', 'dark', 'pastel', 'blur'];
|
|
40
|
+
|
|
39
41
|
export const VIDEO_DIR = 'videos';
|
|
40
42
|
export const KINDS = { pitch: 'promo', tutorial: 'tutorial', course: 'lesson', docs: 'docs' };
|
|
41
43
|
// pxgenius.ai → pxgenius; an IP or a port is kept whole so two internal apps don't share a name
|
|
@@ -110,7 +112,11 @@ export function loadConfig(file, { draft = false } = {}) {
|
|
|
110
112
|
// follow: the camera zooms in on what each step touches and pans from step to step (tutorials)
|
|
111
113
|
const follow0 = pick('follow');
|
|
112
114
|
const follow = follow0 ? { scale: 1.8, ...(typeof follow0 === 'object' ? follow0 : {}) } : null;
|
|
113
|
-
|
|
115
|
+
// frame: false | true | a style name | { style, background: <css> }. Styles: aurora (dark, grain),
|
|
116
|
+
// studio (light, grain), dark, pastel, blur (the page itself, blurred, behind the window)
|
|
117
|
+
const styleOf = (f) => (typeof f === 'string' ? f : f?.style) ?? (typeof P.frame === 'string' ? P.frame : 'aurora');
|
|
118
|
+
const frame = frame0 ? { background: null, ...(typeof frame0 === 'object' ? frame0 : {}), style: styleOf(frame0) } : null;
|
|
119
|
+
if (frame && !FRAME_STYLES.includes(frame.style)) throw new Error(`frame style must be one of ${FRAME_STYLES.join(', ')}`);
|
|
114
120
|
|
|
115
121
|
let voice = null;
|
|
116
122
|
if (raw.voice) {
|
|
@@ -155,6 +161,10 @@ export function loadConfig(file, { draft = false } = {}) {
|
|
|
155
161
|
steps,
|
|
156
162
|
endCard,
|
|
157
163
|
intro,
|
|
164
|
+
article: raw.article ?? null,
|
|
165
|
+
cards: raw.cards ?? 'auto',
|
|
166
|
+
// language of the built-in labels ("Pause and think", "Introduction"): `lang`, else guessed from the captions
|
|
167
|
+
lang: raw.lang ?? (/[\u3040-\u30ff\u3400-\u9fff\uac00-\ud7af]/.test(JSON.stringify(raw.steps ?? [])) ? 'zh' : 'en'),
|
|
158
168
|
frame,
|
|
159
169
|
follow,
|
|
160
170
|
music,
|
package/src/inspect.mjs
CHANGED
|
@@ -118,8 +118,10 @@ export async function describePage(page, viewport, { maxElements = 80 } = {}) {
|
|
|
118
118
|
seen.add(key);
|
|
119
119
|
let target = null;
|
|
120
120
|
if (el.kind === 'role') {
|
|
121
|
+
// sliders, inputs and dropdowns: their label is stable, their value (often their name) is not
|
|
122
|
+
if (['slider', 'textbox', 'combobox', 'searchbox', 'spinbutton'].includes(el.role) && el.label) target = await verify(page.getByLabel(el.label, { exact: true }), { label: el.label, exact: true }, el);
|
|
121
123
|
// our name guess can differ from the browser's accessible name, so fall back gracefully
|
|
122
|
-
if (el.name) target = await verify(page.getByRole(el.role, { name: el.name, exact: true }), { role: el.role, name: el.name, exact: true }, el);
|
|
124
|
+
if (!target && el.name) target = await verify(page.getByRole(el.role, { name: el.name, exact: true }), { role: el.role, name: el.name, exact: true }, el);
|
|
123
125
|
if (!target && el.alt) target = await verify(page.getByAltText(el.alt, { exact: true }), { alt: el.alt, exact: true }, el);
|
|
124
126
|
if (!target && el.name) target = await verify(page.getByRole(el.role, { name: el.name }), { role: el.role, name: el.name }, el);
|
|
125
127
|
if (!target && el.name) target = await verify(page.getByText(el.name, { exact: true }), { text: el.name, exact: true }, el);
|
|
@@ -156,7 +158,7 @@ export async function describePage(page, viewport, { maxElements = 80 } = {}) {
|
|
|
156
158
|
'',
|
|
157
159
|
'## Clickable elements — verified targets (each matches exactly one visible element) for click/move/zoom',
|
|
158
160
|
' Copy targets exactly; `name` is the accessible name, which may differ from the on-screen label.',
|
|
159
|
-
...elements.map((e) => `${fold(e.y)} y=${e.y}\tat (${e.cx}, ${e.cy})\t${yaml(e.target)}${e.role === 'textbox' ? ` (text input ${e.w}×${e.h} — click it, then \`type: …\`)` : e.options ? ` (dropdown — \`select: { target: …, option: "…" }\`; options: ${e.options.map((o) => `"${o.text}"${o.on ? ' (selected)' : ''}`).join(', ')})` : shows(e)}${e.href ? ` → ${e.href}` : ''}${e.kind === 'pointer' ? ' (custom clickable)' : ''}`),
|
|
161
|
+
...elements.map((e) => `${fold(e.y)} y=${e.y}\tat (${e.cx}, ${e.cy})\t${yaml(e.target)}${e.role === 'textbox' ? ` (text input ${e.w}×${e.h} — click it, then \`type: …\`)` : e.role === 'slider' ? ' (slider — `slide: { target: …, to: 0..1 }`)' : e.options ? ` (dropdown — \`select: { target: …, option: "…" }\`; options: ${e.options.map((o) => `"${o.text}"${o.on ? ' (selected)' : ''}`).join(', ')})` : shows(e)}${e.href ? ` → ${e.href}` : ''}${e.kind === 'pointer' ? ' (custom clickable)' : ''}`),
|
|
160
162
|
'',
|
|
161
163
|
'## Images with alt text (target: { alt: "..." })',
|
|
162
164
|
...(info.images.length ? info.images.map((i) => `${fold(i.y)} y=${i.y}\t"${i.alt}"`) : ['(none)']),
|
package/src/outputs.mjs
CHANGED
|
@@ -11,7 +11,7 @@ const srtTime = (t) => {
|
|
|
11
11
|
const clock = (t) => `${Math.floor(t / 60)}:${String(Math.floor(t % 60)).padStart(2, '0')}`;
|
|
12
12
|
const slug = (s) => plain(s).toLowerCase().replace(/[^a-z0-9一-鿿]+/g, '-').replace(/^-|-$/g, '').slice(0, 40);
|
|
13
13
|
|
|
14
|
-
export async function writeOutputs(cfg, { captions, cards = [], segs, narration, total }, { log = console.log } = {}) {
|
|
14
|
+
export async function writeOutputs(cfg, { captions, cards = [], segs, clean, narration, total }, { log = console.log } = {}) {
|
|
15
15
|
const base = cfg.out.replace(/\.[^.]+$/, '');
|
|
16
16
|
const files = [];
|
|
17
17
|
|
|
@@ -27,7 +27,7 @@ export async function writeOutputs(cfg, { captions, cards = [], segs, narration,
|
|
|
27
27
|
if (cfg.chapters && (captions.length || cards.length)) {
|
|
28
28
|
// section cards make the chapters when there are any; otherwise one chapter per caption
|
|
29
29
|
const lines = cards.length
|
|
30
|
-
? [...(cards[0].t > 1 ? [`0:00 Introduction`] : []), ...cards.map((c, k) => `${clock(k === 0 && cards[0].t <= 1 ? 0 : c.t)} ${plain(c.title)}`)]
|
|
30
|
+
? [...(cards[0].t > 1 ? [`0:00 ${cfg.lang === 'zh' ? '开场' : 'Introduction'}`] : []), ...cards.map((c, k) => `${clock(k === 0 && cards[0].t <= 1 ? 0 : c.t)} ${plain(c.title)}`)]
|
|
31
31
|
: captions.map((c, k) => `${clock(k === 0 ? 0 : c.t[0])} ${cfg.stepNumbers ? `${c.n}. ` : ''}${plain(c.text)}`);
|
|
32
32
|
fs.writeFileSync(`${base}.chapters.txt`, `${lines.join('\n')}\n`);
|
|
33
33
|
files.push(`${base}.chapters.txt`);
|
|
@@ -40,12 +40,17 @@ export async function writeOutputs(cfg, { captions, cards = [], segs, narration,
|
|
|
40
40
|
const dir = `${base}-steps`;
|
|
41
41
|
fs.rmSync(dir, { recursive: true, force: true });
|
|
42
42
|
fs.mkdirSync(dir, { recursive: true });
|
|
43
|
-
|
|
43
|
+
// title and intro: `article: { title, intro }`, else the intro card, else the name in words
|
|
44
|
+
const art = cfg.article ?? {};
|
|
45
|
+
const title = art.title ?? cfg.intro?.title ?? cfg.name.replace(/[-_]+/g, ' ').replace(/^./, (c) => c.toUpperCase());
|
|
46
|
+
const intro = art.intro ?? cfg.intro?.subtitle;
|
|
47
|
+
const article = [`# ${plain(title)}`, '', ...(intro ? [String(intro), ''] : [])];
|
|
44
48
|
for (const [k, g] of segs.entries()) {
|
|
45
49
|
const stem = `${String(k + 1).padStart(2, '0')}${g.caption ? `-${slug(g.caption)}` : ''}`;
|
|
46
50
|
const crop = g.crop ? `crop=${g.crop.w}:${g.crop.h}:${g.crop.x}:${g.crop.y},` : '';
|
|
47
51
|
const cw = g.crop?.w ?? cfg.output.width;
|
|
48
|
-
const
|
|
52
|
+
const src = clean && fs.existsSync(clean) ? clean : cfg.out; // the caption-free copy
|
|
53
|
+
const span = ['-ss', g.o0.toFixed(3), '-t', (g.o1 - g.o0).toFixed(3), '-i', src];
|
|
49
54
|
const made = [];
|
|
50
55
|
for (const kind of cfg.clips) {
|
|
51
56
|
const f = path.join(dir, `${stem}.${kind}`);
|
|
@@ -54,7 +59,7 @@ export async function writeOutputs(cfg, { captions, cards = [], segs, narration,
|
|
|
54
59
|
execFileSync('ffmpeg', ['-y', '-loglevel', 'error', ...span, '-filter_complex',
|
|
55
60
|
`${crop}fps=12,scale=${w}:-1:flags=lanczos,split[a][b];[a]palettegen=max_colors=160:stats_mode=diff[p];[b][p]paletteuse=dither=sierra2_4a:diff_mode=rectangle`, f]);
|
|
56
61
|
} else if (kind === 'png') {
|
|
57
|
-
execFileSync('ffmpeg', ['-y', '-loglevel', 'error', '-ss', Math.max(g.o0, g.o1 - 0.2).toFixed(3), '-i',
|
|
62
|
+
execFileSync('ffmpeg', ['-y', '-loglevel', 'error', '-ss', Math.max(g.o0, g.o1 - 0.2).toFixed(3), '-i', src, '-frames:v', '1', ...(crop ? ['-vf', crop.slice(0, -1)] : []), f]);
|
|
58
63
|
} else if (kind === 'webm') {
|
|
59
64
|
execFileSync('ffmpeg', ['-y', '-loglevel', 'error', ...span, '-an', '-vf', `${crop}scale=${Math.min(cw, 1280)}:-2`, '-c:v', 'libvpx-vp9', '-b:v', '0', '-crf', '34', '-row-mt', '1', f]);
|
|
60
65
|
} else {
|
|
@@ -62,7 +67,7 @@ export async function writeOutputs(cfg, { captions, cards = [], segs, narration,
|
|
|
62
67
|
}
|
|
63
68
|
files.push(f); made.push(path.basename(f));
|
|
64
69
|
}
|
|
65
|
-
article.push(`${k + 1}. **${plain(g.caption ?? `Step ${k + 1}`)}**${g.doc ? `\n\n ${
|
|
70
|
+
article.push(`${k + 1}. **${plain(g.caption ?? `Step ${k + 1}`)}**${g.doc ? `\n\n ${String(g.doc)}` : ''}`, ''); // doc text keeps its **bold**
|
|
66
71
|
const show = made.find((m) => m.endsWith('.gif') || m.endsWith('.png'));
|
|
67
72
|
if (show) article.push(` })`, '');
|
|
68
73
|
}
|
package/src/record.mjs
CHANGED
|
@@ -233,13 +233,52 @@ function pageScript({ scrollOffset }) {
|
|
|
233
233
|
const x0 = Math.min(r.left, p.left), y0 = Math.min(r.top, p.top);
|
|
234
234
|
return { x: x0, y: y0, width: Math.max(r.right, p.right) - x0, height: Math.max(r.bottom, p.bottom) - y0 };
|
|
235
235
|
};
|
|
236
|
-
|
|
236
|
+
// native <select> popups are drawn by the OS, not the page, so they never appear in a recording:
|
|
237
|
+
// draw a look-alike menu under the dropdown, return where the chosen option is
|
|
238
|
+
window.__demoMenu = (r, items, pick) => {
|
|
239
|
+
document.getElementById('__demo_menu')?.remove();
|
|
240
|
+
const m = document.createElement('div');
|
|
241
|
+
m.id = '__demo_menu';
|
|
242
|
+
const fs = Math.max(13, Math.min(18, r.height * 0.6));
|
|
243
|
+
Object.assign(m.style, { position: 'fixed', left: `${r.x}px`, top: `${r.y + r.height + 4}px`, minWidth: `${r.width}px`, zIndex: '2147483645', pointerEvents: 'none',
|
|
244
|
+
background: '#fff', border: '1px solid #cbd5e1', borderRadius: '8px', boxShadow: '0 12px 32px rgba(15,23,42,.22)', padding: '4px', font: `${fs}px -apple-system, "Segoe UI", sans-serif`, color: '#0f172a' });
|
|
245
|
+
items.forEach((t, i) => {
|
|
246
|
+
const o = document.createElement('div');
|
|
247
|
+
o.textContent = t;
|
|
248
|
+
Object.assign(o.style, { padding: `${fs * 0.35}px ${fs * 0.7}px`, borderRadius: '5px', whiteSpace: 'nowrap', transition: 'background .15s' });
|
|
249
|
+
o.dataset.i = i;
|
|
250
|
+
m.appendChild(o);
|
|
251
|
+
});
|
|
252
|
+
document.documentElement.appendChild(m);
|
|
253
|
+
// keep it on screen: open upwards if there is no room below
|
|
254
|
+
const b = m.getBoundingClientRect();
|
|
255
|
+
if (b.bottom > innerHeight - 8) m.style.top = `${Math.max(8, r.y - b.height - 4)}px`;
|
|
256
|
+
m.animate([{ opacity: 0, transform: 'translateY(-4px)' }, { opacity: 1, transform: 'none' }], { duration: 160, easing: 'ease-out' });
|
|
257
|
+
const q = m.children[pick].getBoundingClientRect();
|
|
258
|
+
return { x: q.x + Math.min(q.width / 2, 120), y: q.y + q.height / 2 };
|
|
259
|
+
};
|
|
260
|
+
window.__demoMenuHover = (i) => { const m = document.getElementById('__demo_menu'); if (m) [...m.children].forEach((o, k) => { o.style.background = k === i ? '#e0ecff' : ''; }); };
|
|
261
|
+
window.__demoMenuClose = () => document.getElementById('__demo_menu')?.remove();
|
|
262
|
+
// downloads leave no trace in a recording: show a browser-style "downloaded" bubble instead
|
|
263
|
+
window.__demoDownload = (name) => {
|
|
264
|
+
const el = document.createElement('div');
|
|
265
|
+
el.innerHTML = `<span style="display:inline-grid;place-items:center;width:30px;height:30px;border-radius:50%;background:#22c55e;color:#fff;font-weight:700">✓</span><span><b style="display:block;font-weight:600">${name.replace(/[<&]/g, '')}</b><span style="color:#64748b;font-size:14px">Downloaded</span></span>`;
|
|
266
|
+
Object.assign(el.style, { position: 'fixed', right: '24px', top: '20px', zIndex: '2147483646', pointerEvents: 'none', display: 'flex', alignItems: 'center', gap: '12px',
|
|
267
|
+
background: '#fff', color: '#0f172a', font: '16px -apple-system, "Segoe UI", sans-serif', padding: '12px 18px 12px 14px', borderRadius: '12px', boxShadow: '0 12px 32px rgba(15,23,42,.25)', border: '1px solid #e2e8f0' });
|
|
268
|
+
document.documentElement.appendChild(el);
|
|
269
|
+
el.animate([{ opacity: 0, transform: 'translateY(-8px)' }, { opacity: 1, transform: 'none', offset: 0.1 }, { opacity: 1, offset: 0.85 }, { opacity: 0 }], { duration: 2800, easing: 'ease-out' }).onfinish = () => el.remove();
|
|
270
|
+
};
|
|
271
|
+
window.__demoUnnote = (instant) => {
|
|
272
|
+
// detach it first, so a note or mark added during the fade gets a fresh layer instead of vanishing with this one
|
|
237
273
|
const layer = document.getElementById('__demo_notes');
|
|
238
|
-
if (layer
|
|
274
|
+
if (layer && instant) layer.remove();
|
|
275
|
+
else if (layer) { layer.id = ''; layer.style.opacity = '0'; setTimeout(() => layer.remove(), 300); }
|
|
239
276
|
};
|
|
240
|
-
window.__demoUnmark = () => {
|
|
277
|
+
window.__demoUnmark = (instant) => {
|
|
278
|
+
// detach it first, so a note or mark added during the fade gets a fresh layer instead of vanishing with this one
|
|
241
279
|
const layer = document.getElementById('__demo_marks');
|
|
242
|
-
if (layer
|
|
280
|
+
if (layer && instant) layer.remove();
|
|
281
|
+
else if (layer) { layer.id = ''; layer.style.opacity = '0'; setTimeout(() => layer.remove(), 300); }
|
|
243
282
|
};
|
|
244
283
|
// in-page anchor links scroll smoothly instead of jumping. Runs after the page's own handlers
|
|
245
284
|
// (bubble phase) and only when none of them handled the click, so app links like
|
|
@@ -299,9 +338,14 @@ export async function record(cfg, { log = console.log, inspectAfter = null } = {
|
|
|
299
338
|
const browser = await chromium.launch();
|
|
300
339
|
const context = await sessionContext(browser, {
|
|
301
340
|
viewport: cfg.viewport, deviceScaleFactor: cfg.scale, colorScheme: cfg.colorScheme, storageState: cfg.storageState, ignoreHTTPSErrors: cfg.ignoreHTTPSErrors,
|
|
341
|
+
permissions: ['clipboard-read', 'clipboard-write'], // for `paste`
|
|
302
342
|
});
|
|
303
343
|
await context.addInitScript(pageScript, { scrollOffset: cfg.scrollOffset });
|
|
304
344
|
const page = await context.newPage();
|
|
345
|
+
page.on('download', (d) => {
|
|
346
|
+
page.evaluate((n) => window.__demoDownload?.(n), d.suggestedFilename()).catch(() => {});
|
|
347
|
+
marks.push({ t: Date.now() / 1000, box: { x: cfg.viewport.width - 360, y: 10, width: 340, height: 80 } }); // so docs crops include the bubble
|
|
348
|
+
});
|
|
305
349
|
const sleep = (ms) => page.waitForTimeout(ms);
|
|
306
350
|
const now = () => Date.now() / 1000;
|
|
307
351
|
|
|
@@ -309,7 +353,7 @@ export async function record(cfg, { log = console.log, inspectAfter = null } = {
|
|
|
309
353
|
let cur = { x: VW * 0.62, y: VH * 0.55 };
|
|
310
354
|
const ease = (t) => (t < 0.5 ? 4 * t * t * t : 1 - Math.pow(-2 * t + 2, 3) / 2);
|
|
311
355
|
async function moveTo(x, y, ms = 700) {
|
|
312
|
-
if (fast) ms =
|
|
356
|
+
if (fast) ms = Math.min(ms, 160); // still along the real path: menus that open on hover must behave as in the recording
|
|
313
357
|
const from = { ...cur }, steps = Math.max(8, Math.round(ms / 16));
|
|
314
358
|
for (let i = 1; i <= steps; i++) {
|
|
315
359
|
const k = ease(i / steps);
|
|
@@ -322,6 +366,7 @@ export async function record(cfg, { log = console.log, inspectAfter = null } = {
|
|
|
322
366
|
// a pause after an action that changes the page (tutorial pace), counted like a `pause`
|
|
323
367
|
const beat = async () => { if (fast || !cfg.pace.beat) return; holds += cfg.pace.beat / 1000; await sleep(cfg.pace.beat); };
|
|
324
368
|
const clicks = [];
|
|
369
|
+
const gotos = []; // deliberate scene changes (a whoosh); other navigations are the app's own
|
|
325
370
|
async function press() { clicks.push(now()); await page.mouse.down(); await sleep(70); await page.mouse.up(); }
|
|
326
371
|
// Time spent waiting on the page (loads, AI processing…) is tracked separately so
|
|
327
372
|
// compose can compress it hard while keeping cursor motion at a natural speed.
|
|
@@ -394,17 +439,28 @@ export async function record(cfg, { log = console.log, inspectAfter = null } = {
|
|
|
394
439
|
await moveTo(p.x, p.y, opt.ms ?? cfg.pace.cursor);
|
|
395
440
|
break;
|
|
396
441
|
}
|
|
397
|
-
case 'click': {
|
|
442
|
+
case 'click': case 'dblclick': {
|
|
398
443
|
const p = await pointOf(spec, opt);
|
|
399
444
|
const lit = await spot(p.box, hl);
|
|
400
445
|
await moveTo(p.x, p.y, opt.ms ?? cfg.pace.cursor);
|
|
401
446
|
await sleep(lit ? 350 + cfg.pace.beat / 3 : 120);
|
|
402
|
-
await press();
|
|
447
|
+
if (kind === 'dblclick') { clicks.push(now()); await page.mouse.dblclick(p.x, p.y); } else await press();
|
|
403
448
|
if (lit) { await sleep(150); await spotOff(); }
|
|
404
449
|
await settle();
|
|
405
450
|
await beat(); // let the viewer see what the click did
|
|
406
451
|
break;
|
|
407
452
|
}
|
|
453
|
+
case 'hide': {
|
|
454
|
+
// hide: ".chat-widget, [id*=QSI]" — keep pop-ups, survey prompts, feedback tabs and chat widgets
|
|
455
|
+
// out of the video for good, on this page and every page after it (they often appear late)
|
|
456
|
+
const css = `${[].concat(value).join(', ')} { display: none !important; visibility: hidden !important; }`;
|
|
457
|
+
await context.addInitScript((c) => {
|
|
458
|
+
const add = () => { if (!document.getElementById('__demo_hide')) document.documentElement.appendChild(Object.assign(document.createElement('style'), { id: '__demo_hide', textContent: c })); };
|
|
459
|
+
if (document.documentElement) add(); else document.addEventListener('DOMContentLoaded', add);
|
|
460
|
+
}, css);
|
|
461
|
+
await page.evaluate((c) => document.documentElement.appendChild(Object.assign(document.createElement('style'), { textContent: c })), css);
|
|
462
|
+
break;
|
|
463
|
+
}
|
|
408
464
|
case 'dismiss': {
|
|
409
465
|
const l = locate(page, spec);
|
|
410
466
|
if (await l.isVisible().catch(() => false)) { await l.click(); await sleep(400); }
|
|
@@ -413,7 +469,7 @@ export async function record(cfg, { log = console.log, inspectAfter = null } = {
|
|
|
413
469
|
case 'waitFor': await waiting(() => locate(page, spec).waitFor({ state: 'visible', timeout: opt.timeout ?? 180000 })); await settle(); await nudge(); break;
|
|
414
470
|
case 'waitForUrl': await waiting(() => page.waitForURL(value, { timeout: 60000 })); await settle(); await nudge(); break;
|
|
415
471
|
case 'networkIdle': await waiting(() => page.waitForLoadState('networkidle', { timeout: 15000 }).catch(() => {})); await settle(); break;
|
|
416
|
-
case 'goto': await waiting(() => page.goto(new URL(value, cfg.url).href, { waitUntil: 'domcontentloaded' })); await settle(); await nudge(); break;
|
|
472
|
+
case 'goto': gotos.push(now()); await waiting(() => page.goto(new URL(value, cfg.url).href, { waitUntil: 'domcontentloaded' })); await settle(); await nudge(); break;
|
|
417
473
|
case 'scrollTo': {
|
|
418
474
|
const h = await locate(page, spec).elementHandle({ timeout: opt.timeout ?? 15000 });
|
|
419
475
|
await page.evaluate(([el, ms]) => window.__demoScrollTo(el, ms), [h, opt.ms ?? 1200]);
|
|
@@ -423,6 +479,7 @@ export async function record(cfg, { log = console.log, inspectAfter = null } = {
|
|
|
423
479
|
case 'type': {
|
|
424
480
|
// type: "text" or { text } types into whatever has focus; { target, text } clicks the target first
|
|
425
481
|
const v = typeof value === 'object' ? value : { text: value };
|
|
482
|
+
// clear: true empties the field first (the page may have pre-filled it)
|
|
426
483
|
let lit = false;
|
|
427
484
|
if (v.target) {
|
|
428
485
|
const p = await pointOf(v.target, v);
|
|
@@ -430,6 +487,7 @@ export async function record(cfg, { log = console.log, inspectAfter = null } = {
|
|
|
430
487
|
await moveTo(p.x, p.y, v.ms ?? cfg.pace.cursor * 0.85);
|
|
431
488
|
await press();
|
|
432
489
|
}
|
|
490
|
+
if (v.clear) { await page.keyboard.press('ControlOrMeta+A'); await page.keyboard.press('Delete'); }
|
|
433
491
|
await page.keyboard.type(String(v.text), { delay: fast ? 0 : v.delay ?? cfg.pace.type });
|
|
434
492
|
if (lit) await spotOff();
|
|
435
493
|
break;
|
|
@@ -456,19 +514,79 @@ export async function record(cfg, { log = console.log, inspectAfter = null } = {
|
|
|
456
514
|
const p = await pointOf(value.target, value);
|
|
457
515
|
const lit = await spot(p.box, hl);
|
|
458
516
|
await moveTo(p.x, p.y, value.ms ?? cfg.pace.cursor);
|
|
459
|
-
clicks.push(now());
|
|
460
|
-
await page.evaluate(([x, y]) => document.dispatchEvent(new PointerEvent('pointerdown', { clientX: x, clientY: y, bubbles: true })), [p.x, p.y]);
|
|
461
|
-
await sleep(250);
|
|
462
517
|
const l = locate(page, value.target), want = String(value.option);
|
|
463
518
|
const options = await l.evaluate((s) => [...(s.options ?? [])].map((o) => ({ text: o.text.trim(), value: o.value })));
|
|
464
|
-
const
|
|
465
|
-
if (
|
|
466
|
-
|
|
519
|
+
const k = [(o) => o.text === want, (o) => o.value === want, (o) => o.text.toLowerCase().includes(want.toLowerCase())].map((f) => options.findIndex(f)).find((i) => i >= 0);
|
|
520
|
+
if (k == null) throw new Error(`no option "${want}" in that dropdown — options: ${options.map((o) => `"${o.text}"`).join(', ')}`);
|
|
521
|
+
// click it open (a drawn menu, since the real one never shows on video), move to the option, pick it
|
|
522
|
+
await press();
|
|
523
|
+
if (!fast && options.length <= 14) {
|
|
524
|
+
const at = await page.evaluate(([r, items, i]) => window.__demoMenu(r, items, i), [p.box, options.map((o) => o.text), k]);
|
|
525
|
+
await sleep(350);
|
|
526
|
+
await moveTo(at.x, at.y, Math.min(900, cfg.pace.cursor));
|
|
527
|
+
await page.evaluate((i) => window.__demoMenuHover(i), k);
|
|
528
|
+
await sleep(250);
|
|
529
|
+
clicks.push(now());
|
|
530
|
+
await sleep(120);
|
|
531
|
+
await page.evaluate(() => window.__demoMenuClose());
|
|
532
|
+
}
|
|
533
|
+
await page.keyboard.press('Escape').catch(() => {}); // in case the click opened the real one
|
|
534
|
+
await l.selectOption({ value: options[k].value });
|
|
467
535
|
if (lit) { await sleep(300); await spotOff(); }
|
|
468
536
|
await settle();
|
|
469
537
|
await beat();
|
|
470
538
|
break;
|
|
471
539
|
}
|
|
540
|
+
case 'paste': {
|
|
541
|
+
// paste: { target: <field>, text: "..." } or paste: "..." (into whatever has focus) — the text arrives
|
|
542
|
+
// at once, the way a real paste does, with a ⌘V / Ctrl+V badge; real paste events, so editors that
|
|
543
|
+
// reformat on paste (code editors, rich text) behave as they would for a user
|
|
544
|
+
const v = typeof value === 'object' ? value : { text: value };
|
|
545
|
+
if (v.text == null) throw new Error('paste needs text: paste: "..." or paste: { target: <field>, text: "..." }');
|
|
546
|
+
let lit = false;
|
|
547
|
+
if (v.target) {
|
|
548
|
+
const p = await pointOf(v.target, v);
|
|
549
|
+
lit = await spot(p.box, hl);
|
|
550
|
+
await moveTo(p.x, p.y, v.ms ?? cfg.pace.cursor * 0.85);
|
|
551
|
+
await press();
|
|
552
|
+
}
|
|
553
|
+
const text = String(v.text);
|
|
554
|
+
const before = await page.evaluate(() => (document.activeElement?.value ?? document.activeElement?.textContent ?? '').length).catch(() => 0);
|
|
555
|
+
const copied = await page.evaluate((t) => navigator.clipboard.writeText(t).then(() => true, () => false), text).catch(() => false);
|
|
556
|
+
if (!fast) await page.evaluate((l) => window.__demoKeys(l), process.platform === 'darwin' ? '⌘ V' : 'Ctrl V').catch(() => {});
|
|
557
|
+
if (copied) await page.keyboard.press('ControlOrMeta+V');
|
|
558
|
+
await sleep(200);
|
|
559
|
+
const after = await page.evaluate(() => (document.activeElement?.value ?? document.activeElement?.textContent ?? '').length).catch(() => 0);
|
|
560
|
+
if (!copied || after === before) await page.keyboard.insertText(text); // no clipboard access: insert it in one go
|
|
561
|
+
if (lit) await spotOff();
|
|
562
|
+
await settle();
|
|
563
|
+
await beat();
|
|
564
|
+
break;
|
|
565
|
+
}
|
|
566
|
+
case 'slide': {
|
|
567
|
+
// slide: { target: <slider>, to: 0.2 } — drag a slider to a position along its track (0 = start,
|
|
568
|
+
// 1 = end), found from the element instead of pixel positions that break when the layout moves
|
|
569
|
+
if (!value?.target || typeof value?.to !== 'number') throw new Error('slide needs { target: <slider>, to: 0..1 }');
|
|
570
|
+
const p = await pointOf(value.target, value);
|
|
571
|
+
const b = p.box;
|
|
572
|
+
const from = await locate(page, value.target).evaluate((el) => {
|
|
573
|
+
const r = el.type === 'range' ? el : el.querySelector?.('input[type=range]');
|
|
574
|
+
if (r) return (Number(r.value) - Number(r.min || 0)) / ((Number(r.max || 100) - Number(r.min || 0)) || 1);
|
|
575
|
+
const now = Number(el.getAttribute('aria-valuenow')), lo = Number(el.getAttribute('aria-valuemin') ?? 0), hi = Number(el.getAttribute('aria-valuemax') ?? 100);
|
|
576
|
+
return Number.isFinite(now) ? (now - lo) / ((hi - lo) || 1) : 0.5;
|
|
577
|
+
}).catch(() => 0.5);
|
|
578
|
+
const x0 = b.x + b.width * from, x1 = b.x + b.width * Math.max(0, Math.min(1, value.to)), y = b.y + b.height / 2;
|
|
579
|
+
await moveTo(x0, y, value.ms ?? cfg.pace.cursor);
|
|
580
|
+
await sleep(120);
|
|
581
|
+
clicks.push(now());
|
|
582
|
+
await page.mouse.down();
|
|
583
|
+
await moveTo(x1, y, value.dragMs ?? 900);
|
|
584
|
+
await sleep(120);
|
|
585
|
+
await page.mouse.up();
|
|
586
|
+
await settle();
|
|
587
|
+
await beat();
|
|
588
|
+
break;
|
|
589
|
+
}
|
|
472
590
|
case 'mark': {
|
|
473
591
|
// mark: "a phrase" | { text: "a phrase" } (a text range, even inside a paragraph) | any target
|
|
474
592
|
// (e.g. a table row: { css: "tr:nth-child(3)" }) → highlighter sweep, kept until the step ends
|
|
@@ -638,7 +756,8 @@ export async function record(cfg, { log = console.log, inspectAfter = null } = {
|
|
|
638
756
|
await runAll(s.actions, `step ${i + 1}`);
|
|
639
757
|
const end = now(); // before clearing: a step that is held for its narration keeps its marks
|
|
640
758
|
await spotOff();
|
|
641
|
-
|
|
759
|
+
// docs clips start where the step starts: clear at once, so no fading callout leaks into the next clip
|
|
760
|
+
await page.evaluate((i) => { window.__demoUnmark(i); window.__demoUnnote(i); }, cfg.clips.length > 0).catch(() => {});
|
|
642
761
|
let zoomBox = null;
|
|
643
762
|
if (s.zoom?.target) {
|
|
644
763
|
if (isPoint(s.zoom.target)) {
|
|
@@ -658,7 +777,7 @@ export async function record(cfg, { log = console.log, inspectAfter = null } = {
|
|
|
658
777
|
await cdp.send('Page.stopScreencast');
|
|
659
778
|
await browser.close();
|
|
660
779
|
|
|
661
|
-
const timeline = { frames, steps, clicks, navs, scale: cfg.scale };
|
|
780
|
+
const timeline = { frames, steps, clicks, navs, gotos, scale: cfg.scale };
|
|
662
781
|
fs.writeFileSync(path.join(cfg.work, 'timeline.json'), JSON.stringify(timeline, null, 1));
|
|
663
782
|
log(`recorded ${frames.length} frames`);
|
|
664
783
|
return timeline;
|
package/src/setup.mjs
CHANGED
|
@@ -1,6 +1,8 @@
|
|
|
1
|
-
// `storyshot setup
|
|
1
|
+
// `storyshot setup <agent>`: wire Storyshot into a coding agent for this project —
|
|
2
2
|
// the MCP server config plus the playbook as an Agent Skill (both read SKILL.md).
|
|
3
|
+
import { execFileSync } from 'node:child_process';
|
|
3
4
|
import fs from 'node:fs';
|
|
5
|
+
import os from 'node:os';
|
|
4
6
|
import path from 'node:path';
|
|
5
7
|
import { fileURLToPath } from 'node:url';
|
|
6
8
|
|
|
@@ -38,9 +40,59 @@ function mergeMcp(file) {
|
|
|
38
40
|
return file;
|
|
39
41
|
}
|
|
40
42
|
|
|
43
|
+
// one shared copy of the playbook for agents that read instruction files rather than skills folders
|
|
44
|
+
function sharedSkill(dir, skill) {
|
|
45
|
+
const dest = path.join(dir, '.agents', 'skills', 'storyshot', 'SKILL.md');
|
|
46
|
+
fs.mkdirSync(path.dirname(dest), { recursive: true });
|
|
47
|
+
fs.writeFileSync(dest, skill);
|
|
48
|
+
return dest;
|
|
49
|
+
}
|
|
50
|
+
const pointer = () => ['## Videos of this web app (Storyshot)', '',
|
|
51
|
+
'When asked for a demo or promo video, a tutorial or onboarding video, a lesson, or GIFs for docs of a website or web app,',
|
|
52
|
+
'read `.agents/skills/storyshot/SKILL.md` first and follow it. Use the `storyshot` MCP tools when they are available;',
|
|
53
|
+
`otherwise run the CLI: \`${cliCommand()} <command>\` (\`${cliCommand()} guide\` prints the playbook and storyboard reference).`].join('\n');
|
|
54
|
+
// add (or refresh) our section in a shared instructions file, leaving the rest of the file alone
|
|
55
|
+
function upsertBlock(file, body) {
|
|
56
|
+
const block = `<!-- storyshot:start -->\n${body}\n<!-- storyshot:end -->`;
|
|
57
|
+
const re = /<!-- storyshot:start -->[\s\S]*?<!-- storyshot:end -->/;
|
|
58
|
+
const cur = fs.existsSync(file) ? fs.readFileSync(file, 'utf8') : '';
|
|
59
|
+
fs.mkdirSync(path.dirname(file), { recursive: true });
|
|
60
|
+
fs.writeFileSync(file, re.test(cur) ? cur.replace(re, block) : `${cur}${cur && !cur.endsWith('\n') ? '\n\n' : cur ? '\n' : ''}${block}\n`);
|
|
61
|
+
return file;
|
|
62
|
+
}
|
|
63
|
+
// a rules file we own outright
|
|
64
|
+
function writeRule(file, text) {
|
|
65
|
+
fs.mkdirSync(path.dirname(file), { recursive: true });
|
|
66
|
+
fs.writeFileSync(file, text);
|
|
67
|
+
return file;
|
|
68
|
+
}
|
|
69
|
+
function mergeJson(file, key, entry) {
|
|
70
|
+
let json = {};
|
|
71
|
+
if (fs.existsSync(file)) {
|
|
72
|
+
try { json = JSON.parse(fs.readFileSync(file, 'utf8')); } catch { throw new Error(`${file} is not valid JSON — fix or remove it first`); }
|
|
73
|
+
}
|
|
74
|
+
json[key] = { ...(json[key] ?? {}), storyshot: entry };
|
|
75
|
+
fs.mkdirSync(path.dirname(file), { recursive: true });
|
|
76
|
+
fs.writeFileSync(file, `${JSON.stringify(json, null, 2)}\n`);
|
|
77
|
+
return file;
|
|
78
|
+
}
|
|
79
|
+
// Cline keeps MCP servers in its extension storage, per editor (VS Code, Cursor, Windsurf…)
|
|
80
|
+
function clineSettings() {
|
|
81
|
+
const home = os.homedir();
|
|
82
|
+
const base = process.platform === 'darwin' ? path.join(home, 'Library', 'Application Support')
|
|
83
|
+
: process.platform === 'win32' ? (process.env.APPDATA ?? path.join(home, 'AppData', 'Roaming')) : path.join(home, '.config');
|
|
84
|
+
return ['Code', 'Code - Insiders', 'Cursor', 'Windsurf', 'VSCodium']
|
|
85
|
+
.map((ed) => path.join(base, ed, 'User', 'globalStorage', 'saoudrizwan.claude-dev'))
|
|
86
|
+
.filter((d) => fs.existsSync(d))
|
|
87
|
+
.map((d) => path.join(d, 'settings', 'cline_mcp_settings.json'));
|
|
88
|
+
}
|
|
89
|
+
|
|
90
|
+
export const TARGETS = ['cursor', 'claude', 'codex', 'gemini', 'vscode', 'windsurf', 'cline'];
|
|
91
|
+
|
|
41
92
|
export function setup(target, dir = process.cwd()) {
|
|
42
93
|
const skill = fs.readFileSync(path.join(ROOT, 'skills/storyshot/SKILL.md'), 'utf8');
|
|
43
94
|
const written = [];
|
|
95
|
+
const server = serverCommand();
|
|
44
96
|
if (target === 'cursor') {
|
|
45
97
|
// Cursor loads Agent Skills from .cursor/skills/<name>/SKILL.md (same format as Claude Code)
|
|
46
98
|
written.push(mergeMcp(path.join(dir, '.cursor', 'mcp.json')));
|
|
@@ -54,8 +106,35 @@ export function setup(target, dir = process.cwd()) {
|
|
|
54
106
|
fs.mkdirSync(path.dirname(dest), { recursive: true });
|
|
55
107
|
fs.writeFileSync(dest, skill);
|
|
56
108
|
written.push(dest);
|
|
109
|
+
} else if (target === 'codex') {
|
|
110
|
+
// Codex reads AGENTS.md on every task and keeps MCP servers in ~/.codex/config.toml (per user).
|
|
111
|
+
written.push(sharedSkill(dir, skill), upsertBlock(path.join(dir, 'AGENTS.md'), `${pointer()}\n\nStoryshot launches a browser (Chromium) and ffmpeg. Codex's default sandbox blocks the browser: when a Storyshot\ncommand or tool needs it, ask the user to approve running it outside the sandbox. Never work around a blocked browser\nby faking the video.`));
|
|
112
|
+
try {
|
|
113
|
+
execFileSync('codex', ['mcp', 'add', 'storyshot', '--', server.command, ...server.args], { stdio: 'ignore' });
|
|
114
|
+
written.push(`${process.env.CODEX_HOME ?? '~/.codex'}/config.toml (codex mcp add storyshot)`);
|
|
115
|
+
} catch {
|
|
116
|
+
written.push(`(codex CLI not found: add the MCP server yourself with \`codex mcp add storyshot -- ${[server.command, ...server.args].join(' ')}\`)`);
|
|
117
|
+
}
|
|
118
|
+
} else if (target === 'gemini') {
|
|
119
|
+
// Gemini CLI: project settings in .gemini/settings.json, project context in GEMINI.md
|
|
120
|
+
written.push(sharedSkill(dir, skill), mergeJson(path.join(dir, '.gemini', 'settings.json'), 'mcpServers', server), upsertBlock(path.join(dir, 'GEMINI.md'), pointer()));
|
|
121
|
+
} else if (target === 'vscode') {
|
|
122
|
+
// VS Code (GitHub Copilot agent mode): workspace MCP servers in .vscode/mcp.json, instructions in .github/copilot-instructions.md
|
|
123
|
+
written.push(sharedSkill(dir, skill), mergeJson(path.join(dir, '.vscode', 'mcp.json'), 'servers', { type: 'stdio', ...server }),
|
|
124
|
+
upsertBlock(path.join(dir, '.github', 'copilot-instructions.md'), pointer()));
|
|
125
|
+
} else if (target === 'windsurf') {
|
|
126
|
+
// Windsurf: workspace rules in .windsurf/rules/, MCP servers only per user in ~/.codeium/windsurf/mcp_config.json
|
|
127
|
+
written.push(sharedSkill(dir, skill), writeRule(path.join(dir, '.windsurf', 'rules', 'storyshot.md'),
|
|
128
|
+
`---\ntrigger: model_decision\ndescription: Making demo, promo, tutorial, lesson or docs videos/GIFs of a website or web app with Storyshot\n---\n\n${pointer()}\n`));
|
|
129
|
+
written.push(mergeJson(path.join(os.homedir(), '.codeium', 'windsurf', 'mcp_config.json'), 'mcpServers', server));
|
|
130
|
+
} else if (target === 'cline') {
|
|
131
|
+
// Cline: project rules in .clinerules/, MCP servers in the extension's settings (per user, per editor)
|
|
132
|
+
written.push(sharedSkill(dir, skill), writeRule(path.join(dir, '.clinerules', 'storyshot.md'), `${pointer()}\n`));
|
|
133
|
+
const files = clineSettings();
|
|
134
|
+
for (const f of files) written.push(mergeJson(f, 'mcpServers', { ...server, disabled: false }));
|
|
135
|
+
if (!files.length) written.push(`(Cline not found in VS Code, Cursor or Windsurf: in Cline, open MCP Servers → Configure and add "storyshot": ${JSON.stringify(server)})`);
|
|
57
136
|
} else {
|
|
58
|
-
throw new Error(
|
|
137
|
+
throw new Error(`usage: storyshot setup ${TARGETS.join(' | ')}`);
|
|
59
138
|
}
|
|
60
139
|
const gi = ignoreCache(dir);
|
|
61
140
|
if (gi) written.push(`${gi} (added .storyshot/)`);
|