storyshot 0.10.0 → 0.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,19 +1,94 @@
1
1
  # Storyshot
2
2
 
3
- Ask your coding agent for a video of your web app, and get a polished 1080p
4
- MP4: a promo clip, a narrated tutorial, a lesson, or step-by-step GIFs for your
5
- docs.
3
+ Ask your coding agent for a video, and get a polished 1080p MP4. Two kinds of
4
+ film come out of one tool:
6
5
 
6
+ - **An explainer of an idea** — no web app, no footage, nothing recorded. Every
7
+ scene is an animation the agent writes and times to the narration: a
8
+ mechanism, a diagram that builds, a number that lands. Chinese or English.
9
+ *This is what most people come for.*
10
+ - **A video of your web app** — a promo, a narrated tutorial, a lesson, or
11
+ step-by-step GIFs for your docs. A real browser clicks through your site, and
12
+ the recording gets a smooth cursor, automatic zoom, captions, an optional
13
+ voiceover, music and an end card.
14
+
15
+ The agent plans, writes and fixes the storyboard. Storyshot does the rendering,
16
+ and the same storyboard always renders the same video.
17
+
18
+ ## Install it into your coding agent
19
+
20
+ **The skill is the front door.** You do not learn a CLI — your agent does.
21
+
22
+ Open your project in your coding agent (Claude Code, Cursor, Codex, Gemini CLI,
23
+ Copilot in VS Code, Windsurf, Cline) and paste this one line into its chat, with
24
+ your agent's name in place of `claude`:
25
+
26
+ > Set up Storyshot in this project: run `npx -y storyshot setup claude` and do everything it says.
27
+
28
+ Agent names: `claude` · `cursor` · `codex` · `gemini` · `vscode` · `windsurf` · `cline`.
29
+
30
+ That one command installs **the Storyshot skill** into your agent, writes the MCP
31
+ server config, adds `.storyshot/` to `.gitignore`, and downloads the browser it
32
+ needs. The skill is what teaches your agent the whole workflow — how to plan a
33
+ film, write the scenes, read the render back and fix it — so from the next chat
34
+ on you can just ask for a video in plain words.
35
+
36
+ Restart or reload your agent when `setup` says to. Then:
37
+
38
+ > *「做一个视频,讲清楚热泵为什么能把一份电变成三份热」*
39
+ >
40
+ > *"make an explainer about how a heat pump moves three units of heat per unit of electricity"*
41
+ >
7
42
  > *"make a 15 s promo video of https://myapp.com"*
43
+ >
44
+ > *"make a narrated tutorial showing how to export a report"*
8
45
 
9
- Storyshot gives coding agents (Cursor, Claude Code, Codex, Gemini CLI, Copilot in VS Code, Windsurf, Cline) what they need to
10
- do this well. A real browser clicks through your site, and the recording gets
11
- a smooth cursor, automatic zoom, captions, an optional AI voiceover, music
12
- and an end card. Loading time is squeezed out, so a 30-second AI job takes a
13
- second or two on screen.
46
+ or type `/storyshot`.
14
47
 
15
- The agent plans, writes and fixes the storyboard. Storyshot does the
16
- recording and editing, and the same storyboard always renders the same video.
48
+ `npx storyshot doctor` re-checks everything at any time. Storyshot runs on
49
+ macOS, Linux and Windows. Needs Node 18+ and ffmpeg (setup prints the exact
50
+ install command for your OS if it is missing).
51
+
52
+ ### Explainers, in one paragraph
53
+
54
+ An explainer has no `url` and records nothing. You ask for a subject; the agent
55
+ writes a storyboard of **scenes**, and each scene is an HTML animation it draws
56
+ and times to the voice, phrase by phrase. It renders a still of every scene,
57
+ looks at it, and fixes what is wrong before it ever renders video. Free local
58
+ voices in English and Chinese, synth music, subtitles and chapters — no API key
59
+ and no per-video cost. Start here:
60
+ [the explainers in the gallery](https://boheling.github.io/storyshot-gallery/#explainer),
61
+ and the ["Scenes and explainers" playbook](#for-agents-tools-and-playbook).
62
+
63
+ ## What `setup` writes, per agent
64
+
65
+ | agent | what `setup` writes |
66
+ |---|---|
67
+ | Cursor | `.cursor/mcp.json` and the skill in `.cursor/skills/storyshot/` |
68
+ | Claude Code | `.mcp.json` and the skill in `.claude/skills/storyshot/` |
69
+ | Codex | the skill in `.agents/skills/storyshot/`, a short section in `AGENTS.md` pointing to it, and the MCP server through `codex mcp add` (in `~/.codex/config.toml`, for your user). Storyshot launches a browser, which Codex's sandbox blocks: approve it when Codex asks, or run Codex with `--sandbox danger-full-access` |
70
+ | Gemini CLI | `.gemini/settings.json`, a section in `GEMINI.md`, the skill in `.agents/skills/storyshot/` (trust the folder when Gemini asks, or it ignores project settings) |
71
+ | Copilot in VS Code (`vscode`) | `.vscode/mcp.json`, a section in `.github/copilot-instructions.md`, the skill in `.agents/skills/storyshot/` |
72
+ | Windsurf | `.windsurf/rules/storyshot.md`, the skill in `.agents/skills/storyshot/`, and the MCP server in `~/.codeium/windsurf/mcp_config.json` (for your user) |
73
+ | Cline | `.clinerules/storyshot.md`, the skill in `.agents/skills/storyshot/`, and the MCP server in Cline's own settings in each editor it is installed in (for your user) |
74
+
75
+ Existing files are merged, not replaced: other MCP servers stay, and instruction files get one marked
76
+ Storyshot section that a second `setup` updates instead of duplicating.
77
+
78
+ **Tested end to end** (setup, then an agent making a real video from a plain request): **Claude Code,
79
+ Cursor and Codex**. Gemini CLI, Copilot in VS Code, Windsurf and Cline get the configuration their docs
80
+ describe, but haven't been tested with a real session yet; reports are welcome.
81
+
82
+ The agent does the rest:
83
+ - `setup` writes the MCP server config and the Storyshot skill, adds
84
+ `.storyshot/` to `.gitignore`, and downloads the browser it needs.
85
+ - If ffmpeg is missing, setup prints the exact install command for your OS
86
+ (`brew install ffmpeg`, `winget install --id Gyan.FFmpeg -e`,
87
+ `sudo apt-get install -y ffmpeg`). The agent runs it after you approve.
88
+ - The agent reads the skill and starts on your video in the same chat, using
89
+ the CLI.
90
+
91
+ From the next chat on, the agent uses the Storyshot MCP tools automatically.
17
92
 
18
93
  https://github.com/user-attachments/assets/05dd9a73-2edc-46af-9b2b-a8e33c7005b4
19
94
 
@@ -57,51 +132,6 @@ https://github.com/user-attachments/assets/8d3370ca-7a90-48af-b1c9-88189763e95d
57
132
  How each one was made, what we saw, and the caveats:
58
133
  [docs/comparison](docs/comparison/README.md).
59
134
 
60
- ## Quick start
61
-
62
- Open your project in your coding agent and paste this into its chat, with your
63
- agent's name (`cursor`, `claude`, `codex`, `gemini`, `vscode`, `windsurf` or `cline`):
64
-
65
- > Set up Storyshot in this project: run `npx -y storyshot setup cursor` and do
66
- > everything it says. Then make a short promo video of https://myapp.com
67
-
68
- | agent | what `setup` writes |
69
- |---|---|
70
- | Cursor | `.cursor/mcp.json` and the skill in `.cursor/skills/storyshot/` |
71
- | Claude Code | `.mcp.json` and the skill in `.claude/skills/storyshot/` |
72
- | Codex | the skill in `.agents/skills/storyshot/`, a short section in `AGENTS.md` pointing to it, and the MCP server through `codex mcp add` (in `~/.codex/config.toml`, for your user). Storyshot launches a browser, which Codex's sandbox blocks: approve it when Codex asks, or run Codex with `--sandbox danger-full-access` |
73
- | Gemini CLI | `.gemini/settings.json`, a section in `GEMINI.md`, the skill in `.agents/skills/storyshot/` (trust the folder when Gemini asks, or it ignores project settings) |
74
- | Copilot in VS Code (`vscode`) | `.vscode/mcp.json`, a section in `.github/copilot-instructions.md`, the skill in `.agents/skills/storyshot/` |
75
- | Windsurf | `.windsurf/rules/storyshot.md`, the skill in `.agents/skills/storyshot/`, and the MCP server in `~/.codeium/windsurf/mcp_config.json` (for your user) |
76
- | Cline | `.clinerules/storyshot.md`, the skill in `.agents/skills/storyshot/`, and the MCP server in Cline's own settings in each editor it is installed in (for your user) |
77
-
78
- Existing files are merged, not replaced: other MCP servers stay, and instruction files get one marked
79
- Storyshot section that a second `setup` updates instead of duplicating.
80
-
81
- **Tested end to end** (setup, then an agent making a real video from a plain request): **Claude Code,
82
- Cursor and Codex**. Gemini CLI, Copilot in VS Code, Windsurf and Cline get the configuration their docs
83
- describe, but haven't been tested with a real session yet; reports are welcome.
84
-
85
- The agent does the rest:
86
- - `setup` writes the MCP server config and the Storyshot skill, adds
87
- `.storyshot/` to `.gitignore`, and downloads the browser it needs.
88
- - If ffmpeg is missing, setup prints the exact install command for your OS
89
- (`brew install ffmpeg`, `winget install --id Gyan.FFmpeg -e`,
90
- `sudo apt-get install -y ffmpeg`). The agent runs it after you approve.
91
- - The agent reads the skill and starts on your video in the same chat, using
92
- the CLI.
93
-
94
- Restart or reload your agent when convenient (`setup` says how for each one). From the next chat on,
95
- the agent uses the Storyshot MCP tools automatically, and you can just ask:
96
-
97
- - *"make a 15 s promo video of https://myapp.com"*, or type `/storyshot`
98
- - *"make a narrated tutorial showing how to export a report"*
99
- - *"make a lesson video explaining how the AI editor works"*
100
- - *"make docs GIFs for the onboarding flow"*
101
-
102
- `npx storyshot doctor` re-checks everything at any time. Storyshot runs on
103
- macOS, Linux and Windows.
104
-
105
135
  ## What happens when you ask
106
136
 
107
137
  1. **The agent looks at your site** (about 5 s). It reads the page, and checks
@@ -142,8 +172,8 @@ After your next release, re-render it and you have an up-to-date video.
142
172
  | ask for | type (`purpose`) | you get |
143
173
  |---|---|---|
144
174
  | a promo / launch / landing-page clip | `pitch` | short and tightly cut: intro card, the app in a browser frame, music, captions |
145
- | a how-to or onboarding video | `tutorial` | paced to follow along: one narrated action per step, the camera following each click, part cards, a second example, a recap; `.srt` + chapters |
146
- | a lesson | `course` | explains the *why*: concept cards, the idea shown in the app, question cards, a recap, no music; `.srt` + chapters |
175
+ | a how-to or onboarding video | `tutorial` | paced to follow along: one narrated action per step, the camera following each click, part cards, a second example, a recap; `.srt` + chapters. `check` holds the narration to a teaching pace too — one job per step, a budget per line, no bolded payoffs — so a slow film doesn't still read as a pitch |
176
+ | a lesson | `course` | explains the *why*: concept cards, the idea shown in the app, question cards, a recap, no music; `.srt` + chapters. `check` holds it to the tutorial's teaching pace, and to a lesson's own shape: a card names one idea, a question waits for an answer and its step marks it |
147
177
  | help-center snippets | `docs` | silent per-step GIFs and PNGs cropped to the action, plus a ready-to-paste Markdown article |
148
178
  | an animated explainer of how something works | `explainer` | no web app: every step is a **scene**, an animation the agent writes and times to the narration (a mechanism, a diagram that grows, a number); calm music, subtitles, chapters |
149
179
 
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "storyshot",
3
- "version": "0.10.0",
3
+ "version": "0.11.0",
4
4
  "description": "Ask your coding agent for a video of your web app: promo clips, narrated tutorials, lessons and docs GIFs from a YAML storyboard.",
5
5
  "author": "boheling <boheling@gmail.com>",
6
6
  "repository": {
@@ -175,7 +175,9 @@ or scientific tool they have never seen. Storyshot paces `tutorial` and
175
175
  `course` slowly by default (`pace: slow`: slower cursor and typing, a beat
176
176
  after every click, a hold after every narration line, slower voice, real
177
177
  progress shown for waits) and the camera follows the action (`follow`), so
178
- small controls are readable. Your job is the structure:
178
+ small controls are readable. What the tool cannot set for you is the structure
179
+ and the voice. The structure first; "The teaching voice" below is the half that
180
+ decides whether a slow film still sounds like a sales pitch:
179
181
 
180
182
  - **Start with the goal**: an `intro` with `bullets` saying what the viewer
181
183
  will be able to do ("Run a BLAST search", "Read the hits table", "Judge a
@@ -245,6 +247,86 @@ small controls are readable. Your job is the structure:
245
247
  don't pad: if a part is long, split it into more parts rather than
246
248
  speeding it up.
247
249
 
250
+ #### The teaching voice
251
+
252
+ `pace: slow` buys time on screen: slower cursor and typing, a beat after every
253
+ click, a hold after every line. It cannot buy time *inside* a sentence. A
254
+ tutorial that still sounds like an ad read slowly is nearly always a writing
255
+ problem, not a timing one — and the usual cause is density, not length.
256
+ `check` holds a `tutorial` or a `course` to the rules below. A promo is held to
257
+ none of them, which is the point: they are different jobs.
258
+
259
+ 1. **One job per step.** A step either names an idea, does an action, or reads
260
+ a result. Not two of those, and never three. The line that packs all three
261
+ is the commonest failure, and it reads fine on the page:
262
+
263
+ > "The E value is how many hits this good you would expect by pure chance in
264
+ > a database this size. Smaller is better, and anything below 1e-5 usually
265
+ > means a real relationship. Here it is 6e-106, essentially zero."
266
+
267
+ A definition, a rule of thumb and a reading, in eleven seconds. The viewer
268
+ is still working out what "expect by chance" means when the number arrives,
269
+ so they keep none of it. As three steps they keep all three.
270
+ 2. **A step's whole `say:` fits in about 35 words (55 Chinese characters).**
271
+ Past that, split the step.
272
+ 3. **A sentence is about 15 words (25 characters).** `check` speaks up past 18
273
+ (30), so 15 is what to write to, not a line to trip over. Promos are allowed
274
+ 25; a lesson is not.
275
+ 4. **At most three figures in one line.** Nobody holds four numbers by ear. Say
276
+ the one that carries the point, round it out loud ("about eighty percent"),
277
+ and let the screen carry the exact value — which it can, because you marked
278
+ the cell.
279
+ 5. **Say the number, then what it means, then the rule again.** Teaching
280
+ repeats. A script that states each fact exactly once reads as a list of
281
+ facts, however plainly each one is put. The recap at the end repeats them in
282
+ the same words as the first time.
283
+ 6. **Silence is part of the lesson.** After a result appears, the viewer needs a
284
+ beat to find it on their own screen. Hold the step (`pause: 2000`) and say
285
+ nothing. Compose's dead-air warning knows the difference: 3 s is too long in
286
+ a promo, a `tutorial` or `course` is allowed 6 s, and 8 s on a step that
287
+ asked to hold. Resolve a dead-air warning in a lesson by saying what to look
288
+ at, not by cutting the beat.
289
+ 7. **A new term gets a step of its own**, the first time it appears, before any
290
+ number is attached to it.
291
+
292
+ **A tutorial's caption is a label, not a payoff.** A promo bolds the payoff
293
+ because a promo is selling the payoff. Here the caption says where the viewer
294
+ is — "The E value column", "Part 2: read the results" — and the voice does the
295
+ explaining. `Shorter protein, **larger** E value` reaches the conclusion before
296
+ the narration gets there, which is an ad's grammar; `check` warns on `**` in a
297
+ tutorial caption. A `note` works the same way: it labels the control it points
298
+ at in a few words ("FASTA or accession"), it does not recommend it ("Curated,
299
+ fast: good for learning").
300
+
301
+ #### What a lesson adds
302
+
303
+ A tutorial teaches the steps; a lesson (`course`) teaches the idea behind them,
304
+ so the clicking is usually done in `hidden` steps and the film is about reading
305
+ what is on the page. It is held to the whole teaching voice above, and to three
306
+ things of its own. `check` warns on each.
307
+
308
+ 1. **A card names one idea.** A concept card's `say:` gets the same budget as a
309
+ step (about 35 words, 55 Chinese characters), and it is the easiest place to
310
+ break it, because a card feels like the place for the whole explanation:
311
+ definition, threshold and example. Put the name and the plain-words meaning
312
+ on the card. The rule and the number go in the steps after it, where the
313
+ page can show them and you can `mark` them.
314
+ 2. **A question card waits, and its step shows the answer.** A check has three
315
+ parts: the question, time to answer it (`think: 3`, at least 2), and the
316
+ answer marked on the page. Without the wait it is a rhetorical question,
317
+ which a promo asks and a lesson does not. Without the mark, a viewer who
318
+ guessed wrong is told so and never shown where to look. The step a question
319
+ card sits on is the reveal: it points at the answer first, and the reason
320
+ comes in the step after.
321
+ 3. **Ask at least once.** A lesson with no question is a lecture: the viewer
322
+ cannot tell whether they followed it. The best second question is a new case
323
+ the rule has to be applied to (a different search, another input), not the
324
+ first one asked again.
325
+
326
+ The order inside each idea is the one in "Explain in an order the viewer can
327
+ follow" below: a concrete case on screen, then the card that names it, then the
328
+ reading, then the rule, then the question.
329
+
248
330
  ### Writing the narration
249
331
 
250
332
  Viewers notice three things first: lines that run together, phrasing nobody
@@ -254,7 +336,9 @@ They should sound like one person talking a friend through it.
254
336
 
255
337
  **Say it like a person.**
256
338
  - Short sentences, one idea each: about 15 words at most in English, 25
257
- characters in Chinese. If you would run out of breath, split it.
339
+ characters in Chinese. If you would run out of breath, split it. In a
340
+ tutorial or lesson `check` enforces this, along with a budget for the whole
341
+ step — see "The teaching voice".
258
342
  - Spoken words: contractions ("you'll", "it's"), "you", present tense, active
259
343
  voice. In Chinese, 口语 rather than 书面语: "点一下这里", not "进行点击操作".
260
344
  - Use the exact labels on screen, and the same term every time (no synonyms
@@ -314,8 +398,10 @@ Rules of thumb from real renders (promos and tutorials):
314
398
  icon and name, not an address bar.
315
399
  - Other purposes: leave out `duration`. `auto` times each step to its
316
400
  narration and plays motion in real time.
317
- - **Captions ≤ 7 words.** `**bold**` the payoff ("Every text box is
318
- **fully editable**"). Steps with the same caption keep it on screen.
401
+ - **Captions ≤ 7 words.** In a promo, `**bold**` the payoff ("Every text box is
402
+ **fully editable**"); in a tutorial or lesson a caption is a label for where
403
+ the viewer is, with no bold payoff (see "The teaching voice"). Steps with the
404
+ same caption keep it on screen.
319
405
  - **Narration (`say:`) is not a caption.** Write full spoken sentences, one
320
406
  instruction or idea per step. Use normal spelling, and put pronunciations in
321
407
  `voice.pronounce` (e.g. `pxGenius: P X Genius`, `AI: A I`) so subtitles stay
@@ -380,7 +466,9 @@ Notes:
380
466
  automatically (never under 2.5 s); leave out `duration` there.
381
467
  - Compose warns about **dead air**: 3 s or more where nothing moves and
382
468
  nothing is said. Shorten the wait or pause, hide the load in a `hidden`
383
- step, or narrate it. Keep `intro` cards to 6–8 s.
469
+ step, or narrate it. A `tutorial` or `course` is allowed 6 s, and 8 s on a
470
+ step that asked to hold with `pause`, because a learner needs the beat.
471
+ Keep `intro` cards to 6–8 s.
384
472
  - With `subtitles: burn`, the top caption pills can repeat the subtitles;
385
473
  `captions: false` keeps them off the video (chapters and SRT still use them).
386
474
  - `.storyshot/` next to the YAML is a cache of recordings, screenshots and
package/src/check.mjs CHANGED
@@ -9,6 +9,9 @@ import { sentences } from './voice.mjs';
9
9
  import { sceneInfo, filmGround, contrast } from './scenes.mjs';
10
10
  import path from 'node:path';
11
11
 
12
+ const cjkLen = (s) => (s.match(/[぀-ヿ㐀-鿿가-힯]/g) ?? []).length;
13
+ const wordLen = (s) => s.replace(/[぀-ヿ㐀-鿿가-힯]/g, ' ').split(/\s+/).filter((w) => /\w/.test(w)).length;
14
+
12
15
  // Phrases that mark narration as machine-written, and sentences too long to say in one breath.
13
16
  const TELLS = [
14
17
  /\bnot (?:just|only|merely) [^.,;!?]{1,40}?,? but(?: also)?\b/i, // "not just X, but Y"
@@ -28,15 +31,41 @@ export function narrationLint(text) {
28
31
  }
29
32
  let long = null;
30
33
  for (const s of sentences(t)) {
31
- const cjk = (s.match(/[\u3040-\u30ff\u3400-\u9fff\uac00-\ud7af]/g) ?? []).length;
32
- const words = s.replace(/[\u3040-\u30ff\u3400-\u9fff\uac00-\ud7af]/g, ' ').split(/\s+/).filter((w) => /\w/.test(w)).length;
34
+ const cjk = cjkLen(s), words = wordLen(s);
33
35
  if (cjk > 40 && !long) long = `${cjk}-character`;
34
36
  else if (!cjk && words > 25 && !long) long = `${words}-word`;
35
37
  }
36
38
  return { tells, long };
37
39
  }
38
40
 
39
- const TOP = ['name', 'url', 'out', 'purpose', 'aspect', 'viewport', 'output', 'theme', 'music', 'voice', 'sfx', 'storageState', 'ignoreHTTPSErrors', 'intro', 'frame', 'pace', 'follow', 'article', 'cards', 'ground', 'lang', 'colorScheme',
41
+ // How much one step may carry when the viewer is meant to follow along and do it themselves.
42
+ // A promo packs; a tutorial parcels out. The skill has always asked for 15 words a sentence, but the
43
+ // lint above only caught 25, which is a promo's budget — tutorials and lessons are held to the number
44
+ // the skill actually says, and a step gets a budget of its own. Density is the thing nobody notices
45
+ // while writing: three jobs in one breath (name the idea, give the rule, read the number) is what
46
+ // makes a tutorial sound like an ad read at speed, even when every sentence is short.
47
+ // sentenceWords/Chars is the number the skill asks for; the lint only speaks up past the slack, so
48
+ // "about 15 words" stays a target to write to rather than a line to trip over at 16.
49
+ export const TEACH = { words: 35, chars: 55, sentenceWords: 15, sentenceChars: 25, slackWords: 18, slackChars: 30, numbers: 3 };
50
+ const an = (s) => `${/^(?:8|11|18|8\d|11\d|18\d)/.test(s) ? 'an' : 'a'} ${s}`;
51
+ export function teachingLint(text) {
52
+ const t = String(text).replace(/\*\*/g, '');
53
+ const c = cjkLen(t), w = wordLen(t);
54
+ const heavy = c > TEACH.chars ? `${c}-character` : (!c && w > TEACH.words ? `${w}-word` : null);
55
+ let long = null;
56
+ for (const s of sentences(t)) {
57
+ const sc = cjkLen(s), sw = wordLen(s);
58
+ if (sc > TEACH.slackChars && !long) long = `${sc}-character`;
59
+ else if (!sc && sw > TEACH.slackWords && !long) long = `${sw}-word`;
60
+ }
61
+ // figures the viewer has to hold: a digit that starts a number, so an accession (P68871) or a
62
+ // version inside a word is not one, and the comma in "1,000" is part of it while the one after
63
+ // "105," is not. Chinese numerals are spelled out, where the character budget above is what limits them.
64
+ const numbers = [...new Set(t.match(/(?<![A-Za-z\d])\d(?:,?\d)*(?:\.\d+)?(?:e[-+]?\d+)?%?/gi) ?? [])];
65
+ return { heavy, long, numbers };
66
+ }
67
+
68
+ const TOP =['name', 'url', 'out', 'purpose', 'aspect', 'viewport', 'output', 'theme', 'music', 'voice', 'sfx', 'storageState', 'ignoreHTTPSErrors', 'intro', 'frame', 'pace', 'follow', 'article', 'cards', 'ground', 'lang', 'colorScheme',
40
69
  'scrollOffset', 'captionPosition', 'captions', 'highlight', 'stepNumbers', 'subtitles', 'chapters', 'clips', 'maxWait', 'setup', 'steps', 'end_card', 'hyperframes', 'scenes'];
41
70
  const STEP = ['caption', 'say', 'sayAt', 'duration', 'zoom', 'actions', 'hidden', 'sfx', 'highlight', 'captionPosition', 'card', 'follow', 'doc', 'scene'];
42
71
  const SCENE = ['file', 'title', 'lead', 'tail'];
@@ -91,6 +120,8 @@ export function check(file, { draft = false } = {}) {
91
120
  return { ok: false, errors: [`YAML syntax: ${e.message.split('\n')[0]}`], warnings, summary: '' };
92
121
  }
93
122
  unknown(raw, TOP, 'top level');
123
+ const sold = []; // tutorial captions written as a promo's payoff, reported together
124
+ let questions = 0; // question cards (a card that waits with `think`), which a lesson is expected to have
94
125
  for (const [i, a] of (raw.setup ?? []).entries()) checkAction(a, `setup action ${i + 1}`);
95
126
  for (const [i, s] of (raw.steps ?? []).entries()) {
96
127
  const where = `step ${i + 1}`;
@@ -121,8 +152,42 @@ export function check(file, { draft = false } = {}) {
121
152
  if (s.say && kinds.some((k) => k === 'scroll' || k === 'scrollTo') && !kinds.some((k) => ['mark', 'note', 'click', 'type', 'select', 'move', 'hover'].includes(k)) && !s.zoom) {
122
153
  warnings.push(`${where}: scrolls and talks but points at nothing — on a dense page the viewer can't tell what you mean; \`mark\` or \`note\` the row or value the narration is about (the camera then zooms to it)`);
123
154
  }
155
+ // A promo's caption bolds the payoff, because a promo is selling the payoff. A tutorial's caption
156
+ // says where the viewer is, and the narration does the explaining: bold it and the caption gets
157
+ // to the conclusion before the voice does, which is the grammar of an ad, not of a lesson. It is
158
+ // one decision rather than one mistake per step, so it is reported once.
159
+ if (typeof s.caption === 'string' && s.caption.includes('**')) sold.push([i + 1, s.caption]);
160
+ // A callout points at a control and names it. Longer than that and it is arguing for the app.
161
+ for (const a of (s.actions ?? [])) {
162
+ const n = a?.note;
163
+ const t = typeof n === 'string' ? n : n?.text;
164
+ if (t && (wordLen(t) > 8 || cjkLen(t) > 14)) warnings.push(`${where}: the note reads as a sentence ("${t}") — a callout is a label for the control it points at, a few words at most`);
165
+ }
166
+ // A question card is a check, and a check has three parts: the question, time to answer it, and
167
+ // the answer shown on the page. Without the wait it is a rhetorical question, which a promo asks
168
+ // and a lesson does not; without something marked, the viewer who guessed wrong is told they
169
+ // were wrong and never shown where to look.
170
+ const card = s.card && typeof s.card === 'object' ? s.card : null;
171
+ const asks = card && /question|quiz|check|问题|提问|思考|想一想|pregunta|frage/i.test(String(card.kicker ?? ''));
172
+ if (card && (card.think != null || asks)) {
173
+ questions++;
174
+ if (card.think == null) warnings.push(`${where} card: asks a question but does not wait for an answer — give it \`think: 3\` (seconds), so the viewer answers before the reveal`);
175
+ else if (!(card.think >= 2)) warnings.push(`${where} card: think: ${card.think} is too short to answer in — give the viewer at least 2 seconds, 3 to 4 for anything they have to work out`);
176
+ if (!s.zoom && !kinds.some((k) => ['mark', 'note', 'move', 'hover', 'click', 'slide', 'drag', 'type', 'select'].includes(k))) {
177
+ warnings.push(`${where}: this step answers the question card before it, but points at nothing — \`mark\` the row, value or line that is the answer, so a viewer who guessed wrong sees why`);
178
+ }
179
+ }
124
180
  }
125
181
  }
182
+ // A lesson that never asks is a lecture: the viewer cannot tell whether they followed it.
183
+ if (raw.purpose === 'course' && !questions && (raw.steps ?? []).some((s) => !s.hidden && s.say)) {
184
+ warnings.push('a lesson with no question card — check understanding at least once: `card: { kicker: Question, title: "…?", think: 3 }` on the step that then marks the answer');
185
+ }
186
+ if (sold.length) {
187
+ const ns = sold.map(([n]) => n);
188
+ const which = sold.length === 1 ? `step ${ns[0]}` : sold.length > 6 ? `steps ${ns[0]}–${ns.at(-1)} (${sold.length} captions)` : `steps ${ns.join(', ')}`;
189
+ warnings.push(`${which}: the caption bolds a payoff (${sold.slice(0, 3).map(([, c]) => `"${c}"`).join(', ')}${sold.length > 3 ? ', …' : ''}) — that is a promo's grammar, and it reaches the conclusion before the voice does; in a ${raw.purpose} a caption labels where the viewer is ("The E value column", "Part 2: read the results") and the narration explains`);
190
+ }
126
191
  if (raw.end_card) unknown(raw.end_card, END, 'end_card');
127
192
  if (raw.intro) unknown(raw.intro, END, 'intro');
128
193
  if (raw.hyperframes && typeof raw.hyperframes === 'object') unknown(raw.hyperframes, HYPERFRAMES, 'hyperframes');
@@ -226,12 +291,29 @@ export function check(file, { draft = false } = {}) {
226
291
  }
227
292
  // spoken lines that read as machine-written, or run on without a breath (the skill's "Writing the narration")
228
293
  if (cfg.voice) {
229
- const lines = [...visible.flatMap((s) => [[s.say, `step ${cfg.steps.indexOf(s) + 1}`], [s.card?.say, `step ${cfg.steps.indexOf(s) + 1} card`]]),
230
- [cfg.intro?.say, 'intro'], [cfg.endCard?.say, 'end card']].filter(([t]) => t);
231
- for (const [text, where] of lines) {
294
+ const teach = cfg.purpose === 'tutorial' || cfg.purpose === 'course';
295
+ const lines = [...visible.flatMap((s) => [[s.say, `step ${cfg.steps.indexOf(s) + 1}`, true], [s.card?.say, `step ${cfg.steps.indexOf(s) + 1} card`, 'card']]),
296
+ [cfg.intro?.say, 'intro', false], [cfg.endCard?.say, 'end card', false]].filter(([t]) => t);
297
+ for (const [text, where, isStep] of lines) {
232
298
  const { tells, long } = narrationLint(text);
233
299
  if (tells.length) warnings.push(`${where}: the narration sounds machine-written (${tells.map((x) => `"${x}"`).join(', ')}) — say it plainly, as you would out loud`);
234
- if (long) warnings.push(`${where}: a ${long} sentence — split it, so the voice gets a breath between the parts`);
300
+ // a tutorial's own sentence budget is tighter, and it replaces this one
301
+ if (long && !teach) warnings.push(`${where}: a ${long} sentence — split it, so the voice gets a breath between the parts`);
302
+ if (!teach) continue;
303
+ const t = teachingLint(text);
304
+ if (isStep === 'card' && t.heavy) {
305
+ // a concept card names one idea. The rule that goes with it, and the number that shows it,
306
+ // belong to the steps after the card, where the page can carry them
307
+ warnings.push(`${where}: ${an(t.heavy)} line on one card — a card names one idea; say the rule and read the number in the steps after it, where the page shows them`);
308
+ } else if (isStep && t.heavy) {
309
+ warnings.push(`${where}: ${an(t.heavy)} line for one step — one job per step in a ${cfg.purpose} (name the idea, do the action, or read the result); split it`);
310
+ }
311
+ if (t.long) {
312
+ warnings.push(`${where}: ${an(t.long)} sentence — a ${cfg.purpose} keeps them to about ${TEACH.sentenceWords} words (${TEACH.sentenceChars} Chinese characters), so the viewer holds one idea at a time`);
313
+ }
314
+ if (t.numbers.length > TEACH.numbers) {
315
+ warnings.push(`${where}: ${t.numbers.length} figures in one line (${t.numbers.join(', ')}) — nobody holds that many by ear; say the one that carries the point, round it out loud ("about 80 percent"), and leave the exact values on screen`);
316
+ }
235
317
  }
236
318
  }
237
319
  // the Chinese Kokoro voice needs a Python for its G2P: say so now, not after the recording
package/src/compose.mjs CHANGED
@@ -10,6 +10,26 @@ import { sceneSeconds, filmGround, mixColour, shadeColour } from './scenes.mjs';
10
10
  const esc = (s) => String(s).replace(/&/g, '&amp;').replace(/</g, '&lt;').replace(/>/g, '&gt;');
11
11
  const md = (s, accent) => esc(s).replace(/\*\*(.+?)\*\*/g, `<b style="color:${accent};font-weight:700">$1</b>`);
12
12
 
13
+ // The camera path as one ffmpeg expression: a value per keyframe interval, smoothstepped between the
14
+ // two ends. Written as a FLAT SUM of gated terms, not a nested if/else chain — ffmpeg's expression
15
+ // parser guards recursion at 100 levels, and each zoom contributes four keyframes, so a tutorial
16
+ // written to "one job per step" (30-odd zooms, 130-odd intervals) silently exceeded it and the whole
17
+ // compose failed with "Missing ')' or too many args". A `+` chain is parsed iteratively, so the depth
18
+ // is constant however long the film is. The gates partition the timeline, so exactly one term is ever
19
+ // non-zero: the first interval also covers everything before it, and the last keyframe holds to the end.
20
+ export function keyframeExpr(kf, get, fps) {
21
+ const n = kf.length, terms = [];
22
+ for (let i = 0; i < n - 1; i++) {
23
+ const t0 = kf[i][0], t1 = kf[i + 1][0], v0 = get(kf[i]), v1 = get(kf[i + 1]);
24
+ const p = `clip((T-${t0.toFixed(3)})/${Math.max(1e-3, t1 - t0).toFixed(3)},0,1)`;
25
+ const body = v0 === v1 ? `${v0}` : `(${v0}+(${(v1 - v0).toFixed(4)})*${p}*${p}*(3-2*${p}))`;
26
+ const gate = i === 0 ? `lt(T,${t1.toFixed(3)})` : `(gte(T,${t0.toFixed(3)})*lt(T,${t1.toFixed(3)}))`;
27
+ terms.push(`${gate}*${body}`);
28
+ }
29
+ terms.push(`gte(T,${kf[n - 1][0].toFixed(3)})*${get(kf[n - 1])}`);
30
+ return `(${terms.join('+')})`.replaceAll('T', `(on/${fps})`);
31
+ }
32
+
13
33
  export async function compose(cfg, { log = console.log, voiceLens = { steps: {}, endCard: null } } = {}) {
14
34
  const warnings = [];
15
35
  const warn = (m) => { warnings.push(m); log(`warning: ${m}`); };
@@ -192,11 +212,23 @@ export async function compose(cfg, { log = console.log, voiceLens = { steps: {},
192
212
  ...cards.map((c) => [c.t0, c.t1]),
193
213
  ...segs.filter((g) => voiceLens.steps[g.i] != null).map((g) => [g.o0 + g.sayAt, g.o0 + g.sayAt + voiceLens.steps[g.i]]),
194
214
  ];
215
+ // A promo must never stall. A tutorial is the opposite: the viewer is doing this too, and needs a
216
+ // beat to read the result, find the control on their own screen, and catch up. So a lesson gets a
217
+ // longer allowance, and a step that asked to hold (`pause`) means it — only a hold long enough to
218
+ // feel broken is worth a word there.
219
+ const teach = cfg.purpose === 'tutorial' || cfg.purpose === 'course';
220
+ const held = (i) => (cfg.steps[i]?.actions ?? []).some((a) => a && (a.pause != null || a.wait != null));
195
221
  let from = null;
196
222
  const report = (to) => {
197
- if (from != null && to - from >= 3) {
223
+ if (from != null) {
198
224
  const ns = segs.filter((x) => x.o1 > from + 1e-6 && x.o0 < to - 1e-6).map((x) => x.i + 1);
199
- warn(`${ns.length > 1 ? `steps ${ns[0]}–${ns.at(-1)}` : `step ${ns[0] ?? (segs.at(-1)?.i ?? cfg.steps.length - 1) + 1}`}: ${(to - from).toFixed(1)}s of dead air at ${from.toFixed(1)}–${to.toFixed(1)}s (nothing moves and nothing is said) — shorten the wait or pause, hide the load, or narrate what is happening`);
225
+ const limit = !teach ? 3 : ns.length && ns.every((k) => held(k - 1)) ? 8 : 6;
226
+ const fix = teach
227
+ ? 'say what is happening or what to look at, or let the step hold on purpose with `pause`'
228
+ : 'shorten the wait or pause, hide the load, or narrate what is happening';
229
+ if (to - from >= limit) {
230
+ warn(`${ns.length > 1 ? `steps ${ns[0]}–${ns.at(-1)}` : `step ${ns[0] ?? (segs.at(-1)?.i ?? cfg.steps.length - 1) + 1}`}: ${(to - from).toFixed(1)}s of dead air at ${from.toFixed(1)}–${to.toFixed(1)}s (nothing moves and nothing is said) — ${fix}`);
231
+ }
200
232
  }
201
233
  from = null;
202
234
  };
@@ -270,16 +302,7 @@ export async function compose(cfg, { log = console.log, voiceLens = { steps: {},
270
302
  }
271
303
  if (kf.at(-1)[0] < TOTAL) kf.push([TOTAL, 1, C0]);
272
304
  const S = 2; // supersample to avoid zoompan's integer-pixel jitter
273
- const piece = (get) => {
274
- let e = String(get(kf.at(-1)));
275
- for (let i = kf.length - 2; i >= 0; i--) {
276
- const t0 = kf[i][0], t1 = kf[i + 1][0], v0 = get(kf[i]), v1 = get(kf[i + 1]);
277
- const p = `clip((T-${t0.toFixed(3)})/${Math.max(1e-3, t1 - t0).toFixed(3)},0,1)`;
278
- const body = v0 === v1 ? `${v0}` : `(${v0}+(${(v1 - v0).toFixed(4)})*${p}*${p}*(3-2*${p}))`;
279
- e = `if(lt(T,${t1.toFixed(3)}),${body},${e})`;
280
- }
281
- return e.replaceAll('T', `(on/${FPS})`);
282
- };
305
+ const piece = (get) => keyframeExpr(kf, get, FPS);
283
306
  const zExpr = piece((k) => k[1]);
284
307
  const cxExpr = piece((k) => +(k[2][0] * S).toFixed(1));
285
308
  const cyExpr = piece((k) => +(k[2][1] * S).toFixed(1));
package/src/record.mjs CHANGED
@@ -423,7 +423,10 @@ export async function record(cfg, { log = console.log, inspectAfter = null } = {
423
423
  }
424
424
  const { width: VW, height: VH } = cfg.viewport;
425
425
 
426
- const browser = await chromium.launch();
426
+ // STORYSHOT_GPU=1 renders WebGL on the GPU (Metal on macOS) instead of SwiftShader;
427
+ // 3D pages otherwise record at ~10 fps and every step comes out rushed.
428
+ const gpu = /^(1|true|yes)$/i.test(process.env.STORYSHOT_GPU || '');
429
+ const browser = await chromium.launch(gpu ? { args: ['--enable-gpu', '--ignore-gpu-blocklist', ...(process.platform === 'darwin' ? ['--use-angle=metal'] : [])] } : {});
427
430
  const context = await sessionContext(browser, {
428
431
  viewport: cfg.viewport, deviceScaleFactor: cfg.scale, colorScheme: cfg.colorScheme, storageState: cfg.storageState, ignoreHTTPSErrors: cfg.ignoreHTTPSErrors,
429
432
  permissions: ['clipboard-read', 'clipboard-write'], // for `paste`