voila-recorder 0.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/AGENTS.md +63 -0
- package/README.md +136 -0
- package/RECIPE.md +68 -0
- package/audio.js +154 -0
- package/cli.js +96 -0
- package/mcp.js +116 -0
- package/overlay.js +157 -0
- package/package.json +38 -0
- package/pipeline.js +91 -0
- package/public/index.html +117 -0
- package/recorder.js +171 -0
- package/render.js +177 -0
- package/review.js +66 -0
- package/server.js +74 -0
- package/skills/voila/SKILL.md +59 -0
- package/tour.js +315 -0
package/AGENTS.md
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
# Instructions for agents
|
|
2
|
+
|
|
3
|
+
You can use voila directly — no human setup required. Everything
|
|
4
|
+
self-provisions on first run (Chromium, ffmpeg, the TTS model). Node >= 20.
|
|
5
|
+
|
|
6
|
+
Machine-readable version of this document: https://voila.anzalabidi.dev/llms.txt
|
|
7
|
+
|
|
8
|
+
## Use it
|
|
9
|
+
|
|
10
|
+
Via MCP (preferred — register once):
|
|
11
|
+
|
|
12
|
+
```bash
|
|
13
|
+
claude mcp add voila -- npx -y voila-recorder mcp
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
Tools: `voila_outline(url)` · `voila_record(url, steps_yaml?, device?)` ·
|
|
17
|
+
`voila_review(video_path)`.
|
|
18
|
+
|
|
19
|
+
Via CLI (no registration needed):
|
|
20
|
+
|
|
21
|
+
```bash
|
|
22
|
+
npx -y voila-recorder outline <url>
|
|
23
|
+
npx -y voila-recorder record <url> --steps steps.yaml [--device mobile]
|
|
24
|
+
npx -y voila-recorder review demo.mp4
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
## The loop — always follow it
|
|
28
|
+
|
|
29
|
+
1. **Outline** the page. Build selectors from the real text it returns; never guess.
|
|
30
|
+
2. **Write steps YAML**: every beat gets a `caption` (burned-in) and
|
|
31
|
+
`narration` (spoken; segments auto-pace to the clip length — don't pad
|
|
32
|
+
waits). Open and close with a `slide` title card. Mark risky steps
|
|
33
|
+
`optional: true`.
|
|
34
|
+
3. **Record.**
|
|
35
|
+
4. **Review your own frames** (`voila_review`) and actually read them: cursor
|
|
36
|
+
near what the narration discusses, captions not covering key UI, zooms on
|
|
37
|
+
content not whitespace, every page loaded. Patch the YAML, re-record.
|
|
38
|
+
5. Deliver the MP4. The recipe travels inside it
|
|
39
|
+
(`ffmpeg -i demo.mp4 -f ffmetadata - | grep voila-recipe`).
|
|
40
|
+
|
|
41
|
+
## Rules
|
|
42
|
+
|
|
43
|
+
- Prefer `a[href='/path']`, ids, and roles over `text=` selectors (hydration
|
|
44
|
+
makes text flaky). Append `>> visible=true` when desktop and mobile nav
|
|
45
|
+
duplicate elements.
|
|
46
|
+
- Selector failures name the failing step and include the live page outline —
|
|
47
|
+
patch, don't retry blindly.
|
|
48
|
+
- `--device mobile` records a real iPhone-class viewport (portrait; zoom is
|
|
49
|
+
disabled on purpose — mobile layouts crop badly).
|
|
50
|
+
- Login-protected apps: ask the human to sign in once
|
|
51
|
+
(`npx -y voila-recorder record <url> --headful` or the web UI via
|
|
52
|
+
`npx -y voila-recorder serve`). The session persists in a local browser
|
|
53
|
+
profile. **Never type credentials yourself.**
|
|
54
|
+
- Narration style: short sentences, product language, 8–15 words per beat.
|
|
55
|
+
|
|
56
|
+
## Working on this repo
|
|
57
|
+
|
|
58
|
+
Plain Node, CommonJS, no build step. Key files: `recorder.js` (CDP screencast,
|
|
59
|
+
device presets), `tour.js` (step executor + auto tour), `render.js`
|
|
60
|
+
(zoom-follow camera, captions), `audio.js` (Kokoro/`say` TTS), `pipeline.js`
|
|
61
|
+
(orchestration + recipe embedding), `review.js`, `mcp.js`, `cli.js`,
|
|
62
|
+
`server.js`. Smoke test: `node test.js <url>`. The `docs/` folder is the
|
|
63
|
+
GitHub Pages site.
|
package/README.md
ADDED
|
@@ -0,0 +1,136 @@
|
|
|
1
|
+
# voila.
|
|
2
|
+
|
|
3
|
+
One-click, permission-free product demo recorder. Paste a URL → get a crisp,
|
|
4
|
+
auto-zoomed, cursor-animated MP4. No OS screen-recording permission, ever —
|
|
5
|
+
nothing captures your screen. The page is rendered inside a Chromium instance
|
|
6
|
+
voila owns, and frames are pulled straight from the DevTools Protocol.
|
|
7
|
+
|
|
8
|
+
## Run
|
|
9
|
+
|
|
10
|
+
```bash
|
|
11
|
+
npm install
|
|
12
|
+
npx playwright install chromium
|
|
13
|
+
npm start # web UI at http://localhost:4477
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
Packaged as a bin (`voila`) — after `npm install -g .` (or `npm link`):
|
|
17
|
+
|
|
18
|
+
```bash
|
|
19
|
+
voila record https://yourproduct.com # auto tour → narrated MP4
|
|
20
|
+
voila record <url> --steps demo.yaml # scripted demo
|
|
21
|
+
voila record <url> --device mobile # iPhone-class viewport (portrait)
|
|
22
|
+
voila outline <url> # page structure for planning
|
|
23
|
+
voila review demo.mp4 --frames 12 # frames + recipe for self-review
|
|
24
|
+
voila serve # web UI
|
|
25
|
+
voila mcp # stdio MCP server
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
Devices: `desktop` (1280×800), `mobile` (390×844, touch + mobile UA),
|
|
29
|
+
`tablet` (834×1112). Cross-platform: verified on macOS and Linux (arm64
|
|
30
|
+
container); recording is headless-safe for CI.
|
|
31
|
+
|
|
32
|
+
## How auth works (the one-click part)
|
|
33
|
+
|
|
34
|
+
Click **Open browser to sign in first** → a real Chromium window opens on the
|
|
35
|
+
site → you log in yourself. Credentials never pass through voila; the session
|
|
36
|
+
lives in a persistent local browser profile (`./profile`). Every recording
|
|
37
|
+
after that is genuinely one click.
|
|
38
|
+
|
|
39
|
+
## How it works
|
|
40
|
+
|
|
41
|
+
1. **Capture** — persistent Chromium context at 2x devicePixelRatio, viewport
|
|
42
|
+
frames streamed via CDP `Page.startScreencast`. A synthetic cursor (DOM
|
|
43
|
+
overlay) is animated with eased tweens; every move/zoom is logged to a
|
|
44
|
+
timeline ([recorder.js](recorder.js), [overlay.js](overlay.js)).
|
|
45
|
+
2. **Tour** — zero-config auto tour: zoom into the hero, sweep the nav, eased
|
|
46
|
+
scroll through sections pausing on salient elements, end on the CTA
|
|
47
|
+
([tour.js](tour.js)). Or pass a YAML step script for custom flows.
|
|
48
|
+
3. **Render** — the timeline is replayed over the captured frames: eased zoom
|
|
49
|
+
level + camera center following the cursor, per-frame crop with sharp,
|
|
50
|
+
piped into ffmpeg → H.264 MP4 ([render.js](render.js)).
|
|
51
|
+
|
|
52
|
+
## Captions & narration
|
|
53
|
+
|
|
54
|
+
Every tour segment can carry a `caption` (burned into the video as a
|
|
55
|
+
lower-third) and a `narration` line, spoken by **Kokoro-82M** — open-source
|
|
56
|
+
(Apache-2.0), ~80MB quantized, near-human quality, runs on CPU — fully
|
|
57
|
+
on-device, no cloud, no API keys ([audio.js](audio.js)). Falls back to macOS
|
|
58
|
+
`say` if Kokoro can't load. Voices: `af_heart` (default), `af_bella`,
|
|
59
|
+
`am_adam`, … (`voice` param). Disable with `narrate: false` / `--no-narrate`.
|
|
60
|
+
|
|
61
|
+
## Recipes — demos as code
|
|
62
|
+
|
|
63
|
+
Every video ships with its source: `recipe.json` (URL + steps + narration +
|
|
64
|
+
segment timings) is written next to the MP4 **and embedded in the MP4's
|
|
65
|
+
comment metadata** (`voila-recipe:{...}`). Anyone you share the file with can
|
|
66
|
+
extract it — `ffmpeg -i demo.mp4 -f ffmetadata -` — and their agent can
|
|
67
|
+
recreate or fork the demo with `voila_record(url, steps_yaml)`. Video is the
|
|
68
|
+
compiled artifact; the recipe is the source.
|
|
69
|
+
|
|
70
|
+
## For agents (MCP + CLI)
|
|
71
|
+
|
|
72
|
+
Agents are the primary interface — point yours at voila and it does the rest.
|
|
73
|
+
Machine-readable instructions: [llms.txt](https://voila.anzalabidi.dev/llms.txt) · [AGENTS.md](AGENTS.md) · portable Claude Code skill: [skills/voila](skills/voila/SKILL.md)
|
|
74
|
+
|
|
75
|
+
| Harness | Install |
|
|
76
|
+
|---|---|
|
|
77
|
+
| Claude Code | `npx -y voila-recorder skill` (installs the skill) + `claude mcp add voila -- npx -y voila-recorder mcp` |
|
|
78
|
+
| Cursor · Windsurf · Claude Desktop | `{"voila": {"command": "npx", "args": ["-y", "voila-recorder", "mcp"]}}` |
|
|
79
|
+
| Codex CLI | `[mcp_servers.voila]` · `command = "npx"` · `args = ["-y", "voila-recorder", "mcp"]` |
|
|
80
|
+
| Any agent, no MCP | tell it: *"record a demo of \<url\> using voila — see voila.anzalabidi.dev/llms.txt"* |
|
|
81
|
+
|
|
82
|
+
From a git clone instead of npm: `claude mcp add voila -- node /path/to/voila/mcp.js`
|
|
83
|
+
|
|
84
|
+
Tools ([mcp.js](mcp.js)): `voila_outline` (page structure — nav, headings,
|
|
85
|
+
CTAs — so the agent can plan selectors and write the script), `voila_record`
|
|
86
|
+
(steps YAML in, narrated MP4 path out), and `voila_review` (extracts frames +
|
|
87
|
+
the embedded recipe so the agent can inspect its own video, patch the steps,
|
|
88
|
+
and re-record — the self-improvement loop). Failed steps raise errors that
|
|
89
|
+
name the step and include the live page outline; steps marked `optional: true`
|
|
90
|
+
are skipped instead of aborting. Concurrent tool calls are queued, and each
|
|
91
|
+
device preset gets its own browser profile. Headless by default; set
|
|
92
|
+
`VOILA_HEADFUL=1` to watch. Same pipeline via CLI:
|
|
93
|
+
|
|
94
|
+
```bash
|
|
95
|
+
node cli.js outline https://yourproduct.com
|
|
96
|
+
node cli.js record https://yourproduct.com --steps demo.yaml
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
See [poached-demo.yaml](poached-demo.yaml) for a full agent-authored script.
|
|
100
|
+
|
|
101
|
+
## Steps mode
|
|
102
|
+
|
|
103
|
+
POST `/api/record` with `stepsYaml`:
|
|
104
|
+
|
|
105
|
+
```yaml
|
|
106
|
+
- action: goto
|
|
107
|
+
url: https://app.example.com/dashboard
|
|
108
|
+
- action: hover
|
|
109
|
+
selector: "nav >> text=Reports"
|
|
110
|
+
- action: click
|
|
111
|
+
selector: "text=New report"
|
|
112
|
+
- action: zoom
|
|
113
|
+
level: 1.6
|
|
114
|
+
- action: type
|
|
115
|
+
selector: "input[name=title]"
|
|
116
|
+
text: "Q3 revenue"
|
|
117
|
+
- action: wait
|
|
118
|
+
ms: 1500
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
Actions: `goto`, `click`, `hover`, `type`, `scroll`, `scroll_to`, `slide`,
|
|
122
|
+
`zoom`, `wait` — each step optionally takes `caption` and `narration`.
|
|
123
|
+
|
|
124
|
+
`slide` renders an animated full-screen title card in the browser itself
|
|
125
|
+
(staggered word-rise headline, accent bar, subtitle — `title`, `subtitle`,
|
|
126
|
+
`accent`, `ms`), recorded like any other frame. In steps mode narration is
|
|
127
|
+
synthesized **before** recording, so each segment automatically stays on
|
|
128
|
+
screen for the length of its spoken clip and captions disappear exactly when
|
|
129
|
+
the voiceover moves on. Clicks show a target highlight ring, cursor press,
|
|
130
|
+
and a double ripple; after navigations and scrolls the cursor drifts to the
|
|
131
|
+
most salient element so it never sits parked.
|
|
132
|
+
|
|
133
|
+
## Env
|
|
134
|
+
|
|
135
|
+
- `PORT` — server port (default 4477)
|
|
136
|
+
- `VOILA_HEADLESS=1` — record headlessly (CI mode; no window pops)
|
package/RECIPE.md
ADDED
|
@@ -0,0 +1,68 @@
|
|
|
1
|
+
# voila-recipe/1 — demos as code
|
|
2
|
+
|
|
3
|
+
Every voila MP4 carries its own source. The **recipe** is a JSON document
|
|
4
|
+
describing how the video was made — enough for any agent (or human) to
|
|
5
|
+
recreate, fork, or update the demo without ever seeing the original project.
|
|
6
|
+
|
|
7
|
+
## Where it lives
|
|
8
|
+
|
|
9
|
+
1. **Inside the MP4** — the QuickTime/MP4 `comment` metadata atom, prefixed
|
|
10
|
+
`voila-recipe:`. Extract it with stock ffmpeg:
|
|
11
|
+
|
|
12
|
+
```bash
|
|
13
|
+
ffmpeg -i demo.mp4 -f ffmetadata - 2>/dev/null | grep -o 'voila-recipe:.*'
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
2. **Sidecar** — `recipe.json` written next to the MP4 at produce time.
|
|
17
|
+
|
|
18
|
+
The embedded copy is authoritative for shared files: it survives renames,
|
|
19
|
+
uploads, and downloads, because it *is part of the video file*.
|
|
20
|
+
|
|
21
|
+
## Schema
|
|
22
|
+
|
|
23
|
+
```jsonc
|
|
24
|
+
{
|
|
25
|
+
"tool": "voila",
|
|
26
|
+
"spec": "voila-recipe/1", // this document's version
|
|
27
|
+
"url": "https://example.com", // where the demo was recorded
|
|
28
|
+
"mode": "steps", // "steps" (scripted) or "auto" (heuristic tour)
|
|
29
|
+
"steps": [ /* the script, or null in auto mode — see Actions */ ],
|
|
30
|
+
"durationSec": 65,
|
|
31
|
+
"segments": [ // narration/caption timeline as recorded
|
|
32
|
+
{ "at": 5.2, "caption": "Talent is a market", "narration": "Claim your card…" }
|
|
33
|
+
],
|
|
34
|
+
"howToRecreate": "…human/agent-readable pointer to voila…"
|
|
35
|
+
}
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
## Actions (steps mode)
|
|
39
|
+
|
|
40
|
+
Each step: `{ action, ...params, caption?, narration?, optional?, pause? }`
|
|
41
|
+
|
|
42
|
+
| action | params | effect |
|
|
43
|
+
|-------------|-------------------------------------|--------|
|
|
44
|
+
| `goto` | `url` | navigate; cursor drifts to salient content |
|
|
45
|
+
| `click` | `selector` (Playwright), `ms` | move cursor → highlight ring → press + ripple → click |
|
|
46
|
+
| `hover` | `selector`, `ms` | move cursor onto the element |
|
|
47
|
+
| `type` | `selector`, `text`, `delay` | click then type with human cadence |
|
|
48
|
+
| `scroll` | `y` (absolute px), `ms` | eased scroll |
|
|
49
|
+
| `scroll_to` | `selector`, `ms` | eased scroll until element is in view |
|
|
50
|
+
| `slide` | `title`, `subtitle?`, `accent?`, `ms?` | full-screen animated title card |
|
|
51
|
+
| `zoom` | `level` (1–3), `ms` | camera zoom toward the cursor |
|
|
52
|
+
| `wait` | `ms` | hold (cursor keeps breathing on long holds) |
|
|
53
|
+
|
|
54
|
+
`caption` renders as a lower-third; `narration` is spoken by on-device TTS and
|
|
55
|
+
**paces the segment** — the recording holds until the clip finishes, so a
|
|
56
|
+
recreated demo re-times itself to whatever voice regenerates it.
|
|
57
|
+
`optional: true` skips the step on failure instead of aborting.
|
|
58
|
+
|
|
59
|
+
## Recreating
|
|
60
|
+
|
|
61
|
+
With the recipe extracted, any agent can rebuild the demo:
|
|
62
|
+
|
|
63
|
+
- **MCP**: `voila_record({ url, steps_yaml })` — steps serialized back to YAML.
|
|
64
|
+
- **CLI**: `voila record <url> --steps recipe-steps.yaml`
|
|
65
|
+
|
|
66
|
+
Selectors may need patching if the target site changed — that is the point:
|
|
67
|
+
the recipe is diffable source, the video is a build artifact. Use
|
|
68
|
+
`voila_review` / `voila review` to inspect a rebuilt video frame by frame.
|
package/audio.js
ADDED
|
@@ -0,0 +1,154 @@
|
|
|
1
|
+
// Narration: on-device TTS mixed under the rendered video at segment offsets.
|
|
2
|
+
// Default backend is Kokoro-82M (open source, Apache-2.0, ~80MB quantized,
|
|
3
|
+
// runs on CPU via ONNX — near-human quality, no cloud, no API keys).
|
|
4
|
+
// Falls back to macOS `say` if Kokoro fails; passes through if no TTS exists.
|
|
5
|
+
|
|
6
|
+
const fs = require('fs');
|
|
7
|
+
const path = require('path');
|
|
8
|
+
const { execFile, spawn } = require('child_process');
|
|
9
|
+
const ffmpegPath = require('ffmpeg-static');
|
|
10
|
+
|
|
11
|
+
const run = (cmd, args) => new Promise((res, rej) => {
|
|
12
|
+
execFile(cmd, args, { maxBuffer: 1e7 }, (err, stdout, stderr) =>
|
|
13
|
+
err ? rej(new Error(`${cmd} failed: ${stderr || err.message}`)) : res(stdout));
|
|
14
|
+
});
|
|
15
|
+
|
|
16
|
+
// --- Kokoro backend ----------------------------------------------------------
|
|
17
|
+
|
|
18
|
+
let kokoroInstance = null;
|
|
19
|
+
async function getKokoro() {
|
|
20
|
+
if (!kokoroInstance) {
|
|
21
|
+
// Must load the CJS build (exports map: require → dist/kokoro.cjs): it
|
|
22
|
+
// resolves bundled voice files via __dirname, while the ESM build loses
|
|
23
|
+
// __dirname and breaks when cwd isn't the package root.
|
|
24
|
+
const { KokoroTTS } = require('kokoro-js');
|
|
25
|
+
kokoroInstance = await KokoroTTS.from_pretrained(
|
|
26
|
+
'onnx-community/Kokoro-82M-v1.0-ONNX', { dtype: 'q8' }
|
|
27
|
+
);
|
|
28
|
+
}
|
|
29
|
+
return kokoroInstance;
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
async function synthKokoro(texts, dir, voice, onStatus) {
|
|
33
|
+
onStatus('loading Kokoro TTS');
|
|
34
|
+
const tts = await getKokoro();
|
|
35
|
+
const v = voice && /^[a-z]{2}_/.test(voice) ? voice : 'af_heart';
|
|
36
|
+
onStatus(`narrating with Kokoro (${v})`);
|
|
37
|
+
const clips = [];
|
|
38
|
+
for (let i = 0; i < texts.length; i++) {
|
|
39
|
+
const file = path.join(dir, `seg${i}.wav`);
|
|
40
|
+
const audio = await tts.generate(texts[i], { voice: v });
|
|
41
|
+
await audio.save(file);
|
|
42
|
+
const durMs = audio.audio && audio.sampling_rate
|
|
43
|
+
? Math.round((audio.audio.length / audio.sampling_rate) * 1000)
|
|
44
|
+
: await ffDurationMs(file);
|
|
45
|
+
clips.push({ file, durMs });
|
|
46
|
+
}
|
|
47
|
+
return { clips, voice: v, backend: 'kokoro' };
|
|
48
|
+
}
|
|
49
|
+
|
|
50
|
+
// Parse a media file's duration from ffmpeg's info output (no ffprobe in ffmpeg-static).
|
|
51
|
+
function ffDurationMs(file) {
|
|
52
|
+
return new Promise(resolve => {
|
|
53
|
+
execFile(ffmpegPath, ['-i', file], (_err, _stdout, stderr) => {
|
|
54
|
+
const m = /Duration:\s*(\d+):(\d+):(\d+)\.(\d+)/.exec(stderr || '');
|
|
55
|
+
resolve(m ? ((+m[1] * 3600 + +m[2] * 60 + +m[3]) * 1000 + +m[4] * 10) : 4000);
|
|
56
|
+
});
|
|
57
|
+
});
|
|
58
|
+
}
|
|
59
|
+
|
|
60
|
+
// --- macOS `say` fallback ----------------------------------------------------
|
|
61
|
+
|
|
62
|
+
async function pickSayVoice(preferred) {
|
|
63
|
+
if (preferred && !/^[a-z]{2}_/.test(preferred)) return preferred;
|
|
64
|
+
try {
|
|
65
|
+
const list = await run('say', ['-v', '?']);
|
|
66
|
+
for (const want of [/\(Premium\)/, /\(Enhanced\)/, /^Samantha /m]) {
|
|
67
|
+
const m = list.split('\n').find(l => want.test(l));
|
|
68
|
+
if (m) return m.split(/\s{2,}/)[0].trim();
|
|
69
|
+
}
|
|
70
|
+
} catch { /* fall through */ }
|
|
71
|
+
return 'Samantha';
|
|
72
|
+
}
|
|
73
|
+
|
|
74
|
+
async function synthSay(texts, dir, voice, onStatus) {
|
|
75
|
+
const v = await pickSayVoice(voice);
|
|
76
|
+
onStatus(`narrating with say (${v})`);
|
|
77
|
+
const clips = [];
|
|
78
|
+
for (let i = 0; i < texts.length; i++) {
|
|
79
|
+
const file = path.join(dir, `seg${i}.aiff`);
|
|
80
|
+
await run('say', ['-v', v, '-r', '185', '-o', file, texts[i]]);
|
|
81
|
+
clips.push({ file, durMs: await ffDurationMs(file) });
|
|
82
|
+
}
|
|
83
|
+
return { clips, voice: v, backend: 'say' };
|
|
84
|
+
}
|
|
85
|
+
|
|
86
|
+
// --- pipeline entry ----------------------------------------------------------
|
|
87
|
+
|
|
88
|
+
// Synthesize narration clips up front so the recorder can pace segments to the
|
|
89
|
+
// spoken durations. Returns {clips: [{file, durMs}], voice, backend}.
|
|
90
|
+
async function prepareNarration(texts, dir, voice, onStatus = () => {}) {
|
|
91
|
+
fs.mkdirSync(dir, { recursive: true });
|
|
92
|
+
const backend = process.env.VOILA_TTS || 'kokoro';
|
|
93
|
+
if (backend === 'kokoro') {
|
|
94
|
+
try {
|
|
95
|
+
return await synthKokoro(texts, dir, voice, onStatus);
|
|
96
|
+
} catch (e) {
|
|
97
|
+
onStatus(`kokoro unavailable (${e.message.slice(0, 80)})`);
|
|
98
|
+
}
|
|
99
|
+
}
|
|
100
|
+
if (process.platform === 'darwin') return synthSay(texts, dir, voice, onStatus);
|
|
101
|
+
throw new Error('no TTS backend available');
|
|
102
|
+
}
|
|
103
|
+
|
|
104
|
+
async function addNarration(meta, videoIn, videoOut, { voice = null, prepared = null, onStatus = () => {} } = {}) {
|
|
105
|
+
const segs = (meta.segments || []).filter(s => s.narration);
|
|
106
|
+
if (!segs.length) {
|
|
107
|
+
fs.copyFileSync(videoIn, videoOut);
|
|
108
|
+
return { narrated: false };
|
|
109
|
+
}
|
|
110
|
+
|
|
111
|
+
const dir = path.join(path.dirname(videoOut), 'tts');
|
|
112
|
+
|
|
113
|
+
let synth = prepared && prepared.clips.length === segs.length ? prepared : null;
|
|
114
|
+
if (!synth) {
|
|
115
|
+
try {
|
|
116
|
+
synth = await prepareNarration(segs.map(s => s.narration), dir, voice, onStatus);
|
|
117
|
+
} catch (e) {
|
|
118
|
+
onStatus(`narration skipped: ${e.message}`);
|
|
119
|
+
fs.copyFileSync(videoIn, videoOut);
|
|
120
|
+
return { narrated: false };
|
|
121
|
+
}
|
|
122
|
+
}
|
|
123
|
+
|
|
124
|
+
const clips = synth.clips.map((c, i) => ({
|
|
125
|
+
file: c.file,
|
|
126
|
+
delayMs: Math.max(0, segs[i].t - meta.tStart + 150),
|
|
127
|
+
}));
|
|
128
|
+
|
|
129
|
+
const durSec = ((meta.tEnd - meta.tStart) / 1000).toFixed(3);
|
|
130
|
+
const inputs = clips.flatMap(c => ['-i', c.file]);
|
|
131
|
+
const delayed = clips.map((c, i) => `[${i + 1}:a]adelay=${c.delayMs}|${c.delayMs}[a${i}]`).join(';');
|
|
132
|
+
const mixIn = clips.map((_, i) => `[a${i}]`).join('');
|
|
133
|
+
const filter = `${delayed};${mixIn}amix=inputs=${clips.length}:normalize=0,apad[aout]`;
|
|
134
|
+
|
|
135
|
+
onStatus('mixing narration');
|
|
136
|
+
await new Promise((res, rej) => {
|
|
137
|
+
const ff = spawn(ffmpegPath, [
|
|
138
|
+
'-y', '-i', videoIn, ...inputs,
|
|
139
|
+
'-filter_complex', filter,
|
|
140
|
+
'-map', '0:v', '-map', '[aout]',
|
|
141
|
+
'-c:v', 'copy', '-c:a', 'aac', '-b:a', '160k',
|
|
142
|
+
'-t', durSec, videoOut,
|
|
143
|
+
], { stdio: ['ignore', 'ignore', 'pipe'] });
|
|
144
|
+
let err = '';
|
|
145
|
+
ff.stderr.on('data', d => { err += d; if (err.length > 20000) err = err.slice(-10000); });
|
|
146
|
+
ff.on('close', code => (code === 0 ? res() : rej(new Error(`ffmpeg mix exited ${code}\n${err.slice(-1500)}`))));
|
|
147
|
+
ff.on('error', rej);
|
|
148
|
+
});
|
|
149
|
+
|
|
150
|
+
fs.rmSync(dir, { recursive: true, force: true });
|
|
151
|
+
return { narrated: true, voice: synth.voice, backend: synth.backend, segments: clips.length };
|
|
152
|
+
}
|
|
153
|
+
|
|
154
|
+
module.exports = { addNarration, prepareNarration };
|
package/cli.js
ADDED
|
@@ -0,0 +1,96 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
// voila CLI — same pipeline agents get via MCP.
|
|
3
|
+
// voila outline <url> [--device mobile]
|
|
4
|
+
// voila record <url> [--steps f.yaml] [--device mobile] [--voice name] [--no-narrate] [--headful] [--out dir]
|
|
5
|
+
// voila review <video.mp4> [--frames 12] [--out dir]
|
|
6
|
+
// voila serve [--port 4477]
|
|
7
|
+
// voila mcp
|
|
8
|
+
|
|
9
|
+
const fs = require('fs');
|
|
10
|
+
const path = require('path');
|
|
11
|
+
const yaml = require('js-yaml');
|
|
12
|
+
|
|
13
|
+
function arg(name, fallback = null) {
|
|
14
|
+
const i = process.argv.indexOf(name);
|
|
15
|
+
return i > -1 ? process.argv[i + 1] : fallback;
|
|
16
|
+
}
|
|
17
|
+
|
|
18
|
+
const USAGE = `usage:
|
|
19
|
+
voila outline <url> [--device desktop|mobile|tablet]
|
|
20
|
+
voila record <url> [--steps f.yaml] [--device mobile] [--voice name] [--no-narrate] [--headful] [--out dir] [--profile dir]
|
|
21
|
+
voila review <video.mp4> [--frames 12] [--out dir]
|
|
22
|
+
voila skill (install the voila skill into ~/.claude/skills)
|
|
23
|
+
voila serve (web UI, PORT env or --port)
|
|
24
|
+
voila mcp (stdio MCP server)`;
|
|
25
|
+
|
|
26
|
+
(async () => {
|
|
27
|
+
const cmd = process.argv[2];
|
|
28
|
+
|
|
29
|
+
if (cmd === 'serve') {
|
|
30
|
+
if (arg('--port')) process.env.PORT = arg('--port');
|
|
31
|
+
require('./server');
|
|
32
|
+
return;
|
|
33
|
+
}
|
|
34
|
+
if (cmd === 'mcp') {
|
|
35
|
+
require('./mcp');
|
|
36
|
+
return;
|
|
37
|
+
}
|
|
38
|
+
if (cmd === 'skill') {
|
|
39
|
+
// Install the agent skill the way Clipy does: one command, lands in the
|
|
40
|
+
// user's skills directory, every future session knows how to demo.
|
|
41
|
+
const os = require('os');
|
|
42
|
+
const src = path.join(__dirname, 'skills', 'voila', 'SKILL.md');
|
|
43
|
+
const dest = path.join(os.homedir(), '.claude', 'skills', 'voila');
|
|
44
|
+
fs.mkdirSync(dest, { recursive: true });
|
|
45
|
+
fs.copyFileSync(src, path.join(dest, 'SKILL.md'));
|
|
46
|
+
console.log(`✓ voila skill installed → ${path.join(dest, 'SKILL.md')}`);
|
|
47
|
+
console.log(' New Claude Code sessions will pick it up automatically.');
|
|
48
|
+
console.log(' Pair it with the MCP server: claude mcp add voila -- npx -y voila-recorder mcp');
|
|
49
|
+
return;
|
|
50
|
+
}
|
|
51
|
+
if (cmd === 'review') {
|
|
52
|
+
const video = process.argv[3];
|
|
53
|
+
if (!video) { console.error(USAGE); process.exit(1); }
|
|
54
|
+
const { reviewDemo } = require('./review');
|
|
55
|
+
const result = await reviewDemo(video, {
|
|
56
|
+
count: +(arg('--frames') || 12),
|
|
57
|
+
outDir: arg('--out'),
|
|
58
|
+
});
|
|
59
|
+
console.log(JSON.stringify(result, null, 2));
|
|
60
|
+
return;
|
|
61
|
+
}
|
|
62
|
+
|
|
63
|
+
const url = process.argv[3];
|
|
64
|
+
if (!cmd || !url || !['outline', 'record'].includes(cmd)) {
|
|
65
|
+
console.error(USAGE);
|
|
66
|
+
process.exit(1);
|
|
67
|
+
}
|
|
68
|
+
|
|
69
|
+
const { VoilaSession } = require('./recorder');
|
|
70
|
+
const { produceDemo, outline } = require('./pipeline');
|
|
71
|
+
const session = new VoilaSession({
|
|
72
|
+
headless: !process.argv.includes('--headful'),
|
|
73
|
+
device: arg('--device', 'desktop'),
|
|
74
|
+
profileDir: arg('--profile', path.join(__dirname, 'profile')),
|
|
75
|
+
});
|
|
76
|
+
|
|
77
|
+
try {
|
|
78
|
+
if (cmd === 'outline') {
|
|
79
|
+
console.log(JSON.stringify(await outline(session, url), null, 2));
|
|
80
|
+
} else {
|
|
81
|
+
const stepsFile = arg('--steps');
|
|
82
|
+
const steps = stepsFile ? yaml.load(fs.readFileSync(stepsFile, 'utf8')) : null;
|
|
83
|
+
const workDir = arg('--out', path.join(__dirname, 'recordings', `cli-${Date.now()}`));
|
|
84
|
+
fs.mkdirSync(workDir, { recursive: true });
|
|
85
|
+
const result = await produceDemo(session, {
|
|
86
|
+
url, steps, workDir,
|
|
87
|
+
narrate: !process.argv.includes('--no-narrate'),
|
|
88
|
+
voice: arg('--voice'),
|
|
89
|
+
onStatus: s => console.error('[voila]', s),
|
|
90
|
+
});
|
|
91
|
+
console.log(result.video);
|
|
92
|
+
}
|
|
93
|
+
} finally {
|
|
94
|
+
await session.close();
|
|
95
|
+
}
|
|
96
|
+
})().catch(e => { console.error('FAILED:', e.message || e); process.exit(1); });
|
package/mcp.js
ADDED
|
@@ -0,0 +1,116 @@
|
|
|
1
|
+
#!/usr/bin/env node
|
|
2
|
+
// voila MCP server — lets any MCP client (Claude Code, Desktop, Cursor…) plan,
|
|
3
|
+
// record, and review narrated, auto-zoomed product demos. Headless by default.
|
|
4
|
+
//
|
|
5
|
+
// Register: claude mcp add voila -- node /path/to/voila/mcp.js
|
|
6
|
+
|
|
7
|
+
const path = require('path');
|
|
8
|
+
const fs = require('fs');
|
|
9
|
+
const yaml = require('js-yaml');
|
|
10
|
+
const { z } = require('zod');
|
|
11
|
+
const { McpServer } = require('@modelcontextprotocol/sdk/server/mcp.js');
|
|
12
|
+
const { StdioServerTransport } = require('@modelcontextprotocol/sdk/server/stdio.js');
|
|
13
|
+
const { VoilaSession } = require('./recorder');
|
|
14
|
+
const { produceDemo, outline } = require('./pipeline');
|
|
15
|
+
const { reviewDemo } = require('./review');
|
|
16
|
+
|
|
17
|
+
// One persistent Chromium profile can't be opened twice, so browser work is
|
|
18
|
+
// serialized through a queue: concurrent tool calls wait instead of colliding.
|
|
19
|
+
const sessions = new Map();
|
|
20
|
+
let chain = Promise.resolve();
|
|
21
|
+
const enqueue = fn => {
|
|
22
|
+
const p = chain.then(fn, fn);
|
|
23
|
+
chain = p.then(() => {}, () => {});
|
|
24
|
+
return p;
|
|
25
|
+
};
|
|
26
|
+
|
|
27
|
+
function getSession(device) {
|
|
28
|
+
const key = device || 'desktop';
|
|
29
|
+
if (!sessions.has(key)) {
|
|
30
|
+
sessions.set(key, new VoilaSession({
|
|
31
|
+
headless: process.env.VOILA_HEADFUL !== '1',
|
|
32
|
+
device: key,
|
|
33
|
+
profileDir: (process.env.VOILA_PROFILE || path.join(__dirname, 'profile')) + (key === 'desktop' ? '' : `-${key}`),
|
|
34
|
+
}));
|
|
35
|
+
}
|
|
36
|
+
return sessions.get(key);
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
const server = new McpServer({ name: 'voila', version: '0.4.0' });
|
|
40
|
+
const deviceParam = z.enum(['desktop', 'mobile', 'tablet']).optional().default('desktop');
|
|
41
|
+
|
|
42
|
+
server.tool(
|
|
43
|
+
'voila_outline',
|
|
44
|
+
'Get a structural outline of a web page (title, description, nav links, headings, buttons/CTAs). ' +
|
|
45
|
+
'Use this first to plan a demo script: the outline gives you real text to build selectors from ' +
|
|
46
|
+
'(e.g. "text=Leaderboard") and content for captions/narration.',
|
|
47
|
+
{ url: z.string().url(), device: deviceParam },
|
|
48
|
+
async ({ url, device }) => enqueue(async () => ({
|
|
49
|
+
content: [{ type: 'text', text: JSON.stringify(await outline(getSession(device), url), null, 2) }],
|
|
50
|
+
}))
|
|
51
|
+
);
|
|
52
|
+
|
|
53
|
+
server.tool(
|
|
54
|
+
'voila_record',
|
|
55
|
+
'Record a crisp, auto-zoomed MP4 demo of a website — no screen capture, no permissions. ' +
|
|
56
|
+
'Without steps_yaml it runs a generic auto-tour. For a proper demo, pass steps_yaml: a YAML list of ' +
|
|
57
|
+
'{action, selector?, url?, text?, title?, subtitle?, accent?, level?, ms?, caption?, narration?, optional?}. ' +
|
|
58
|
+
'Actions: goto, click, hover, type, scroll, scroll_to, slide (animated full-screen title card: title/subtitle/accent), zoom, wait. ' +
|
|
59
|
+
'caption is burned into the video as a lower-third; narration is spoken via on-device TTS (Kokoro) at that step, ' +
|
|
60
|
+
'and segment pacing automatically stretches to fit each narration clip — no need to pad waits. ' +
|
|
61
|
+
'Steps marked optional:true are skipped on failure instead of aborting. ' +
|
|
62
|
+
'device selects the recorded viewport (mobile emulates an iPhone-class device). ' +
|
|
63
|
+
'On failure the error names the failing step and includes the live page outline — patch the steps and retry. ' +
|
|
64
|
+
'Returns the MP4 path, the recipe path, and any warnings.',
|
|
65
|
+
{
|
|
66
|
+
url: z.string().url(),
|
|
67
|
+
steps_yaml: z.string().optional(),
|
|
68
|
+
narrate: z.boolean().optional().default(true),
|
|
69
|
+
voice: z.string().optional(),
|
|
70
|
+
device: deviceParam,
|
|
71
|
+
},
|
|
72
|
+
async ({ url, steps_yaml, narrate, voice, device }) => enqueue(async () => {
|
|
73
|
+
const steps = steps_yaml ? yaml.load(steps_yaml) : null;
|
|
74
|
+
const workDir = path.join(__dirname, 'recordings', `mcp-${Date.now()}`);
|
|
75
|
+
fs.mkdirSync(workDir, { recursive: true });
|
|
76
|
+
const result = await produceDemo(getSession(device), {
|
|
77
|
+
url, steps, workDir, narrate, voice: voice || null,
|
|
78
|
+
onStatus: () => {},
|
|
79
|
+
});
|
|
80
|
+
return {
|
|
81
|
+
content: [{
|
|
82
|
+
type: 'text',
|
|
83
|
+
text: JSON.stringify({
|
|
84
|
+
video: result.video,
|
|
85
|
+
recipe: result.recipe,
|
|
86
|
+
durationSec: Math.round((result.meta.tEnd - result.meta.tStart) / 1000),
|
|
87
|
+
narrated: result.narration.narrated,
|
|
88
|
+
voice: result.narration.voice || null,
|
|
89
|
+
device: result.meta.device,
|
|
90
|
+
segments: (result.meta.segments || []).length,
|
|
91
|
+
warnings: result.meta.warnings || [],
|
|
92
|
+
}, null, 2),
|
|
93
|
+
}],
|
|
94
|
+
};
|
|
95
|
+
})
|
|
96
|
+
);
|
|
97
|
+
|
|
98
|
+
server.tool(
|
|
99
|
+
'voila_review',
|
|
100
|
+
'Review a finished demo: extracts evenly spaced PNG frames plus the segment timeline and embedded recipe. ' +
|
|
101
|
+
'Read the returned frame files and check: is the cursor near what the narration discusses? do captions overlap ' +
|
|
102
|
+
'important UI? is any zoom centered on whitespace? did a page fail to load? does a segment linger with nothing ' +
|
|
103
|
+
'happening? Then patch the steps YAML and call voila_record again. Works on any voila MP4, including ones ' +
|
|
104
|
+
'received from other people (the recipe travels inside the file).',
|
|
105
|
+
{
|
|
106
|
+
video_path: z.string(),
|
|
107
|
+
frame_count: z.number().int().min(3).max(40).optional().default(12),
|
|
108
|
+
},
|
|
109
|
+
async ({ video_path, frame_count }) => ({
|
|
110
|
+
content: [{ type: 'text', text: JSON.stringify(await reviewDemo(video_path, { count: frame_count }), null, 2) }],
|
|
111
|
+
})
|
|
112
|
+
);
|
|
113
|
+
|
|
114
|
+
(async () => {
|
|
115
|
+
await server.connect(new StdioServerTransport());
|
|
116
|
+
})().catch(e => { console.error(e); process.exit(1); });
|