retake-demos 0.14.1 → 0.14.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +1 -1
- package/dist/cli.js +8 -3
- package/dist/record.js +37 -1
- package/dist/render.js +2 -2
- package/dist/routine.js +7 -2
- package/dist/ui/chat.html +5 -5
- package/dist/ui/server.js +13 -3
- package/dist/voice.js +56 -7
- package/package.json +2 -1
- package/scripts/local-voice.py +98 -0
- package/site/releases.json +12 -0
package/README.md
CHANGED
|
@@ -324,7 +324,7 @@ On a new workspace, `retake ui` opens the first-demo flow and writes the manifes
|
|
|
324
324
|
|
|
325
325
|
**Scenes** are the spine: `{ action: scene, label, caption }` at each beat. They take their timestamps from the actual run, so captions land on the frames they describe even when the app took 3 seconds this time and 11 the last. **Labels must be unique** — `thumbnail`, `until`, the stills and the proof log all address scenes by label, so two scenes called the same thing make every one of those ambiguous. `validate` rejects it.
|
|
326
326
|
|
|
327
|
-
**Two kinds of take.** `mode: demo` (the default) proves how something works — a client walkthrough, a PR clip, a lesson — and keeps everything but the product out of the frame. `mode: launch` presents the product in public and may carry a title card, music, emphasis and a branded ending. `validate` warns if a demo is carrying launch furniture. **The launch cut.** `compressIdle: true` shows the app's long waits as ~1.5 s each (your own pacing `wait`s are never touched); `voiceover: true` reads the captions aloud (edge-tts, `pipx install edge-tts` once); `typing: brisk` makes typed text land fast with the pauses carrying the meaning. All render-time except typing. **Music.** `music: track.mp3` (or `{ file, gainDb, fadeOutMs }`) mixes a bed under the video at render time, looped or trimmed to fit and faded out at the end. Bring your own track — it ships inside your video, so it must be one you may use (CC0 or licensed). **Cards and callouts.** `intro:`/`outro:` add a title card in Retake's own look — rendered at render time, so changing a word is a re-render, and the intro's settled frame is written as `cover.png`, the poster. `{ action: callout, selector, label, ms }` draws an animated ring and label around an element while the take holds — recorded coordinates, drawn at render, moving with the camera. Sparingly: the captions carry the story.
|
|
327
|
+
**Two kinds of take.** `mode: demo` (the default) proves how something works — a client walkthrough, a PR clip, a lesson — and keeps everything but the product out of the frame. `mode: launch` presents the product in public and may carry a title card, music, emphasis and a branded ending. `validate` warns if a demo is carrying launch furniture. **The launch cut.** `compressIdle: true` shows the app's long waits as ~1.5 s each (your own pacing `wait`s are never touched); `voiceover: true` reads the captions aloud (edge-tts, `pipx install edge-tts` once). **Local voices** run on an Apple Silicon Mac and send nothing anywhere: `retake routine set voice heart` (Kokoro, female, ~300 MB), `michael` (Kokoro, male) or `aiden` (Qwen3, ~1.8 GB). They need [uv](https://docs.astral.sh/uv/) (`brew install uv`); the model downloads once, the first time a video is narrated. Kokoro reads demo words the demo way ("press re-CORD", "deh-mohz") and every local voice is levelled to Edge's loudness; `typing: brisk` makes typed text land fast with the pauses carrying the meaning. All render-time except typing. **Music.** `music: track.mp3` (or `{ file, gainDb, fadeOutMs }`) mixes a bed under the video at render time, looped or trimmed to fit and faded out at the end. Bring your own track — it ships inside your video, so it must be one you may use (CC0 or licensed). **Cards and callouts.** `intro:`/`outro:` add a title card in Retake's own look — rendered at render time, so changing a word is a re-render, and the intro's settled frame is written as `cover.png`, the poster. `{ action: callout, selector, label, ms }` draws an animated ring and label around an element while the take holds — recorded coordinates, drawn at render, moving with the camera. Sparingly: the captions carry the story.
|
|
328
328
|
|
|
329
329
|
**Seeds** put the app in a known state before the camera: write a JSON file, run JS in the page, or run a shell command. **Setup** is the stuff that runs before recording and is trimmed off the front — logins live there. **`tempo`** speeds the finished video up or down at render time; **`lock`** names a shared resource so two demos that touch the same backend don't record over each other.
|
|
330
330
|
|
package/dist/cli.js
CHANGED
|
@@ -1685,8 +1685,8 @@ routine
|
|
|
1685
1685
|
const parsed = RoutineSchema.safeParse(next);
|
|
1686
1686
|
if (!parsed.success)
|
|
1687
1687
|
throw new Error(parsed.error.issues.map((i) => `${i.path.join(".")}: ${i.message}`).join("; "));
|
|
1688
|
-
if (key === "voice" && value !== "silent" && !VOICES[value] && !/^[a-z]{2}-[A-Z]{2}-/.test(value)) {
|
|
1689
|
-
say(`ℹ "${value}" is not one of ${Object.keys(VOICES).join(", ")} and does not look like an edge-tts id (en-US-AriaNeural). \`retake routine voices\` lists what is here.`);
|
|
1688
|
+
if (key === "voice" && value !== "silent" && !VOICES[value] && !/^[a-z]{2}-[A-Z]{2}-/.test(value) && !/^(kokoro|qwen):[a-z_]+$/.test(value)) {
|
|
1689
|
+
say(`ℹ "${value}" is not one of ${Object.keys(VOICES).join(", ")} and does not look like an edge-tts id (en-US-AriaNeural) or a local one (kokoro:af_heart). \`retake routine voices\` lists what is here.`);
|
|
1690
1690
|
}
|
|
1691
1691
|
if (typeof parsed.data.music === "string" && !fs.existsSync(path.resolve(parsed.data.music))) {
|
|
1692
1692
|
say(`ℹ ${parsed.data.music} is not there yet — the render will fail until it is. It ships inside your video, so it must be a track you may use.`);
|
|
@@ -1701,7 +1701,7 @@ routine
|
|
|
1701
1701
|
.option("--line <text>", "what they should say", "This is the voice your demos will use, at the pace they will use it.")
|
|
1702
1702
|
.option("--rate <percent>", "try a speed", "0")
|
|
1703
1703
|
.action(async (opts) => {
|
|
1704
|
-
const { synthesize } = await import("./voice.js");
|
|
1704
|
+
const { synthesize, LOCAL_VOICE, localModelReady } = await import("./voice.js");
|
|
1705
1705
|
const dir = path.resolve("outputs", ".voices");
|
|
1706
1706
|
fs.mkdirSync(dir, { recursive: true });
|
|
1707
1707
|
const rate = Number(opts.rate);
|
|
@@ -1710,6 +1710,11 @@ routine
|
|
|
1710
1710
|
for (const [name, v] of Object.entries(VOICES)) {
|
|
1711
1711
|
const out = path.join(dir, `${name}.mp3`);
|
|
1712
1712
|
process.stdout.write(` ${name.padEnd(14)} ${v.id} … `);
|
|
1713
|
+
// A sample is not worth a download: local voices are heard once chosen.
|
|
1714
|
+
if (LOCAL_VOICE.test(v.id) && !localModelReady(v.id)) {
|
|
1715
|
+
process.stdout.write(`local, not downloaded yet — ${v.note}\n`);
|
|
1716
|
+
continue;
|
|
1717
|
+
}
|
|
1713
1718
|
try {
|
|
1714
1719
|
synthesize(opts.line, v.id, out, rate);
|
|
1715
1720
|
process.stdout.write(`${path.relative(process.cwd(), out)}\n`);
|
package/dist/record.js
CHANGED
|
@@ -1003,7 +1003,26 @@ export async function record(m, opts) {
|
|
|
1003
1003
|
fs.rmSync(path.join(opts.outDir, "reached.json"), { force: true });
|
|
1004
1004
|
}
|
|
1005
1005
|
catch { /* fine */ }
|
|
1006
|
-
|
|
1006
|
+
// The start is marked by painting the empty tab (paintFirstFrame), which
|
|
1007
|
+
// holds on an ordinary machine. On a loaded one — CI, a laptop running a
|
|
1008
|
+
// test suite — the browser can begin the video before that paint lands,
|
|
1009
|
+
// and the cut falls on the blank page. So check the picture itself: if the
|
|
1010
|
+
// take's first frame is flat while the page was proven painted, move the
|
|
1011
|
+
// start to the first real frame and every step with it.
|
|
1012
|
+
let late = 0;
|
|
1013
|
+
if (video) {
|
|
1014
|
+
late = blankLead(video, sec(setupEnd - t0));
|
|
1015
|
+
if (late > 0) {
|
|
1016
|
+
for (const e of timeline) {
|
|
1017
|
+
e.start = sec(e.start + late);
|
|
1018
|
+
e.end = sec(e.end + late);
|
|
1019
|
+
}
|
|
1020
|
+
for (const e of pageErrors)
|
|
1021
|
+
e.at = sec(e.at + late);
|
|
1022
|
+
log(`start: the video began ${late.toFixed(2)}s before the marked start — every step moved to match the picture`);
|
|
1023
|
+
}
|
|
1024
|
+
}
|
|
1025
|
+
const trimBefore = sec(setupEnd - t0 + late);
|
|
1007
1026
|
// The recording keeps rolling while the browser is torn down, so the raw
|
|
1008
1027
|
// video always has a tail of nothing after the last step. The take ends
|
|
1009
1028
|
// where the demo ends (plus a breath), never where the recorder stopped —
|
|
@@ -1566,6 +1585,23 @@ export function describe(step) {
|
|
|
1566
1585
|
return `scene: ${step.label}`;
|
|
1567
1586
|
}
|
|
1568
1587
|
}
|
|
1588
|
+
/** Seconds of flat, empty picture from `at` onward, before the first frame
|
|
1589
|
+
with anything on it — 0 when `at` already shows the page. Bounded: past
|
|
1590
|
+
4s it is the app, not the clock, and the take is left alone. */
|
|
1591
|
+
export function blankLead(video, at, maxSeconds = 4) {
|
|
1592
|
+
try {
|
|
1593
|
+
const out = execFileSync(ffmpegBin(), ["-hide_banner", "-ss", at.toFixed(3), "-t", String(maxSeconds), "-i", video,
|
|
1594
|
+
"-vf", "fps=20,signalstats,metadata=print:file=-", "-f", "null", "-"], { encoding: "utf8", stdio: ["ignore", "pipe", "ignore"], timeout: 30_000 });
|
|
1595
|
+
const frames = [...out.matchAll(/pts_time:([\d.]+)[\s\S]*?YMIN=(\d+)[\s\S]*?YMAX=(\d+)/g)].map((m) => ({ t: Number(m[1]), flat: Number(m[3]) - Number(m[2]) <= 8 }));
|
|
1596
|
+
if (!frames.length || !frames[0].flat)
|
|
1597
|
+
return 0;
|
|
1598
|
+
const first = frames.find((f) => !f.flat);
|
|
1599
|
+
return first ? Math.round(first.t * 1000) / 1000 : 0;
|
|
1600
|
+
}
|
|
1601
|
+
catch {
|
|
1602
|
+
return 0;
|
|
1603
|
+
}
|
|
1604
|
+
}
|
|
1569
1605
|
/** Paint the empty tab and return the wall-clock ms of that paint — the
|
|
1570
1606
|
video's first frame. Null if the page would not paint (then the old
|
|
1571
1607
|
estimate stands). It happens during setup, so it is trimmed away. */
|
package/dist/render.js
CHANGED
|
@@ -24,7 +24,7 @@ import { bandHeightFor, maxCharsFor, wrap } from "./captions.js";
|
|
|
24
24
|
import { renderCard, renderCalloutOverlay, renderPoster, renderTitledCover } from "./cards.js";
|
|
25
25
|
import { describeIdle, planIdle, warpFilterArgs, warpTake } from "./pace.js";
|
|
26
26
|
import { holdFilterArgs, holdTake, planHolds } from "./hold.js";
|
|
27
|
-
import { DEFAULT_VOICE, audioSeconds, cueGroups, cuesPerParagraph, cutPoints, silences, synthesize, synthesizeWithCues } from "./voice.js";
|
|
27
|
+
import { DEFAULT_VOICE, LOCAL_VOICE, LOCAL_VOICE_VERSION, audioSeconds, cueGroups, cuesPerParagraph, cutPoints, silences, synthesize, synthesizeWithCues } from "./voice.js";
|
|
28
28
|
import { expandEnv, resolve, Manifest } from "./manifest.js";
|
|
29
29
|
import { ffmpegBin } from "./media-bin.js";
|
|
30
30
|
export { ffmpegBin } from "./media-bin.js";
|
|
@@ -41,7 +41,7 @@ export function renderHash(m, take) {
|
|
|
41
41
|
}
|
|
42
42
|
catch {
|
|
43
43
|
return null;
|
|
44
|
-
} }) : null], idle: m.compressIdle, vo: m.voiceover, music: m.music ? [m.music, (() => { try {
|
|
44
|
+
} }) : null], idle: m.compressIdle, vo: m.voiceover, voEngine: typeof m.voiceover === "object" && LOCAL_VOICE.test(m.voiceover.voice) ? LOCAL_VOICE_VERSION : undefined, music: m.music ? [m.music, (() => { try {
|
|
45
45
|
const f = typeof m.music === "string" ? m.music : m.music.file;
|
|
46
46
|
const st = fs.statSync(path.resolve(f));
|
|
47
47
|
return [st.size, Math.round(st.mtimeMs)];
|
package/dist/routine.js
CHANGED
|
@@ -60,9 +60,14 @@ export const VOICES = {
|
|
|
60
60
|
brian: { id: "en-US-BrianNeural", note: "Casual and sincere — least like a narrator." },
|
|
61
61
|
emma: { id: "en-US-EmmaNeural", note: "Clear and conversational, good over fast UI." },
|
|
62
62
|
aria: { id: "en-US-AriaNeural", note: "A news read: even, flat, no colour." },
|
|
63
|
+
// Local: these run on this Mac (Apple Silicon) and send nothing anywhere.
|
|
64
|
+
heart: { id: "kokoro:af_heart", note: "Local (Kokoro). Warm, natural, female. First use downloads ~300 MB." },
|
|
65
|
+
michael: { id: "kokoro:am_michael", note: "Local (Kokoro). Steady, male. Shares Heart's ~300 MB download." },
|
|
66
|
+
aiden: { id: "qwen:aiden", note: "Local (Qwen3). Natural, male. First use downloads ~1.8 GB." },
|
|
63
67
|
};
|
|
64
|
-
/** A routine's `voice` is a name above, "silent",
|
|
65
|
-
* (`vi-VN-HoaiMyNeural`) for a language the named set does not cover
|
|
68
|
+
/** A routine's `voice` is a name above, "silent", a raw edge-tts voice id
|
|
69
|
+
* (`vi-VN-HoaiMyNeural`) for a language the named set does not cover, or a
|
|
70
|
+
* raw local id (`kokoro:af_bella`, `qwen:ryan`). */
|
|
66
71
|
export function voiceId(voice) {
|
|
67
72
|
return VOICES[voice]?.id ?? voice;
|
|
68
73
|
}
|
package/dist/ui/chat.html
CHANGED
|
@@ -520,16 +520,16 @@
|
|
|
520
520
|
<p class="ob-lead">The first draft stays silent. Narration comes after you approve the recording and script.</p>
|
|
521
521
|
<div class="voice-grid" role="radiogroup" aria-label="Narration choice">
|
|
522
522
|
<button class="voice-card selected" type="button" data-voice="silent" role="radio" aria-checked="true">
|
|
523
|
-
<span class="vicon" aria-hidden="true">×</span><b>No voiceover</b><small>Skip this for now. You can
|
|
523
|
+
<span class="vicon" aria-hidden="true">×</span><b>No voiceover</b><small>Skip this for now. You can choose a voice later.</small><span class="tag">No setup</span>
|
|
524
524
|
</button>
|
|
525
525
|
<button class="voice-card" type="button" data-voice="edge" data-sample="edge" role="radio" aria-checked="false">
|
|
526
526
|
<span class="vicon" aria-hidden="true">∞</span><b>Edge</b><small>Online. Small install, many voices, and fast generation. Your script goes to Microsoft's service.</small><span class="tag">Tap to hear</span>
|
|
527
527
|
</button>
|
|
528
528
|
<button class="voice-card" type="button" data-voice="aiden" data-sample="aiden" role="radio" aria-checked="false">
|
|
529
|
-
<span class="vicon" aria-hidden="true">A</span><b>Aiden</b><small>Local Qwen voice. The natural
|
|
529
|
+
<span class="vicon" aria-hidden="true">A</span><b>Aiden</b><small>Local Qwen voice. The most natural one, and the biggest: about 1.8 GB. Runs on your Mac, so your script never leaves it.</small><span class="tag">Tap to hear</span>
|
|
530
530
|
</button>
|
|
531
531
|
<button class="voice-card" type="button" data-voice="kokoro" data-sample="kokoro" role="radio" aria-checked="false">
|
|
532
|
-
<span class="vicon" aria-hidden="true">K</span><b>Kokoro</b><small>Local and much smaller
|
|
532
|
+
<span class="vicon" aria-hidden="true">K</span><b>Kokoro</b><small>Local and much smaller: about 300 MB, many voices. Runs on your Mac, so your script never leaves it.</small><span class="tag">Tap to hear</span>
|
|
533
533
|
</button>
|
|
534
534
|
</div>
|
|
535
535
|
<div class="voice-demo" id="voice-demo"><video id="voice-demo-video" controls playsinline preload="metadata"></video><small id="voice-demo-label"></small></div>
|
|
@@ -642,8 +642,8 @@ function renderOnboarding() {
|
|
|
642
642
|
consent.classList.toggle('show', needsConsent);
|
|
643
643
|
box.checked = onboardingState.consent;
|
|
644
644
|
if (onboardingState.voice === 'edge') $('voice-consent-copy').textContent = 'I understand that Edge sends the narration script to Microsoft’s online speech service.';
|
|
645
|
-
if (onboardingState.voice === 'aiden') $('voice-consent-copy').textContent = 'I understand that Aiden needs a
|
|
646
|
-
if (onboardingState.voice === 'kokoro') $('voice-consent-copy').textContent = 'I understand that Kokoro needs a
|
|
645
|
+
if (onboardingState.voice === 'aiden') $('voice-consent-copy').textContent = 'I understand that Aiden needs a one-time local model download of about 1.8 GB, the first time a video is narrated. Nothing downloads on this screen.';
|
|
646
|
+
if (onboardingState.voice === 'kokoro') $('voice-consent-copy').textContent = 'I understand that Kokoro needs a one-time local model download of about 300 MB, the first time a video is narrated. Nothing downloads on this screen.';
|
|
647
647
|
$('onboard-next').disabled =
|
|
648
648
|
(onboardingStep === 1 && !normalUrl($('onboard-url').value)) ||
|
|
649
649
|
(onboardingStep === 2 && brandRead.state === 'reading') ||
|
package/dist/ui/server.js
CHANGED
|
@@ -54,7 +54,7 @@ import { captureMatchesTake, readPreviewReceipt } from "../preview.js";
|
|
|
54
54
|
import { STYLES, VOICES, RoutineSchema, environmentFor, readRoutines, starter, voiceId, writeRoutines } from "../routine.js";
|
|
55
55
|
import { loadEnvSpec } from "../recording-env.js";
|
|
56
56
|
import { describeSchedule, readSchedule, scheduleLoaded } from "../schedule.js";
|
|
57
|
-
import { synthesize, voiceCachePath } from "../voice.js";
|
|
57
|
+
import { synthesize, voiceCachePath, LOCAL_VOICE, localModelReady } from "../voice.js";
|
|
58
58
|
const ROOT = PROJECT_ROOT;
|
|
59
59
|
const DEMOS = path.join(ROOT, "demos");
|
|
60
60
|
/** Where takes land. Default outputs/ in the project; changeable in Settings
|
|
@@ -789,6 +789,16 @@ export function serve(port) {
|
|
|
789
789
|
if (!Number.isInteger(rate) || rate < -50 || rate > 50)
|
|
790
790
|
return json(res, 400, { error: "rate is a whole percent, -50 to 50" });
|
|
791
791
|
const id = voiceId(voice);
|
|
792
|
+
// A local voice is never downloaded just to be previewed (and Aiden is
|
|
793
|
+
// ~30s to speak even when present, which would hold this whole window
|
|
794
|
+
// up): play the sample that ships with Retake instead.
|
|
795
|
+
if (LOCAL_VOICE.test(id) && (id.startsWith("qwen:") || !localModelReady(id))) {
|
|
796
|
+
const f = path.join(PKG_ROOT, "assets", "voice-samples", id.startsWith("qwen:") ? "aiden.mp4" : "kokoro.mp4");
|
|
797
|
+
if (!fs.existsSync(f))
|
|
798
|
+
return json(res, 404, { error: "no sample for this voice yet" });
|
|
799
|
+
res.writeHead(200, { "content-type": "video/mp4", "content-length": fs.statSync(f).size, "cache-control": "no-store" });
|
|
800
|
+
return stream(f, res);
|
|
801
|
+
}
|
|
792
802
|
const line = "This is the voice your demos will use, at the pace they will use it.";
|
|
793
803
|
const out = voiceCachePath(line, id, rate);
|
|
794
804
|
try {
|
|
@@ -1231,8 +1241,8 @@ export function serve(port) {
|
|
|
1231
1241
|
: narration === "edge"
|
|
1232
1242
|
? "Make the first draft silent and do not add captions. The person chose Edge for the final narration. Do not synthesize it until they approve both the recording and one continuous script."
|
|
1233
1243
|
: narration === "aiden"
|
|
1234
|
-
? "Make the first draft silent and do not add captions. The person chose the local Aiden
|
|
1235
|
-
: "Make the first draft silent and do not add captions. The person chose the
|
|
1244
|
+
? "Make the first draft silent and do not add captions. The person chose the local Aiden voice for the final narration: `voiceover.voice: qwen:aiden` (a routine says `voice: aiden`). It runs on their Mac; its first use downloads a ~1.8 GB model. Do not narrate until they approve the recording and one continuous script, and say the download is coming before the first narrated render."
|
|
1245
|
+
: "Make the first draft silent and do not add captions. The person chose the local Kokoro voice for the final narration: `voiceover.voice: kokoro:af_heart` (a routine says `voice: heart`). It runs on their Mac; its first use downloads a ~300 MB model. Do not narrate until they approve the recording and one continuous script, and say the download is coming before the first narrated render.";
|
|
1236
1246
|
const describe = `${b.describe.trim()}\n\n${sound}`;
|
|
1237
1247
|
const prov = pickProvider();
|
|
1238
1248
|
const which = prov?.name === "codex" ? "codex" : prov?.name === "claude-code" ? "claude-code" : null;
|
package/dist/voice.js
CHANGED
|
@@ -3,23 +3,56 @@
|
|
|
3
3
|
* scene's line is synthesized once and placed at the scene's start, so the
|
|
4
4
|
* narration is honest to the take by construction.
|
|
5
5
|
*
|
|
6
|
-
*
|
|
7
|
-
* CLI (`edge-tts`, pipx/pip) or module (`python3 -m edge_tts`)
|
|
8
|
-
*
|
|
6
|
+
* Engines: edge-tts (Microsoft's neural voices — free, network), found as a
|
|
7
|
+
* CLI (`edge-tts`, pipx/pip) or module (`python3 -m edge_tts`); and two local
|
|
8
|
+
* voices that run on this Mac and send nothing anywhere — Kokoro
|
|
9
|
+
* (`kokoro:af_heart`) and Qwen3 (`qwen:aiden`) — through scripts/local-voice.py
|
|
10
|
+
* under uv. Retake bundles neither; the errors say how to get them.
|
|
9
11
|
*/
|
|
10
12
|
import fs from "node:fs";
|
|
11
13
|
import path from "node:path";
|
|
12
14
|
import os from "node:os";
|
|
13
15
|
import { createHash } from "node:crypto";
|
|
14
16
|
import { execFileSync, spawnSync } from "node:child_process";
|
|
17
|
+
import { createRequire } from "node:module";
|
|
18
|
+
import { PKG_ROOT } from "./paths.js";
|
|
15
19
|
export const DEFAULT_VOICE = "en-US-JennyNeural";
|
|
16
20
|
/** Content-addressed speech cache. Narration text and voice are immutable
|
|
17
21
|
* inputs, so generating the same line again is pure wasted network time. */
|
|
18
22
|
export function voiceCachePath(text, voice, ratePercent = 0) {
|
|
19
23
|
const root = process.env.RETAKE_TTS_CACHE || path.join(os.homedir(), ".cache", "retake-tts", "edge");
|
|
20
|
-
|
|
24
|
+
// A local voice's output changes when its pronunciation or levelling does;
|
|
25
|
+
// the version is part of the key so an improvement is heard, not cached away.
|
|
26
|
+
const v = LOCAL_VOICE.test(voice) ? `${voice}\0${LOCAL_VOICE_VERSION}` : voice;
|
|
27
|
+
const key = createHash("sha256").update(`${v}\0${ratePercent}\0${text}`).digest("hex");
|
|
21
28
|
return path.join(root, `${key}.mp3`);
|
|
22
29
|
}
|
|
30
|
+
/** Voices that run on this Mac: `kokoro:<voice>` and `qwen:<speaker>`. */
|
|
31
|
+
export const LOCAL_VOICE = /^(kokoro|qwen):[a-z_]+$/;
|
|
32
|
+
/** Bump when local pronunciation or levelling changes: it re-renders them. */
|
|
33
|
+
export const LOCAL_VOICE_VERSION = "local-voice-1";
|
|
34
|
+
const LOCAL_MODEL = { kokoro: { repo: "models--mlx-community--Kokoro-82M-8bit", size: "~300 MB" }, qwen: { repo: "models--mlx-community--Qwen3-TTS-12Hz-0.6B-CustomVoice-8bit", size: "~1.8 GB" } };
|
|
35
|
+
/** The local voice engine: Retake's script under uv, with the MLX audio stack. */
|
|
36
|
+
export function localEngine(voice, platform = process.platform, arch = process.arch) {
|
|
37
|
+
if (platform !== "darwin" || arch !== "arm64")
|
|
38
|
+
throw new Error(`voiceover: ${voice} is a local voice, and local voices run on Apple Silicon Macs for now — use an Edge voice here (andrew, ava, emma…)`);
|
|
39
|
+
if (spawnSync("uvx", ["--version"], { stdio: "ignore" }).status !== 0)
|
|
40
|
+
throw new Error(`voiceover: ${voice} runs locally through uv, which is not installed — install it once (\`brew install uv\`), then re-render`);
|
|
41
|
+
const kind = voice.split(":")[0];
|
|
42
|
+
if (!localModelReady(voice))
|
|
43
|
+
process.stderr.write(`… first use of ${voice}: downloading its voice model (${LOCAL_MODEL[kind].size}, one time). Nothing you narrate leaves this computer.\n`);
|
|
44
|
+
return { cmd: "uvx", args: ["--quiet", "--python", "3.12", "--from", "mlx-audio==0.5.5", "--with", "soundfile", "--with", "misaki[en]", "python", path.join(PKG_ROOT, "scripts", "local-voice.py")] };
|
|
45
|
+
}
|
|
46
|
+
/** Whether a local voice's model is already on this computer. */
|
|
47
|
+
export function localModelReady(voice) {
|
|
48
|
+
const kind = voice.split(":")[0];
|
|
49
|
+
if (!LOCAL_MODEL[kind])
|
|
50
|
+
return false;
|
|
51
|
+
return fs.existsSync(path.join(process.env.HF_HOME || path.join(os.homedir(), ".cache", "huggingface"), "hub", LOCAL_MODEL[kind].repo));
|
|
52
|
+
}
|
|
53
|
+
function engineFor(voice) {
|
|
54
|
+
return LOCAL_VOICE.test(voice) ? localEngine(voice) : engine();
|
|
55
|
+
}
|
|
23
56
|
function engine() {
|
|
24
57
|
const home = process.env.HOME ?? "";
|
|
25
58
|
for (const c of ["edge-tts", path.join(home, ".local", "bin", "edge-tts")]) {
|
|
@@ -44,23 +77,39 @@ export function edgeFailure(stderr, cmd) {
|
|
|
44
77
|
return `voiceover: edge-tts failed${last ? ` — ${last.slice(0, 200)}` : ""}`;
|
|
45
78
|
}
|
|
46
79
|
function runEdge(e, args, timeout) {
|
|
80
|
+
const local = e.cmd === "uvx";
|
|
47
81
|
try {
|
|
48
|
-
execFileSync(e.cmd, [...e.args, ...args], { stdio: "pipe", timeout });
|
|
82
|
+
execFileSync(e.cmd, [...e.args, ...args], { stdio: "pipe", timeout: local ? Math.max(timeout, 900_000) : timeout, env: local ? { ...process.env, FFMPEG_BIN: ffmpegPath() } : process.env });
|
|
49
83
|
}
|
|
50
84
|
catch (err) {
|
|
51
85
|
const x = err;
|
|
86
|
+
if (local) {
|
|
87
|
+
const last = String(x.stderr ?? "").trim().split("\n").filter(Boolean).pop() ?? "";
|
|
88
|
+
throw new Error(`voiceover: the local voice failed${last ? ` — ${last.slice(0, 240)}` : ""}`);
|
|
89
|
+
}
|
|
52
90
|
if (x.code === "ETIMEDOUT" || x.signal === "SIGTERM")
|
|
53
91
|
throw new Error("voiceover: Microsoft's voice service did not answer in time — try again in a minute");
|
|
54
92
|
throw new Error(edgeFailure(String(x.stderr ?? ""), e.cmd));
|
|
55
93
|
}
|
|
56
94
|
}
|
|
95
|
+
/** ffmpeg for the local voice's levelling: the one Retake renders with. */
|
|
96
|
+
function ffmpegPath() {
|
|
97
|
+
try {
|
|
98
|
+
const require = createRequire(import.meta.url);
|
|
99
|
+
const p = require("ffmpeg-static");
|
|
100
|
+
if (p && fs.existsSync(p))
|
|
101
|
+
return p;
|
|
102
|
+
}
|
|
103
|
+
catch { /* system ffmpeg */ }
|
|
104
|
+
return "ffmpeg";
|
|
105
|
+
}
|
|
57
106
|
export function synthesize(text, voice, out, ratePercent = 0) {
|
|
58
107
|
const cached = voiceCachePath(text, voice, ratePercent);
|
|
59
108
|
if (fs.existsSync(cached) && fs.statSync(cached).size >= 200) {
|
|
60
109
|
fs.copyFileSync(cached, out);
|
|
61
110
|
return;
|
|
62
111
|
}
|
|
63
|
-
const e =
|
|
112
|
+
const e = engineFor(voice);
|
|
64
113
|
const temp = `${out}.tmp-${process.pid}`;
|
|
65
114
|
try {
|
|
66
115
|
runEdge(e, ["--voice", voice, "--rate", `${ratePercent >= 0 ? "+" : ""}${ratePercent}%`, "--text", text, "--write-media", temp], 60_000);
|
|
@@ -108,7 +157,7 @@ export function synthesizeWithCues(text, voice, out, ratePercent = 0) {
|
|
|
108
157
|
fs.copyFileSync(cached, out);
|
|
109
158
|
return parseSrt(fs.readFileSync(cachedSrt, "utf8"));
|
|
110
159
|
}
|
|
111
|
-
const e =
|
|
160
|
+
const e = engineFor(voice);
|
|
112
161
|
const temp = `${out}.tmp-${process.pid}`, srt = `${out}.srt`;
|
|
113
162
|
try {
|
|
114
163
|
runEdge(e, ["--voice", voice, "--rate", `${ratePercent >= 0 ? "+" : ""}${ratePercent}%`, "--text", text, "--write-media", temp, "--write-subtitles", srt], 120_000);
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "retake-demos",
|
|
3
|
-
"version": "0.14.
|
|
3
|
+
"version": "0.14.2",
|
|
4
4
|
"description": "Demo-as-code for coding agents: a YAML manifest in, a product walkthrough video out. Rerun the demo instead of re-recording it.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"demo",
|
|
@@ -39,6 +39,7 @@
|
|
|
39
39
|
"docs/stitch.md",
|
|
40
40
|
"LICENSE",
|
|
41
41
|
"scripts/patch-testreel.mjs",
|
|
42
|
+
"scripts/local-voice.py",
|
|
42
43
|
"AGENTS.md",
|
|
43
44
|
"llms.txt",
|
|
44
45
|
"site/releases.json"
|
|
@@ -0,0 +1,98 @@
|
|
|
1
|
+
"""Retake's local voices: Kokoro and Qwen3 (Aiden), on this Mac.
|
|
2
|
+
|
|
3
|
+
Speaks the way edge-tts does, so narration does not care which it got:
|
|
4
|
+
--voice kokoro:<voice> | qwen:<speaker> --rate +N% --text ...
|
|
5
|
+
--write-media out.mp3 [--write-subtitles out.srt]
|
|
6
|
+
Each sentence is its own take (the voice keeps its shape), joined with short
|
|
7
|
+
breaths; the SRT says where each falls. Levelled to Edge's loudness so a
|
|
8
|
+
comparison hears the voice, not the volume.
|
|
9
|
+
|
|
10
|
+
--pronounce "text" print the text as Kokoro will be given it (no models)
|
|
11
|
+
|
|
12
|
+
Run by Retake through uv: uvx --from mlx-audio==0.5.5 --with soundfile
|
|
13
|
+
--with "misaki[en]" python local-voice.py ... Apple Silicon only (MLX).
|
|
14
|
+
"""
|
|
15
|
+
import argparse, os, re, subprocess, sys, tempfile
|
|
16
|
+
|
|
17
|
+
# ---------- words Kokoro's guesser gets wrong in demo narration ----------
|
|
18
|
+
# "record" is a heteronym: the guesser hears a music RE-cord in "press record".
|
|
19
|
+
# In a demo script it is almost always the verb (re-CORD); after a word that
|
|
20
|
+
# makes it a noun ("a record", "the record") it is left alone. "demos" came
|
|
21
|
+
# out "dee-moss". Overrides use misaki's own [word](/phonemes/) form.
|
|
22
|
+
NOUN_BEFORE = {"a", "an", "the", "this", "that", "my", "your", "our", "their", "its", "his", "her",
|
|
23
|
+
"any", "no", "new", "old", "track", "world", "public", "permanent", "every"}
|
|
24
|
+
FIXED = {"demos": "dˈɛmOz", "demo": "dˈɛmO"}
|
|
25
|
+
VERB = {"record": "ɹəkˈɔɹd", "records": "ɹəkˈɔɹdz"}
|
|
26
|
+
|
|
27
|
+
|
|
28
|
+
def pronounce(text: str) -> str:
|
|
29
|
+
out, prev = [], ""
|
|
30
|
+
for tok in re.split(r"(\W+)", text):
|
|
31
|
+
low = tok.lower()
|
|
32
|
+
if low in VERB and prev not in NOUN_BEFORE:
|
|
33
|
+
tok = f"[{tok}](/{VERB[low]}/)"
|
|
34
|
+
elif low in FIXED:
|
|
35
|
+
tok = f"[{tok}](/{FIXED[low]}/)"
|
|
36
|
+
if tok.strip() and re.match(r"\w", tok):
|
|
37
|
+
prev = low
|
|
38
|
+
out.append(tok)
|
|
39
|
+
return "".join(out)
|
|
40
|
+
|
|
41
|
+
|
|
42
|
+
EDGE_LUFS = -19.8 # measured: Edge's Andrew reading the same lines
|
|
43
|
+
SR = 24000
|
|
44
|
+
MODELS = {"kokoro": "mlx-community/Kokoro-82M-8bit", "qwen": "mlx-community/Qwen3-TTS-12Hz-0.6B-CustomVoice-8bit"}
|
|
45
|
+
|
|
46
|
+
|
|
47
|
+
def main() -> None:
|
|
48
|
+
p = argparse.ArgumentParser()
|
|
49
|
+
p.add_argument("--voice"); p.add_argument("--rate", default="+0%"); p.add_argument("--text")
|
|
50
|
+
p.add_argument("--write-media"); p.add_argument("--write-subtitles"); p.add_argument("--pronounce")
|
|
51
|
+
a = p.parse_args()
|
|
52
|
+
if a.pronounce is not None:
|
|
53
|
+
print(pronounce(a.pronounce))
|
|
54
|
+
return
|
|
55
|
+
|
|
56
|
+
import numpy as np, soundfile as sf
|
|
57
|
+
from mlx_audio.tts.utils import load_model
|
|
58
|
+
|
|
59
|
+
engine, name = a.voice.split(":", 1)
|
|
60
|
+
speed = 1 + int(re.sub(r"[%+]", "", a.rate) or 0) / 100
|
|
61
|
+
model = load_model(MODELS[engine])
|
|
62
|
+
|
|
63
|
+
def say(sentence: str):
|
|
64
|
+
if engine == "kokoro":
|
|
65
|
+
gen = model.generate(text=pronounce(sentence), voice=name, speed=speed, lang_code="a" if name[0] == "a" else "b")
|
|
66
|
+
else:
|
|
67
|
+
gen = model.generate(text=sentence, voice=name, speed=speed, lang_code="english", verbose=False)
|
|
68
|
+
return np.concatenate([np.array(r.audio, dtype=np.float32).reshape(-1) for r in gen])
|
|
69
|
+
|
|
70
|
+
paras = [x.strip() for x in re.split(r"\n\s*\n|\n", a.text) if x.strip()] or [a.text]
|
|
71
|
+
audio, cues, t = [], [], 0.0
|
|
72
|
+
for para in paras:
|
|
73
|
+
for sentence in [s for s in re.split(r"(?<=[.!?])\s+", para) if s.strip()]:
|
|
74
|
+
chunk = say(sentence)
|
|
75
|
+
audio.append(chunk); cues.append((t, t + len(chunk) / SR, sentence)); t += len(chunk) / SR
|
|
76
|
+
audio.append(np.zeros(int(SR * 0.3), dtype=np.float32)); t += 0.3
|
|
77
|
+
audio.append(np.zeros(int(SR * 0.35), dtype=np.float32)); t += 0.35
|
|
78
|
+
|
|
79
|
+
ff = os.environ.get("FFMPEG_BIN", "ffmpeg")
|
|
80
|
+
wav = tempfile.mktemp(suffix=".wav")
|
|
81
|
+
sf.write(wav, np.concatenate(audio), SR)
|
|
82
|
+
try:
|
|
83
|
+
meas = subprocess.run([ff, "-hide_banner", "-nostats", "-i", wav, "-af", "ebur128=framelog=quiet", "-f", "null", "-"], capture_output=True, text=True).stderr
|
|
84
|
+
found = re.findall(r"I:\s+(-?[\d.]+) LUFS", meas)
|
|
85
|
+
gain = EDGE_LUFS - float(found[-1]) if found else 0.0
|
|
86
|
+
subprocess.run([ff, "-hide_banner", "-loglevel", "error", "-y", "-i", wav, "-af", f"volume={gain:.2f}dB,alimiter=limit=0.97",
|
|
87
|
+
"-c:a", "libmp3lame", "-b:a", "128k", "-f", "mp3", a.write_media], check=True)
|
|
88
|
+
finally:
|
|
89
|
+
os.remove(wav)
|
|
90
|
+
if a.write_subtitles:
|
|
91
|
+
ts = lambda s: f"{int(s // 3600):02d}:{int(s % 3600 // 60):02d}:{int(s % 60):02d},{int(round((s % 1) * 1000)) % 1000:03d}"
|
|
92
|
+
with open(a.write_subtitles, "w") as f:
|
|
93
|
+
for i, (st, en, tx) in enumerate(cues, 1):
|
|
94
|
+
f.write(f"{i}\n{ts(st)} --> {ts(en)}\n{tx}\n\n")
|
|
95
|
+
|
|
96
|
+
|
|
97
|
+
if __name__ == "__main__":
|
|
98
|
+
sys.exit(main())
|
package/site/releases.json
CHANGED
|
@@ -1,6 +1,18 @@
|
|
|
1
1
|
{
|
|
2
2
|
"_readme": "What's new, written for the person using Retake — not for whoever wrote it. Both the landing page and the Retake window read THIS file; there is no second copy. Rules for adding one: say what changed for them, name no files, no flags they will not type, no words like refactor or regression. If a line only makes sense to someone who has read the source, it does not belong here. Newest first. Add an entry on every release that changes what people see.",
|
|
3
3
|
"releases": [
|
|
4
|
+
{
|
|
5
|
+
"version": "0.14.2",
|
|
6
|
+
"date": "2026-10-16",
|
|
7
|
+
"headline": "The local voices you can pick are now the voices you get",
|
|
8
|
+
"notes": [
|
|
9
|
+
"Kokoro and Aiden, the two local voices offered when you first open Retake, now narrate your videos. Before, choosing one only told your agent your preference: Retake itself could speak only Edge. They run on your Mac (Apple Silicon), so your script never leaves it, and they come out exactly as loud as Edge.",
|
|
10
|
+
"Pick them by name: heart (Kokoro, female), michael (Kokoro, male) or aiden. The model downloads once, the first time a video is narrated: about 300 MB for Kokoro, 1.8 GB for Aiden.",
|
|
11
|
+
"Kokoro now says demo words the demo way: “press record” as re-CORD, not the music kind, and “demos” as deh-mohz.",
|
|
12
|
+
"Hearing a voice before you pick it never downloads anything, and never freezes the window.",
|
|
13
|
+
"On a busy computer, a recording could still start on a blank frame. Retake now checks the picture itself and starts on the first real frame."
|
|
14
|
+
]
|
|
15
|
+
},
|
|
4
16
|
{
|
|
5
17
|
"version": "0.14.1",
|
|
6
18
|
"date": "2026-10-16",
|