storyshot 0.5.0 → 0.6.0

package/README.md

@@ -153,6 +153,7 @@ Rendering is free and runs locally. Only the voiceover can cost money:
 |---|---|---|
 | none | free | captions only |
 | `kokoro` | free | local, English and Mandarin Chinese (`zf_001`); downloads a ~90 MB model on first use (Chinese: ~130 MB, and needs Python 3.10–3.12) |
+| `qwen` | free | local, Apple Silicon Macs only; the most natural free Chinese voice (`dylan`, `serena`); downloads a ~2.3 GB model and needs Python 3.10–3.12 |
 | ElevenLabs | your credits | the most natural voice; many languages including Chinese. Put `ELEVENLABS_API_KEY` in `.env`. |
 | OpenAI | your credits | `OPENAI_API_KEY` |
 
@@ -394,9 +395,10 @@ mix is normalized to -14 LUFS.
 | provider | key (env or `.env`) | notes |
 |---|---|---|
 | `kokoro` (default) | none | Local and free. English voices (`af_heart`, `am_michael`, `bf_emma`, …) and Mandarin Chinese ones (`zf_001`, see below). Needs `npm i kokoro-js`; the first run downloads an ~90 MB model. |
+| `qwen` | none | Local and free, Apple Silicon Macs only: Qwen3-TTS, the most natural free Chinese voice here (`dylan`, `serena`, see below). |
 | `elevenlabs` | `ELEVENLABS_API_KEY` | Most natural, many languages including Chinese. `voice` is a voice ID; `model` defaults to `eleven_multilingual_v2`. The key needs Text to Speech access, plus Voices → Read to list voices. |
 | `openai` | `OPENAI_API_KEY` | `voice: alloy`, `model: gpt-4o-mini-tts`; optional `instructions` for tone. |
-| `say` | none | macOS built-in voices, robotic; fine for drafts. For Chinese, use `kokoro` with `zf_001` rather than `Tingting`. |
+| `say` | none | macOS built-in voices, robotic; fine for drafts. For Chinese, use `qwen` (Apple Silicon) or `kokoro` with `zf_001` rather than `Tingting`. |
 
 **Chinese narration.** `voice: { provider: kokoro, voice: zf_001 }` speaks Mandarin with Kokoro
 v1.1-zh, free and local, and is what `kokoro` uses when the `say:` lines are Chinese and no voice is
@@ -409,6 +411,20 @@ whether it's ready. English words and letters in a line (`GeoGebra`, `y 等于 m
 English, a 2 before a measure word is read 两 (`2 个单位`), and `=`, `+`, `×`, `÷`, `²` are spoken;
 subtitles keep the line as written. Known issue: with every Kokoro voice, 逗号 can sound like 度号.
 
+**Chinese on an Apple Silicon Mac: Qwen3-TTS.** `voice: { provider: qwen }` speaks with
+[Qwen3-TTS 0.6B CustomVoice](https://huggingface.co/Qwen/Qwen3-TTS-12Hz-0.6B-CustomVoice) (Apache-2.0)
+through [mlx-audio](https://github.com/Blaizzy/mlx-audio). It reads English words, letters and numbers
+in a Chinese line more reliably than `zf_001`. Voices: `dylan` (male, Beijing accent; the default
+for Chinese tutorials, lessons and docs), `serena` (female, warm; the default for Chinese promos),
+`vivian` (female, bright), `uncle_fu` (male, mellow), `eric` (male, Sichuan accent), and `ryan`, `aiden` (English), `ono_anna` (Japanese), `sohee` (Korean). On
+first use Storyshot creates a venv in `~/.cache/storyshot/qwen-venv` (Python 3.10–3.12, about 400 MB)
+and downloads the ~2.3 GB model into the Hugging Face cache. It speaks at about 2x real time on an M4 Pro
+and peaks at about 8 GB of memory. The model samples its speech, so a line now and then runs on into
+invented words or stops short. Storyshot regenerates such a take (it judges by length) and warns if
+one still looks off. A scientific number like `2e-105` is spelled out in Chinese, because it sets the
+model babbling. On other machines the same storyboard renders with `kokoro` (`zf_001`, or `draftVoice`),
+with a note.
+
 Other fields: `speed` (default 1), `volume` (default 1), and `pronounce`, a
 map of word → spoken form used only for TTS, e.g.
 `pronounce: { pxGenius: P X Genius, PPTX: P P T X }`. Subtitles keep the real

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "storyshot",
-  "version": "0.5.0",
+  "version": "0.6.0",
   "description": "Turn a short YAML storyboard into a polished product demo video of your web app — smooth cursor, auto zoom, captions, music.",
   "author": "boheling <boheling@gmail.com>",
   "repository": {

package/skills/storyshot/SKILL.md

@@ -88,8 +88,13 @@ frames or stock footage presented as a recording of their app.
    and Mandarin Chinese: for Chinese lines use `voice: { provider: kokoro,
    voice: zf_001 }` (the default for Chinese `say:` lines), not macOS `say -v
    Tingting`, which sounds robotic. It needs Python 3.10–3.12 and sets up its
-   own venv on first use (`doctor` reports it). For other languages, use
-   ElevenLabs.
+   own venv on first use (`doctor` reports it). On an Apple Silicon Mac, the
+   best free Chinese voice is `voice: { provider: qwen }` (Qwen3-TTS: `dylan`,
+   a male teacher's voice, for tutorials and lessons; `serena`, female, for
+   promos; those are the defaults): clearer than
+   zf_001 with English words and letters, about 2x real time, a ~2.3 GB model
+   on first use. Elsewhere a `qwen` storyboard renders with kokoro. For other
+   languages, use ElevenLabs.
 3. **Login.** If the flow needs a logged-in page:
    - Tell the user a browser window will open and that they should log in
      there, preferably with a demo account (the video shows whatever is on

package/src/check.mjs

@@ -4,6 +4,7 @@ import YAML from 'yaml';
 import { loadConfig } from './config.mjs';
 import { gitIgnored } from './auth.mjs';
 import { isZhVoice, zhVoiceStatus } from './kokoro-zh.mjs';
+import { qwenStatus } from './qwen-tts.mjs';
 
 const TOP = ['name', 'url', 'out', 'purpose', 'viewport', 'output', 'theme', 'music', 'voice', 'sfx', 'storageState', 'ignoreHTTPSErrors', 'intro', 'frame', 'pace', 'follow', 'article', 'cards', 'lang', 'colorScheme',
   'scrollOffset', 'captionPosition', 'captions', 'highlight', 'stepNumbers', 'subtitles', 'chapters', 'clips', 'maxWait', 'setup', 'steps', 'end_card', 'hyperframes'];
@@ -163,6 +164,11 @@ export function check(file, { draft = false } = {}) {
       const zh = zhVoiceStatus();
       if (zh.problem) errors.push(zh.problem);
     }
+    if (v?.provider === 'qwen') {
+      const q = qwenStatus();
+      if (q.problem) errors.push(q.problem);
+    }
+    if (v?.fallback) warnings.push(v.fallback);
     if (!draft && cfg.voice && ['elevenlabs', 'openai'].includes(cfg.voice.provider)) {
       const env = cfg.voice.provider === 'elevenlabs' ? 'ELEVENLABS_API_KEY' : 'OPENAI_API_KEY';
       if (!process.env[env]) errors.push(`voice provider ${cfg.voice.provider} needs ${env} (environment or a .env file next to the storyboard) — or render with draft mode to use the free local voice`);

package/src/config.mjs

@@ -2,7 +2,8 @@
 import fs from 'node:fs';
 import path from 'node:path';
 import YAML from 'yaml';
-import { VOICE_DEFAULTS, kokoroVoiceFor } from './voice.mjs';
+import { VOICE_DEFAULTS, kokoroVoiceFor, qwenVoiceFor } from './voice.mjs';
+import { QWEN_VOICES, qwenSupported, qwenVoice } from './qwen-tts.mjs';
 
 // Built-in labels drawn into the video, by `lang` (e.g. es, es-MX); other languages get English
 export const LABELS = {
@@ -148,10 +149,22 @@ export function loadConfig(file, { draft = false } = {}) {
     const provider = raw.voice.provider ?? 'kokoro';
     if (!VOICE_DEFAULTS[provider]) throw new Error(`voice.provider must be one of ${Object.keys(VOICE_DEFAULTS).join(', ')}`);
     const pace0 = PACES[pick('pace')] ?? PACES.normal;
-    // kokoro's default voice follows the narration's language (zf_001 for Chinese, af_heart otherwise)
+    // kokoro's and qwen's default voices follow the narration's language (zf_001 / serena for Chinese)
     const kokoroVoice = kokoroVoiceFor(raw);
-    voice = { speed: pace0.voiceSpeed, volume: 1, ...VOICE_DEFAULTS[provider], ...(provider === 'kokoro' ? { voice: kokoroVoice } : {}), ...raw.voice, provider };
-    if (draft && provider !== 'kokoro') {
+    const byLang = { kokoro: kokoroVoice, qwen: qwenVoiceFor(raw) }[provider];
+    voice = { speed: pace0.voiceSpeed, volume: 1, ...VOICE_DEFAULTS[provider], ...(byLang ? { voice: byLang } : {}), ...raw.voice, provider };
+    if (provider === 'qwen') {
+      const v = qwenVoice(voice.voice);
+      if (!v) throw new Error(`voice: qwen has no voice "${voice.voice}" (one of ${QWEN_VOICES.join(', ')})`);
+      voice.voice = v;
+      // MLX needs Apple Silicon: anywhere else the same storyboard renders with kokoro
+      if (!qwenSupported()) {
+        const k = raw.voice.draftVoice ?? kokoroVoice;
+        voice = { ...voice, provider: 'kokoro', voice: k, fallback: `the qwen voice needs an Apple Silicon Mac; using kokoro ${k} instead` };
+      }
+    }
+    // free local voices stay; a paid one is swapped for kokoro
+    if (draft && !['kokoro', 'qwen'].includes(voice.provider)) {
       voice = { ...voice, provider: 'kokoro', voice: raw.voice.draftVoice ?? kokoroVoice, model: undefined };
     }
   }

package/src/doctor.mjs

@@ -7,6 +7,7 @@ import { createRequire } from 'node:module';
 import path from 'node:path';
 import { hasFilter, hasEncoder, NEEDED, OPTIONAL } from './ffmpeg.mjs';
 import { zhVoiceStatus } from './kokoro-zh.mjs';
+import { qwenStatus } from './qwen-tts.mjs';
 import { hyperframesCli, HF_VERSION } from './hyperframes.mjs';
 
 const has = (cmd, args = ['--version']) => { try { execFileSync(cmd, args, { stdio: 'ignore' }); return true; } catch { return false; } };
@@ -78,6 +79,11 @@ export async function doctor({ log = console.log } = {}) {
   lines.push(zh.ready ? `✔ Chinese voice (kokoro zf_001): Python ${zh.python?.version ?? '?'} venv in ${zh.venv}`
     : zh.python ? `· Chinese voice (kokoro zf_001): Python ${zh.python.version} found; storyshot sets up its venv in ${zh.venv} on first use`
       : '· Chinese voice (kokoro zf_001) needs Python 3.10–3.12 — optional: brew install python@3.12, or uv python install 3.12 (or set STORYSHOT_PYTHON)');
+  const q = qwenStatus();
+  lines.push(!q.supported ? '· Qwen3-TTS voice (provider: qwen) needs an Apple Silicon Mac; storyboards that use it fall back to kokoro here'
+    : q.ready ? `✔ Qwen3-TTS voice (provider: qwen): venv in ${q.venv}${q.model ? ', model downloaded' : '; the model (~2.3 GB) downloads on first use'}`
+      : q.python ? `· Qwen3-TTS voice (provider: qwen): Python ${q.python.version} found; storyshot sets up its venv in ${q.venv} and downloads the model (~2.3 GB) on first use`
+        : '· Qwen3-TTS voice (provider: qwen) needs Python 3.10–3.12 — optional: brew install python@3.12, or uv python install 3.12 (or set STORYSHOT_PYTHON)');
   const hf = hyperframesCli();
   lines.push(hf.error ? `· HyperFrames cards (hyperframes: true) unavailable — optional: ${hf.error}`
     : hf.missing ? `· HyperFrames cards (hyperframes: true): ${HF_VERSION} installs itself on first use`

package/src/qwen-tts.mjs

@@ -0,0 +1,134 @@
+// Narration with Qwen3-TTS 0.6B CustomVoice (Apache-2.0): nine preset voices, five of them Chinese
+// (serena, vivian, uncle_fu, dylan with a Beijing accent, eric with a Sichuan one), free and local.
+// It runs through mlx-audio, so on Apple Silicon only; elsewhere config.mjs falls back to kokoro.
+// Python (qwen_tts.py) runs in storyshot's own venv, once per render for all uncached lines.
+import { execFileSync, spawn, spawnSync } from 'node:child_process';
+import crypto from 'node:crypto';
+import fs from 'node:fs';
+import os from 'node:os';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { findPython } from './kokoro-zh.mjs';
+
+export const QWEN_VOICES = ['serena', 'vivian', 'uncle_fu', 'dylan', 'eric', 'ryan', 'aiden', 'ono_anna', 'sohee'];
+export const qwenSupported = () => process.platform === 'darwin' && process.arch === 'arm64';
+// `Uncle Fu`, `Serena` → uncle_fu, serena (null when it isn't one of the nine)
+export const qwenVoice = (name) => {
+  const v = String(name ?? '').trim().toLowerCase().replace(/[\s-]+/g, '_');
+  return QWEN_VOICES.includes(v) ? v : null;
+};
+
+const REPO = 'mlx-community/Qwen3-TTS-12Hz-0.6B-CustomVoice-bf16';
+const REVISION = '6415d95f88be018ff9e46813119dc3bc12261328';
+// 0.9 (the model's default) now and then runs on into made-up words; 0.7 keeps the same voice steadier
+const TEMPERATURE = 0.7;
+const REQUIREMENTS = ['mlx-audio==0.6.3', 'mlx==0.32.4', 'mlx-metal==0.32.4', 'transformers==5.22.0', 'huggingface-hub==2.4.0',
+  'tokenizers==0.23.3', 'safetensors==0.8.0', 'numpy==2.5.4'];
+// Part of every cached line's hash: change it when the model, the packages or qwen_tts.py's sampling change.
+export const QWEN_TTS_ID = `${REPO}@${REVISION.slice(0, 7)}|${REQUIREMENTS[0]}|t${TEMPERATURE}|guard-1`;
+const CACHE = path.join(os.homedir(), '.cache', 'storyshot');
+export const QWEN_VENV = path.join(CACHE, 'qwen-venv');
+const HELPER = path.join(path.dirname(fileURLToPath(import.meta.url)), 'qwen_tts.py');
+const HUB = process.env.HF_HUB_CACHE ?? path.join(process.env.HF_HOME ?? path.join(os.homedir(), '.cache', 'huggingface'), 'hub');
+const MODEL_FILE = path.join(HUB, `models--${REPO.replace('/', '--')}`, 'snapshots', REVISION, 'model.safetensors');
+
+// --- text (TTS only: subtitles keep the original) --------------------------------------------------
+// The model reads = + 2个 第2行 x² itself; a minus sign before a number it drops, and a number like
+// 2e-105 sets it babbling, so those are spelled out in Chinese lines.
+export function qwenSpeakText(text, lang = qwenLang(text)) {
+  let t = String(text).replace(/\s*\n+\s*/g, ' ');
+  if (lang === 'chinese') {
+    t = t.replace(/(\d(?:\.\d+)?)[eE]\s*([-−+]?)\s*(\d+)/g, (_, m, s, e) => `${m}乘10的${s && s !== '+' ? '负' : ''}${e}次方`)
+      .replace(/(?<![\w)²³]\s*)−\s*(?=\d)/g, '负');
+  }
+  return t;
+}
+
+// The language the model is told: the script decides; Latin text is left to the model.
+export function qwenLang(text) {
+  const s = String(text).replace(/\d+(?:\.\d+)?(?:[eE][-−+]?\d+)?/g, ' '); // the e in 2e-105 is no English word
+  if (/[぀-ヿ]/.test(s)) return 'japanese';
+  if (/[가-힯]/.test(s)) return 'korean';
+  const han = (s.match(/[㐀-鿿]/g) ?? []).length, latin = (s.match(/[A-Za-z]+/g) ?? []).length;
+  return han > latin ? 'chinese' : 'auto';
+}
+
+// --- the venv ---------------------------------------------------------------------------------------
+const venvPython = (dir) => path.join(dir, 'bin', 'python');
+const STAMP = 'storyshot-requirements.txt';
+const venvReady = () => {
+  try { return fs.readFileSync(path.join(QWEN_VENV, STAMP), 'utf8') === REQUIREMENTS.join('\n') && fs.existsSync(venvPython(QWEN_VENV)); } catch { return false; }
+};
+
+const NO_PYTHON = () => 'the Qwen3-TTS voice needs Python 3.10–3.12 to run (brew install python@3.12, or uv python install 3.12, ' +
+  `or set STORYSHOT_PYTHON); storyshot then keeps its packages in its own venv (${QWEN_VENV}), nothing is installed globally`;
+
+// For doctor and check: can this machine use it, and what's still to come on first use?
+export function qwenStatus() {
+  if (!qwenSupported()) return { supported: false };
+  const model = fs.existsSync(MODEL_FILE);
+  if (venvReady()) return { supported: true, ready: true, venv: QWEN_VENV, model };
+  const python = findPython();
+  return { supported: true, ready: false, venv: QWEN_VENV, model, python, problem: python ? null : NO_PYTHON() };
+}
+
+function ensureVenv(log) {
+  if (venvReady()) return venvPython(QWEN_VENV);
+  const py = findPython();
+  if (!py) throw new Error(NO_PYTHON());
+  log(`setting up the Qwen3-TTS voice (one time): a Python ${py.version} venv in ${QWEN_VENV} with mlx-audio (~400 MB, 1–2 min)…`);
+  const run = (cmd, args, what) => {
+    const r = spawnSync(cmd, args, { encoding: 'utf8', stdio: ['ignore', 'pipe', 'pipe'] });
+    if (r.status !== 0) throw new Error(`${what} failed: ${(r.stderr || r.stdout || r.error?.message || '').trim().split('\n').slice(-6).join('\n')}`);
+  };
+  fs.rmSync(QWEN_VENV, { recursive: true, force: true });
+  fs.mkdirSync(CACHE, { recursive: true });
+  run(py.exe, ['-m', 'venv', QWEN_VENV], `creating the venv with ${py.exe} -m venv`);
+  run(venvPython(QWEN_VENV), ['-m', 'pip', 'install', '--disable-pip-version-check', '--no-input', '-q', ...REQUIREMENTS], 'installing mlx-audio into the venv');
+  fs.writeFileSync(path.join(QWEN_VENV, STAMP), REQUIREMENTS.join('\n')); // written last: a half-built venv is rebuilt next time
+  return venvPython(QWEN_VENV);
+}
+
+// --- synthesis --------------------------------------------------------------------------------------
+// entries: [{ text, file }] (text already through qwenSpeakText) → a WAV at each `file`.
+export async function synthQwen(v, entries, { log = console.log } = {}) {
+  if (!entries.length) return;
+  if (!qwenSupported()) throw new Error('the Qwen3-TTS voice runs on Apple Silicon Macs only (it uses MLX)');
+  const py = ensureVenv(log);
+  if (!fs.existsSync(MODEL_FILE)) log(`downloading the Qwen3-TTS model (one time, ~2.3 GB) into ${HUB}…`);
+  const lines = entries.map((e) => ({
+    text: e.text, file: `${e.file}.raw.wav`, voice: v.voice, lang: qwenLang(e.text),
+    // the same line gets the same take: a seed from the text
+    seed: parseInt(crypto.createHash('sha1').update(e.text).digest('hex').slice(0, 7), 16),
+  }));
+  const job = JSON.stringify({ repo: REPO, revision: REVISION, temperature: TEMPERATURE, lines });
+  const { code, stdout, tail } = await new Promise((resolve, reject) => {
+    const p = spawn(py, ['-I', HELPER], { env: { ...process.env, HF_HUB_DISABLE_TELEMETRY: '1', TRANSFORMERS_VERBOSITY: 'error' } });
+    let out = '', err = '', buf = '';
+    p.stdout.on('data', (d) => { out += d; });
+    p.stderr.on('data', (d) => {
+      err = (err + d).slice(-4000);
+      buf += d;
+      let nl;
+      while ((nl = buf.search(/[\r\n]/)) >= 0) {
+        const m = buf.slice(0, nl).match(/^line (\d+)\/\d+ (\S+)( \d+ tries)?/);
+        buf = buf.slice(nl + 1);
+        if (m) log(`  tts (qwen/${v.voice}, ${m[2]}${m[3] ? `,${m[3]}` : ''}): "${entries[m[1] - 1].text}"`);
+      }
+    });
+    p.on('error', reject);
+    p.on('close', (c) => resolve({ code: c, stdout: out, tail: err }));
+    p.stdin.end(job);
+  });
+  if (code !== 0) throw new Error(`Qwen3-TTS failed (${py} ${HELPER}): ${tail.trim().split('\n').slice(-8).join('\n')}`);
+  const { results } = JSON.parse(stdout);
+  for (const [i, e] of entries.entries()) {
+    if (results[i]?.suspect) log(`  ⚠ qwen: "${e.text}" still sounds off after ${results[i].tries} tries (${results[i].seconds.toFixed(1)}s) — listen to it, or rephrase the line / add a pronounce entry`);
+    const raw = lines[i].file, tmp = `${e.file}.tmp.wav`;
+    // the model has no speed control: ffmpeg's pitch-preserving atempo does it
+    if ((v.speed ?? 1) !== 1) execFileSync('ffmpeg', ['-loglevel', 'error', '-y', '-i', raw, '-af', `atempo=${v.speed}`, tmp]);
+    else fs.renameSync(raw, tmp);
+    fs.rmSync(raw, { force: true });
+    fs.renameSync(tmp, e.file);
+  }
+}

package/src/qwen_tts.py

@@ -0,0 +1,70 @@
+# Qwen3-TTS CustomVoice through mlx-audio (Apple Silicon), for `voice: { provider: qwen }`. Runs in
+# storyshot's own venv (qwen-tts.mjs), one process per render: the model loads once for all lines.
+#
+# stdin:  {"repo", "revision", "temperature", "lines": [{"text", "file", "voice", "lang", "seed"}]}
+# stderr: "line i/n <seconds>s [<tries> tries]" per line, plus whatever the libraries log
+# stdout: {"results": [{"seconds", "tries", "suspect"}]}
+#
+# The model samples its speech tokens, so now and then a line runs on into made-up words (a
+# scientific number like 2e-105 does it) or stops early. A take far longer or shorter than the text
+# should need is sampled again with another seed, and the closest one is kept.
+import contextlib
+import json
+import os
+import re
+import sys
+
+os.environ.setdefault('HF_HUB_DISABLE_TELEMETRY', '1')
+os.environ.setdefault('TRANSFORMERS_VERBOSITY', 'error')
+
+TRIES = 4
+
+
+def expected(text):
+    """Rough speaking time in seconds: ~3.6 Chinese characters a second, Latin words and numbers."""
+    han = len(re.findall(r'[㐀-鿿]', text))
+    words = len(re.findall(r'[A-Za-z]+', text))
+    nums = len(re.findall(r'\d+(?:\.\d+)?', text))
+    return han / 3.6 + words * 0.4 + nums * 0.5 + 0.3
+
+
+def plausible(seconds, est):
+    return 0.35 * est - 0.5 <= seconds <= 2.0 * est + 2.0
+
+
+def main():
+    job = json.load(sys.stdin)
+    out = sys.stdout
+    # the libraries print their own notes on stdout: keep it for the JSON answer
+    with contextlib.redirect_stdout(sys.stderr):
+        import math
+        import mlx.core as mx
+        import numpy as np
+        from huggingface_hub import snapshot_download
+        from mlx_audio.audio_io import write
+        from mlx_audio.tts.utils import load_model
+
+        model = load_model(snapshot_download(job['repo'], revision=job['revision']))
+        results, n = [], len(job['lines'])
+        for i, line in enumerate(job['lines']):
+            est = expected(line['text'])
+            best = None
+            for k in range(TRIES):
+                mx.random.seed(line['seed'] + k * 7919)
+                parts = model.generate(text=line['text'], voice=line['voice'], lang_code=line['lang'],
+                                       temperature=job['temperature'])
+                audio = np.concatenate([np.array(r.audio, dtype=np.float32) for r in parts])
+                seconds = len(audio) / model.sample_rate
+                score = abs(math.log(max(seconds, 0.1) / est))
+                if best is None or score < best[0]:
+                    best = (score, audio, seconds)
+                if plausible(seconds, est):
+                    break
+            write(line['file'], best[1], model.sample_rate)
+            results.append({'seconds': best[2], 'tries': k + 1, 'suspect': not plausible(best[2], est)})
+            print(f'line {i + 1}/{n} {best[2]:.1f}s' + (f' {k + 1} tries' if k else ''), file=sys.stderr, flush=True)
+    out.write(json.dumps({'results': results}))
+
+
+if __name__ == '__main__':
+    main()

package/src/voice.mjs

@@ -1,15 +1,18 @@
 // Voiceover: text-to-speech for each step's `say:` line, cached on disk.
-// Providers: kokoro (local, free: English, and Mandarin with the v1.1-zh voices), elevenlabs, openai, say (macOS).
+// Providers: kokoro (local, free: English, and Mandarin with the v1.1-zh voices), qwen (local, free, Apple
+// Silicon: Qwen3-TTS, best for Chinese), elevenlabs, openai, say (macOS).
 import { execFileSync } from 'node:child_process';
 import crypto from 'node:crypto';
 import fs from 'node:fs';
 import os from 'node:os';
 import path from 'node:path';
 import { isZhVoice, synthZh, zhSpeakText, ZH_DEFAULT_VOICE, ZH_TTS_ID } from './kokoro-zh.mjs';
+import { qwenSpeakText, QWEN_TTS_ID, synthQwen } from './qwen-tts.mjs';
 
 const SR = 48000;
 export const VOICE_DEFAULTS = {
   kokoro: { voice: 'af_heart' },
+  qwen: { voice: 'dylan' },
   elevenlabs: { voice: 'JBFqnCBsd6RMkjVDRZzb', model: 'eleven_multilingual_v2' },
   openai: { voice: 'alloy', model: 'gpt-4o-mini-tts' },
   say: { voice: 'Samantha' },
@@ -19,10 +22,24 @@ export const VOICE_DEFAULTS = {
 // characters than Latin words; no kana or hangul), or, with no narration to go by, `lang: zh`.
 // English (af_heart) otherwise.
 export function kokoroVoiceFor(raw) {
-  const says = [...(raw.steps ?? []).flatMap((s) => [s?.say, s?.card?.say]), raw.intro?.say, raw.end_card?.say].filter(Boolean).join(' ');
+  return narrationIsChinese(raw) ? ZH_DEFAULT_VOICE : VOICE_DEFAULTS.kokoro.voice;
+}
+
+// qwen without a named voice, for Chinese: dylan (a steady teacher's voice) in tutorials, lessons and
+// docs, serena (warmer) in promos; ono_anna for Japanese, sohee for Korean, aiden otherwise
+export function qwenVoiceFor(raw) {
+  const says = narration(raw);
+  if (/[\u3040-\u30ff]/.test(says) || /^ja/i.test(String(raw.lang ?? ''))) return 'ono_anna';
+  if (/[\uac00-\ud7af]/.test(says) || /^ko/i.test(String(raw.lang ?? ''))) return 'sohee';
+  if (!narrationIsChinese(raw)) return 'aiden';
+  return (raw.purpose ?? 'pitch') === 'pitch' ? 'serena' : VOICE_DEFAULTS.qwen.voice;
+}
+
+const narration = (raw) => [...(raw.steps ?? []).flatMap((s) => [s?.say, s?.card?.say]), raw.intro?.say, raw.end_card?.say].filter(Boolean).join(' ');
+function narrationIsChinese(raw) {
+  const says = narration(raw);
   const han = (says.match(/[\u3400-\u9fff]/g) ?? []).length, latin = (says.match(/[A-Za-z]+/g) ?? []).length;
-  const zh = says.trim() ? han > latin && !/[\u3040-\u30ff\uac00-\ud7af]/.test(says) : /^zh/i.test(String(raw.lang ?? ''));
-  return zh ? ZH_DEFAULT_VOICE : VOICE_DEFAULTS.kokoro.voice;
+  return says.trim() ? han > latin && !/[\u3040-\u30ff\uac00-\ud7af]/.test(says) : /^zh/i.test(String(raw.lang ?? ''));
 }
 
 let kokoro = null;
@@ -72,7 +89,7 @@ async function synth(v, text, file) {
       execFileSync('say', ['-v', v.voice, '-r', String(Math.round(185 * v.speed)), '-o', file, text]);
       return;
     default:
-      throw new Error(`unknown voice provider "${v.provider}" (use kokoro, elevenlabs, openai or say)`);
+      throw new Error(`unknown voice provider "${v.provider}" (use kokoro, qwen, elevenlabs, openai or say)`);
   }
 }
 
@@ -86,26 +103,28 @@ function pronounce(v, text) {
   return t;
 }
 
-// The cache file for a line, and the text the engine is given. The Mandarin voice reads a rewritten
-// text (2个 → 两个, …) and has its model and G2P version in the hash.
+// The cache file for a line, and the text the engine is given. The Mandarin Kokoro voice reads a
+// rewritten text (2个 → 两个, …) and has its model and G2P version in the hash; so does qwen.
 function cached(cfg, line) {
   const v = cfg.voice;
   const dir = path.join(path.dirname(cfg.work), 'tts-cache');
   fs.mkdirSync(dir, { recursive: true });
-  const zh = v.provider === 'kokoro' && isZhVoice(v.voice);
-  const text = zh ? zhSpeakText(pronounce(v, line)) : pronounce(v, line);
-  const ext = { kokoro: 'wav', say: 'aiff' }[v.provider] ?? 'mp3';
-  const key = zh ? [v.provider, v.voice, ZH_TTS_ID, v.speed, text] : [v.provider, v.voice, v.model, v.speed, v.instructions, text];
+  const zh = v.provider === 'kokoro' && isZhVoice(v.voice), qwen = v.provider === 'qwen';
+  const text = zh ? zhSpeakText(pronounce(v, line)) : qwen ? qwenSpeakText(pronounce(v, line)) : pronounce(v, line);
+  const ext = { kokoro: 'wav', qwen: 'wav', say: 'aiff' }[v.provider] ?? 'mp3';
+  const key = zh ? [v.provider, v.voice, ZH_TTS_ID, v.speed, text] : qwen ? [v.provider, v.voice, QWEN_TTS_ID, v.speed, text]
+    : [v.provider, v.voice, v.model, v.speed, v.instructions, text];
   const hash = crypto.createHash('sha1').update(JSON.stringify(key)).digest('hex').slice(0, 16);
-  return { text, file: path.join(dir, `${hash}.${ext}`), zh };
+  return { text, file: path.join(dir, `${hash}.${ext}`), zh, qwen };
 }
 
 // Returns stereo Float32 PCM at 48 kHz with leading/trailing silence trimmed.
 export async function speak(cfg, line, { log = console.log } = {}) {
   const v = cfg.voice;
-  const { text, file, zh } = cached(cfg, line);
+  const { text, file, zh, qwen } = cached(cfg, line);
   if (!fs.existsSync(file)) {
     if (zh) await synthZh(v, [{ text, file }], { log });
+    else if (qwen) await synthQwen(v, [{ text, file }], { log });
     else {
       log(`  tts (${v.provider}): "${text}"`);
       await synth(v, text, file);
@@ -129,11 +148,13 @@ export function tempo(pcm, factor) {
 export async function measureVoice(cfg, { log = console.log } = {}) {
   const lens = { steps: {}, endCard: null, intro: null, cards: {} };
   if (!cfg.voice) return lens;
-  if (cfg.voice.provider === 'kokoro' && isZhVoice(cfg.voice.voice)) {
-    // synthesize every uncached Mandarin line in one batch, so the Python G2P starts once
+  if (cfg.voice.fallback) log(`note: ${cfg.voice.fallback}`);
+  const zh = cfg.voice.provider === 'kokoro' && isZhVoice(cfg.voice.voice);
+  if (zh || cfg.voice.provider === 'qwen') {
+    // synthesize every uncached line in one batch, so Python (the G2P, or the Qwen model) starts once
     const lines = [...cfg.steps.filter((s) => !s.hidden).flatMap((s) => [s.say, s.card?.say]), cfg.intro?.say, cfg.endCard?.say].filter(Boolean);
     const todo = new Map(lines.map((l) => cached(cfg, l)).filter((c) => !fs.existsSync(c.file)).map((c) => [c.file, c]));
-    await synthZh(cfg.voice, [...todo.values()], { log });
+    await (zh ? synthZh : synthQwen)(cfg.voice, [...todo.values()], { log });
   }
   for (const [i, s] of cfg.steps.entries()) {
     if (s.say && !s.hidden) lens.steps[i] = (await speak(cfg, s.say, { log })).length / 2 / SR;