@mekuru/voisona 0.0.0-stage → 0.1.2

package/README.jp.md

@@ -0,0 +1,123 @@
+# @mekuru/voisona
+
+[English](README.md)
+
+Mekuru Movie Maker のセリフを、VoiSona Talk の REST API で読み上げる `Voisona` 部品です。
+
+## セットアップ
+
+Windows・macOS で VoiSona Talk 1.3.7 以降にサインインし、ボイスライブラリを取得します。環境設定の「API」タブで API パスワードを設定し、「REST APIを有効にする」をオンにします（ポート `32766`）。アプリは起動したまま使います。[公式の設定手順](https://manual.voisona.com/ja/talk/pc/2b6e9bc7efb180ea86ccc6c7347e9ca6)。
+
+Bun を動かすシェルで、サインインのメールアドレスと「API」タブのパスワードを設定します。
+
+```sh
+export MEKURU_VOISONA_USER='you@example.com'
+export MEKURU_VOISONA_PASSWORD='your-api-password'
+```
+
+例では `tanaka-san_ja_JP`（田中傘）を使います。取得済みのライブラリを指定し、その利用規約に従ってください。
+
+Bun 1.4 以上・libx264 付き ffmpeg・ffprobe・日本語フォントを用意し、プロジェクトに追加します。
+
+```sh
+bun add -d @mekuru/core @mekuru/cli @mekuru/voisona
+```
+
+`tsconfig.json` に `jsx: "react-jsx"` と `jsxImportSource: "@mekuru/core"` を設定します。`bun create @mekuru` では設定済みです。
+
+## 簡単な使い方
+
+`src/main.tsx` に保存します。
+
+```tsx
+import { writeXml } from "@mekuru/core";
+import { Voisona } from "@mekuru/voisona";
+
+const text = "こんにちは。";
+
+await writeXml(
+  ".mekuru/main.xml",
+  <mekuru size="1280x720" fps={30}>
+    <stack>
+      <Voisona voice="tanaka-san_ja_JP">{text}</Voisona>
+      <text x="50%" y="50%" anchor="center" size={60} color="#FFFFFF">{text}</text>
+    </stack>
+  </mekuru>,
+);
+```
+
+```sh
+bun src/main.tsx
+bunx mekuru-server .mekuru/main.xml
+```
+
+`http://localhost:3333` でプレビューします。書き出しは `bunx mekuru-compiler .mekuru/main.xml -o video.mp4`。
+
+## キャラクター
+
+`<Voisona>` に渡すキャラクターごとの設定をエクスポートしています。展開して使い、ほかの props も追加できます。
+
+```tsx
+import { Voisona, tanaka } from "@mekuru/voisona";
+
+<Voisona {...tanaka}>こんにちは。</Voisona>;
+```
+
+| エクスポート | 名前 | `voice` |
+| --- | --- | --- |
+| `tanaka` | 田中傘 | `"tanaka-san_ja_JP"` |
+| `chisa` | 知声 | `"chis-a_ja_JP"` |
+| `sasara` | さとうささら | `"sato-sasara_ja_JP"` |
+| `tsuzumi` | すずきつづみ | `"suzuki-tsudumi_ja_JP"` |
+| `takahashi` | タカハシ | `"takahashi_ja_JP"` |
+| `rikka` | 小春六花 | `"koharu-rikka_ja_JP"` |
+| `karin` | 夏色花梨 | `"natsuki-karin_ja_JP"` |
+| `chifuyu` | 花隈千冬 | `"hanakuma-chifuyu_ja_JP"` |
+| `minato` | 双葉湊音 | `"futaba-minato_ja_JP"` |
+
+## 詳しい使い方
+
+### 声の設定 (`VoisonaProps`)
+
+- `voice`: 必須: `"tanaka-san_ja_JP"` などのボイスライブラリ名、または `"田中傘"` などの表示名。
+- `version`: ボイスライブラリのバージョン。既定は選択した声のインストール済みの最新版。
+- `language`: `"ja_JP"` などの言語コード。既定はその声の一覧の最初の言語。
+- `speed`: API の単位での話す速さ。
+- `pitch`: 声の高さの調整（セント）。
+- `volume`: 合成時の音量（dB）。
+- `alp`: 声質のパラメータ（`alp`）。
+- `huskiness`: 声のかすれ具合。
+- `intonation`: 抑揚の強さ。
+- `style`: スタイル名と強さの辞書。`happy` がある声なら `{ happy: 0.5 }` など。指定しないスタイルはその声の既定の強さを使います。
+- `max-chars`: 1回の合成リクエストの最大文字数。既定は `500`。
+- `url`: API のベース URL。既定は `http://localhost:32766`。`/api/talk/v1` は含めません。
+
+省略した数値設定は API の既定値を使います。`/api/talk/v1/voices` で声・バージョン・言語、`/api/talk/v1/voices/<voice_name>/<voice_version>` で `style_names`・`default_style_weights` を確認できます。表示名が重複する場合は `voice_name` を指定します。長文は句読点の近くで分割して一つの WAV に結合し、`gap` は全体の後に入れます。
+
+### テキストと時間
+
+- `children`: セリフ。文字列・数値・フラグメント・`<br />`。空の読み上げはエラー。
+- `name`: タイムラインの話者名。既定: `voice`。
+- `reading`: 読みの辞書。例: `{ Mekuru: "めくる" }`。長いキーを優先し、表示は変えない。
+- `gap`: セリフの後の無音（フレーム）。既定: `Math.round(0.3 * fps)`（30 fps で9）。
+- `gain-db`: 再生音量の調整（dB）。既定: 調整なし。
+- `len`: 無音を含む全体のフレーム数。既定: 音声の長さを切り上げた値 + `gap`。話速は変えない。
+- `id`・`at`: 生成する `<dur>` の識別名と開始時刻。[時間の属性](../../docs/reference/Syntax.jp.md#時間の属性)を参照。
+
+ルビ `｜表示《よみ》` は「よみ」を読み、タイムラインには「表示」を残します。半角 `|` と `漢字《かんじ》` も使用できます。`<br />` は空白になり、ほかの要素は入れられません。セリフの連続再生は `<seq>`、字幕との同時表示は `<stack>`、口パクは[立ち絵の `Talk`](../portrait/README.jp.md) を使います。
+
+### キャッシュとプレビュー
+
+- 音声はエンジン・声の設定・読み上げテキストをキーに `.mekuru/cache/tts/` へ保存。キャッシュ済みならエンジン停止中も使用できます。
+- 接続先や再生設定の変更では再合成しません。同名のモデルを更新した場合などは、該当する WAV を削除して再生成します。
+- `writeXml()` は合成中に無音の下書きを出し、完了後に XML を更新します。動画の書き出し前に `await` してください。
+
+### 接続
+
+接続先は `url` → `MEKURU_VOISONA_URL` → 既定 URL の順で決まります。Bun と同じ環境から確認します。
+
+```sh
+curl --user "$MEKURU_VOISONA_USER" http://localhost:32766/api/talk/v1/voices
+```
+
+curl に API パスワードを入力します。HTTP 401 はメールアドレスと API パスワードを確認してください。接続できない場合は起動状態とポートを確認します。Windows のエンジンには [WSL の mirrored ネットワーク](https://learn.microsoft.com/en-us/windows/wsl/networking#mirrored-mode-networking)で `127.0.0.1` から接続できます。`localhost` で失敗する場合は `MEKURU_VOISONA_URL=http://127.0.0.1:32766` を指定します。

package/README.md

@@ -1,3 +1,123 @@
-# Temporary Holding Version
+# @mekuru/voisona
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+[日本語](README.jp.md)
+
+The `Voisona` component reads Mekuru Movie Maker dialogue through the VoiSona Talk REST API.
+
+## Setup
+
+Install VoiSona Talk 1.3.7 or later on Windows or macOS, sign in, and download a voice library. In Preferences → API, set an API password and enable the REST API (port `32766`). Keep the app running. See the [official setup guide](https://manual.voisona.com/en/talk/pc/2b6e9bc7efb18014b922c93fcaa8aac4).
+
+Set these variables in the shell running Bun. Use your sign-in email and the password set in the API tab:
+
+```sh
+export MEKURU_VOISONA_USER='you@example.com'
+export MEKURU_VOISONA_PASSWORD='your-api-password'
+```
+
+The example uses `tanaka-san_ja_JP` (田中傘). Use an installed library and follow its terms.
+
+Requires Bun 1.4+, ffmpeg with libx264, ffprobe, and Japanese fonts. Install in the project folder:
+
+```sh
+bun add -d @mekuru/core @mekuru/cli @mekuru/voisona
+```
+
+In `tsconfig.json`, set `jsx: "react-jsx"` and `jsxImportSource: "@mekuru/core"` (already set by `bun create @mekuru`).
+
+## Quick start
+
+Save as `src/main.tsx`:
+
+```tsx
+import { writeXml } from "@mekuru/core";
+import { Voisona } from "@mekuru/voisona";
+
+const text = "こんにちは。";
+
+await writeXml(
+  ".mekuru/main.xml",
+  <mekuru size="1280x720" fps={30}>
+    <stack>
+      <Voisona voice="tanaka-san_ja_JP">{text}</Voisona>
+      <text x="50%" y="50%" anchor="center" size={60} color="#FFFFFF">{text}</text>
+    </stack>
+  </mekuru>,
+);
+```
+
+```sh
+bun src/main.tsx
+bunx mekuru-server .mekuru/main.xml
+```
+
+Preview: `http://localhost:3333`. Export with `bunx mekuru-compiler .mekuru/main.xml -o video.mp4`.
+
+## Characters
+
+The package exports each character's settings for `<Voisona>`. Spread one and add other props as needed.
+
+```tsx
+import { Voisona, tanaka } from "@mekuru/voisona";
+
+<Voisona {...tanaka}>こんにちは。</Voisona>;
+```
+
+| Export | Name | `voice` |
+| --- | --- | --- |
+| `tanaka` | 田中傘 | `"tanaka-san_ja_JP"` |
+| `chisa` | 知声 | `"chis-a_ja_JP"` |
+| `sasara` | さとうささら | `"sato-sasara_ja_JP"` |
+| `tsuzumi` | すずきつづみ | `"suzuki-tsudumi_ja_JP"` |
+| `takahashi` | タカハシ | `"takahashi_ja_JP"` |
+| `rikka` | 小春六花 | `"koharu-rikka_ja_JP"` |
+| `karin` | 夏色花梨 | `"natsuki-karin_ja_JP"` |
+| `chifuyu` | 花隈千冬 | `"hanakuma-chifuyu_ja_JP"` |
+| `minato` | 双葉湊音 | `"futaba-minato_ja_JP"` |
+
+## Detailed usage
+
+### Voice (`VoisonaProps`)
+
+- `voice`: Required. Voice library name, e.g. `"tanaka-san_ja_JP"`, or display name, e.g. `"田中傘"`.
+- `version`: Voice library version. Default: newest installed version of the selected voice.
+- `language`: Language code such as `"ja_JP"`. Default: the voice's first listed language.
+- `speed`: Speaking speed, in the API's units.
+- `pitch`: Pitch adjustment in cents.
+- `volume`: Synthesis volume in dB.
+- `alp`: Voice character parameter (`alp`).
+- `huskiness`: Voice huskiness.
+- `intonation`: Intonation strength.
+- `style`: Map of style names to weights, e.g. `{ happy: 0.5 }` if that voice has a `happy` style. Unspecified weights keep the voice's defaults.
+- `max-chars`: Maximum characters per synthesis request. Default: `500`.
+- `url`: Base URL; default `http://localhost:32766`. Do not include `/api/talk/v1`.
+
+Omitted numeric voice parameters use the API defaults. `/api/talk/v1/voices` lists voices, versions, and languages; `/api/talk/v1/voices/<voice_name>/<voice_version>` provides `style_names` and `default_style_weights`. Use `voice_name` if display names overlap. Long lines are split near punctuation and joined into one WAV; `gap` follows the complete line.
+
+### Text and timing
+
+- `children`: Dialogue: strings, numbers, fragments, or `<br />`. Empty speech is an error.
+- `name`: Timeline speaker label. Default: `voice`.
+- `reading`: Pronunciation dictionary, e.g. `{ Mekuru: "めくる" }`. Longer keys win; displayed text is unchanged.
+- `gap`: Trailing silence in frames. Default: `Math.round(0.3 * fps)` (9 at 30 fps).
+- `gain-db`: Playback gain in dB. Default: no adjustment.
+- `len`: Total frames, including the gap. Default: rounded-up audio duration + `gap`. Does not change speaking speed.
+- `id`, `at`: Identifier and start timing for the generated `<dur>`; see [timing](../../docs/reference/Syntax.md#timing-attributes).
+
+Ruby `｜表示《よみ》` reads `よみ` and keeps `表示` in the timeline. ASCII `|` and `漢字《かんじ》` also work. `<br />` becomes a space; other nested elements are rejected. Use `<seq>` for successive lines and `<stack>` for speech with subtitles. For lip sync, wrap a line in a [portrait's `Talk`](../portrait/README.md).
+
+### Cache and preview
+
+- Audio is cached in `.mekuru/cache/tts/` by engine, voice settings, and spoken text. Cached lines work without a running engine.
+- Connection URLs and playback settings do not invalidate audio. Remove the cached WAV to regenerate speech after changing a server or updating a model under the same name.
+- `writeXml()` writes a silent draft while synthesizing, then replaces it with completed XML. Await it before video export.
+
+### Connection
+
+`url` → `MEKURU_VOISONA_URL` → the default URL, in that order. Check from the same environment as Bun:
+
+```sh
+curl --user "$MEKURU_VOISONA_USER" http://localhost:32766/api/talk/v1/voices
+```
+
+curl prompts for the API password. For HTTP 401, check the email and API password. For connection errors, check the engine and port. On Windows, use [WSL mirrored networking](https://learn.microsoft.com/en-us/windows/wsl/networking#mirrored-mode-networking) to reach the engine through `127.0.0.1`. Set `MEKURU_VOISONA_URL=http://127.0.0.1:32766` if `localhost` fails.

package/package.json

@@ -1,6 +1,25 @@
 {
   "name": "@mekuru/voisona",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
-}
+  "version": "0.1.2",
+  "description": "The speech component <Voisona> of Mekuru Movie Maker: lines of speech read by VoiSona Talk, through its REST API",
+  "type": "module",
+  "exports": {
+    ".": "./src/index.tsx"
+  },
+  "files": [
+    "src"
+  ],
+  "sideEffects": false,
+  "peerDependencies": {
+    "@mekuru/core": "^0.1.2"
+  },
+  "devDependencies": {
+    "@mekuru/core": "0.1.2"
+  },
+  "dependencies": {
+    "@mekuru/speech": "^0.1.2"
+  },
+  "engines": {
+    "bun": ">=1.4"
+  }
+}

package/src/characters.ts

@@ -0,0 +1,11 @@
+import type { VoisonaProps } from "./index.tsx";
+
+export const tanaka = { name: "田中傘", voice: "tanaka-san_ja_JP" } satisfies Partial<VoisonaProps>;
+export const chisa = { name: "知声", voice: "chis-a_ja_JP" } satisfies Partial<VoisonaProps>;
+export const sasara = { name: "さとうささら", voice: "sato-sasara_ja_JP" } satisfies Partial<VoisonaProps>;
+export const tsuzumi = { name: "すずきつづみ", voice: "suzuki-tsudumi_ja_JP" } satisfies Partial<VoisonaProps>;
+export const takahashi = { name: "タカハシ", voice: "takahashi_ja_JP" } satisfies Partial<VoisonaProps>;
+export const rikka = { name: "小春六花", voice: "koharu-rikka_ja_JP" } satisfies Partial<VoisonaProps>;
+export const karin = { name: "夏色花梨", voice: "natsuki-karin_ja_JP" } satisfies Partial<VoisonaProps>;
+export const chifuyu = { name: "花隈千冬", voice: "hanakuma-chifuyu_ja_JP" } satisfies Partial<VoisonaProps>;
+export const minato = { name: "双葉湊音", voice: "futaba-minato_ja_JP" } satisfies Partial<VoisonaProps>;

package/src/index.tsx

@@ -0,0 +1,197 @@
+import type { JSX } from "@mekuru/core";
+import { httpClient, limit, splitText } from "@mekuru/speech/engine";
+import { joinWavs, line, voiceOf, type SpeechProps } from "@mekuru/speech";
+
+const DEFAULT_URL = "http://localhost:32766";
+const API = "/api/talk/v1";
+const TIMEOUT = 300_000;
+
+export interface VoisonaProps extends SpeechProps {
+  voice: string;
+  version?: string;
+  language?: string;
+  speed?: number;
+  pitch?: number;
+  volume?: number;
+  alp?: number;
+  huskiness?: number;
+  intonation?: number;
+  style?: Readonly<Record<string, number>>;
+  "max-chars"?: number;
+  url?: string;
+}
+
+interface Voice {
+  voice_name: string;
+  voice_version: string;
+  languages: string[];
+  display_names?: { language: string; name: string }[];
+}
+
+interface VoiceDetail {
+  style_names?: string[];
+  default_style_weights?: number[];
+}
+
+interface Synthesis {
+  state: "queued" | "running" | "succeeded" | "failed";
+}
+
+const HELP =
+  "  - start VoiSona Talk (1.3.7 or later), sign in, and turn on 「REST APIを有効にする」 in the API tab of its preferences\n" +
+  "  - it listens on port 32766; if it is elsewhere (WSL2 without mirrored networking), set `url` on `<Voisona>`\n" +
+  "    or the MEKURU_VOISONA_URL environment variable";
+
+const CREDENTIALS =
+  "VoiSona Talk needs the email you sign in with and the API password (set in the API tab of its preferences):\n" +
+  "  set the MEKURU_VOISONA_USER and MEKURU_VOISONA_PASSWORD environment variables";
+
+const names = (v: Voice): string[] => [v.voice_name, ...(v.display_names ?? []).map((d) => d.name)];
+
+const newer = (a: string, b: string): number => {
+  const x = a.split(".").map(Number);
+  const y = b.split(".").map(Number);
+  for (let i = 0; i < Math.max(x.length, y.length); i++) {
+    const d = (y[i] ?? 0) - (x[i] ?? 0);
+    if (d) return d;
+  }
+  return 0;
+};
+
+class Server {
+  private readonly limited = limit(1);
+
+  private readonly raw: ReturnType<typeof httpClient>;
+
+  private voices?: Promise<Voice[]>;
+
+  private readonly details = new Map<string, Promise<VoiceDetail>>();
+
+  constructor(url: string) {
+    this.raw = httpClient(url, "VoiSona Talk", HELP);
+  }
+
+  private async call(path: string, init: RequestInit = {}): Promise<Response> {
+    const user = process.env.MEKURU_VOISONA_USER;
+    const password = process.env.MEKURU_VOISONA_PASSWORD;
+    if (!user || !password) throw new Error(CREDENTIALS);
+    const headers = new Headers(init.headers);
+    headers.set("Authorization", `Basic ${Buffer.from(`${user}:${password}`).toString("base64")}`);
+    try {
+      return await this.raw(API + path, { ...init, headers });
+    } catch (e) {
+      const message = e instanceof Error ? e.message : String(e);
+      throw /: HTTP 401\b/.test(message) ? new Error(`${message}\n${CREDENTIALS}`) : e;
+    }
+  }
+
+  private async voice(props: VoisonaProps): Promise<{ voice: Voice; language: string }> {
+    this.voices ??= this.call("/voices")
+      .then(async (res) => ((await res.json()) as { items: Voice[] }).items)
+      .catch((e: unknown) => {
+        this.voices = undefined;
+        throw e;
+      });
+    const all = await this.voices;
+    const found = all.filter((v) => names(v).includes(props.voice)).sort((a, b) => newer(a.voice_version, b.voice_version));
+    if (found.length === 0) {
+      const have = [...new Set(all.map((v) => `${v.voice_name} (${names(v).slice(1).join(", ")})`))];
+      throw new Error(`VoiSona Talk has no voice \`${props.voice}\` (it has ${have.join(", ") || "none"}; download voices in the app)`);
+    }
+    if (new Set(found.map((v) => v.voice_name)).size > 1) {
+      throw new Error(`VoiSona Talk has several voices named \`${props.voice}\`: write one of ${[...new Set(found.map((v) => v.voice_name))].join(", ")}`);
+    }
+    const voice = props.version === undefined ? found[0] : found.find((v) => v.voice_version === props.version);
+    if (!voice) throw new Error(`VoiSona Talk has no version ${props.version} of \`${props.voice}\` (it has ${found.map((v) => v.voice_version).join(", ")})`);
+    const language = props.language ?? voice.languages[0];
+    if (!voice.languages.includes(language)) {
+      throw new Error(`the VoiSona Talk voice \`${props.voice}\` does not speak ${language} (it speaks ${voice.languages.join(", ")})`);
+    }
+    return { voice, language };
+  }
+
+  private async styleWeights(voice: Voice, style: Readonly<Record<string, number>>): Promise<number[]> {
+    const key = `${voice.voice_name}/${voice.voice_version}`;
+    let detail = this.details.get(key);
+    if (!detail) {
+      detail = this.call(`/voices/${encodeURIComponent(voice.voice_name)}/${encodeURIComponent(voice.voice_version)}`).then(
+        async (res) => (await res.json()) as VoiceDetail,
+      );
+      detail.catch(() => this.details.delete(key));
+      this.details.set(key, detail);
+    }
+    const { style_names: styles = [], default_style_weights: defaults = [] } = await detail;
+    const weights = styles.map((_, i) => defaults[i] ?? 0);
+    for (const [name, weight] of Object.entries(style)) {
+      const i = styles.indexOf(name);
+      if (i < 0) throw new Error(`the VoiSona Talk voice \`${voice.voice_name}\` has no style \`${name}\` (it has ${styles.join(", ") || "none"})`);
+      weights[i] = weight;
+    }
+    return weights;
+  }
+
+  private async speak(body: object, text: string): Promise<Buffer> {
+    const res = await this.call("/speech-syntheses", {
+      method: "POST",
+      headers: { "Content-Type": "application/json" },
+      body: JSON.stringify({ ...body, text }),
+    });
+    const { uuid } = (await res.json()) as { uuid: string };
+    try {
+      const until = Date.now() + TIMEOUT;
+      for (;;) {
+        const { state } = (await (await this.call(`/speech-syntheses/${uuid}`)).json()) as Synthesis;
+        if (state === "succeeded") break;
+        if (state === "failed") throw new Error(`VoiSona Talk could not read: ${text.slice(0, 40)}`);
+        if (Date.now() > until) throw new Error(`VoiSona Talk did not finish in ${TIMEOUT / 1000}s: ${text.slice(0, 40)}`);
+        await Bun.sleep(100);
+      }
+      return Buffer.from(await (await this.call(`/speech-syntheses/${uuid}/wav`)).arrayBuffer());
+    } finally {
+      await this.call(`/speech-syntheses/${uuid}`, { method: "DELETE" }).catch(() => {});
+    }
+  }
+
+  private async write(props: VoisonaProps, text: string, out: string): Promise<void> {
+    const { voice, language } = await this.voice(props);
+    const global: Record<string, number | number[]> = {};
+    for (const key of ["speed", "pitch", "volume", "alp", "huskiness", "intonation"] as const) {
+      const value = props[key];
+      if (value !== undefined) global[key] = value;
+    }
+    if (props.style) global.style_weights = await this.styleWeights(voice, props.style);
+    const body = {
+      language,
+      voice_name: voice.voice_name,
+      voice_version: voice.voice_version,
+      global_parameters: global,
+      destination: "memory",
+      force_enqueue: true,
+    };
+    const wavs: Buffer[] = [];
+    for (const part of splitText(text, props["max-chars"] ?? 500)) wavs.push(await this.speak(body, part));
+    await Bun.write(out, wavs.length === 1 ? wavs[0] : joinWavs(wavs));
+  }
+
+  synthesize(props: VoisonaProps, text: string, out: string): Promise<void> {
+    return this.limited(() => this.write(props, text, out));
+  }
+}
+
+const servers = new Map<string, Server>();
+
+const server = (url: string | undefined): Server => {
+  const at = (url ?? process.env.MEKURU_VOISONA_URL ?? DEFAULT_URL).replace(/\/+$/, "");
+  const found = servers.get(at);
+  if (found) return found;
+  const created = new Server(at);
+  servers.set(at, created);
+  return created;
+};
+
+export const Voisona = async (props: VoisonaProps): Promise<Awaited<JSX.Element>> => {
+  const { url, ...voice } = voiceOf(props);
+  return line("voisona", props, props.voice, voice, (say, out) => server(url).synthesize(props, say, out));
+};
+
+export * from "./characters.ts";